Data Studio environments and libraries

At the moment, Data Studio offers a set of predefined libraries curated by Seven Bridges bioinformaticians, which are automatically available every time an analysis is started.

The list of available libraries depends on the environment you are using (JupyterLab or RStudio) and the selected environment setup (set of preinstalled libraries that is available each time an analysis is started).

Both of these settings are selected in the analysis creation wizard and cannot be changed once the analysis has been created.

JupyterLab

Depending on the purpose and objective of your JupyterLab analysis, you can select an environment setup that you find most suitable for the given analysis.

The following table shows the available JupyterLab environment setups and some details about available tools and libraries in each of them:

All available environment setups also contain sevenbridges-python and sevenbridges-r API libraries, as well as htop and openvpn as general-purpose tools. The libraries are installed using conda, as JupyterLab supports multiple programming languages and conda is a language-agnostic package manager. You can also install libraries directly from the notebook and use them during the execution of your analysis.

For optimal performance and avoidance of potential conflicts, we recommend using conda when installing libraries within your analyses. However, unlike default libraries, libraries installed in that way will not be automatically available next time the analysis is started.

Spark initialization and loading of Parquet/VCF files for the SB Data Science - Spark 3.5.1, Python 3.11 environment setup

To initialize Spark in the Spark 3.5.1, Python 3.11 environment, use the following code:

from pyspark.sql import SparkSession
import glow
spark = SparkSession \
    .builder \
    .appName("PythonPi") \
    .config("spark.jars.packages", "io.projectglow:glow-spark3_2.12:2.0.0") \
    .config("spark.hadoop.io.compression.codecs", "io.projectglow.sql.util.BGZFCodec") \
    .getOrCreate()
spark = glow.register(spark)

When loading Parquet or VCF files, use the following pattern:

df = spark.read.parquet('/path/to/example.parquet')
df_vcf = spark.read.format('vcf').load('/path/to/file.vcf')

Spark initialization and loading of Parquet/VCF files for the SB Data Science - Spark 3.1.2, Python 3.9 environment setup

To initialize Spark in the Spark 3.1.2, Python 3.9 environment, use the following code:

from pyspark.sql import SparkSession
import glow
spark = SparkSession \
    .builder \
    .appName("PythonPi") \
    .config("spark.jars.packages", "io.projectglow:glow-spark3_2.12:1.1.2") \
    .config("spark.hadoop.io.compression.codecs", "io.projectglow.sql.util.BGZFCodec") \
    .getOrCreate()
spark = glow.register(spark)

When loading Parquet or VCF files, use the following pattern:

df = spark.read.parquet('/path/to/example.parquet')
df_vcf = spark.read.format('vcf').load('/path/to/file.vcf')

RStudio

If you select RStudio as the analysis environment, you can also select one of the available environment setups depending on the purpose of your analysis. This will help you optimize analysis setup and time to getting a fully-functional environment that suits your needs by having the needed libraries preinstalled in the selected environment setup. Here are the available options:

All available environment setups also contain the sevenbridges-r API library, as well as htop and openvpn as general-purpose tools.