## Description In the QuickStart tutorial Part I you’ll work through a mock study in a SB project to answer the burning question “How does a student’s height affect their grades?”. The study is made up, but the steps you will take, though simple, mirror those in a more complex, real study. In Part II, which is optional, you will create an app (a command line tool) from a Python script and chain the two apps together to make a workflow. The raw data is a TSV spreadsheet of heights and grades for 86 middle-school students. Data includes each student’s grades for language arts, math, and science. ### What you will do Each QuickStart step introduces basic platform functionality that you can use in your own analysis journey. #### Part I: Platform QuickStart 1. Upload the raw student grades data (TSV) to your project 2) Run an app ( calculate-average-1) to calculate the average GPA from each student’s three subject grades. 3. Import the processed grades TSV to a Jupyter notebook for plotting. #### Part II: Create an app (optional) 4. (optional) Convert the notebook analysis to an app (CL tool). 5) (optional) Modify calculate-average-1 to input the name of the new column in the studentgrades_processed TSV. 6. (optional exercise) Chain the two command line tools together in a workflow. ### Requirements - Make a copy of this project under your own Billing Group - Code is in Python, though you should not need to know Python to do the exercises. ## Part I: Platform QuickStart ### Step 1. Upload data _\[JAC] I want to include a simple section here where people can download the studentgrades.tsv from public storage and upload it to their copy of the project. _ ### S**tep-by-step instructions** See Upload your data to the platform in the Knowledge base. For larger files, see Upload via the Command Line. Once the file is uploaded, take a moment to preview it and familiarize yourself with the contents. ### Step 2. Process raw data Once you have the studentgrades.tsv data in your Files tab, the first step in your analysis is to calculate each student’s average grade from their three subject grades (in the studentgrades.tsv file). This is a processing step, similar to alignment or variant calling sequencer reads (but much less computationally expensive!). Luckily, your colleagues have already created an app (command line tool) that does this step. You will find `calculate-average-1` in the app tab of your project. The `calculate-average-1` tool takes in a tsv and a string for the output filename and outputs a modified TSV with an additional column, average_GPA that is the average of the three subject grades for each student. ### Step by step instructions to run an app For instructions on how to run the analysis, see Run a task in the Knowledge Center. ### Step 3. Plot results Now that you have your processed raw data, the next step is to view the results in an interactive Jupyter notebook running in a Data Studio environment. An interactive analysis lets you adjust parameters and plot results in real time. In the QuickStart, you will run a notebook called plot-data-from-tsv.ipynb in a pre-configured Jupyter Data Studio. ### Step by step instructions to plot results in Data Studio For instructions on how to run the analysis, see Run a Data Studio analysis in the Knowledge Center. ## Part II: Create an app tutorial (optional) ### When to use Data Studio versus app You may want to run an analysis interactively in Data Studio, so you can tweak plotting parameters and see the changes to your plots in real time, for example. However, to scale and automate your analysis, it’s preferable to create an app to do the job. ### Overview: How to create an app To create an app from a script or Python/R code from a notebook you’ll use the visual code editor in SB to wrap the code in CWL. This will involve updating the code slightly and setting the inputs and outputs explicitly. Before you start, you need to make sure your basic code is working, by running in a notebook, JupyterLab, RStudio, or terminal, for example. Once you know the code works, you are ready to convert the codebase to a command-line tool (CWL app in Seven Bridges). In this tutorial, you will be turning the notebook code you ran to plot results in the QuickStart into a Python script and wrapping it in CWL to make an app on the platform. ### Creating a tool (app) in Seven Bridges You'll use the tool editor in Seven Bridges to wrap the tool in CWL. If your tool is already installed in a Docker, you may be able to reference the existing Docker and simply define inputs and outputs in the Tool Editor. For step-by-step instructions, see Introduction to tool wrapping. ### 1. Create a tool and open in tool editor **1.1. **Go to the **Apps** tab in your project and click the **Create app** button. **1.2.** In the **Create new app** popup, select **Command line tool**, give the app a **name** and click the blue **Create** button at the bottom right. ### 2. Add the Docker and base command in the Visual Editor fields **2.1. **Scroll down to the **Docker Image** and **Base Command** sections. **2.2. **Fill in the Docker.
Like when you’re running a script in a terminal, you need to include the command language, such as Python 3 or R, first. ### Docker recommendations You can use a Docker from any repository by including it here. To keep costs and time low, use a small Docker, such as python:3.10-slim, and add the specific packages you need for plotting and statistics. **2.3.** Fill in the **Base command** name ### Base command naming The base command is the command you want to app to run. This can be a Python or R script, for example. You can name the base command whatever you want (you’ll add the script further down in the edit tool), for example `my_code.py`. ### 3. Add inputs **3.1.** Scroll down to the **Input Ports** section. You will add two inputs, one for the data file (TSV with student grades and heights) and one for the name of the output plot. **3.2.** For each input, click the **+ Add an Input** link (bottom left in the imports ports section) and fill in the **ID** and **Type** in the input popup on the right-hand side (screenshots below). 💡**Make sure to toggle inputs as “required” -** otherwise they don’t show up when you run the tool!

PLOT_INPUT “Required” toggled on Type equal “File”	OUTPUT_FILE_NAME “Required” toggled on Type equal “String”

### 4. Add an output port for the plot (i.e., png file) **4.1. **Fill in the following values in the output popup on the right-hand side. **4.2.** Don’t forget to define the Glob to match the filename to a type of file. **4.3. **If you’re using an input string to name it (as in this case where it’s an input variable), remember to use a wildcard.

	\<- ID: Choose a name \<- Type: File \<- Glob: \*.png

5\. Create the base command code (from notebook or script) If you’ve used a Data Studio instance to troubleshoot your code or have a working script to convert to an app, you already have the outline of the base command. However, you will need to adjust the code to work as an app.   **Remove extra code**
Notebooks often have a lot of interactive fluff – places where you interrogate the data to make sure it’s what you expect. This makes sense when you are running a script line by line, but not when you’ll be running it in batch mode.   Thus, the first step will be to strip out extra code and save the script as a text file. You’ll use this codebase (i.e., myscript.py) when creating the command line app.   See the example below of the Python codebase for the first app (initially run in a notebook), calculate-average-1. Note that it is often worth keeping comments, to help with reusability.   _Example: Stripped Python codebase for calculate-average-1 (toggle open to see)_ 1   import pandas as pd 2    3   # Get all the data from the student table and load into a pandas dataframe 4   studentgrades = pd.read_csv("../project-files/student-grades.txt", sep="\t") 5   6   # Rename the 'id' column to be more descriptive 7   studentgrades.rename(columns = \{'id':'student_ID'\}, inplace = True) 8     9   # calculate the average GPA for each student and add to the tsv as a new column called average_GPA   10   studentgrades\['average_GPA']=(studentgrades.GPA_language_arts+studentgrades.GPA_maths+studentgrades.GPA_science)/3 11   12   # write dataframe to new tsv file studentgrades_withaverage.tsv 13   studentgrades.to_csv("studentgrades_withaverage.tsv", index=False, sep="\t") _HINT: Stripped Python codebase for plot-it-1.py (toggle open to see)_ import import pandas as pd import matplotlib.pyplot as plt import seaborn as sns    # Load all the data from the processed .tsv file into a pandas dataframe studentgrades = pd.read_csv("../project-files/studentgrades_processed.tsv", sep="\t")     # Rename the 'id' column to be more descriptive studentgrades.rename(columns = \{'id':'student_ID'\}, inplace = True) 10      11 # Define bivariate plotting function to visualize the distribution between two continuous traits 12 plt.rcParams\["figure.figsize"] = \[6,4] 13 def bivariateDistributionPlot(data, var1, var2, kind = "scatter"):     with sns.axes_style("whitegrid"): 14 jplot = sns.jointplot(x = data\[var1], y = data\[var2], kind=kind, color="k", s=200) 15 jplot.set_axis_labels(var1, var2) 16       17 # Plot relationship between height and average_GPA 18 bivariateDistributionPlot(studentgrades, var1 = "Height", var2 = "average_GPA", kind = "scatter") 6\. Add the Python script (“File Requirements”) Once you have the basic codebase, you’re ready to add your f**ile requirements**.   ⛔ A note about special characters        The CWL editor cannot handle special characters, like dashes or pound signs. When naming variables and files, it’s best to use an underscore instead. 1. Scroll down to **FILE REQUIREMENTS**, click the blue **Add** button and select **File**.

2) In the popup window on the right, add the **File Name** (must the file referenced in the **base command** section) and click** \** (bottom right)** **to open the script editor.
3. Copy the base code (script) into the Expression Editor.

7\. Modify pip installs   📘If you’ve included all the necessary libraries and packages in a custom Docker, you won’t need to add them manually. To dd them in the base command will require some extra code. You’ll need to add some extra code to pip installs to make them work in an app. See the **Python** example below.   **💡Note that you need to repeat lines 3 and 4 (below) for _each pip install_.** **Example code in Data Studio** (Python) import pandas as pd **Example code in CL tool **(i.e., script.py) import subprocess   import sys   subprocess.check_call(\[sys.executable, "-m", "pip", "install", "pandas"])    import pandas as pd 8. Modify file input commands 📘Remember that your files in Data Studio are stored in the ephemeral Data Studio VM, not in glob storage (i.e., project “Files”). Each storage has a different file structure and file paths, and you will adjust the Python or script code to reflect this.    In general, you’ll need to be careful when referencing files in an app since files are stored in different directories in project storage (Files tab) versus Data Studio. You’ll use a **relative path in Data Studio**, but you’ll use an **input variable** in a CL tool.   **Reading in a file (Data Studio example):**    studentgrades = pd.read_csv("**../project-files/student-grades.txt**", sep="\t") **Reading in a file (CL tool example):** myfile = "$(inputs.input_student_grades.path)" Studentgrades = pd.read_csv(myfile, sep="\t" ) **💡**The best way to get the right inputs (the “inputs.input_student_grades.path” above) is by using the Editor tool.   ⛔ Don’t forget to append “.path” at the end, so the command can find the file!! 1. Click on the code file link under **File Requirements**.

2) Click inside the file contents box to pop out the code editor:

3. Create the input file variable(s), for example, myfile = “$()” and myoutputfile = “$()” in the example below.
4. Use the variable (i.e., myfile) instead of the relative file path (i.e., ../project-files/student-grades.txt) in the base command code.   6) Toggle on “Required”   **💡** Inputs need to be **required** (toggled on). Otherwise, you won’t have an opportunity to enter or select them when you run the app.    9. Change the output coding in the base command.   You’ll write to an output port in the app, which you’ve already set up in the Visual Editor. You need to include a step to explicitly write to a file (.png). And because you’re not using relative paths to define where the output goes, you will also want to be define a variable for the output file.   See examples of the difference between the code used in Data Studio versus an app below.   **Example: Plotting in a notebook (Data Studio):**   \# Plot function defined above bivariateDistributionPlot(studentgrades, var1 = "Height", var2 = "average_GPA", kind = "scatter") **Example: Writing to file (CL tool):**   \# Define the distribution between two continuous traits def bivariateDistributionPlot(data, var1, var2, kind = "scatter"):
with sns.axes_style("whitegrid"):
jplot = sns.jointplot(x = data\[var1], y = data\[var2], kind=kind, color="k", s=200)
jplot.set_axis_labels(var1, var2)      return jplot
\# Plot function defined above jplot = bivariateDistributionPlot(studentgrades, var1 = "Height", var2 = "average_GPA", kind = "scatter")
\# Define variable for output plot   myoutput="(inputs.plot_name).png"
\# Write to output file jplot.savefig(myoutput, dpi=150, bbox_inches="tight") 10. Add a hint To help with troubleshooting, it is helpful to output SB logs to standard output. In the Hints section, choose sbg:SaveLogs from the **Class** dropdown and add stdout.txt in the **Value** field. 11. Save and run app in Seven Bridges Run the newly created app ![Screenshot of adding a Docker in the visual tool editor](https://files.readme.io/80f8b80ab0555aa11c668774c7c7a834e9d7c6644b6cd3c2a943bc8145812e06-Adding-Docker-in-the-Tool-Editor_Screenshot.png)