Import from a DRS server

OVERVIEW

CAVATICA allows you to import DRS files from either open external sources or the following compatible Seven Bridges environments:

After importing the files, you can then use them as any other file on CAVATICA. Please note that the actual files physically remain hosted on the source platform and are only linked to CAVATICA.

PREREQUISITE

In order to be able to import a DRS file, your account on CAVATICA has to be connected to the DRS server of the environment you are importing data from.

Learn more about this procedure.

Access the import feature

  1. Open the project to which you're importing files.
  2. Select the Files tab and then click + Add Files.
  3. Click Add Files > GA4GH Data Repository Service (DRS), as shown below.
447447

The page for importing files is displayed. The following two options are available:

867867
  • Paste DRS URIs - use this option to paste the list of DRS URIs of the files you want to import (see below).
  • From a manifest file - use this option to import files by using a manifest (see below).

Import files by pasting DRS URIs

  1. Enter DRS URIs into the text area (see Form a DRS URI below).

📘

Important notes

The maximum number of links you can put inside the text area is 500. To import again, you will have to wait for 5 minutes.

In addition, please bear in mind that all other API calls you make within this time frame will also count toward this limit. URLs should be entered one per row.

  1. Optional: Enter tags in the Tag files box (see below). 
  2. Resolve naming conflicts-  choose the method for resolving a naming conflict; read more below.

📘

Allowed characters in file names are all alphanumerics and special characters except slash (/). File names can’t be longer than 255 characters.Allowed characters in folder names are a-z, A-Z, 0-9, and special characters (_), (-), and (.). Folder names can’t be longer than 255 characters.

  1. Give your consent regarding importing sensitive data by checking the related option.
  2. Click Import to start the upload.

Import from a manifest file

To import files from a manifest file:

  1. Access your project.
  2. Click the Files tab.
  3. Click Add files and choose option "GA4GH Data Repository Service (DRS)".
866866
  1. Click Browse manifest and choose the manifest file from your computer.
865865

5, (Optional) Tag your files (see below).

  1. Choose the method for resolving naming conflicts (see below).
  2. Give your consent regarding importing sensitive data by checking the related option.
  3. Click Import.

The files are imported to your project.

Manifest file format

The following rules apply for creating the manifest file.

Field

Description

drs_uri
mandatory

The DRS uri of the file you are importing.

name
optional

The name of the file.

subsequent fields

All other fields in the manifest file are treated as metadata.

ADD TAGS TO THE FILES

To make it easier to visually identify and organize files you can tag them before the import. Separate tags with commas or press enter.

All tags which are already in the selected project are autosuggested in the drop-down menu. Learn more about tagging your files

RESOLVE NAMING CONFLICTS

Naming conflicts happen when you try to import an item (file or folder) from a DRS server to a project's root folder or subfolder, where an item with the same name already exists. The list below illustrates the possible cases of naming conflicts and the way they are resolved depending on which option is selected in the Resolve naming conflicts dropdown:

  1. Importing a file that has the same name as an existing file at the destination path:
    • Skip (default): The file won't be imported. This is the default behavior.
    • Auto-rename: The imported file will be automatically renamed by adding a prefix (underscore and serial number).
    • Overwrite: The existing file will be overwritten by the file imported from the server.
  2. Importing a file that has the same name as an existing folder at the destination path:
    • Skip (default): The file won't be imported.
    • Auto-rename: The imported file will be automatically renamed by adding a prefix (underscore and serial number).
    • Overwrite: The file won't be imported.

The following table illustrates the entire naming conflict resolution mechanism:

471471

DURING UPLOAD

Track the import progress under the import status icon on the right-hand side of the main menu. When the upload completes, review your files in the Files tab of your project.

Form a DRS URI on a compatible Seven Bridges environment

This step takes place on the CGC or BioData Catalyst Powered by Seven Bridges.

A DRS URI is formed by appending the ID of the file you want to import, to the DRS endpoint of the environment you are importing the files from.

To form a DRS URI for importing a file:

  1. Use the DRS endpoint for the environment you are importing the file from:
    CGC Powered by Seven Bridges: drs://cgc-ga4gh-api.sbgenomics.com
    BioData Catalyst Powered by Seven Bridges: drs://ga4gh-api.sb.biodatacatalyst.nhlbi.nih.gov
    You can also find the endpoint when logged in to the chosen environment, in the Account Settings page under Dataset Access.
  2. Get the ID of the file from the URL, once you have clicked on a file to see its details.
878878
  1. Form the DRS URI by following the <drs-endpoint>/<file-id> pattern. For example, a full DRS URI for a file you want to import from BioData Catalyst Powered by Seven Bridges would be: drs://ga4gh-api.sb.biodatacatalyst.nhlbi.nih.gov/5bcd123ce4b0359df2c239ae.
  2. (Optional) In the Add tags field add the keywords (tags) that describe the imported items.
  3. Resolve naming conflicts - Select the action to be taken if a naming conflict occurs. Available actions are Skip (default option), Auto-rename and Overwrite. Read more about naming conflicts resolution.
  4. Click the checkbox to confirm that you understand the terms of use.
  5. Click Import. The file import process starts.

Did this page help you?