CWL v1.0 improvements over sbg:draft-2
Overview
The following command line tool changes are implemented for CWL v1.0 in comparison to sbg:draft-2.
No more $job for BiX
$job
does not exist in JavaScript expressions. Available contexts are $inputs
and $runtime
.
sbg:draft-2 | CWL v1.0 |
---|---|
$job.inputs.input_bam.path | $(inputs.input_bam.path) |
$job.allocatedResources.cpu | $(runtime.cores) |
Entering a JS expression
Expressions are denoted by the syntax $(...)
or ${...}
. A code fragment wrapped in the $(...)
syntax is used as a one-line expression. A code fragment wrapped in ${...}
behaves like expressions in sbg:draft-2.
sbg:draft-2 | CWL v1.0 |
---|---|
$job.inputs.input_bam.path + ‘.vcf’ | $(inputs.input_bam.path).vcf |
{ return $job.inputs.input_bam.path + ‘.vcf’ } | ${ return inputs.input_bam.path + ‘.vcf’} |
Commands relating to file paths (basename, dirname, nameroot, nameext)
Experience improved commands relating to file paths. For instance, use .nameroot
to get the input basename. Learn more from CWL's documentation.
Expression | Return |
---|---|
$(inputs.input_bam.path) | /path/to/file.sorted.bam |
$(inputs.input_bam.basename) | file.sorted.bam |
$(inputs.input_bam.dirname) | /path/to |
$(inputs.input_bam.nameroot) | file.sorted |
$(inputs.input_bam.nameext) | bam |
One-line expressions
If the input is a BAM and the output is a VCF, you can define the output name as $(inputs.input_bam.nameroot).vcf
.
No expressions in the base command
All the expressions must be inserted via arguments.
This means that if you want to insert pre-commands (such as un-TAR reference files), you need to define the whole command through arguments and leave the base command empty.
Expressions in secondary files
Secondary files can now be defined with an expression.
For example, if the input is VCF or VCF.GZ, the secondary file is either .idx or .tbi.: $(self.nameext == 'gz' ? "tbi" : "idx")
.
StepInputExpressionRequirement
Workflow connection type incompatibility can now be resolved by applying JS expressions on step inputs. This simple example converts an incoming file into a list with one file, because the tool expects a list of files. Any custom value manipulation can be performed, such as converting single-item list to a single element, converting a single element into a list, list flattening, removing nulls from a list and much more. Whatever you need to make the connected tools compatible without changing the tools.
The value of self
is the incoming value. in case of multiple connections to one input port, linkMerge is applied first. In case of scatter, scatter is applied first and valueFrom
is performed separately for each element in the scattered list.
in:
- id: array_of_files
source: single_file
valueFrom: "$(self ? [self] : self)"
InitialWorkDirRequirement
You can create a file in workdir
on runtime or make a file available in the workdir
on runtime using InitialWorkDirRequirement
.
Create literal content file example
requirements:
- class: InitialWorkDirRequirement
listing:
- entry: $(JSON.stringify(inputs))
entryname: cwl.inputs.json
Create expression content file example:
requirements:
- class: InitialWorkDirRequirement
listing:
- entry: |-
Some contents
entryname: dynamic_input.txt
Stage an input example:
requirements:
- class: InitialWorkDirRequirement
listing:
- "$(inputs.bam)"
- "$(inputs.fastq_list)"
Example with staging inputs and creating files
requirements:
- class: InitialWorkDirRequirement
listing:
- entry: $(JSON.stringify(inputs))
entryname: cwl.inputs.json
- entry: |-
${
return JSON.stringify(inputs)
}
entryname: '${return "_1_cwl.inputs.json"}'
- "$(inputs.bam)"
- "$(inputs.fastq_list)"
ExpressionLibRequirement
Use ExpressionLibRequirement
to write a JS function in one place and use it in multiple places.
Define a function as:
requirements:
- class: InlineJavascriptRequirement
expressionLib:
- var ext = function(){
var x = inputs.vcf.nameext == 'gz' ? "tbi" : "idx";
return x
};
And call it in a different place:
secondaryFiles:
${
return ext()
}
SchemaDefRequirement
Custom input and output structures can be defined in SchemaDefRequirement
.
class: SchemaDefRequirement
types:
- name: FileRecord
type: record
fields:
- name: file
type: File
- name: metadata
type: map
values: string
shellQuote
Keep this as false.
EnvVarRequirement
Instead of starting a command with export MY_CUSTOM_VARIABLE=DzoniJovanovic
, use EnvVarRequirement
.
requirements:
EnvVarRequirement:
envDef:
FILIP: $(inputs.last_name)
ExpressionTool
Similar to command-line tool except it has no command line and does not start a Docker container. The only purpose of this is to reshape stuff.
Example:
- https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/decider_bwa_expression.cwl
- https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/sort_scatter_expression.cwl
Input/Output Type Directory
The input can be type directory.
Input/Output Union Type
An input or output can be defined to be one or more possible types.
Example: Intervals input string or BED file
Input/Output Format Ontology
inputs:
aligned_sequences:
type: File
label: Aligned sequences in BAM format
format: edam:format_2572
$namespaces:
edam: http://edamontology.org/
$schemas:
- http://edamontology.org/EDAM_1.18.owl
SoftwareRequirement
hints:
SoftwareRequirement:
packages:
interproscan:
specs: [ "https://identifiers.org/rrid/RRID:SCR_005829" ]
version: [ "5.21-60" ]
Updated less than a minute ago