{"_id":"5a830eb8175a9c002ef0cfd7","project":"5773dcfc255e820e00e1cd4d","version":{"_id":"5773dcfc255e820e00e1cd50","__v":26,"project":"5773dcfc255e820e00e1cd4d","createdAt":"2016-06-29T14:36:44.812Z","releaseDate":"2016-06-29T14:36:44.812Z","categories":["5773dcfc255e820e00e1cd51","5773df36904b0c0e00ef05ff","577baf92451b1e0e006075ac","577bb183b7ee4a0e007c4e8d","577ce77a1cf3cb0e0048e5ea","577d11865fd4de0e00cc3dab","578e62792c3c790e00937597","578f4fd98335ca0e006d5c84","578f5e5c3d04570e00976ebb","57bc35f7531e000e0075d118","57f801b3760f3a1700219ebb","5804d55d1642890f00803623","581c8d55c0dc651900aa9350","589dcf8ba8c63b3b00c3704f","594cebadd8a2f7001b0b53b2","59a562f46a5d8c00238e309a","5a2aa096e25025003c582b58","5a2e79566c771d003ca0acd4","5a3a5166142db90026f24007","5a3a52b5bcc254001c4bf152","5a3a574a2be213002675c6d2","5a3a66bb2be213002675cb73","5a3a6e4854faf60030b63159","5c8a68278e883901341de571","5cb9971e57bf020024523c7b","5cbf1683e2a36d01d5012ecd"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"category":{"_id":"577baf92451b1e0e006075ac","__v":0,"project":"5773dcfc255e820e00e1cd4d","version":"5773dcfc255e820e00e1cd50","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-07-05T13:01:06.540Z","from_sync":false,"order":6,"slug":"developer-hub","title":"Developer tools"},"user":"5613e4f8fdd08f2b00437620","githubsync":"","__v":0,"parentDoc":null,"metadata":{"title":"","description":"","image":[]},"updates":[],"next":{"pages":[],"description":""},"createdAt":"2018-02-13T16:13:44.607Z","link_external":false,"link_url":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":3,"body":"##Overview\n\nThe following command line tool changes are implemented for CWL v1.0 in comparison to sbg:draft-2.\n\n## No more $job for BiX8\n`$job` does not exist in JavaScript expressions. Available contexts are `$inputs` and `$runtime`.\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"sbg:draft-2\",\n    \"h-1\": \"CWL v1.0\",\n    \"0-0\": \"`$job.inputs.input_bam.path`\",\n    \"0-1\": \"`$(inputs.input_bam.path)`\",\n    \"1-0\": \"`$job.allocatedResources.cpu`\",\n    \"1-1\": \"`$(runtime.cores)`\"\n  },\n  \"cols\": 2,\n  \"rows\": 2\n}\n[/block]\n##Entering a JS expression\n\nExpressions are denoted by the syntax `$(...)` or `${...}`. A code fragment wrapped in the `$(...)` syntax is used as a one-line expression. A code fragment wrapped in `${...}` behaves like expressions in sbg:draft-2.\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"sbg:draft-2\",\n    \"h-1\": \"CWL v1.0\",\n    \"0-0\": \"`$job.inputs.input_bam.path + ‘.vcf’`\",\n    \"0-1\": \"`$(inputs.input_bam.path).vcf`\",\n    \"1-0\": \"`{ return $job.inputs.input_bam.path + ‘.vcf’ }`\",\n    \"1-1\": \"`${ return inputs.input_bam.path + ‘.vcf’}`\"\n  },\n  \"cols\": 2,\n  \"rows\": 2\n}\n[/block]\n##Commands relating to file paths (basename, dirname, nameroot, nameext)\n\nExperience improved commands relating to file paths. For instance, use `.nameroot` to get the input basename. Learn more from CWL's [documentation](http://www.commonwl.org/v1.0/CommandLineTool.html#File).\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"Expression\",\n    \"h-1\": \"Return\",\n    \"0-0\": \"`$(inputs.input_bam.path)`\",\n    \"0-1\": \"`/path/to/file.sorted.bam`\",\n    \"1-0\": \"`$(inputs.input_bam.basename)`\",\n    \"1-1\": \"`file.sorted.bam`\",\n    \"2-0\": \"`$(inputs.input_bam.dirname)`\",\n    \"2-1\": \"`/path/to`\",\n    \"3-0\": \"`$(inputs.input_bam.nameroot)`\",\n    \"3-1\": \"`file.sorted`\",\n    \"4-0\": \"`$(inputs.input_bam.nameext)`\",\n    \"4-1\": \"`bam`\"\n  },\n  \"cols\": 2,\n  \"rows\": 5\n}\n[/block]\n##One-line expressions\n\nIf the input is a BAM and the output is a VCF, you can define the output name as `$(inputs.input_bam.nameroot).vcf`.\n\n##No expressions in the base command\n\nAll the expressions must be inserted via arguments.\n\nThis means that if you want to insert pre-commands (such as un-TAR reference files), you need to define the whole command through arguments and leave the base command empty.\n\n##Expressions in secondary files\n\nSecondary files can now be defined with an expression.\n\nFor example, if the input is VCF or VCF.GZ, the secondary file is either .idx or .tbi.: `$(self.nameext == 'gz' ? \"tbi\" : \"idx\")`.\n\n##InitialWorkDirRequirement\n\nYou can create a file in `workdir` on runtime or make a file available in the `workdir` on runtime using `InitialWorkDirRequirement`.\n\n###Create literal content file example\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n \\n  - class: InitialWorkDirRequirement\\n    listing:\\n    - entry: $(JSON.stringify(inputs))\\n      entryname: cwl.inputs.json\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n###Create expression content file example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n \\n  - class: InitialWorkDirRequirement\\n    listing:\\n    - entry: |-\\n \\n         Some contents\\n \\n      entryname: dynamic_input.txt\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n###Stage an input example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n \\n - class: InitialWorkDirRequirement\\n   listing:\\n   - \\\"$(inputs.bam)\\\"\\n   - \\\"$(inputs.fastq_list)\\\"\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n###Example with staging inputs and creating files\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n \\n - class: InitialWorkDirRequirement\\n   listing:\\n   - entry: $(JSON.stringify(inputs))\\n     entryname: cwl.inputs.json\\n   - entry: |-\\n     ${\\n        return JSON.stringify(inputs)\\n     }\\n     entryname: '${return \\\"_1_cwl.inputs.json\\\"}'\\n   - \\\"$(inputs.bam)\\\"\\n   - \\\"$(inputs.fastq_list)\\\"\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##ExpressionLibRequirement\n\nUse `ExpressionLibRequirement` to write a JS function in one place and use it in multiple places.\n\nDefine a function as:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n - class: InlineJavascriptRequirement\\n \\n   expressionLib:\\n     - var ext = function(){\\n       var x = inputs.vcf.nameext == 'gz' ? \\\"tbi\\\" : \\\"idx\\\";\\n       return x\\n     };\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\nAnd call it in a different place:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"secondaryFiles:\\n ${\\n \\n    return ext()\\n \\n  }\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##SchemaDefRequirement\n\nCustom input and output structures can be defined in `SchemaDefRequirement`.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"class: SchemaDefRequirement\\ntypes:\\n- name: FileRecord\\n  type: record\\n  fields:\\n    - name: file\\n      type: File\\n    - name: metadata\\n      type: map   \\n      values: string\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##shellQuote\n\nKeep this as false.\n\n##EnvVarRequirement\n\nInstead of starting a command with `export MY_CUSTOM_VARIABLE=DzoniJovanovic`, use `EnvVarRequirement`.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"requirements:\\n EnvVarRequirement:\\n   envDef:\\n     FILIP: $(inputs.last_name)\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##ExpressionTool\n\nSimilar to command-line tool except it has no command line and does not start a Docker container. The only purpose of this is to reshape stuff.\n\nExample:\n* https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/decider_bwa_expression.cwl\n* https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/sort_scatter_expression.cwl\n\n##Input/Output Type Directory\n\nThe input can be type directory.\n\n##Input/Output Union Type\n\nAn input or output can be defined to be one or more possible types.\n\nExample: Intervals input string or BED file\n\n##Input/Output Format Ontology\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"inputs:\\n aligned_sequences:\\n   type: File\\n   label: Aligned sequences in BAM format\\n   format: edam:format_2572\\n \\n \\n \\n$namespaces:\\n edam: http://edamontology.org/\\n$schemas:\\n - http://edamontology.org/EDAM_1.18.owl\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]\n##SoftwareRequirement\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"hints:\\n SoftwareRequirement:\\n   packages:\\n     interproscan:\\n       specs: [ \\\"https://identifiers.org/rrid/RRID:SCR_005829\\\" ]\\n       version: [ \\\"5.21-60\\\" ]\",\n      \"language\": \"yaml\"\n    }\n  ]\n}\n[/block]","excerpt":"","slug":"cwl-v10-improvements-over-sbgdraft-2","type":"basic","title":"CWL v.1.0 improvements over sbg:draft-2"}

CWL v.1.0 improvements over sbg:draft-2


##Overview The following command line tool changes are implemented for CWL v1.0 in comparison to sbg:draft-2. ## No more $job for BiX8 `$job` does not exist in JavaScript expressions. Available contexts are `$inputs` and `$runtime`. [block:parameters] { "data": { "h-0": "sbg:draft-2", "h-1": "CWL v1.0", "0-0": "`$job.inputs.input_bam.path`", "0-1": "`$(inputs.input_bam.path)`", "1-0": "`$job.allocatedResources.cpu`", "1-1": "`$(runtime.cores)`" }, "cols": 2, "rows": 2 } [/block] ##Entering a JS expression Expressions are denoted by the syntax `$(...)` or `${...}`. A code fragment wrapped in the `$(...)` syntax is used as a one-line expression. A code fragment wrapped in `${...}` behaves like expressions in sbg:draft-2. [block:parameters] { "data": { "h-0": "sbg:draft-2", "h-1": "CWL v1.0", "0-0": "`$job.inputs.input_bam.path + ‘.vcf’`", "0-1": "`$(inputs.input_bam.path).vcf`", "1-0": "`{ return $job.inputs.input_bam.path + ‘.vcf’ }`", "1-1": "`${ return inputs.input_bam.path + ‘.vcf’}`" }, "cols": 2, "rows": 2 } [/block] ##Commands relating to file paths (basename, dirname, nameroot, nameext) Experience improved commands relating to file paths. For instance, use `.nameroot` to get the input basename. Learn more from CWL's [documentation](http://www.commonwl.org/v1.0/CommandLineTool.html#File). [block:parameters] { "data": { "h-0": "Expression", "h-1": "Return", "0-0": "`$(inputs.input_bam.path)`", "0-1": "`/path/to/file.sorted.bam`", "1-0": "`$(inputs.input_bam.basename)`", "1-1": "`file.sorted.bam`", "2-0": "`$(inputs.input_bam.dirname)`", "2-1": "`/path/to`", "3-0": "`$(inputs.input_bam.nameroot)`", "3-1": "`file.sorted`", "4-0": "`$(inputs.input_bam.nameext)`", "4-1": "`bam`" }, "cols": 2, "rows": 5 } [/block] ##One-line expressions If the input is a BAM and the output is a VCF, you can define the output name as `$(inputs.input_bam.nameroot).vcf`. ##No expressions in the base command All the expressions must be inserted via arguments. This means that if you want to insert pre-commands (such as un-TAR reference files), you need to define the whole command through arguments and leave the base command empty. ##Expressions in secondary files Secondary files can now be defined with an expression. For example, if the input is VCF or VCF.GZ, the secondary file is either .idx or .tbi.: `$(self.nameext == 'gz' ? "tbi" : "idx")`. ##InitialWorkDirRequirement You can create a file in `workdir` on runtime or make a file available in the `workdir` on runtime using `InitialWorkDirRequirement`. ###Create literal content file example [block:code] { "codes": [ { "code": "requirements:\n \n - class: InitialWorkDirRequirement\n listing:\n - entry: $(JSON.stringify(inputs))\n entryname: cwl.inputs.json", "language": "yaml" } ] } [/block] ###Create expression content file example: [block:code] { "codes": [ { "code": "requirements:\n \n - class: InitialWorkDirRequirement\n listing:\n - entry: |-\n \n Some contents\n \n entryname: dynamic_input.txt", "language": "yaml" } ] } [/block] ###Stage an input example: [block:code] { "codes": [ { "code": "requirements:\n \n - class: InitialWorkDirRequirement\n listing:\n - \"$(inputs.bam)\"\n - \"$(inputs.fastq_list)\"", "language": "yaml" } ] } [/block] ###Example with staging inputs and creating files [block:code] { "codes": [ { "code": "requirements:\n \n - class: InitialWorkDirRequirement\n listing:\n - entry: $(JSON.stringify(inputs))\n entryname: cwl.inputs.json\n - entry: |-\n ${\n return JSON.stringify(inputs)\n }\n entryname: '${return \"_1_cwl.inputs.json\"}'\n - \"$(inputs.bam)\"\n - \"$(inputs.fastq_list)\"", "language": "yaml" } ] } [/block] ##ExpressionLibRequirement Use `ExpressionLibRequirement` to write a JS function in one place and use it in multiple places. Define a function as: [block:code] { "codes": [ { "code": "requirements:\n - class: InlineJavascriptRequirement\n \n expressionLib:\n - var ext = function(){\n var x = inputs.vcf.nameext == 'gz' ? \"tbi\" : \"idx\";\n return x\n };", "language": "yaml" } ] } [/block] And call it in a different place: [block:code] { "codes": [ { "code": "secondaryFiles:\n ${\n \n return ext()\n \n }", "language": "yaml" } ] } [/block] ##SchemaDefRequirement Custom input and output structures can be defined in `SchemaDefRequirement`. [block:code] { "codes": [ { "code": "class: SchemaDefRequirement\ntypes:\n- name: FileRecord\n type: record\n fields:\n - name: file\n type: File\n - name: metadata\n type: map \n values: string", "language": "yaml" } ] } [/block] ##shellQuote Keep this as false. ##EnvVarRequirement Instead of starting a command with `export MY_CUSTOM_VARIABLE=DzoniJovanovic`, use `EnvVarRequirement`. [block:code] { "codes": [ { "code": "requirements:\n EnvVarRequirement:\n envDef:\n FILIP: $(inputs.last_name)", "language": "yaml" } ] } [/block] ##ExpressionTool Similar to command-line tool except it has no command line and does not start a Docker container. The only purpose of this is to reshape stuff. Example: * https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/decider_bwa_expression.cwl * https://github.com/NCI-GDC/gdc-dnaseq-cwl/blob/master/tools/sort_scatter_expression.cwl ##Input/Output Type Directory The input can be type directory. ##Input/Output Union Type An input or output can be defined to be one or more possible types. Example: Intervals input string or BED file ##Input/Output Format Ontology [block:code] { "codes": [ { "code": "inputs:\n aligned_sequences:\n type: File\n label: Aligned sequences in BAM format\n format: edam:format_2572\n \n \n \n$namespaces:\n edam: http://edamontology.org/\n$schemas:\n - http://edamontology.org/EDAM_1.18.owl", "language": "yaml" } ] } [/block] ##SoftwareRequirement [block:code] { "codes": [ { "code": "hints:\n SoftwareRequirement:\n packages:\n interproscan:\n specs: [ \"https://identifiers.org/rrid/RRID:SCR_005829\" ]\n version: [ \"5.21-60\" ]", "language": "yaml" } ] } [/block]