{"metadata":{"image":[],"title":"","description":""},"api":{"url":"","auth":"required","params":[],"results":{"codes":[]},"settings":""},"next":{"description":"","pages":[]},"title":"Create and upload your Docker image","type":"basic","slug":"upload-your-docker-image","excerpt":"","body":"## Overview\n\nIn order to use your own tools on Cavatica, you need to install each tool in an individual [Docker](doc:docker-basics) image, and then upload the image to [the Cavatica Image Registry](doc:the-image-registry) or to Docker Hub.\n\n## Prerequisites\n\nBefore you can create and upload a Docker image, you need to [install Docker](doc:install-docker). You also need to make sure Docker is running, as follows:\n* **Mac OS 10.10.3 Yosemite or newer**: run Docker for Mac and start a terminal of your choice.\n* **Mac OS 10.8 Mountain Lion or newer**: run Docker Quickstart terminal to start Docker Machine.\n* **Windows 10**: run Docker for Windows and start a terminal of your choice.\n* **Windows 7** or **8**: run Docker Quickstart Terminal to start Docker Machine. \n* **Linux**: no action required.\n\n## Steps\n\nTo create and upload a Docker image:\n\n1. Run `docker login pgc-images.sbgenomics.com` and enter your Cavatica credentials. Don't forget: enter your [authentication token](doc:get-your-authentication-token) when prompted for a password.\n2. [Open a Docker base image](#section-open-a-docker-image).\n3. [Install your tool](#section-install-your-tool-inside-a-docker-container) in the image.\n4. [Commit your image](#section-commit-your-image).\n5. [Push your image](#section-push-your-image) to the Cavatica registry.\n\n## Open a Docker image\nIf you are installing a tool, you'll need to start from a [base image](doc:docker-basics). On the other hand, if you are modifying a tool that you have already uploaded, you can pull the image containing the tool from [the Cavatica registry](doc:the-image-registry).\n\nTo install a tool, start from a base image. This can be any base image from Docker Hub, but starting with a plain operating system image like Ubuntu is generally recommended. To use this base image, enter:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker run -ti ubuntu\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nTo pull an image from the Cavatica registry, enter the repository that the image is stored in, followed by the image tag (if any), separated by a colon. The full repository path has the format `pgc-images.sbgenomics.com/<username>/<repository_name>[:tag]`. For example, if the user `rfranklin` wanted to open the image tagged `1.3` from her `picard` repository, she would enter\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"docker-run -ti pgc-images.sbgenomics.com/rfranklin/picard:1.3\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## Install your tool inside a Docker container\nInside a container you can install your chosen tool or modify an existing tool. Do this in the way that you would normally, using methods appropriate for your tool, e.g. `apt-get`. Files from the local host can be accessed in the directory `/mountedcwd`. So, if the software you want to install is stored in a remote repository, you can `git pull` your code and mount it into the container.\n\nWhen you've finished installing a tool, leave the container by typing `exit`.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"root:::at:::container$ exit\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## Commit your image\nAfter you exit the container, you can commit the image of it.\nFirst, list all your local containers, so that you can commit an image (snapshot) of the ubuntu container that you just created. The `-a` option here lists all containers, include those that are not currently running; you should see a container that was just created recently. This is the one you want.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$ docker ps -a\\n\\nCONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES\\n\\nc52b82b497eb        ubuntu              \\\"/bin/bash\\\"              2 minutes ago       Exited (0) 2 minutes ago                        kickass_liskov\\nae194ed75819        debian              \\\"/bin/bash\\\"              7 hours ago         Exited (0) 7 hours ago                          jovial_swanson\\n34ff1377fbee        hello-world         \\\"/hello\\\"                 26 hours ago        Exited (0) 26 weeks ago                         high_almeida\\n54240578230c        c0bfb9c8e377        \\\"/bin/sh -c '/usr/gam\\\"   26 hours ago        Exited (0) 26 weeks ago                         serene_pare\\n517904a42f3d        docker/whalesay     \\\"cowsay hhLinked Appl\\\"   27 hours ago        Exited (0) 27 weeks ago                         romantic_bhaskara\\n1aad55d740cd        docker/whalesay     \\\"cowsay docker/whales\\\"   27 hours ago        Exited (0) 27 weeks ago                         cocky_bhaskara\\n7bfb18e0d18a        hello-world         \\\"/hello\\\"                 28 hours ago        Exited (0) 28 weeks ago                         stupefied_williams\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nGrab the `CONTAINER ID` of the `ubuntu` image that was created 2 minutes ago. It's `c52b82b497eb`.\n\nNow, we'll commit an image of that container. This also gives you the opportunity to name your image. You must name it with the format `pgc-images.sbgenomics.com/<username>/<repository_name>[:tag]`. For example, if the user `rfranklin` wanted to commit her changes to a container in a repository named `picard`, with tag `1.4` she would name the image `pgc-images.sbgenomics.com/rfranklin/picard:1.4`.\n\nCommit the image as follows:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$ docker commit c52b82b497eb pgc-images.sbgenomics.com/rfranklin/picard:1.4\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nIf you want to confirm that the image has been named, you can list all of your local images.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$ docker images \\n\\nREPOSITORY                                TAG                 IMAGE ID            CREATED             VIRTUAL SIZE\\n\\npgc-images.sbgenomics.com/rfranklin/picard   1.4              0fe5d1d1aaec        10 minutes ago      125.1 MB\\nrfranklin/test                            latest              0fe5d1d1aaec        18 minutes ago      125.1 MB\\ndebian                                    latest              7a01cc5f27b1        7 hours ago         125.1 MB\\nubuntu                                    latest              6cc0fc2a5ee3        8 months ago        187.9 MB\\nhello-world                               latest              0a6ba66e537a        8 months ago        960 B\\ndocker/whalesay                           latest              ded5e192a685        8 months ago        247 MB\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n## Push your image\nTo push your image to the Cavatica image registry, run the command `docker push pgc-images.sbgenomics.com/<username>/<repository_name>[:tag]`, where `<username>/<repository_name>[:tag]` refers to the image that you have already committed. For example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$ docker push pgc-images.sbgenomics.com/rfranklin/picard:1.4\\nThe push refers to a repository [pgc-images.sbgenomics.com/rfranklin/picard] (len: 1)\\[email protected]: pushed\\n1.4: digest: sha256:afc9023f29719ffd361cdcbc334fe4ec2c041997ee501a15a86ed7f6e9277008 size: 3990\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nThe progress of the image upload will be shown in the terminal. When it has completed, you will see the message `pushed`.\n\n## Delete a local Docker image\nIf you want to delete a Docker image, use `docker rmi pgc-images.sbgenomics.com/<username>/<repository_name>[:tag]`. For example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$ docker rmi pgc-images.sbgenomics.com/rfranklin/picard:1\\nDeleted 02c8c0913b94a09053dccded886512b77fbd2ecbe18930d1b059bb573f13afd1\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]","updates":[],"order":3,"isReference":false,"hidden":false,"sync_unique":"","link_url":"","link_external":false,"_id":"589dcff97510ce0f00a982de","parentDoc":null,"category":{"sync":{"isSync":false,"url":""},"pages":[],"title":"Docker","slug":"docker","order":7,"from_sync":false,"reference":false,"_id":"5eaff69e844d67003642a020","createdAt":"2020-05-04T11:03:58.395Z","version":"5773dcfc255e820e00e1cd50","project":"5773dcfc255e820e00e1cd4d","__v":0},"__v":0,"createdAt":"2017-02-10T14:36:41.948Z","githubsync":"","project":"5773dcfc255e820e00e1cd4d","user":"575e85ac41c8ba0e00259a44","version":{"version":"1.0","version_clean":"1.0.0","codename":"","is_stable":true,"is_beta":false,"is_hidden":false,"is_deprecated":false,"categories":["5773dcfc255e820e00e1cd51","5773df36904b0c0e00ef05ff","577baf92451b1e0e006075ac","577bb183b7ee4a0e007c4e8d","577ce77a1cf3cb0e0048e5ea","577d11865fd4de0e00cc3dab","578e62792c3c790e00937597","578f4fd98335ca0e006d5c84","578f5e5c3d04570e00976ebb","57bc35f7531e000e0075d118","57f801b3760f3a1700219ebb","5804d55d1642890f00803623","581c8d55c0dc651900aa9350","589dcf8ba8c63b3b00c3704f","594cebadd8a2f7001b0b53b2","59a562f46a5d8c00238e309a","5a2aa096e25025003c582b58","5a2e79566c771d003ca0acd4","5a3a5166142db90026f24007","5a3a52b5bcc254001c4bf152","5a3a574a2be213002675c6d2","5a3a66bb2be213002675cb73","5a3a6e4854faf60030b63159","5c8a68278e883901341de571","5cb9971e57bf020024523c7b","5cbf1683e2a36d01d5012ecd","5dc15666a4f788004c5fd7d7","5eaff69e844d67003642a020","5eb00899b36ba5002d35b0c1","5eb0172be179b70073dc936e","5eb01b42b36ba5002d35ebba","5eb01f202654a20136813093","5eb918ef149186021c9a76c8"],"_id":"5773dcfc255e820e00e1cd50","__v":33,"createdAt":"2016-06-29T14:36:44.812Z","releaseDate":"2016-06-29T14:36:44.812Z","project":"5773dcfc255e820e00e1cd4d"}}

Create and upload your Docker image


## Overview In order to use your own tools on Cavatica, you need to install each tool in an individual [Docker](doc:docker-basics) image, and then upload the image to [the Cavatica Image Registry](doc:the-image-registry) or to Docker Hub. ## Prerequisites Before you can create and upload a Docker image, you need to [install Docker](doc:install-docker). You also need to make sure Docker is running, as follows: * **Mac OS 10.10.3 Yosemite or newer**: run Docker for Mac and start a terminal of your choice. * **Mac OS 10.8 Mountain Lion or newer**: run Docker Quickstart terminal to start Docker Machine. * **Windows 10**: run Docker for Windows and start a terminal of your choice. * **Windows 7** or **8**: run Docker Quickstart Terminal to start Docker Machine. * **Linux**: no action required. ## Steps To create and upload a Docker image: 1. Run `docker login pgc-images.sbgenomics.com` and enter your Cavatica credentials. Don't forget: enter your [authentication token](doc:get-your-authentication-token) when prompted for a password. 2. [Open a Docker base image](#section-open-a-docker-image). 3. [Install your tool](#section-install-your-tool-inside-a-docker-container) in the image. 4. [Commit your image](#section-commit-your-image). 5. [Push your image](#section-push-your-image) to the Cavatica registry. ## Open a Docker image If you are installing a tool, you'll need to start from a [base image](doc:docker-basics). On the other hand, if you are modifying a tool that you have already uploaded, you can pull the image containing the tool from [the Cavatica registry](doc:the-image-registry). To install a tool, start from a base image. This can be any base image from Docker Hub, but starting with a plain operating system image like Ubuntu is generally recommended. To use this base image, enter: [block:code] { "codes": [ { "code": "docker run -ti ubuntu", "language": "text" } ] } [/block] To pull an image from the Cavatica registry, enter the repository that the image is stored in, followed by the image tag (if any), separated by a colon. The full repository path has the format `pgc-images.sbgenomics.com/<username>/<repository_name>[:tag]`. For example, if the user `rfranklin` wanted to open the image tagged `1.3` from her `picard` repository, she would enter [block:code] { "codes": [ { "code": "docker-run -ti pgc-images.sbgenomics.com/rfranklin/picard:1.3", "language": "text" } ] } [/block] ## Install your tool inside a Docker container Inside a container you can install your chosen tool or modify an existing tool. Do this in the way that you would normally, using methods appropriate for your tool, e.g. `apt-get`. Files from the local host can be accessed in the directory `/mountedcwd`. So, if the software you want to install is stored in a remote repository, you can `git pull` your code and mount it into the container. When you've finished installing a tool, leave the container by typing `exit`. [block:code] { "codes": [ { "code": "[email protected]$ exit", "language": "text" } ] } [/block] ## Commit your image After you exit the container, you can commit the image of it. First, list all your local containers, so that you can commit an image (snapshot) of the ubuntu container that you just created. The `-a` option here lists all containers, include those that are not currently running; you should see a container that was just created recently. This is the one you want. [block:code] { "codes": [ { "code": "$ docker ps -a\n\nCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES\n\nc52b82b497eb ubuntu \"/bin/bash\" 2 minutes ago Exited (0) 2 minutes ago kickass_liskov\nae194ed75819 debian \"/bin/bash\" 7 hours ago Exited (0) 7 hours ago jovial_swanson\n34ff1377fbee hello-world \"/hello\" 26 hours ago Exited (0) 26 weeks ago high_almeida\n54240578230c c0bfb9c8e377 \"/bin/sh -c '/usr/gam\" 26 hours ago Exited (0) 26 weeks ago serene_pare\n517904a42f3d docker/whalesay \"cowsay hhLinked Appl\" 27 hours ago Exited (0) 27 weeks ago romantic_bhaskara\n1aad55d740cd docker/whalesay \"cowsay docker/whales\" 27 hours ago Exited (0) 27 weeks ago cocky_bhaskara\n7bfb18e0d18a hello-world \"/hello\" 28 hours ago Exited (0) 28 weeks ago stupefied_williams", "language": "text" } ] } [/block] Grab the `CONTAINER ID` of the `ubuntu` image that was created 2 minutes ago. It's `c52b82b497eb`. Now, we'll commit an image of that container. This also gives you the opportunity to name your image. You must name it with the format `pgc-images.sbgenomics.com/<username>/<repository_name>[:tag]`. For example, if the user `rfranklin` wanted to commit her changes to a container in a repository named `picard`, with tag `1.4` she would name the image `pgc-images.sbgenomics.com/rfranklin/picard:1.4`. Commit the image as follows: [block:code] { "codes": [ { "code": "$ docker commit c52b82b497eb pgc-images.sbgenomics.com/rfranklin/picard:1.4", "language": "text" } ] } [/block] If you want to confirm that the image has been named, you can list all of your local images. [block:code] { "codes": [ { "code": "$ docker images \n\nREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE\n\npgc-images.sbgenomics.com/rfranklin/picard 1.4 0fe5d1d1aaec 10 minutes ago 125.1 MB\nrfranklin/test latest 0fe5d1d1aaec 18 minutes ago 125.1 MB\ndebian latest 7a01cc5f27b1 7 hours ago 125.1 MB\nubuntu latest 6cc0fc2a5ee3 8 months ago 187.9 MB\nhello-world latest 0a6ba66e537a 8 months ago 960 B\ndocker/whalesay latest ded5e192a685 8 months ago 247 MB", "language": "text" } ] } [/block] ## Push your image To push your image to the Cavatica image registry, run the command `docker push pgc-images.sbgenomics.com/<username>/<repository_name>[:tag]`, where `<username>/<repository_name>[:tag]` refers to the image that you have already committed. For example: [block:code] { "codes": [ { "code": "$ docker push pgc-images.sbgenomics.com/rfranklin/picard:1.4\nThe push refers to a repository [pgc-images.sbgenomics.com/rfranklin/picard] (len: 1)\[email protected]: pushed\n1.4: digest: sha256:afc9023f29719ffd361cdcbc334fe4ec2c041997ee501a15a86ed7f6e9277008 size: 3990", "language": "text" } ] } [/block] The progress of the image upload will be shown in the terminal. When it has completed, you will see the message `pushed`. ## Delete a local Docker image If you want to delete a Docker image, use `docker rmi pgc-images.sbgenomics.com/<username>/<repository_name>[:tag]`. For example: [block:code] { "codes": [ { "code": "$ docker rmi pgc-images.sbgenomics.com/rfranklin/picard:1\nDeleted 02c8c0913b94a09053dccded886512b77fbd2ecbe18930d1b059bb573f13afd1", "language": "text" } ] } [/block]