Transfer Data to Terra
To transfer BICCN data into Terra to run batch pipelines (existing or your own) or scalable Jupyter notebooks, start by going to the NeMO archive. If you are in Terra, the NeMO archives can be found in the data library as a data set. Both navigating to NeMO through this link or going to NeMO data set in the Terra Library, will take you to the NeMO data portal. Search for data as detailed in this wiki, and then transfer data to Terra. The instructions below pick up at the last step shown in the "Search for Data in the NeMO Archives" page.
Continues the "Search for Data in the NeMO Archives" page.
Click on the shopping cart to view the contents of the shopping cart. You can transfer the files to Terra which includes metadata for leveraging pipelines.
Click on the Terra button to transfer data. You may also optionally download the data and metadata with the download manifest option.
- If you have an existing workspace you want to append data, feel free to select an existing workspace.
- If not, select the option to create a workspace. Before creating a workspace you will need to have a billing project available to you in Terra (to select in the create view). You either need to create a billing project or ask for access to an existing billing object. These costs are for charges from the underlying cloud computing platform (eg. computing, storage, and egress).
Fill out the workspace name, select a billing project, and give an optional description. Workspace name and billing association will be permanent and not changeable (outside of deleting the workspace).
Congratulations! You are done, now what?
- You have data, data is found under the data tab.
- "Files" goes directly to the files in the underlying storage, "Table" are metadata tables that can be used to run many pipelines at once. These are updated when outputs are made to link inputs and metadata to output files.
- Workflows gives you access to workflows from Dockstore and from the Terra methods registry, feel free to import them into you workspace to use them. There are also curated workflows/analysis under the hamburger on the upper left and in the library. Here is some workflow documentation to get started with workflows.
- Notebooks are able to be added/turned on for your workspace to work with you data by cliking on Notebook Runtime at the upper right of your workspace.
- Learn more in general about Terra at our support environment. A quick overview on some subjects can be found in these videos.
