Skip to content

Commit

Permalink
Release 1.1.0 (#43)
Browse files Browse the repository at this point in the history
* Add reticulate to renv.lock

* Rename example notebook

* Add content to README

* Add support for Synapse PAT (fixes #42)

* Add documentation to render HTML doc

* Remove reference to latex in README

* Update README

* Update conda env

* Fix lint checks
  • Loading branch information
tschaffter authored May 13, 2021
1 parent fe91a47 commit 59882ba
Show file tree
Hide file tree
Showing 13 changed files with 208 additions and 162 deletions.
6 changes: 2 additions & 4 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
# RStudio
RSTUDIO_USER=rstudio
RSTUDIO_PASSWORD=yourpasswordhere
RSTUDIO_PASSWORD=yourpassword
RSTUDIO_USERID=1000
RSTUDIO_GROUPID=1000
RSTUDIO_ROOT=FALSE

# Synapse
#SYNAPSE_USERNAME=yourusername
#SYNAPSE_API_KEY=yourapikey # TODO replace by PAT
#SYNAPSE_PAT=yourtokenhere
SYNAPSE_TOKEN=yourtoken

# Port on the host that connects to RStudio
HOST_PORT=80
6 changes: 0 additions & 6 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,15 @@ updates:
schedule:
interval: "monthly"
target-branch: "main"
reviewers:
- "tschaffter"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
target-branch: "main"
reviewers:
- "tschaffter"

- package-ecosystem: "pip"
directory: "/conda/sage"
schedule:
interval: "monthly"
target-branch: "main"
reviewers:
- "tschaffter"
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
args: Dockerfile
- name: Check that packages in requirements.txt files are in asc order
run: |
sort -f --check conda/sage/requirements.txt
sort -f --check conda/sage-bionetworks/requirements.txt
test:
needs: [lint]
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/lock.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: 'Lock Threads'
name: Lock threads

on:
schedule:
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
## TBA
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ RUN curl -fsSLO https://repo.anaconda.com/miniconda/Miniconda3-${miniconda3_vers
# Create conda environments
COPY conda /tmp/conda
RUN conda init bash \
&& conda env create -f /tmp/conda/sage/environment.yaml \
&& conda env create -f /tmp/conda/sage-bionetworks/environment.yaml \
&& rm -fr /tmp/conda \
# Fix libssl issue that affects conda env used with reticulate
&& cp /usr/lib/x86_64-linux-gnu/libssl.so.1.1 \
Expand Down
230 changes: 90 additions & 140 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,25 +7,18 @@

Docker image for analyses using RStudio and Python-Conda

## Motivations
## Overview

- Provides a versionized development environment (R, Python)
- Renders R notebooks to HTML and PDF programmatically (GH Action, etc.)
This project provides

## Specification

- Extends the Docker image [rocker/rstudio]
- Behaves the same as [rocker/rstudio] and offer extra features (see below)
- Includes R packages to render HTML notebooks and use Python/conda (`reticulate`)
- Renders HTML and PDF notebooks from .Rmd files programmatically
- Comes with [Miniconda] installed
- Specifies the version of the R packages installed using `renv`
- Uses [GitHub Dependabot] to check Docker and pip dependencies
- a development environment where R and Python codes work together
- a portable development environment using Docker
- a versionized development environment

## Usage

1. Create and edit the file that contains the future environment variables. You
can initially start RStudio using this configuration as-is.
1. Create and edit the file that contains the environment variables. You can
initially start RStudio using this configuration as-is.

cp .env.example .env

Expand All @@ -36,117 +29,45 @@ Docker image for analyses using RStudio and Python-Conda
RStudio is now available at http://localhost. On the login page, enter the
default username (`rstudio`) and the password specified in `.env`.

To stop the API service, enter `Ctrl+C` followed by `docker-compose down`. If
running in detached mode, you will only need to enter `docker-compose down`.

## Versioning

### GitHub tags

This repository uses [semantic versioning] to track the releases of this
project. This repository uses "non-moving" GitHub tags, that is, a tag will
always point to the same git commit once it has been created.

### Docker tags

The artifact published by this repository is a Docker image. The versions of the
image are aligned with the versions of [rocker/rstudio], not the versions of
Stubby or the GitHub tags of this repository.

The table below describes the image tags available.

| Tag name | Moving | Description
|---------------------------------|--------|------------
| `latest` | Yes | Latest stable release.
| `edge` | Yes | Lastest commit made to the default branch.
| `<major>` | Yes | Latest stable release for RStudio version `<major>`.
| `<major>.<minor>` | Yes | Latest stable release for RStudio version `<major>.<minor>`.
| `<major>.<minor>.<patch>` | Yes | Latest stable release for RStudio version `<major>.<minor>.<patch>`.
| `<major>.<minor>.<patch>-<sha>` | No | Same as above but with the reference to the git commit.

You should avoid using a moving tag like `latest` when deploying containers in
production, because this makes it hard to track which version of the image is
running and hard to roll back.

## Contributing

Thinking about contributing to this project? Get started by reading our
[Contributor Guide](CONTRIBUTING.md).

## License

[Apache License 2.0]

<!-- ## Starts RStudio
docker run --rm -p 8787:8787 -e PASSWORD=yourpassword tschaffter/rstudio
### Starts RStudio using docker-compose
This repository provides a `docker-compose.yml` to enable you to store your
configuration variables to file and start RStudio with a single command.
1. Copy *.env.example* to *.env*
2. Update the variables in *.env*
3. Start RStudio
docker-compose up -d
## Change the default username
The default username set by [rocker/rstudio] is `rstudio`. It is recommended to
use a different username for better security. This is achieved by setting the
environment variable `USER`
docker run --rm -p 8787:8787 \
--name rstudio \
-e USER=yourusername \
-e PASSWORD=yourpassword \
tschaffter/rstudio
## Set user / group identifiers
When using volumes (`-v` flags) permissions issues can arise between the host OS
and the container, [rocker/rstudio] avoid this issue by allowing you to specify
the user `USERID` and group `GROUPID`.
To stop RStudio, enter `Ctrl+C` followed by `docker-compose down`. If running
in detached mode, you will only need to enter `docker-compose down`.

Ensure any volume directories on the host are owned by the same user you specify
and any permissions issues will vanish like magic.
## Accessing logs

In this instance `USERID=1000` and `GROUPID=1000`, to find yours use id user as
below:
Follow the logs using `docker logs`

$ id username
uid=1000(abc) gid=1000(abc) groups=1000(abc)
docker logs --follow rstudio

## Give the user root permissions
Rotating log files are available in `/var/log/rstudio`.

Set the environment variable `ROOT=true` (default is `false`).
## Setting user / group identifiers

## Set Synapse credentials
When using Docker volumes, permissions issues can arise between the host OS and
the container. You can avoid these issues by letting RStudio know the User ID
(UID) and Group ID (GID) it should use when creating and editting files so that
these IDs match yours, which you can get using the command `id`:

Set the environment variables `SYNAPSE_USERNAME` and `SYNAPSE_API_KEY`. If both
variables are set, they will be used to create the configuration file
`~/.synapseConfig`.
$ id
uid=1000(archer) gid=1000(archer) groups=1000(archer)

This Docker image comes with [Miniconda] installed (see below) and a conda
environment named `sage`. This environment provides the [Synapse Python client]
that you can use to interact with the collaborative platform [Synapse].
In this example, we would set `RSTUDIO_USERID=1000` and `RSTUDIO_GROUPID=1000`.

## Access logs
## Giving the user root permissions

Follow the logs using `docker logs`
Set the environment variable `ROOT=TRUE` (default is `FALSE`).

docker logs --follow rstudio
## Setting Synapse credentials

Rotating log files are available in `/var/log/rstudio`.
Set the environment variables `SYNAPSE_TOKEN` to the value of one of your
Synapse Personal Access Tokens. If this variable is set, it will be used to
create the configuration file `~/.synapseConfig` when the container starts.

## Using Conda

The image [rocker/rstudio] comes with Python2 and Python3 installed. Here we
want to give the user the freedom to use any versions of Python and packages
using conda environments. Conda environments, through the isolation of Python
dependecies, also contribute to the reproducibility of experiements.
This Docker image comes with [Miniconda] installed (see below) and an example
Conda environment named `sage-bionetworks`. This environment includes packages
used to interact with the collaborative platform [Synapse] developed by [Sage
Bionetworks].

### From the terminal

Expand All @@ -158,57 +79,83 @@ or create a new one.
host $ docker exec -it rstudio bash
container # su yourusername
container $ conda env list
container $ conda activate sage
container $ conda activate sage-bionetworks

> Note: Use `conda config --set auto_activate_base false` to prevent conda from
automatically activating the default environment after logging in.
### In R
### From RStudio

The R code below lists the environment available before activating the existing
environment named `sage`.
environment named `sage-bionetworks`.

> library(reticulate)
> conda_list()
name python
1 miniconda /opt/miniconda/bin/python
2 sage /opt/miniconda/envs/sage/bin/python
> use_condaenv("sage", required = TRUE)
If the environment variables `SYNAPSE_USERNAME` and `SYNAPSE_API_KEY` were set
when the container started, you should be able to login to Synapse using the
[Synapse Python client].
> synapseclient <- reticulate::import('synapseclient')
> syn <- synapseclient$Synapse()
> syn$login()
Welcome, Max Caulfield!
2 sage-bionetworks /opt/miniconda/envs/sage/bin/python
> use_condaenv("sage-bionetworks", required = TRUE)

## Render an HTML and PDF notebook programmatically
## Rendering an HTML notebook using a CLI

This Docker image can be used to generate HTML and PDF notebooks from *.Rmd*
files programmatically. The command below mounts the folder `$(pwd)/notebooks`
to the container and instructs the program to render the notebook
[notebooks/notebook.Rmd](notebooks/notebook.Rmd) to HTML. The notebook generated
is saved to the same directory as the input notebook and has the same name but
with the extension `.nb.html`.
This Docker image provides the command `render` that generates an HTML or PDF
notebook from an R notebook (*.Rmd*). Run the command below from the host to
mount the directory `$(pwd)/notebooks` where the R notebook is and generate the
HTML notebook that will be saved to the same directory with the extension
`.nb.html`.

docker run --rm \
-v $(pwd)/notebooks:/data \
-e RENDER_INPUT="/data/notebook.Rmd" \
-e RENDER_INPUT="/data/example.Rmd" \
tschaffter/rstudio \
render

Run this command to convert the notebook to PDF (TBA)
## Versioning

### GitHub tags

This repository uses [semantic versioning] to track the releases of this
project. This repository uses "non-moving" GitHub tags, that is, a tag will
always point to the same git commit once it has been created. The tags of the
GitHub releases are available [here](https://github.com/tschaffter/rstudio/releases).

### Docker tags

The artifact published by this repository is the Docker image
[tschaffter/rstudio]. The versions of the image are aligned with the versions of
[rocker/rstudio], not the GitHub tags of this repository.

The table below describes the image tags available.

| Tag name | Moving | Description
|---------------------------------|--------|------------
| `latest` | Yes | Latest stable release.
| `edge` | Yes | Lastest commit made to the default branch.
| `<major>` | Yes | Latest stable major release `<major>`.
| `<major>.<minor>` | Yes | Latest stable minor release `<major>.<minor>`.
| `<major>.<minor>.<patch>` | Yes | Latest stable patch release `<major>.<minor>.<patch>`.
| `<major>.<minor>.<patch>-<sha>` | No | Same as above but with the reference to the git commit.

You should avoid using a moving tag like `latest` when deploying containers in
production, because this makes it hard to track which version of the image is
running and hard to roll back.

## Contributing

Thinking about contributing to this project? Get started by reading our
[Contributor Guide](CONTRIBUTING.md).

## License

[Apache License 2.0]

<!--
Similarly, run this command to convert the notebook to PDF.
docker run --rm \
-v $(pwd)/notebooks:/data \
-e RENDER_INPUT="/data/notebook.Rmd" \
-e RENDER_INPUT="/data/example.Rmd" \
-e RENDER_OUTPUT_FORMAT="pdf_document" \
tschaffter/rstudio \
render -->


render
-->

<!-- Links -->

Expand All @@ -220,3 +167,6 @@ Run this command to convert the notebook to PDF (TBA)
[semantic versioning]: https://semver.org/
[rocker/rstudio]: https://hub.docker.com/r/rocker/rstudio
[Apache License 2.0]: https://github.com/tschaffter/rstudio/blob/main/LICENSE
[Sage Bionetworks]: https://sagebionetworks.org
[reticulate]: https://rstudio.github.io/reticulate
[tschaffter/rstudio]: https://hub.docker.com/repository/docker/tschaffter/rstudio
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: sage
name: sage-bionetworks
channels:
- defaults
dependencies:
Expand Down
File renamed without changes.
3 changes: 1 addition & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ services:
- USERID=${RSTUDIO_USERID}
- GROUPID=${RSTUDIO_GROUPID}
- ROOT=${RSTUDIO_ROOT}
- SYNAPSE_USERNAME
- SYNAPSE_API_KEY
- SYNAPSE_TOKEN
volumes:
- .:/home/rstudio/awesome-project:rw
ports:
Expand Down
Loading

0 comments on commit 59882ba

Please sign in to comment.