From d11cc3f5bf1f8c7a31036eb1bc99d9bbedb9faa6 Mon Sep 17 00:00:00 2001 From: Alasdair Wilson Date: Fri, 12 Jan 2024 17:03:44 +0000 Subject: [PATCH 1/3] small fixes to docs --- README.md | 4 ++-- docs/about.md | 12 +++++++++--- docs/deployment.md | 44 ++++++++++++++++++++++++++++++-------------- docs/index.md | 4 ++-- 4 files changed, 43 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index aafdee1..0d3b274 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ This is a [Next.js](https://nextjs.org/) project bootstrapped with [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app). -To see our full documentation, please visit [our documentation site](oxfordrse.github.io/gutenberg). +To see our full documentation, please visit [our documentation site](https://blog.oxrse.uk/gutenberg/). To get a development environment running, follow the instructions below: @@ -61,4 +61,4 @@ dotenv -e .env.local npx prisma studio Gutenberg is automatically deployed to fly.io by a [GitHub action](https://github.com/OxfordRSE/gutenberg/actions/workflows/deploy.yml) whenever a new commit is pushed to the `main` branch. -To see how to deploy the application, see our [deployment guide](https://oxfordrse.github.io/docs/deployment/). +To see how to deploy the application, see our [deployment guide](https://blog.oxrse.uk/gutenberg/deployment). diff --git a/docs/about.md b/docs/about.md index ec21c95..31e16df 100644 --- a/docs/about.md +++ b/docs/about.md @@ -7,7 +7,7 @@ Gutenberg is being developed by the [Oxford Research Software Engineering team]( The aim is to provide an interactive training environment for the delivery of courses on various software engineering concepts, at all levels. -The application is made of two key components: +The application consists of two key components: - The course material. - The tools to interact with it. @@ -15,11 +15,17 @@ The application is made of two key components: ## Course Material [Course material](https://github.com/UNIVERSE-HPC/course-material) is provided currently from the [HPC Universe Project](https://universe-hpc.github.io/) though this can reasonably be exchanged for any similarly styled sets of course material in the form of markdown files. -The structrure of the markdown content is described in the HPC universe [contribution guide](https://github.com/UNIVERSE-HPC/course-material/blob/main/CONTRIBUTING.md). +The structure of the markdown content is described in the HPC universe [contribution guide](https://github.com/UNIVERSE-HPC/course-material/blob/main/CONTRIBUTING.md). + +The course material which the application will use can be configured to be any git repository, or multiple repositories. ## Teaching Tools -The basic operation of Gutenberg is in the design of courses called Events; +The basic operation of Gutenberg is in the design and management of courses called Events; Events are built from a collection of in person or remote lectures or labs with associated pages from the course material for students to work through. Events are then shared with the students, who can enrol and then follow along, attending the lectures at the listed times and places. Instructors can monitor the progress of students as they complete embedded problems in the material. In addition, students can provide feedback in the form of comments on the material, which are embedded threads in the appropriate pages, designed to report problems, have illuminating conversations about the material, or just comment on the concepts. + +To see how to get started with Gutenberg, see our [deployment guide](/deployment.md). + +For a introduction to using Gutenber to host an "event", see our [user guide](guide). diff --git a/docs/deployment.md b/docs/deployment.md index 38722e7..277c2f9 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -15,9 +15,9 @@ If you wish to see how to deploy the application to a development environment, p The deployment uses Docker and the Dockerfile is included in the repository, you can use this to deploy the application to any environment that supports Docker, e.g.: - A server with a public IP address to which you can deploy. -- A space to host the application, such as Fly.io or [Heroku](https://www.heroku.com/). +- A space to host the application, such as [Fly.io](https://fly.io) or [Heroku](https://www.heroku.com/). -In addition, you will need to supply a postgres database, which we intercat with via the prisma ORM. +In addition, you will need to supply a [Postgres](https://www.postgresql.org/) database, which we interact with via the [Prisma](https://www.prisma.io/) ORM. This can be hosted on the same server as the application, or on a different server, as long as the application can connect to it. ## Deploying via Fly.io @@ -28,10 +28,12 @@ After you have deployed your application for the first time, you can set the nec ### Automatic Deployment -The reccomended way to deploy is via a GitHub action. This action is already set up in the repository and can be run at any timne to deply the current `main` branch to fly.io. +The recommended way to deploy is via a GitHub action. +This action is already set up in the repository to run anytime `main` is updated or can be run at any time to deploy the current `main` branch to fly.io. +It may also be appropriate to tie the action to run automatically by e.g. tying it to running on tags or releases. -To enable this action, simply set the "FLY_API_TOKEN" secret in the repository settings to your API key. -You can now run the action manually on the default branch. +To enable use of this action, simply set the "FLY_API_TOKEN" secret in the repository settings to your API key. +You can now successfully run the action manually on the default branch. ### Manual Deployment @@ -43,11 +45,11 @@ fly deploy ``` This will build and deploy the app to fly. -N.B. you can, of course, set the API key in your env permanently via the .env file. +N.B. you can, of course, set the API key in your env permanently via an env file (e.g. .env.local), see the [env configuration documentation]({{ "/config/" | relative_url }}) for more details. #### Staging -In addition, the repository contains a workflow that will automatically deploy the application to a staging server, also hostred on, fly.io whenever a new commit is pushed to the `main` branch. +In addition, the repository contains a workflow that will automatically deploy the application to a staging server, also hosted on, fly.io whenever a new commit is pushed to the `main` branch. If you wish to use the staging application then please set the `DATABASE_URL_TEST` environment variable to the connection string for your staging database. ## Deploying via Docker @@ -66,24 +68,38 @@ docker build -t gutenberg . You can then deploy this docker image and set up appropriate forwarding to your machine. You must set the environment variables as described in the [configuration documentation]({{ "/config/" | relative_url }}) for the application to work properly. -It is reccomended that you use reverse proxy, such as [nginx](https://www.nginx.com/), to expose the application to the internet. +It is recommended that you use reverse proxy, such as [nginx](https://www.nginx.com/), to expose the application to the internet. ## Connecting to a Database Regardless of how you wish to deploy, the application will need to connect to a database. This can be done by setting the `DATABASE_URL` environment variable to the connection string for your database. -The Oxford deployment uses a PostgreSQL database hosted on Fly.io, but you can use any database that supports the prisma ORM. If you are deploying to docker it is recommended to use a dockerised database on the same virtual network. As such, your docker network will consist of the application container, the database container and the reverse proxy container. +The Oxford deployment uses a PostgreSQL database hosted on Fly.io, but the database can also be hosted elsewhere. +If you are deploying via Docker it is best to use a dockerised database on the same virtual network. +As such, your docker network will consist of the application container, the database container, the reverse proxy container and, optionally, the Qdrant database. ## Enabling Search -The material in gutenberg is searchable via a semantic search on a vector database, we use [qdrant](https://qdrant.tech/) for this and the vector embeddings are provided by openAI's [embedding API endpoint](https://platform.openai.com/docs/api-reference/embeddings). +The material in gutenberg is searchable via a semantic search on a vector database, we use [Qdrant](https://Qdrant.tech/) for this and the vector embeddings are provided by openAI's [embedding API endpoint](https://platform.openai.com/docs/api-reference/embeddings). -To enable search, you will need an openAI account, with credits available, then set the `OPENAI_API_KEY` environment variable to your openAI API key,the `QDRANT__API_KEY` environment variable to your qdrant API key, and the `QDRANT_DB_URL` environment variable to the URL of your qdrant database. -We host a qdrant database on fly.io, but you can use any qdrant database that is accessible to the application. -The vector database can be also be deployed via docker, see the [qdrant documentation](https://qdrant.tech/docs/deployment/docker.html) for more details. +To enable search, you will need an openAI account, with credits available, then set the `OPENAI_API_KEY` environment variable to your openAI API key, the `QDRANT_API_KEY` environment variable to your Qdrant API key, and the `QDRANT_DB_URL` environment variable to the URL of your Qdrant database. +We host our Qdrant database on fly.io, but you can use any Qdrant database that can be accessed by the application. +The vector database can be also be deployed via docker, see the [Qdrant documentation](https://qdrant.tech/docs/deployment/docker.html) for more details. The existence of the `COLLECTION_NAME` environment variable allows one vector database to be shared by any number of applications. +Once you have configured these environment variables, you should enable the search functionality in the frontend by setting the `NEXT_PUBLIC_ENABLE_SEARCH` environment variable to `true`. +If you do not wish to use the search functionality then you can leave this variable unset or set it to `false`. + +## Customisation + +You should customise your deployment by creating a configuration yaml file, see the [configuration documentation]({{ "/config/template" | relative_url }}) for more details. + ## Configuration -The application can be configured via environment variables, while we have touched on some key ones here, namely the database connection string and the search related variables, a full list of the environment variables can be found in the [env vars config section]({{ "/config/vars" | relative_url }}). +The application can be further configured via environment variables. +While we have touched on some key ones here, namely the database connection string and the search related variables, a full list of the environment variables and their purpose can be found in the [env vars config section]({{ "/config/vars" | relative_url }}). + +## Using the site + +Once you are deployed, you can visit our [guide](/guide) for an overview of how to set up and manage a course event. diff --git a/docs/index.md b/docs/index.md index c22895d..b7ec355 100644 --- a/docs/index.md +++ b/docs/index.md @@ -10,7 +10,7 @@ This is the documentation for `Gutenberg`. ## Quick Start -To get `Gutenberg` running follow our qucik start guide to deploying and configuring the app. +To get `Gutenberg` running follow our quick start guide to deploying and configuring the app. Once you are up and running, check out the user guide for instructions on how to use the application to deliver a course. - [Deploying Gutenberg](/deployment.md) @@ -19,4 +19,4 @@ Once you are up and running, check out the user guide for instructions on how to ## Development -If you wish to contribute to `Gutenberg` or simply want to customise the site further please refer to our [development guide](/development/). +If you wish to contribute to `Gutenberg` or simply want to customise the site further please refer to our [development guide](development). From e10a20de371a6c0de625720585bc16a2219223e0 Mon Sep 17 00:00:00 2001 From: Alasdair Wilson Date: Mon, 15 Jan 2024 16:43:18 +0000 Subject: [PATCH 2/3] updated deployment docs for multi repos --- docs/development/index.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/development/index.md b/docs/development/index.md index 63cf3cf..8228763 100644 --- a/docs/development/index.md +++ b/docs/development/index.md @@ -52,6 +52,20 @@ If it is your first time running the application, you will need to run the migra yarn prisma migrate dev ``` +## Providing course material + +Course material is provided by git repositories defined by the "repos" in the [configuration yaml]({{ "/config/template" | relative_url }}). + +Once you have installed the dependencies, you can populate the material directory with: + +```bash +yarn pullmat +``` + +This will pull the course material into the "MATERIAL_DIR" directory, which defaults to `.material`. + +Any course material in this directory will be automatically rendered by gutenberg at `/material`. + ## Running the development server The development server can now be started with: @@ -64,6 +78,15 @@ npm run dev Open [http://localhost:3000](http://localhost:3000) with your browser to see the result! +Alternatively, to test a production build: + +```bash +yarn build +yarn start +``` + +Will build and run a production optimised version of the application. + ## Prisma Studio Prisma Studio is a GUI for viewing and editing the database. It can be started with: From 5371ada876c5808694e7c7391748fb56ab92ceae Mon Sep 17 00:00:00 2001 From: Alasdair Wilson Date: Mon, 15 Jan 2024 17:02:34 +0000 Subject: [PATCH 3/3] prettier --- docs/deployment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/deployment.md b/docs/deployment.md index 277c2f9..c699a8b 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -97,7 +97,7 @@ You should customise your deployment by creating a configuration yaml file, see ## Configuration -The application can be further configured via environment variables. +The application can be further configured via environment variables. While we have touched on some key ones here, namely the database connection string and the search related variables, a full list of the environment variables and their purpose can be found in the [env vars config section]({{ "/config/vars" | relative_url }}). ## Using the site