From 924fd1ad916ed1249dbed59b2ea9cdcfcf06ecc8 Mon Sep 17 00:00:00 2001 From: Agah Date: Mon, 27 May 2024 10:17:42 +0100 Subject: [PATCH] Add info on nfs --- .gitignore | 1 + source/.DS_Store | Bin 8196 -> 0 bytes source/FULLSTACK.md | 112 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 113 insertions(+) delete mode 100644 source/.DS_Store diff --git a/.gitignore b/.gitignore index fd712b5..7645caf 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,4 @@ _build build .vscode .DS_Store +**/.DS_Store \ No newline at end of file diff --git a/source/.DS_Store b/source/.DS_Store deleted file mode 100644 index 363b4d36087590ce8beff24e3ec4a2b3d94ac1c5..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 8196 zcmeHMO;8*~6z;bW=$Qm)kPJT|$rdXWR6^JVCB{G&Shdi^P+9&%LUwmX7@3{Ho!MP} z#6k~RYOYZ$Q4b!}6rPM)<-vodQe}D2sz{a5i{uZEg^(r=92Bi;vW*%^SmhqydDW9pg*x^lZKVV*5}FDP1b&g zkARPWkARPWkARQB{{aC!vw5OhocluSmp%eM0=Fdse18aH<}&KZWg*?I0}J;GfU+37 z-Gp_zaMnX z2}FhTOCJFrfz1f;yL%DQNt#S0^Xu>LhHg7;ZC^v6sCY+7DWx)%%j7fhX)7Oh5>D3c zO=M2<)57dq`k8x7-{QVY6&ieBjNUzU}vO#X-T3rP2sNobLsho#iy2@z9>QhZ(AT0 z8$!HvGd|T$8A+TWH}yg`ro!=*l@^Z`6@UG*PGjf##A8NaJ~=DExKRK@l6T4vJuLT+ z>M6^N>kHZ>tFRG^eryh8v>e7glCq3!+O#KGWh!B$vPQzudJKab+oO&)=kVINZDpK3 z*Km@ttgW5W&`!^3F|C0$Xs+79jcQz^Vpp7xHglBly z-LgD5gUifpNw~Hil9eGNk<<*PO2VzZLsmv|A~g1>*LnDUSsBG;N#m65N@(%AySf$S zEFbxJ!gfy1Y9`L=GaR1Bt~4kyO2@diM^!ad3Pqa*PHvbDQL?wB0 zo?IX=lQ+oW=E&`$Ekv;rkwluC3vQ zRl9{)>$&6-NHD}@PJtPMtx8R8UHu+;?^YZiLau3ZloN)PuDaI2`bLTzVbpEPP4x#P zBr*e`ZK}VAQWg-p*7`OHxyy=#u3f&DB1_p0p*<`gk&wr%SZKTCV-)GiN>=M?YL%W| zS!RU)tfY;++mdyc$$R7zWZbvpJLK7~TrLUVs-swd%iUXQ$U~tjyhCJtpVk8Pl-LDg07__hM+j z+-?Ml#DKN@{{P_C@BiC9Eq^>80Uv>FL;#9o@mMboxYdTt@7h7k=P>ic?UsdfU9h<2 qApqNJ|1iXQ5KXyH)RW6XVh^Tme+cl;e?)!lZ%+UGcUgG*6@LRZy#$W{ diff --git a/source/FULLSTACK.md b/source/FULLSTACK.md index 8b0d089..098e817 100644 --- a/source/FULLSTACK.md +++ b/source/FULLSTACK.md @@ -4,6 +4,118 @@ NeuroLibre operates two servers dedicated to serving static files and API endpoi The purpose of this section is to offer an overview of each component within the NeuroLibre full-stack server. This overview will provide a solid understanding of its architecture before delving into hands-on development. It serves as a helpful guide to familiarize oneself with the system's structure and functionalities, facilitating a smoother and more informed development process. +### Data Storage for Test Server Stack (May 2024) + +As of May 2024, all data associated with the test server stack is stored in the neurolibre-data volume, which has a capacity of 1TB and is identified by the ID 05fe3035-**. This volume includes: + +* The book-artifacts directory, which contains the static HTML files generated from myst/jupyter-book builds. +* Other directories storing submission data, managed by the repo2data tool. + +Needless to say, please do NOT delete this volume. It would not be the end of the world, yet this would hamper ongoing submissions. + +#### Volume Mounting + +On the instance where the `neurolibre/full-stack-server` is deployed with preview configurations, the `neurolibre-data` volume is mounted on `/DATA`. You can verify this configuration by viewing the `/etc/fstab` file with the command: + +``` +sudo cat /etc/fstab +``` + +#### Network File Sharing (NFS) Setup + +The /DATA directory is shared between both the full-stack-server and binder-test instances via a Network File Sharing (NFS) protocol. In this setup: + +* The `full-stack-server` acts as the NFS server. +* The `binder-test` instance acts as the NFS client. + +This configuration ensures that both instances have access to the same data directory (`/DATA`), facilitating seamless data management and sharing. + +##### Setting up the NFS server + +###### Mounting and Installing NFS Server + +1. Ensure `/DATA` is Mounted + +Confirm that the `/DATA` directory has been properly mounted the `full-stack-server` instance. + +2. Install NFS Kernel Server + +Update the package list and install nfs-kernel-server: + +``` +sudo apt-get update +sudo apt install nfs-kernel-server +sudo systemctl start nfs-kernel-server +``` + +confirm the status: + +``` +sudo systemctl start nfs-kernel-server +``` + +> [!NOTE] +> Make sure that the instance is spawned from a [Ubuntu Cloud image](https://cloud-images.ubuntu.com/) wih **generic** kernel. Instances spawned from a linux-kvm (Kernel-based Virtual Machine) optimized kernel will not support the installation of nfs-kernel-server due to the lack of necessary NFS drivers (nfsd). + +###### Configure NFS Exports + +3. Edit the /etc/exports File + +Open the /etc/exports file for editing: + +``` +sudo nano /etc/exports +``` + +4. Add the following line to share the `/DATA` directory over the internal network: + +``` +/DATA 192.168.73.0/24(rw,sync,no_root_squash,no_all_squash) +``` + +Put the following content in `/etc/exports` (use `sudo nano /etc/exports`): + +``` +/DATA 192.168.73.0/24(rw,sync,no_root_squash,no_all_squash) +``` + +This configuration shares the `/DATA` directory with all machines on the `192.168.73.0/24` subnet, allowing read and write access while disabling root squashing. + +##### Setting up an NFS client + +1. Install `nfs-common`: + +``` +sudo apt-get update +sudo apt install nfs-common +``` + +2. To mount shared volume temporarily (disappears upon reboot of the client): + +``` +sudo mount -t nfs 192.168.73.179:/DATA /DATA +``` + +If you are asking the heck is that `192.168.73.179`, it is the internal IP address of the nfs server. You can either find it out on the cloudflare dashboard or by running `hostname -i` on the nfs server instance. + +2. To mount shared volume permanently: + +Put the following content in the last line of the `/etc/fstab` (use `sudo nano /etc/fstab`): + +``` +192.168.73.179:/DATA /DATA nfs rw,noatime,nolock,hard,tcp 0 0 +``` + +**NOTE:** If you are deploying BinderHub test server using terraform, you don't have to deal with this manually, just ensure that you pass the IP address using the following variable: + +```json +variable "sftp_ip_address" { + description = "Internal IP address of the SFTP instance on openstack." +} +``` + +For more details on deploying BinderHub on openstack using Terraform, see the relevant section of the developer documentation. + ### Static files Static files are the reproducible preprint content (HTML, CSS, JS, etc.) that are generated in one of the following cases: