lando: add lando functionality (bug 1885346)

README.md

@@ -2,42 +2,44 @@
 
 This repository contains Docker configuration files to start up a local
 installation of most of the parts of Mozilla's code-review-and-landing
-system, collectively known as "Conduit".  This includes
+system, collectively known as "Conduit". This includes
 
-* BMO, Mozilla's Bugzilla fork
-* Phabricator, including extensions and patches
-* Lando, both API and UI
-* Transplant, the service that lands commits
-* A Mercurial server
-* A container ("local-dev") with various command-line tools including MozPhab
+- BMO, Mozilla's Bugzilla fork
+- Phabricator, including extensions and patches
+- Lando
+- Transplant, the service that lands commits
+- A Mercurial server
+- A container ("local-dev") with various command-line tools including MozPhab
 
 The suite allows only some services to be started up, if the whole
-system is not needed.  It also provides the option of using both local
+system is not needed. It also provides the option of using both local
 clones and hosted images, so you need only have the code checked out
 for the service(s) you need to modify.
 
 This suite can be used to demo Conduit services and to aid in
-development.  This repository, however, should not be viewed as a
+development. This repository, however, should not be viewed as a
 substitute for self-contained tests in individual repositories.
 
 ## Installation
 
 ### Prerequisites
 
 1. You need to have [docker](https://docs.docker.com/install/) and
-   [docker-compose](https://docs.docker.com/compose/install/) installed.
+   [docker compose](https://docs.docker.com/compose/install/) installed.
 1. For Lando, an Auth0 developer account. See the
-   [lando-ui README.md](https://github.com/mozilla-conduit/lando-ui/blob/master/README.md)
+   [lando README.md](https://github.com/mozilla-conduit/lando/blob/master/README.md)
    for instructions on how to set that up.
 
 ### Steps
 
-* Pull the repository into a separate (e.g. `conduit`) directory.
-* Go to the `conduit/suite` directory
-* Depending on what services you plan to run, you may need to create a
-  `docker-compose.override.yml` file. At the moment, this is only
+- Pull the repository into a separate (e.g. `conduit`) directory.
+- Go to the `conduit/suite` directory
+- Depending on what services you plan to run, you may need to create a
+  `docker compose.override.yml` file. At the moment, this is only
   required for Lando and Transplant. If in doubt, please refer to the
-  relevant projects.  Here is a sample file:
+  relevant projects. Here is a sample file:
+
+XXX This example is outdated
 
 ```
 version: '3.4'
@@ -73,22 +75,14 @@ services:
     environment: *transplant_secret
 ```
 
-* Run `docker-compose build`
-
-### First run of Lando
-
-If you are running Lando, you will need to first initialize the database:
-
-```shell
-docker-compose exec lando-api lando-cli db upgrade
-```
+- Run `docker compose build`
 
 ## Using the local-dev service
 
 The "local-dev" container includes command-line tools used to interact
 with Conduit services.
 
-To set up the container run `docker-compose run local-dev`.
+To set up the container run `docker compose run --rm local-dev`.
 You will be placed inside of a repository cloned from http://hg.test. You can
 use it as a normal local development repository.
 
@@ -119,11 +113,12 @@ proxy, or run a preconfigured Firefox.
 
 ### Websites provided by the suite
 
- * Phabricator - http://phabricator.test
- * Lando - http://lando-ui.test
- * Lando API - http://lando-api.test/ui via Swagger UI.
- * Bugzilla - http://bmo.test
- * Mercurial - http://hg.test
+- Bugzilla - http://bmo.test
+- Phabricator - http://phabricator.test
+- Lando - http://lando.test
+- (Legacy) Lando - http://lando-ui.test
+- (Legacy) Lando API - http://lando-api.test/ui via Swagger UI.
+- Mercurial - http://hg.test
 
 ## Running apps from local clone
 
@@ -142,11 +137,10 @@ repositories you wish to use locally to the `conduit` directory.
 ```shell
 git clone [email protected]:mozilla-conduit/arcanist.git
 git clone [email protected]:mozilla-bteam/bmo.git
-git clone [email protected]:mozilla-conduit/lando-api.git
-git clone [email protected]:mozilla-conduit/lando-ui.git
+git clone [email protected]:mozilla-conduit/lando.git
 git clone [email protected]:mozilla-conduit/phabricator.git
 git clone [email protected]:mozilla-conduit/phabricator-emails.git
-git clone [email protected]:mozilla-conduit/review.git
+git clone [email protected]:mozilla-conduit/review.git # moz-phab
 ```
 
 If you've installed all of the above projects, your directory structure
@@ -158,8 +152,7 @@ conduit
 ├── arcanist/
 ├── bmo/
 ├── suite/
-├── lando-api/
-├── lando-ui/
+├── lando/
 ├── phabricator/
 ├── phabricator-emails/
 └── review/
@@ -174,74 +167,66 @@ the phabricator code from a local repository instead of the
 
 ```shell
 # Build the containers
-$ docker-compose \
-  -f docker-compose.yml \
-  -f docker-compose.phabricator.yml \
-  -f docker-compose.override.yml \
+$ docker compose \
+  -f docker compose.yml \
+  -f docker compose.phabricator.yml \
+  -f docker compose.override.yml \
   build
 # Start the containers
-$ docker-compose \
-  -f docker-compose.yml \
-  -f docker-compose.phabricator.yml \
-  -f docker-compose.override.yml \
+$ docker compose \
+  -f docker compose.yml \
+  -f docker compose.phabricator.yml \
+  -f docker compose.override.yml \
   up -d
 ```
 
 You can also use multiple apps from local repositories. For example,
 to work on both Phabricator and Bugzilla,
 
 ```shell
-docker-compose \
-  -f docker-compose.yml \
-  -f docker-compose.phabricator.yml \
-  -f docker-compose.bmo.yml \
-  -f docker-compose.override.yml \
+docker compose \
+  -f docker compose.yml \
+  -f docker compose.phabricator.yml \
+  -f docker compose.bmo.yml \
+  -f docker compose.override.yml \
   up --build -d
 ```
 
-And for example to work on lando-ui and lando-api,
+And for example to work on lando,
 
 ```shell
-docker-compose \
-  -f docker-compose.yml \
-  -f docker-compose.lando-api.yml \
-  -f docker-compose.lando-ui.yml \
-  -f docker-compose.override.yml \
+docker compose \
+  -f docker compose.yml \
+  -f docker compose.lando.yml \
+  -f docker compose.override.yml \
   up --build -d
 ```
 
-Note that normally you must have `-f docker-compose.yml` as the first
-option and `-f docker-compose.override.yml` as the last one.
+Note that normally you must have `-f docker compose.yml` as the first
+option and `-f docker compose.override.yml` as the last one.
 
 To work on a local version of the Arcanist fork, load the
-`docker-compose.cinnabarc.yml` configuration. This will modify the
-`arc` command in the `local-dev` service. Similarly, to load a local version 
-of the ARC wrapper "review" , load the `docker-compose.review.yml`. 
+`docker compose.cinnabarc.yml` configuration. This will modify the
+`arc` command in the `local-dev` service. Similarly, to load a local version
+of the ARC wrapper "review" , load the `docker compose.review.yml`.
 
 If you don't want to spin up all configured containers, you can
 specify the ones you'd like to work on. The command below runs
-`phabricator.test`, `phabricator`, `phabdb`, `lando-api.test`,
-`lando-api` and `lando-api.db` to allow the verification of the
+`phabricator.test`, `phabricator`, `phab.db`, `lando.test`,
 integration between Phabricator and Lando API:
 
-`docker-compose up phabricator.test lando-api.test`
+`docker compose up phabricator.test lando.test`
 
 ## Preconfigured users:
 
-For performing administration tasks in Phabricator, first log out of
-Phabricator and then go to http://phabricator.test/auth/start/?admin and log 
-in with
-
-`user:admin`, `password:password123456789!`
-
 To log in as a normal test user, you will need to use BMO for
 auth delegation. Log out of Phabricator and then click on 'Log In or
 Register'. You will be redirected to BMO's login page.
 
 `user:[email protected]`, `password:password123456789!`
 
 We also have a `ConduitReviewer` account that can be opened in a second private
-browser window for performing the other half of the review dance.  On the BMO 
+browser window for performing the other half of the review dance. On the BMO
 login page enter:
 
 `user:[email protected]`, `password:password123456789!`
@@ -256,6 +241,17 @@ credentials:
 
 `user:[email protected]`, `password:password012!`
 
+For performing administration tasks in Phabricator, first log out of
+Phabricator and then go to http://phabricator.test/auth/start/?admin and log
+in with
+
+`user:admin`, `password:password123456789!`
+
+A local Git server is also available at http://git.test. The `conduit` user can
+log in with the credentials above. For administrative tasks, the account details are as follows:
+
+`user: git-admin`, `password:password123456789!`
+
 ## Updating the preloaded Phabricator database
 
 As noted in [this Phabricator ticket](https://secure.phabricator.com/T5310),
@@ -264,27 +260,42 @@ the application database with the settings we want.
 
 To update the preloaded database with new settings:
 
- 1. **Important:** Run `docker-compose down` and
+1.  **Important:** Run `docker compose down` and
     `docker volume rm suite_phabricator-mysql-db` to ensure you have a
     fresh DB!
- 1. Start the application with `docker-compose up` and log in with the
+1.  Start the application with `docker compose up` and log in with the
     appropriate user ("admin" to update global settings, "phab-bot" for
     things like API keys).
- 1. Change the desired setting.
- 1. Run `docker-compose run phabricator dump > demo.sql` to dump the
+1.  Change the desired setting.
+1.  Run `docker compose run phabricator dump > demo.sql` to dump the
     database.
- 1. Edit `demo.sql` and delete the extra shell output at the beginning and at
+1.  Edit `demo.sql` and delete the extra shell output at the beginning and at
     the end of the file.
- 1. `$ gzip demo.sql`
- 1. `$ mv demo.sql.gz docker/phabricator/demo.sql.gz`
- 1. Submit a [PR](https://github.com/mozilla-conduit/suite/pulls) with
+1.  `$ gzip demo.sql`
+1.  `$ mv demo.sql.gz docker/phabricator/demo.sql.gz`
+1.  Submit a [PR](https://github.com/mozilla-conduit/suite/pulls) with
     the changes.
 
+## Updating the git repositories
+
+A backup of the repositories is stored in the `docker/gogs` directory. It is
+restored automatically by the one-shot gogs-init service when spinning up a
+fresh stack.
+
+If you need to update the repositories, you can simply work against
+http://git.test (or in the local-dev container) and push the changes to the
+repositories. You can then update the backup by running:
+
+    docker compose exec git.test /scripts/gogs-backup.sh
+
+which will update the backup in `docker/gogs`. You can then commit the changes
+and submit a PR.
+
 ## Clone the test repository
 
 The `local-dev` service uses repositories cloned from http://hg.test/.
 You will need to re-clone them every time Mercurial server images are
-created.  There is a bash script which will remove the existing
+created. There is a bash script which will remove the existing
 directories and clone the repositories using `hg` and `git-cinnabar`:
 
 `# ./clone_repositories.sh`
@@ -294,25 +305,24 @@ directories and clone the repositories using `hg` and `git-cinnabar`:
 Start the suite:
 
 ```shell
-docker-compose up -d
-docker-compose exec lando-api lando-cli db upgrade
+docker compose up -d
 ```
 
 Create a diff:
 
 ```shell
-$ docker-compose run local-dev
-# ./clone-repositories.sh
+$ docker compose run --rm local-dev
+# cd repos
 # cd test-repo
 # echo test >> README
 # hg commit -m "test info added"
 # moz-phab install-certificate
 # moz-phab submit -b 1
 ```
 
-Log in to http://lando-ui.test.
+Log in to http://lando.test.
 
-Navigate to http://lando-ui.test/revisions/D2.
+Navigate to http://lando.test/revisions/D2.
 
 Confirm the warning and click on the `Land` button.
 

docker-compose.autolandhg.yml

@@ -2,7 +2,6 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-version: '3.4'
 services:
   autoland.hg-init:
     build:
@@ -24,9 +23,6 @@ services:
     environment:
       SSHD_PORT: "8022"
       SSH_PUBLIC_KEY: "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHWpDYI3qzSxV8tbjH8C4z3ef6QU2yBKGTHTFf6OrPrqDIO++ixfr4LW3yGmW4Q3CRJcx8KZ6hnx2FPnDOCsrqw= app"
-    ports:
-      - "8000:8000"
-      - "8022:8022"
     volumes:
       - autoland.hg:/repos
       - autoland.ssh:/home/app/.ssh

docker-compose.bmo.yml

@@ -2,8 +2,6 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-version: '3.4'
-
 services:
   bmo.test:
     build:

docker-compose.cinnabarc.yml

@@ -2,7 +2,6 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-version: '3.4'
 services:
   local-dev:
     volumes: