fixing webapp build (#162)

* fixes #161

* changelog

* testing pypi workflow

* syntax fix

* changing npm to nodejs in conda build

* updating contributing guide

* some fixes

* testing publish workflow

* fixes

* disabling pypi workflow on this branch

* temporarily disabling conda workflow

* temporarily disabling codeql

* testing without node setup

* fix gitignore and remove entire build dir contents

* fix yarn install, update README

* add yarn to github workflow, upgrade node to v16

* forgot to add uses:yarn to tests.yml

* fix tests.yml webapp build, reverting pypi.yml

* Removing branch name in publish workflow

* Re-enabling codeql

* Re-enabling conda workflow

* Re-enabling image push to ECR

* fixing conda

* testing fix

* testing npm and yarn within container

* Removing npm and yarn test

* Testing npm and yarn install

* Replacing setup-node step with npm-yarn install

* Fixed duplicate container names

* Testing one container

* Trying install within build webapp

* Fixing syntax errors

* Fixing syntax errors

* Testing node install again

* Removing sudo

* Removing flag

* Adding all container types

* Adding all container types again

* Adding python-38-centos7

* Syntax error fix

* Syntax error fix

* Separate step for centos

* Retrying centos install

* Fix syntax error

* Fix syntax error

* Fix syntax error

* Checking permissions

* Checking permissions

* Removing duplicate conditions

* Reverting steps

* Fixes

* Fixes

* More fixes

* More fixes

* More fixes

* Fixes

* Fixes

* Fixes

* Fixes on centos

* Fixes on centos

* Fixes on centos

* fixing tests

* adding build validation to tests

* testing a fix

* testing a fix

* testing a fix

* testing mac

* testing mac

* testing mac

* re-enabling all workflows

Co-authored-by: Valentin Kostadinov <[email protected]>
Co-authored-by: Okechukwu Ochia <[email protected]>

.github/workflows/condabuild.yml

@@ -42,4 +42,3 @@ jobs:
           conda create --name buildenv python=3.8
           $CONDA/bin/activate buildenv
           conda build --override-channels .
-        

.github/workflows/pypi.yml

@@ -39,6 +39,15 @@ jobs:
         run: |
           python -m pip install --upgrade pip
           pip install twine
+      - name: Set up Node
+        uses: actions/setup-node@v2
+        with:
+          node-version: 16
+      - name: Build Webapp
+        run: |
+          cd ./covalent_ui/webapp
+          yarn install
+          yarn build
       - name: Build Distribution
         run: python setup.py sdist
       - name: Validate Distribution

.github/workflows/tests.yml

@@ -38,7 +38,8 @@ jobs:
           - python:3.8-bullseye
           - python:3.9-buster
           - python:3.9-bullseye
-          - centos/python-38-centos7
+          - image: "centos/python-38-centos7"
+            options: "--user root"
         experimental: [false]
         include:
           - os: macos-latest
@@ -60,6 +61,37 @@ jobs:
         run: |
           pip install --no-cache-dir -r ./requirements.txt
           pip install --no-cache-dir -r ./tests/requirements.txt
+      - name: Install NPM and Yarn on Ubuntu
+        if: matrix.container == 'python:3.8-buster' || matrix.container == 'python:3.8-bullseye' || matrix.container == 'python:3.9-buster' || matrix.container == 'python:3.9-bullseye'
+        run: |
+          apt-get update && apt-get install -y curl
+          curl -fsSL https://deb.nodesource.com/setup_16.x | bash -
+          apt-get install -y nodejs
+          npm install -g yarn
+      - name: Install NPM and Yarn on CentOS
+        if: matrix.container.image == 'centos/python-38-centos7'
+        run: |
+          yum remove -y npm nodejs
+          curl -fsSL https://rpm.nodesource.com/setup_16.x | bash -
+          yum install -y npm nodejs
+          npm install -g yarn
+      - name: Build webapp
+        run: |
+          cd ./covalent_ui/webapp
+          yarn install
+          yarn build
+      - name: Build distribution
+        run: python setup.py sdist
+      - name: Validate distribution
+        run: |
+          VERSION="$(cat ./VERSION)"
+          cd dist
+          tar xzf cova-${VERSION}.tar.gz
+          diff -r cova-${VERSION}/covalent ../covalent
+          diff -r cova-${VERSION}/covalent_dispatcher ../covalent_dispatcher
+          diff -x README.md cova-${VERSION}/covalent_ui ../covalent_ui 
+          diff -r cova-${VERSION}/covalent_ui/webapp/build ../covalent_ui/webapp/build
+          rm -rf cova-${VERSION}/
       - name: Install Covalent
         run: pip install -e .
       - name: Start Covalent dispatcher server

CHANGELOG.md

@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.29.3] - 2022-03-09
+
+### Fixed
+
+- Covalent UI is built in the Dockerfile, the setup file, the pypi workflow, the tests workflow, and the conda build script.
+
 ## [0.29.2] - 2022-03-09
 
 ### Added

CONTRIBUTING.md

@@ -20,7 +20,8 @@ conda activate covalent-dev
 
 Install Covalent's core requirements as well as the developer requirements:
 ```shell
-conda install setuptools pip
+conda install setuptools pip nodejs yarn
+python setup.py webapp
 pip install -e .
 pip install -r tests/requirements.txt
 pre-commit install

Dockerfile

@@ -20,16 +20,22 @@
 
 FROM python:3.8-slim-buster
 
-RUN apt-get update && apt-get install -y \
-  gcc \
+RUN apt-get update \
+  && apt-get install -y curl gcc \
+  && curl -sL https://deb.nodesource.com/setup_16.x | bash - \
+  && apt-get install -y nodejs \
+  && npm install --global yarn \
   && rm -rf /var/lib/apt/lists/*
 
 RUN mkdir -p /opt/covalent
 COPY . /opt/covalent
 RUN pip install --no-cache-dir --use-feature=in-tree-build /opt/covalent
+RUN cd /opt/covalent/covalent_ui/webapp \
+  && yarn install  \
+  && yarn build
 
 WORKDIR /opt/covalent
 
 EXPOSE 8080
 ENTRYPOINT [ "python" ]
-CMD ["/opt/covalent/covalent_ui/app.py", "--port", "8080"]
+CMD ["/opt/covalent/covalent_ui/app.py", "--port", "8080"]

VERSION

@@ -1 +1 @@
-0.29.2
+0.29.3

covalent_ui/README.md

@@ -1,32 +1,51 @@
 # Covalent UI
 
-## Running UI server
+## Setup
 
--   Start UI server
+-   Activate conda environment and install required packages:
 
 ```shell
-cd covalent_ui
-flask run --port 48008
+conda activate <covalent-env>
+pip install -r requirement.txt
 ```
 
--   Open `http://localhost:48008` in your browser.
--   Dispatch workflows to explore them in the UI.
+-   Install `node` (v16 or later) and `npm`:
 
-## Setup details
+```shell
+# Linux
+curl -sL https://deb.nodesource.com/setup_16.x | bash -
+apt-get install -y nodejs
 
--   Ensure python environment is properly set up with all required packages installed.
+# macOS
+brew install node
+```
+
+-   Install `yarn`:
 
 ```shell
-conda activate <covalent-env>
-pip install -r requirement.txt
+npm install --global yarn
 ```
 
--   The UI server does not require the **Covalent Dispatcher** to be running but it receives execution updates from it.
+## Build web app
 
--   Results are currently persisted in the browser client's `localStorage`.
+```
+cd covalent_ui/webapp
+yarn install
+yarn build
+```
+
+## Run server
+
+```shell
+cd covalent_ui
+python app.py -p <port>
+```
+
+-   Open `http://0.0.0.0:<port>` in your browser.
+-   Dispatch workflows to explore them in the UI.
 
 ## Frontend development
 
--   The optimized production build of the UI web app lives under `covalent_ui/webapp/build`. It is statically served by the UI server by default.
+-   The optimized production build of the UI web app lives under `covalent_ui/webapp/build`. It is statically served by the server by default.
 
--   See `covalent_ui/webapp/README.me` for details on how to build and run the development.
+-   For the development version of the web app, see `covalent_ui/webapp/README.me`

covalent_ui/app.py

@@ -153,4 +153,4 @@ def serve(path):
     else:
         port = DEFAULT_PORT
     debug = True if args.develop is True else False
-    socketio.run(app, debug=debug, host="0.0.0.0", port=port)
+    socketio.run(app, debug=debug, host="0.0.0.0", port=port, use_reloader=True)

covalent_ui/webapp/.gitignore

@@ -11,8 +11,8 @@
 # testing
 /coverage
 
-# production
-!build
+# production build
+/build
 
 # misc
 .DS_Store