Home

Overview

This project can be devided into two steps, the first step is running Python scripts located in /aiida-registry to parse plugins.yaml file, collect the website data, and store the results in a JSON file, and the second step is to move that JSON file the /aiida-registry-app, the location of the React app, and then execute the React app which reads that JSON file and render the website with the data collected.

Data Collection

In plugins.yaml, plugins developers provide us with some information and links about their plugins, like source code, pip url, the metadata json file, etc. We use this data to collect more information, this done by parsing the provided files, making API calls to PyPI and Github, and using docker to run plugins in aiida-core docker image to retrieve the workflows/calculations information.

Fetch Metadata (aiida-registry fetch)

The aiida-registry fetch command invokes the make_pages() function, which generates the fetch_metadata.json file. This file contains general information extracted from various sources:

• General information from PyPI (e.g., author, author email, version).
• Data extracted from the setup.json provided in plugins.yaml.
• Commit count obtained from API calls to GitHub (or local Git clone if the plugin is not hosted on GitHub).
• Recent release dates sourced from PyPI. For plugins not hosted on PyPI, we employ an alternate method to determine release dates by tracking version changes over time. This requires cloning the last fetched file from the repository, hence requiring two input files: plugins.yaml and cloned_plugins_metadata.json.

JSON file structure

The final structure of the fetch_metadata.json file is organized as follows:

fetch_metadata.json

{
    "plugins": { ...
    },
    "globalsummary": [...
    ],
    "status_dict": {...
    },
    "entrypointtypes": {...
    },
}

Get Calculations/Workflows Information (aiida-registry test-install)

The aiida-registry test-install command executes the test_install_all() function, which installs each plugin within the aiida-core Docker container and gathers specifications information for entry points associated with calculations and workflows.

Upon collecting this information, we open the plugins_metadata.json file generated by aiida-registry fetch and append the entry point information to the corresponding plugin as follows:

plugins_metadata["plugins"][plugin_name]["entry_points"][ep_group][ep_name] = extracted_info

an example of extracted_info for an entry point is as follows:

{
  "description": [...
  ],

  "class": string,

  "spec": {
    "inputs": [
      {
        "name": string,
        "required": bool,
        "valid_types": string,
        "info": string
      },
    ],

      "outputs": [
      {
        "name": string,
        "required": bool,
        "valid_types": string,
        "info": string
      },
    ]
    "exit_codes": [
      {
        "status":number,
        "message":string,
      },
    ]
  }
}

Following this step, we save the final version of the plugins_metadata.json file, now containing all required information. The next step involves making this file accessible to the React application, which is accomplished by moving the file to the React app directory.

Frontend Structure

After moving the JSON file to aiida-registry-app/src/, it becomes accessible for import and use within the React application.
Inside the aiida-registry-app directory, we install and build the react app using npm as follows:

npm install
npm run build

This produces the directory that will be deployed aiida-registry-app/dist/

TODO:

• read and process the file in javascript.
• establish application essential routes.
• how the ui updated?
• Features
• Styling

This flow diagram provides a visual representation of the project's architecture and data flow.

How to fix registry warnings and errors

You may want to check that your plugin is correctly registered before making the changes to your plugin repository. To do so, you can run the registry check locally.

Clone this repository and install the dependencies:

git clone https://github.com/aiidateam/aiida-registry.git
cd aiida-registry
pip install -e .

Fetch the metadata and run installation test or your plugin

aiida-registry fetch <plugin-name>
aiida-registry test-install

You'll see the warnings and errors as what you can see from the registry page.