I think we shouldn't remove the installation directly from the workflow, let's actually use the workflow matrix set to run different dependencies as different runs.

Here is an example: https://github.com/asyml/forte-wrappers/blob/main/.github/workflows/main.yml#L84

"use the workflow matrix set to run different dependencies as different runs."

Do you mean in the future we will have different versions of forte-wrapper, and you want to define a matrix to test them all?

1. No, not different versions of forte-wrapper. I show you the forte-wrapper repository workflow as an example of how to set up a workflow matrix.
2. In this repo, ForteHealth, we can set up a similar matrix, since not all processors in this repo can be installed and run at the same time, they can be tested individually. Note that you could use the extra_install method in setup.py to specify different sets of dependencies.

You may wonder how we make them work together in the future, that's another issue (we have solutions such as RemoteProcessor or Docker images). Right now we can focus on making them each testable.

I still have a question: @Piyush13y seems to want to set forte.spacy as an extra_install. If so, forte.spacy and neurocoref will be in the same extra_install list, example:

setuptools.setup(
....
    install_requires=[
        "forte~=0.2.0",
    ],
    extras_require={
        .....
        "coreference": [
            "forte.spacy",  # TODO: version
            "cython>=0.25",
            "neuralcoref @ git+https://[email protected]/huggingface/[email protected]#egg=neuralcoref",
        ],
    },
    include_package_data=True,
    python_requires=">=3.6",
)

I think pip won't make sure that forte.spacy is install ahead of neuralcoref.

@Piyush13y seems to want to set forte.spacy as an extra_install.

Emmm... It's a little bit confusing because now forte.spacy is not even in the setup.py in master branch: https://github.com/asyml/ForteHealth/blob/master/setup.py

Installing a spacy version compatible with neuralcoref causes the failure of SpacyProcessor. Since SpacyProcessor is needed in unit test, this seems to be a dead end before we can manage to run the modules separately.

now we come back to my initial suggestion, use the matrix set up to run the module separately

Bad news. A spacy version compatible with neuralcoref is not compatible with my RTX 3060.

I checked the issues in neuralcoref regarding the spacy version problem, almost everyone is suggesting building neuralcoref from source.

The latest release of neuralcoref is in 2019. Maybe it is worth considering changing a better package.

huh, actually for now we are simply looking for a coreference module that works. Maybe we can directly add this function to AllenNLPProcessor, or other packages you find easy

We can deal with this through setup.py using extra requires.

Can we use more standard way in setup.py