Fixing incompatibilities with newer libraries versions

[limacarvalho]

Hi @frasermarlow, @yuhan and community!

First of all, great work on this super helpful solution built on top of dagster. I am sure this is a great help and a starting point for lots of people who are interested in analyzing the authenticity of a particular GitHub repository.

I decided to follow the README instructions to bring up the fake-star-detector, both by building from scratch and using the docker compose up, unfortunately, both failed for me.

Let me describe a bit more what happened:

What

---

When following the provided instructions the docker container finishes up building just fine and remains up without any issues, however when accessing the web UI via http://localhost:3000 it is possible to see something is wrong.

The following error is shown:

More info available in the error log/message:
error_message.log

Why

---

Without doing a proper root cause analysis, it may not be possible to identify the exact issue, however, I feel this can be related to newer python packages' versions being installed (compared to the ones installed before while testing) thus causing conflicts with other packages that are not frequently updated.

How

---

To work around the current issue and make sure this solution is still installable/usable at the current date, I hard-coded some packages' versions in the setup.py file. This seems to work around the issue, at least temporarily, and I was able to run the tool without any problems.
Of course, this would be a temporary solution, and implementing a proper fix using the latest dependencies versions would be the ideal/proper fix.

How to replicate the current issue

---

• git clone https://github.com/dagster-io/fake-star-detector.git
• cd fake-star-detector
• # Adding the github token and then moving on
• docker compose up -d --build
• # accessing the link http://localhost:3000

How to work around the issue

• git clone https://github.com/dagster-io/fake-star-detector.git
• cd fake-star-detector
• # Adding the github token and then moving on
• Replace the contents of setup.py with:

import glob

from setuptools import find_packages, setup

setup(
    name="fake_star_detector",
    packages=find_packages(exclude=["fake_star_detector_tests"]),
    # package data paths are relative to the package key
    package_data={"fake_star_detector": ["../" + path for path in glob.glob("dbt_project/**", recursive=True)]},
    install_requires=[
        "dagster==1.2.2",
        "dagster-cloud[serverless]==1.2.2",
        "dagster-dbt==0.18.2",
        "PyGithub",
        "Pydantic<2",
        "pandas==1.5.3",
        "matplotlib",
        "nbconvert",
        "nbformat",
        "ipykernel",
        "jupytext",
        "dbt-core==1.4",
        "dbt-bigquery==1.4",
        # packaging v22 has build compatibility issues with dbt as of 2022-12-07
        "packaging<22.0",
        "pendulum<3.0",
    ],
    extras_require={"dev": ["dagit==1.2.2", "pytest"]},
)

• docker compose up -d --build
• # accessing the link http://localhost:3000

Thanks a lot! Cheers 🍻