Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add github action for linting jupyter #1289

Merged
merged 15 commits into from
Feb 10, 2025
Merged

Conversation

JoshuaSBrown
Copy link
Collaborator

@JoshuaSBrown JoshuaSBrown commented Feb 5, 2025

PR Description

This PR adds a linter to the GitHub actions to make sure that our jupyternotebooks will work correctly. In addition, the discovered linting items were addressed.

i.e.

nbQA failed to process 2_Data_Records_with_Solutions.ipynb with exception "JSONDecodeError('Expecting value: line 12 column 13 (char 582)')"
nbQA failed to process 4_Collections_Queries_with_Solutions.ipynb with exception "JSONDecodeError('Expecting value: line 9 column [13](https://github.com/ORNL/DataFed/actions/runs/13193136321/job/36829631988?pr=1291#step:5:14) (char 367)')"
nbQA failed to process 1_Basics_with_Solutions.ipynb with exception "JSONDecodeError('Expecting value: line 11 column 13 (char 494)')"
nbQA failed to process 2_Data_Records.ipynb with exception "JSONDecodeError('Expecting value: line 12 column 13 (char 582)')"
nbQA failed to process 3_Data_Transfer.ipynb with exception "JSONDecodeError('Expecting value: line 12 column 13 (char 434)')"
nbQA failed to process 0_verify.ipynb with exception "JSONDecodeError('Expecting value: line [15](https://github.com/ORNL/DataFed/actions/runs/13193136321/job/36829631988?pr=1291#step:5:16) column 13 (char 461)')"
nbQA failed to process 5_Collaborative_Exercise.ipynb with exception "JSONDecodeError('Expecting value: line 7 column 17 (char 132)')"
nbQA failed to process 4_Collections_Queries.ipynb with exception "JSONDecodeError('Expecting value: line 9 column 13 (char 367)')"
nbQA failed to process 3_Data_Transfer_with_Solutions.ipynb with exception "JSONDecodeError('Expecting value: line 12 column 13 (char 434)')"
nbQA failed to process 1_Basics.ipynb with exception "JSONDecodeError('Expecting value: line 11 column 13 (char 494)')"

Tasks

  • - A description of the PR has been provided, and a diagram included if it is a new feature.
  • - Formatter has been run
  • - Labels have been assigned to the pr
  • - A reviwer has been added
  • - A user has been assigned to work on the pr

Summary by Sourcery

CI:

  • Add a GitHub Actions workflow to lint Jupyter notebooks with flake8 and format them with black using nbQA.

Copy link

sourcery-ai bot commented Feb 5, 2025

Reviewer's Guide by Sourcery

This pull request introduces a new GitHub Actions workflow to lint Jupyter Notebooks. The workflow checks and formats notebooks using nbQA, flake8, and black.

Sequence diagram for Jupyter Notebook Linting Workflow

sequenceDiagram
    participant G as GitHub
    participant A as GitHub Actions
    participant P as Python Environment
    participant N as nbQA

    G->>A: Push event triggered
    A->>A: Checkout repository
    A->>P: Setup Python 3.10
    A->>P: Install dependencies
    Note over P: Install nbQA, flake8, black
    A->>N: Run flake8 on notebooks
    A->>N: Run black on notebooks
    N-->>A: Return linting results
    A-->>G: Report completion status
Loading

File-Level Changes

Change Details Files
Added a new workflow file to lint Jupyter Notebooks.
  • Configured a new GitHub Actions workflow named 'Jupyter Notebook Linter'.
  • The workflow is triggered on push events.
  • The workflow runs on ubuntu-latest.
  • The workflow checks out the repository, sets up Python 3.10, and installs nbqa, flake8, and black.
  • The workflow then runs flake8 and black on notebooks in the ./jupyter_notebooks directory.
  • The workflow reports the completion status.
.github/workflows/jupyternotebook-lint.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@JoshuaSBrown JoshuaSBrown self-assigned this Feb 5, 2025
@JoshuaSBrown JoshuaSBrown added the Type: Docs - User User / admin documentation label Feb 5, 2025
Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @JoshuaSBrown - I've reviewed your changes - here's some feedback:

Overall Comments:

  • Please complete the PR description and checklist items to help reviewers understand the changes and ensure all requirements are met.
  • Consider adding path filters to the workflow trigger (e.g., on: push: paths: ['jupyter_notebooks/**']) to avoid running notebook linting when no notebooks have changed.
Here's what I looked at during the review
  • 🟡 General issues: 1 issue found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

.github/workflows/jupyternotebook-lint.yml Outdated Show resolved Hide resolved
JoshuaSBrown and others added 3 commits February 5, 2025 08:12
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
@JoshuaSBrown JoshuaSBrown changed the title Add notebook for linting jupyter Add github action for linting jupyter Feb 5, 2025
@AronPerez
Copy link
Collaborator

Question: Which linter config is being used to lint these? I ask because we should be able to correct these locally with an easy setup that can be pulled from a known source

@JoshuaSBrown
Copy link
Collaborator Author

Question: Which linter config is being used to lint these? I ask because we should be able to correct these locally with an easy setup that can be pulled from a known source

It is using the defaults for the tools. I'll make it explicit and consistent with the rest of the repo good callout.

@JoshuaSBrown
Copy link
Collaborator Author

@AronPerez looks like the default for black was set to 88, and not 100. I don't want to reformat a bunch of things outside of the jupyternotebooks at the moment. I'll get this passing, and we can address all the files that do not adhere to the 100 line limit in a separate PR.

@JoshuaSBrown
Copy link
Collaborator Author

#1294

Copy link
Collaborator

@AronPerez AronPerez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on CI passing

@JoshuaSBrown JoshuaSBrown merged commit 13f4b1a into devel Feb 10, 2025
14 checks passed
@JoshuaSBrown JoshuaSBrown deleted the 1288-notebook-linter branch February 10, 2025 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Docs - User User / admin documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Jupyternotebook, CI] - Notebooks need to be linted as part of github actions
2 participants