Skip to content

Conversation

@bollwyvl
Copy link
Contributor

@bollwyvl bollwyvl commented Apr 8, 2021

Binder

  • add a basic "development" binder (we can later add a "pinned" one long-lived branch?)
    • environment.yml: normal stuff, also jupyterlab-tour and nbgitpuller
    • postBuild: pretty much what CI does
    • add "legacy" magic function names for launching under lab 3 with jupyter notebook
  • add anonymous github provider
    • configured by default on binder by copying .binder/jupyter_config.example.json
    • docs
  • add README badge
    • update with an "interesting" pre-loaded link to reduce API calls when API limited
  • VERY WIP config API change
    • just hands around the PRConfig object instead of unpacking it
    • remove entirely?
  • testing
    • appear to have broken things

@codecov-io
Copy link

Codecov Report

Merging #44 (378432b) into master (909cad7) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master      #44   +/-   ##
=======================================
  Coverage   61.11%   61.11%           
=======================================
  Files          33       33           
  Lines        1638     1638           
  Branches       94       94           
=======================================
  Hits         1001     1001           
  Misses        632      632           
  Partials        5        5           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 909cad7...378432b. Read the comment docs.

@bollwyvl
Copy link
Contributor Author

bollwyvl commented Apr 8, 2021

Does what it says on the tin:
Screenshot from 2021-04-08 12-31-26

It does pop the nbdime build nag... i'm wondering if we actually need it, or can

  • disable the nbdime labextension (other stuff seems to initialize)
  • disable the build message with tornado cruft

@bollwyvl
Copy link
Contributor Author

bollwyvl commented Apr 9, 2021

Took some doing, but things seem to work up to viewing (locally). Will post a Binder screenshot... probably broke a bunch of stuff, and may need to make some separate PRs (that might be good to land before 3.0 final, perhaps).

@bollwyvl
Copy link
Contributor Author

bollwyvl commented Apr 9, 2021

Cool, github-anonymous works all the way up to viewing text diffs, and correctly doesn't try to paginate across all of jupyterlab's pull requests:

Screenshot from 2021-04-09 12-46-48

I was able to look at about 10 pull requests (and each of their files) before it started dying with 403:

Screenshot from 2021-04-09 12-48-47

@bollwyvl
Copy link
Contributor Author

@fcollonval Haven't gotten around to warming this up, but the approach has legs, and it's worth checking out the binder.

As this PR has gotten rather large (like i do), I think an effective path forward might be to do some refactoring before trying to land this:

Some of those could go on 3.x...

  • normalize the python unit test approaches
    • some pytest.fixtures would drastically reduce some test boilerplate
  • use entry_points for the manager providers
    • this way, someone could add a your-dvcs-provider-here if they are able to test it better (e.g. bitbucket)
  • lean more heavily on the PRConfig object rather than fancy names in the manager constructors
  • introduce some concept of scoped searches
  • investigate some built-in caching (sadly, there is no tornado-cache a la requests-cache, but let's not go to threadtown unless we have to)
    • we could also (optionally) use the tornado curl client, it is generally better, just slightly harder to install on windows

but some of the bigger ones might need to be 4.x...

  • allow multiple, simultaneous providers

@fcollonval
Copy link
Member

@fcollonval Haven't gotten around to warming this up, but the approach has legs, and it's worth checking out the binder.

Awesome work @bollwyvl - sorry to not look into this earlier.

As this PR has gotten rather large (like i do), I think an effective path forward might be to do some refactoring before trying to land this

Agreed, I'll help open small PRs.

introduce some concept of scoped searches

vscode GitHub plugin uses the GitHub search API. And the user can customize groups like this:

"githubIssues.queries": [
        {
            "label": "My Issues",
            "query": "default"
        },
        {
            "label": "All Bugs",
            "query": "state:open repo:${owner}/${repository} sort:updated-desc label:type:Bug"
        },
        {
            "label": "All Features",
            "query": "state:open repo:${owner}/${repository} sort:updated-desc label:type:Enhancement"
        }
    ]

We could add a config button in the frontend panel that would open a form allowing the user to customize the search in a similar way. The question being do we want it to be a setting or a state?

  • investigate some built-in caching (sadly, there is no tornado-cache a la requests-cache, but let's not go to threadtown unless we have to)

What do you think of aiocache?

but some of the bigger ones might need to be 4.x...

  • allow multiple, simultaneous providers

Yep this definitely needs redesign of the extension code.


new_ = json.loads(result)
if next_url is not None:
if has_pagination and next_url is not None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has_pagination should not be added - but if you have a better API idea it would be great.

As the next_url contains already all required parameters, has_pagination is passed as False when calling the next page (see line 217). But if there are more than 2 pages, requiring has_pagination to be True will block calling the page 3, 4,... .

Unfortunately bringing the pagination in the frontend will be tricky to pull all the discussions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, unfortuntately i don' think we can just always assume it's okay to pull down 1000 PRs indiscriminantly...


# Reset cache
self._merge_requests_cache = {}
self._merge_requests_cache = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you reset the cache to None and not an empty dictionary? The idea to set as an empty dictionary is to allow to use the get method line 455.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be a screwup...


# Reset cache
self._pull_requests_cache = {}
self._pull_requests_cache = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you reset the cache to None and not an empty dictionary? The idea to set as an empty dictionary is to allow to use the get method line 298.

This was referenced Apr 19, 2021
fcollonval added a commit to fcollonval/pull-requests that referenced this pull request Apr 19, 2021
@bollwyvl
Copy link
Contributor Author

vscode GitHub plugin uses the GitHub search API. And the user can customize groups like this:

anonymous can't use the github search API, so we at least need some approach that is compatible with that, but that's just a subset of the larger problem.

setting or a state?

I'm pretty sure a it should be settings on the frontend, as it would need an "interesting" UI to make it useful, but only the provider knows what the fields are.

Basically, we'll want each search provider to create a JSON schema for "what is a search" that can be mapped to whatever the search provider has, and the provider would have to provide what it knows. So anonymous would only know about:

  • org (a string)
  • repo (a string)
  • state (an enum)

A full github/gitlab/bitbucket/gitea provider knows about the above, and (probably) a ton of other things, plus it could go off and do other queries to provide autocomplete values.

Anyhow, so the client would PUT a body that conforms to that, and then have a search that existed for the duration of the server lifetime. When the server restarts, you'd chunk back into it.

@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 90.90909% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.27%. Comparing base (909cad7) to head (7dde21a).
⚠️ Report is 59 commits behind head on master.

Files with missing lines Patch % Lines
...pyterlab_pullrequests/managers/github_anonymous.py 57.14% 6 Missing ⚠️
jupyterlab_pullrequests/handlers.py 64.28% 5 Missing ⚠️
jupyterlab_pullrequests/managers/manager.py 90.90% 1 Missing ⚠️
jupyterlab_pullrequests/tests/conftest.py 95.83% 1 Missing ⚠️
...ab_pullrequests/tests/test_handlers_integration.py 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #44      +/-   ##
==========================================
+ Coverage   61.11%   61.27%   +0.16%     
==========================================
  Files          33       34       +1     
  Lines        1638     1681      +43     
  Branches       94       94              
==========================================
+ Hits         1001     1030      +29     
- Misses        632      646      +14     
  Partials        5        5              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

4 participants