add API for crates.io to trigger rebuilds #2534

pflanze · 2024-06-24T13:28:16Z

relates to #2442.

adds config variable DOCSRS_TRIGGER_REBUILD_TOKEN /
Config.trigger_rebuild_token
adds build_trigger_rebuild_handler and route
"/crate/:name/:version/rebuild"

Note: does not yet contain any kind of rate limiting!

syphar

some first comments.

I'm not 100% sure about the Json/APi error design, but I need a less tired moment to think about it.

src/config.rs

src/web/builds.rs

As per suggestion on rust-lang#2534, although `id` was ambiguous thus using `count(*)`.

pflanze · 2024-06-25T19:39:08Z

I think I've handled all of your suggestions, but I forgot about Clippy. I'll need to get back to that another evening.

pflanze · 2024-06-25T19:44:30Z

Oh build failures:

error: set DATABASE_URL to use query macros online, or run cargo sqlx prepare to update the query cache

Not sure right now what to do about this (need to stop for tonight).

syphar · 2024-06-26T04:22:09Z

Oh build failures:

error: set DATABASE_URL to use query macros online, or run cargo sqlx prepare to update the query cache

Not sure right now what to do about this (need to stop for tonight).

I've put the necessary command into our Justfile, so calling just sqlx-prepare is fine

src/web/builds.rs

src/web/crate_details.rs

src/web/error.rs

src/web/builds.rs

syphar · 2024-06-26T04:46:40Z

Thank you for all the work! This will be awesome for crate authors

pflanze · 2024-06-27T05:13:10Z

I've so far avoided to force-push and instead pushed 3 commits (those with subject lines starting with "sq", standing for squash) with changes that I will squash into the earlier commits where they belong; the idea is that this makes it easier for you to see what I changed; this assumes that diffing is easier as a last step after I do the squashing together with the right commits and force-pushing at the end. Let me know once you're happy and I'll do that. And if that was useful or rather a complication.

As per suggestion on rust-lang#2534, although `id` was ambiguous thus using `count(*)`.

pflanze · 2024-06-27T05:55:29Z

I've realized that you can still see the commit from before me squashing the commits anyway and have now force-pushed db8b363...b7deb0d (i.e. if you'd like to see the incremental changes I did easily, look at db8b363, for the state I wish to be merged b7deb0d).

pflanze · 2024-06-30T19:02:29Z

I've pushed 3 commits with (a) the 'optimized' SQL query, FWIW, (b)+(c) add and use the constant_time_eq crate for token comparisons. Without it, it would likely be possible for attackers to find the token via response times.

syphar

This looks really awesome, thank you for the continued work on this.

I'll do some local testing and then merge & deploy this.

( and probably already start using it to queue rebuilds )

syphar · 2024-07-10T05:37:58Z

@pflanze do you want to squash some commits before I merge?

- web: `axum_cached_redirect`: change return type to be concrete - web/routes: add `post_internal` - test: add `TestFrontent::post`

This aims to satisfy these wishes: - Use the same error enum (`AxumNope`) for both handlers returning HTML or JSON. - Yet allow those handlers to specify whether HTML or JSON should be returned. - Not change the exising code returning `AxumNope`--still convert to HTML by default. Because `AxumNope` also contains a case (`Search`) that is fixed to producing HTML, but shouldn't ever be used by handlers returning JSON, that works with a runtime error in case the latter assumption isn't being followed. The approach is to add a new wrapper around `AxumNope` called `JsonAxumNope` and matching type def `JsonAxumResult`. The wrapper also implements `IntoResponse`, but produces JSON. Endpoints wishing to return JSON errors need to use that wrapper and specify `JsonAxumResult` as their return type.

This resolves rust-lang#2442. - adds config variable `DOCSRS_TRIGGER_REBUILD_TOKEN` / `Config.trigger_rebuild_token` - adds `build_trigger_rebuild_handler` and route "/crate/:name/:version/rebuild" Note: does not yet contain any kind of rate limiting!

This removes the intermediate `ErrorResponse` type and instead pattern matches AxumNope on both levels, using a new function `redirect_with_policy` to avoid duplication. (Saves 11 lines of code, 7 comment lines, and the indirection via intermediate type, but adds the function instead.)

pflanze · 2024-07-11T07:35:40Z

@pflanze do you want to squash some commits before I merge?

@syphar, yes, some more squashing was warranted. I think now it's at the proper resolution.

pflanze · 2024-07-11T16:52:50Z

This looks really awesome, thank you for the continued work on this.

Thank you, too, for your help during https://rustfest.ch/ and your feedback afterwards! It was the perfect introduction.

Now I just need to get a bit more time.

pflanze requested a review from a team as a code owner June 24, 2024 13:28

github-actions bot added the S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed label Jun 24, 2024

syphar requested changes Jun 24, 2024

View reviewed changes

src/config.rs Outdated Show resolved Hide resolved

src/web/builds.rs Outdated Show resolved Hide resolved

src/web/builds.rs Outdated Show resolved Hide resolved

src/web/builds.rs Outdated Show resolved Hide resolved

src/web/builds.rs Outdated Show resolved Hide resolved

syphar added S-waiting-on-author Status: This PR is incomplete or needs to address review comments and removed S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed labels Jun 24, 2024

syphar changed the title ~~Rebuild api~~ add API for crates.io to trigger crate rebuilds Jun 24, 2024

syphar changed the title ~~add API for crates.io to trigger crate rebuilds~~ add API for crates.io to trigger rebuilds Jun 24, 2024

pflanze added a commit to pflanze/docs.rs that referenced this pull request Jun 25, 2024

web/builds: add crate_version_exists

8fc46ab

As per suggestion on rust-lang#2534, although `id` was ambiguous thus using `count(*)`.

pflanze added a commit to pflanze/docs.rs that referenced this pull request Jun 25, 2024

web/builds: add crate_version_exists

5cd562c

As per suggestion on rust-lang#2534, although `id` was ambiguous thus using `count(*)`.

pflanze force-pushed the rebuild_api branch from 8fc46ab to 5cd562c Compare June 25, 2024 19:36

pflanze requested a review from syphar June 25, 2024 19:42

github-actions bot added S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed and removed S-waiting-on-author Status: This PR is incomplete or needs to address review comments labels Jun 25, 2024

syphar requested changes Jun 26, 2024

View reviewed changes

src/web/builds.rs Show resolved Hide resolved

src/web/crate_details.rs Outdated Show resolved Hide resolved

src/web/error.rs Show resolved Hide resolved

src/web/builds.rs Outdated Show resolved Hide resolved

syphar added S-waiting-on-author Status: This PR is incomplete or needs to address review comments and removed S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed labels Jun 26, 2024

pflanze force-pushed the rebuild_api branch from a01ce7c to 0268b98 Compare June 27, 2024 00:15

pflanze requested a review from syphar June 27, 2024 05:33

github-actions bot added S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed and removed S-waiting-on-author Status: This PR is incomplete or needs to address review comments labels Jun 27, 2024

pflanze added a commit to pflanze/docs.rs that referenced this pull request Jun 27, 2024

web/builds: add crate_version_exists

449ec34

As per suggestion on rust-lang#2534, although `id` was ambiguous thus using `count(*)`.

pflanze force-pushed the rebuild_api branch from db8b363 to b7deb0d Compare June 27, 2024 05:53

syphar approved these changes Jul 10, 2024

View reviewed changes

pflanze added 8 commits July 11, 2024 09:28

preparations for API to request rebuild of a crate version

bb0212b

- web: `axum_cached_redirect`: change return type to be concrete - web/routes: add `post_internal` - test: add `TestFrontent::post`

web/error: add AxumNope::Unauthorized

cf58d70

web/builds: add crate_version_exists to query the database directly

005a072

Cargo,web/builds: use constant_time_eq for security

247d86b

Run "cargo sqlx prepare"

108356a

pflanze force-pushed the rebuild_api branch from 0487f1c to 108356a Compare July 11, 2024 07:30

syphar merged commit 2eb0516 into rust-lang:master Jul 11, 2024
8 checks passed

github-actions bot added S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it and removed S-waiting-on-review Status: This pull request has been implemented and needs to be reviewed labels Jul 11, 2024

syphar removed the S-waiting-on-deploy This PR is ready to be merged, but is waiting for an admin to have time to deploy it label Jul 12, 2024

syphar mentioned this pull request Jul 17, 2024

docs.rs: allow POST requests rust-lang/simpleinfra#444

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add API for crates.io to trigger rebuilds #2534

add API for crates.io to trigger rebuilds #2534

pflanze commented Jun 24, 2024 •

edited by syphar

Loading

syphar left a comment

pflanze commented Jun 25, 2024

pflanze commented Jun 25, 2024

syphar commented Jun 26, 2024

syphar commented Jun 26, 2024

pflanze commented Jun 27, 2024

pflanze commented Jun 27, 2024

pflanze commented Jun 30, 2024

syphar left a comment

syphar commented Jul 10, 2024

pflanze commented Jul 11, 2024

pflanze commented Jul 11, 2024

add API for crates.io to trigger rebuilds #2534

add API for crates.io to trigger rebuilds #2534

Conversation

pflanze commented Jun 24, 2024 • edited by syphar Loading

syphar left a comment

Choose a reason for hiding this comment

pflanze commented Jun 25, 2024

pflanze commented Jun 25, 2024

syphar commented Jun 26, 2024

syphar commented Jun 26, 2024

pflanze commented Jun 27, 2024

pflanze commented Jun 27, 2024

pflanze commented Jun 30, 2024

syphar left a comment

Choose a reason for hiding this comment

syphar commented Jul 10, 2024

pflanze commented Jul 11, 2024

pflanze commented Jul 11, 2024

pflanze commented Jun 24, 2024 •

edited by syphar

Loading