Replace support for Node.js http(s) Agents with documentation on using fetch dispatchers instead

[gr2m]

A problem that I haven't foreseen surfaced in this pull request to @octokit/core.js: Node's native fetch method does not support the usage of a Node http(s) agent, because it's not built on top of Node's http module (it's built on top of the net module instead).

I researched a bit and found that the native fetch method supports a dispatcher option that can be set to an instance of undici.Agent - not to mix up with Node's http.Agent.

My recommendations

1. Remove the request.agent option
2. Add documentation how to create a custom fetch function with a dispatcher preset (it's easy, see answer to this discussion)
3. Add documentation how to intercept all requests sent using Node's native fetch method by setting

Impact

This will cause a breaking change of @octokit/request and all of packages that depend on it (which is nearly all of them 🤷🏼). I'm sorry I missed this one.

Steps

1. Replace the tests to implement a client proxy and certificate authentication using a fetch dispatcher: feat: v5 core.js#563

   I think I've read somewhere that client authentication using a certificate is not supported by Node's native fetch. If that's the case we can recommend to use node-fetch with the agent option and implement the test that way

2. Update @octokit/graphql, then @octokit/core, then all plugins, authentication strategies, and @octokit/auth- packages that depend on @octokit/request

3. Update all packages that depend on @octokit/core

@octokit/js @octokit/js-community

[wolfy1339]

I don't see a problem with that. It's too bad we have to make another breaking change right after we just pushed some.

[nickfloyd]

Thanks @gr2m for catching this and doing the footwork on all of this!

@wolfy1339 I will assemble a tracking issue for this (for sanity sake). Detailing all of the above but, more importantly, detailing the steps to get these things out the door:

1. List all of the inflight breaking changes for dropping node v14/v16 support
2. Make the change for request.js in a beta branch and create a breaking release for that
3. Update all of the dependencies either in the existing beta branches (listed from step 1) or create new beta branches and breaking releases
4. Use those breaking releases to update all of the other dependencies
5. Ship all request/fetch and dropping node v14/v16 breaking changes - when ready and in order.

Does that sound good to you both?

[wolfy1339]

Sounds good

[wolfy1339]

Since we cannot update @octokit/request in all the repos, and it's causing some issues for users because of mismatched types, how do we want to handle that?
I don't see that there is a good way forward here except telling users to revert to the previous major version and deprecating those releases until this can be properly sorted

[gr2m]

sorry I'm not following, what the type mismatch related to this discussion? The plan is to update @octokit/request in all the repos eventually, rolling it out the same way we roll out the bump of required Node versions?

[wolfy1339]

I understand the goal is to update @octokit/request in all the repos, I'm just talking about the present

Since the release of @octokit/core is blocked, plugins pull in the current stable version of @octokit/core, which in turn pull in @octokit/types v9, while the plugins were updated to v10.

octokit/rest.js#282 (comment)
#2439 (comment)

[gr2m]

Ugh yeah thanks to typescript plugins have a dependency on @octokit/core, I hate this.

I'd just try to push this through as fast as possible. Worst case we release a breaking change for @octokit/request where we remove the agent option, and take care of adding documentation on how to implement proxies and client certificate authentication later, so that we unblock the @octokit/core release

[nickfloyd]

Tracking issue: #2486

[oscard0m]

I'm very late to the party if there is any scoped task I can take to help on this, count me in!

[wolfy1339]

Currently we have a PR in the works, and discussions happening over in the @octokit/request repo: octokit/request.js#599

We still need documentation on how to use the new fetch API equivalent to http agents.
If you'd like you can take that up