Additional headers not being used correctly

[MaxProfesor]

Hi issue I am encountering is in regards with additional_headers.
Firstly when connection is made the opening handshake requests headers don't match my custom headers
Also in the second request (101 switching protocols) while the additional_headers I set are all present, there is 2 user agent headers. One is my custom one from additional_headers the other (is the problem) it shouldn't be there "Python/3.9 websockets/14.2"
I am unsure but I think in previous versions that was not the case.

[MaxProfesor]

I used fiddler to inspect the requests going out

[MaxProfesor]

Found out the reason for the duplicate user_agent. In previous versions of websockets I passed useragent with extra_headers. Now user_agent_header is an argument in connect(). So by not passing it it was defaulting to the python/3.9... one. While in additional_headers (previously extra_headers) I had also user_agent so both were passed. This might be something to be fixed (checking if extra_headers already contains useragent string before defaulting). Now I still dont know why in the initial request the headers are not custom any info would be appreciated meanwhile I will try to dig deeper.

[aaugustin]

The first request is performed by your intercepting proxy, Fiddler, probably to fetch the certificate from the target server. websockets isn't involved there.

For the rest, sorry, it's a long story:

• Early versions of websockets didn't provide a way to override the User-Agent header (other than monkey-patching).
• websockets 4.0 made it possible to override it with extra_headers = {"User-Agent": "..."} (5592994).
• websockets 10.4 made it possible to remove it with user_agent_header=None (2a07325). Additionally, this provided a new, more straightforward way to override it, and you found it already.
• websockets 13.0 added an alternative way to configure it, with an environment variable (bbb3161).

---

You raise the question of what should be done in case User-Agent is provided explicitly in additional_headers (new implementations) or extra_headers (legacy implementation) in addition to the default provided implicitly in user_agent_header.

The legacy implementation doesn't create a duplicate:

request_headers.setdefault("User-Agent", self.user_agent_header)

The new asyncio and threading implementations create a duplicate:

self.request.headers["User-Agent"] = user_agent_header

You're suggesting that the new implementations should behave like the legacy implementation. This sounds sensible.

---

I /think/ that my reasoning at the time was "additional_headers is for advanced customization; whatever you put in there, you're in charge, and I won't go down the path of validating it".

The counter argument is: extra_headers={"User-Agent": "..."} was the official API at some point and keeping the same behavior has no cost.

[aaugustin]

This will be fixed in the next version & documented in the upgrade guide.