+ sign in URL (query) encoded as %2B finaly is replaced by SPACE

[juzwos]

Ocelot version 23.3.4
Problem

• sign in URL(query) encoded as %2B finaly is replaced by SPACE

Hi
How to solve this problem
when I tried to pass the string with plus sign (+) properly encoded as %2B it was finally interpreted as white space ' '

I used
{catchAll}
{everything}
param={param}
param=%2BTEST+TEST

in each case the +/%2B was replaced by ' '(white space)

I know that + is the symbol of white space in URL and I think it is the clue of the issue

Direct call to the target api works fine

[raman-m]

But which upstream paths are you utilizing?
What does your ocelot.json look like?

So, you've attempted:

• /{everything}
• path/?param={param}

Have you tried path?{everything}? → docs: Catch All Query String

[raman-m]

Also please include logs

[juzwos]

Hi
thank you for the answer

I tried using {everything} - without any success

comparision
ver: 21.0.0
SOURCE: param=param_5315095.%5Bż%40%24%2B
DESTINATION:param=param_5315095.%5b%C5%BC%40%24%2b

ver:23.3.4
SOURCE: param=param_5315095.%5B%C5%BC%40%24%2B
DESTINATION:param=param_5315095.[%C5%BC@$+

as you can see in newest version %2B was send as + and + was translated to white space

[mponiedzialek]

To simplify above description
In version 21.0.0 when upstream service send query parameter with %2B then it was forwarded as %2B to downstream service. Currently (ver: 23.3.x) %2B is decoding to + and sending to downstream service.

Is it possible to pass parameters as it is to downstream service without decoding? (as it was in previous version)

[mponiedzialek]

After deeper investigation we found that issue appears in
https://github.com/ThreeMammals/Ocelot/blob/ce0227c059f220761acc2d880e554174bdb3c544/src/Ocelot/DownstreamUrlCreator/Middleware/DownstreamUrlCreatorMiddleware.cs (line 81)
If method: MergeQueryStringsWithoutDuplicateValues not executing problem not appears

[raman-m]

Indeed, my friend! This isn't a problem; it's a feature! 😄
We have something similar in our backlog. I'll share the numbers with you later.

[mponiedzialek]

In my humble opinion "the feature" is here:
https://github.com/ThreeMammals/Ocelot/blob/ce0227c059f220761acc2d880e554174bdb3c544/src/Ocelot/DownstreamUrlCreator/Middleware/DownstreamUrlCreatorMiddleware.cs (line 120):
private static string MapQueryParameter(KeyValuePair<string, string> pair) => $"{pair.Key}={pair.Value}";
it can be replaced by: $"{pair.Key}={HttpUtility.UrlEncode(pair.Value, Encoding.UTF8)}" or more suitable encoding method should be used.

[raman-m]

I've asked to fix this (de)coding problem of param values in PR #2150 but contributors are lazy as usual. I know this problem.
More info about:

• Regex exception when route template placeholder contains invalid character ) #2116
• #2116 Escaping unsafe pattern values of Regex constructor ​​derived from URL query parameter values containing special Regex chars #2150 (comment)
• TODO and other follow-up tasks #2141 (comment) subtask 4

[raman-m]

@mponiedzialek wrote on Oct 23

private static string MapQueryParameter(KeyValuePair<string, string> pair) => $"{pair.Key}={pair.Value}";
it can be replaced by: $"{pair.Key}={HttpUtility.UrlEncode(pair.Value, Encoding.UTF8)}" or more suitable encoding method should be used.

🆗 This change might resolve the issue. Are you planning to open a PR?

Regarding the problem, does it still occur when using the Catch All aka /{everything} upstream path? And does the same issue arise with the Catch All Query String?
It appears we've correctly fixed the forwarding of the OData filter query string. However, the OData filter contains a canonical valid string that is URL-decoded and free of special characters. In your user scenario, you input everything as URL-encoded, but the values are decoded during the parameter merging stage, correct?

[mponiedzialek]

Finally I prepared PR - how to get access to push it? :)

[raman-m]

You have to fork Ocelot repository into your account as our dev process states in docs:

• Development Process | RST docs
• Development Process | Read the Docs

The 2nd step directs a contributor to fork the repository and create a feature branch from the develop branch. In your repository, you can make any changes you desire and push any updates to your feature branch.

The 3rd step involves creating a pull request to the develop branch of the upstream repository.
Is this explanation clear?

[raman-m]

@mponiedzialek on Oct 30

Any problems with forking?

[mponiedzialek]

Sorry, I had no time in last week. Currently I read the instruction but still I don't know how or who should create branch where I can push my changes. Locally I created branch: bug/encoding-query-parameter-fix-discussion-2181 (started from develop) but I don't have permissions to create it on remote

[raman-m]

🤣