Allowing disabling percent encoding for some HTTP header fields

[zunda-pixel]

Question

I want to redirect to domain2.com from domain1.com.

I can redirect same domain.
ex: fromdomain1.com/login to domain1.com/account.

responses:
  '303':
    headers:
      location:
        schema:
          type: string

func login(_ input: Operations.login.Input) async throws -> Operations.login.Output {
  // login
  return .seeOther(.init(headers: .init(location: "https:/example2.com")))
}

[czechboy0]

Hi @zunda-pixel,

just to confirm, you're writing a server here, right? You can return a completely arbitrary URL in the response's Location header, can you clarify why you say you can't redirect to a different domain?

[zunda-pixel]

Yes, I am writing a server.

Base url is keeping...

http://127.0.0.1:8080/login -> http://127.0.0.1:8080/https%3A%2F%2Fexample2.com

[czechboy0]

Hmm, baseURL is only a client-side concept in Swift OpenAPI Generator, not a server one. Can you describe in detail how you're testing this and how you're getting this URL?

[zunda-pixel]

This is sample repository that has issue I told. Please check.
https://github.com/zunda-pixel/LoginServer

openapi: '3.1.0'
info:
  title: LoginService
  version: 1.0.0
servers:
  - url: https://example.com/
    description: Example service deployment.
paths:
  /login:
    get:
      operationId: login
      responses:
        '303':
          description: A success response Login
          headers:
            location:
              schema:
                type: string

import OpenAPIRuntime
import OpenAPIVapor
import Vapor

struct Handler: APIProtocol {
  func login(_ input: Operations.login.Input) async throws -> Operations.login.Output {
    return .seeOther(.init(headers: .init(location: "https://apple.com")))
  }
}

@main struct LoginServer {
  static func main() async throws {
    let app = Vapor.Application()
    let transport = VaporTransport(routesBuilder: app)
    let handler = Handler()
    try handler.registerHandlers(on: transport)
    try await app.execute()
  }
}

[czechboy0]

Can you clarify what the issue is? The code in the project all looks correct.

What are the steps you're taking, what is the result you see, and what is the result you expect? That'll help us understand where the mismatch is.

[zunda-pixel]

Current Result

1. http://localhost:8080/login
2. http://127.0.0.1:8080/https%3A%2F%2Fapple.com

Expecting Result

1. http://localhost:8080/login
2. https://apple.com

[czechboy0]

Which HTTP client are you using? A web browser? curl?

[zunda-pixel]

I use a web browser.

[czechboy0]

Thank you @zunda-pixel, I was able to isolate the issue.

The problem is that OpenAPI-defined headers are serialized according to the rules of RFC6570 (details here), which dictate that non-reserved characters need to be percent encoded. However, in the Location header, that causes the URL in the response header to be percent-encoded, and the web browser client doesn't remove the percent encoding, it seems.

As a workaround, add a middleware that removes the percent encoding of the Location header:

import OpenAPIRuntime
import OpenAPIVapor
import Vapor
import HTTPTypes

struct Handler: APIProtocol {
  func login(_ input: Operations.login.Input) async throws -> Operations.login.Output {
    return .seeOther(.init(headers: .init(location: "https://apple.com")))
  }
}

@main struct LoginServer {
  static func main() async throws {
    let app = Vapor.Application()
    let transport = VaporTransport(routesBuilder: app)
    let handler = Handler()
    try handler.registerHandlers(on: transport, middlewares: [
        UnescapeLocationHeaderMiddleware()
    ])
    try await app.execute()
  }
}

struct UnescapeLocationHeaderMiddleware: ServerMiddleware {
    func intercept(
        _ request: HTTPRequest,
        body: HTTPBody?,
        metadata: ServerRequestMetadata,
        operationID: String,
        next: (HTTPRequest, HTTPBody?, ServerRequestMetadata) async throws -> (HTTPResponse, HTTPBody?)
    ) async throws -> (HTTPResponse, HTTPBody?) {
        var (response, responseBody) = try await next(request, body, metadata)
        guard let location = response.headerFields[.location] else {
            return (response, responseBody)
        }
        response.headerFields[.location] = location.removingPercentEncoding
        return (response, responseBody)
    }
}

Now, thanks for reporting this. It's a bit troubling, and I suspect we'll need some way to make this more compatible with clients that don't percent-decode header fields.

If you don't mind, I'll repurpose this issue to track improving Swift OpenAPI Generator this way and rename it.

[zunda-pixel]

Thank you for looking into this issue, and providing a workaround.

[liambutler-lawrence]

👋 Is there any progress on this issue? We just started using swift-openapi-generator to generate a Swift client for our existing Lambda-based backend, and were surprised that Swift decided to unilaterally URL-encode header params, which is not part of the OpenAPI spec, and neither our backend nor existing clients do so.

---

In case it helps anyone else, here's the middleware we're using at the moment (to fix all the headers at once):

    private struct HeaderEncodingMiddleware: ClientMiddleware {
        func intercept(
            _ request: HTTPRequest, body: HTTPBody?, baseURL: URL, operationID: String,
            next: (HTTPRequest, HTTPBody?, URL) async throws -> (HTTPResponse, HTTPBody?)
        ) async throws -> (HTTPResponse, HTTPBody?) {
            var modifiedRequest = request
            modifiedRequest.headerFields = HTTPFields(
                request.headerFields.map { field in
                    HTTPField(name: field.name, value: field.value.removingPercentEncoding!)
                })
            return try await next(modifiedRequest, body, baseURL)
        }
    }

[czechboy0]

Hi @liambutler-lawrence,

percent-encoding of special characters in header fields is prescribed by OpenAPI, see this comment for details and links to the RFC: #529 (comment)

This issue tracks some possible ways to opt out of the behavior, because some services write their OpenAPI docs after their initial implementation, and they don't correctly remove percent encoding. That's a bug on any service that does that, but it's common enough that we're open to helping our users handle that situation better to interop with such legacy services.

If you'd like to see progress here, please propose a solution that we can discuss (how exactly should the opt out work?), and maybe you could ever open a PR for it later 🙂

[liambutler-lawrence]

@czechboy0 Thanks for following up and linking that doc. I stand corrected; however, we currently have no plans to change our server or our other clients to (unnecessarily IMHO) URL-encode header params. I would suggest adding a configuration flag that can be passed when initializing the generated Client, e.g.

let client = Client(serverURL: foo, configuration: .init(urlEncodeHeaderParams: false), transport: bar)

[czechboy0]

Our goal is to conform to the OpenAPI specification. For non-compliant implementations, there's the middleware solution from above. If we support some sort of opt-out, it's likely that it'd be specific to certain headers that historically haven't used percent encoding. But for any new headers, a compliant OpenAPI implementation should include percent encoding. Unilaterally diverging from the specification would break our compatibility with other OpenAPI tools.