Skip to content
This repository has been archived by the owner on Jan 13, 2021. It is now read-only.

Problem sending JSON data as request body #206

Open
lily-mara opened this issue Feb 26, 2016 · 11 comments · May be fixed by #208
Open

Problem sending JSON data as request body #206

lily-mara opened this issue Feb 26, 2016 · 11 comments · May be fixed by #208

Comments

@lily-mara
Copy link

I am trying to do some testing using the http2 version of httbin: https://nghttp2.org/httpbin/. Posting to the url https://nghttp2.org/httpbin/post will respond with a JSON object containing information about the request. This JSON includes the body and data sent with the request. Using hyper, it appears that the data that I'm sending as the request body is not being sent. Here's an example:

from hyper import HTTPConnection
conn = HTTPConnection('nghttp2.org:443')

req = conn.request(
    'POST',
    '/httpbin/post',
    body=b'{"foo": "bar"}',
)

resp = conn.get_response()
print(resp.read())

Using a different http2 client, I get the expected response:

{
  "args": {},
  "data": "{\"foo\": \"bar\"}",  <-- critical bit right here
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "no-cache",
    "Content-Length": "14",
    "Content-Type": "application/json",
    "Csp": "active",
    "Host": "nghttp2.org",
    "Origin": "chrome-extension://aicmkgpgakddgnaphhhpliifpcfhicfo",
    "Postman-Token": "70fa24f0-9e35-82ef-c838-7c15461ba3bd",
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36",
    "Via": "2 nghttpx"
  },
  "json": {
    "foo": "bar"
  },
  "origin": "68.180.88.224",
  "url": "https://nghttp2.org/httpbin/post"
}

But using the python given above, I get the response:

{
  "args": {},
  "data": "",
  "files": {},
  "form": {},
  "headers": {
    "Host": "nghttp2.org",
    "Transfer-Encoding": "chunked",
    "Via": "2 nghttpx"
  },
  "json": null,
  "origin": "68.180.88.224",
  "url": "https://nghttp2.org/httpbin/post"
}

I'm using version 0.5.0 of hyper, Python 3.4.4. Am I doing something wrong with my request, or is there a way to get more verbose output from hyper so that I can see what's really going on?
@Lukasa
Copy link
Member

Lukasa commented Feb 26, 2016

@natemara Can you quickly test whether this problem is fixed by using the code from the master branch on GitHub?

@lily-mara
Copy link
Author

I installed directly from master, and I'm getting the same results.

@Lukasa
Copy link
Member

Lukasa commented Feb 26, 2016

Interesting. So this seems only to happen with nghttp2.org: with http2bin.org, the same problem is not observed.

@tatsuhiro-t, do you have any theories about why nghttp2.org/httpbin might not be receiving data?

@tatsuhiro-t
Copy link

The difference between working client (e.g., nghttp) and hyper client is that former sends content-length, but hyper does not. nghttpx (which forwards request to backend httpbin) uses Transfer-Encoding: chunked in HTTP/1, but httpbin seems to not support chunked request. I have not verified httpbin code yet. I just checked that request body without content-length was sent to backend using go HTTP server as nghttpx's backend. So probably, httpbin does not handle request body without content-length yet?

@Lukasa
Copy link
Member

Lukasa commented Feb 28, 2016

@tatsuhiro-t That's interesting: what WSGI server are you using to run httpbin?

@tatsuhiro-t
Copy link

Gunicorn. Does it answer your question?

@Lukasa
Copy link
Member

Lukasa commented Feb 28, 2016

@tatsuhiro-t Hmm, that's interesting. http2bin.org uses H2O and gunicorn and doesn't display this problem. Weird.

@tatsuhiro-t
Copy link

Will check how h2o behaves in this situation. I think it is possible to calculate content-length if all request body is buffered.
BTW, why doesn't hyper client set content-length in request? Certainly, there is no way to set, but in this case, it knows whole request body, so it can set content-length.

@Lukasa
Copy link
Member

Lukasa commented Feb 28, 2016

@tatsuhiro-t Yeah, I think the lack of Content-Length in this case in hyper is a bug. That's ok. =)

@tatsuhiro-t
Copy link

I checked what h2o does. As I guessed, h2o sets content-length if the request body fits in its buffer.

@Lukasa
Copy link
Member

Lukasa commented Feb 29, 2016

Good intuition! =)

@lily-mara lily-mara linked a pull request Mar 2, 2016 that will close this issue
Open
@michal-niklas michal-niklas mentioned this issue Apr 27, 2017
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add content-length header for http2 connections
3 participants
@tatsuhiro-t @Lukasa @lily-mara