Stream parser throws away data if too much read from socket #218
Comments
|
This is exactly what happens. We worked around it with by overriding the def once(self):
"""
Performs the operation of reading from the underlying
connection in order to feed the stream of bytes.
We start with a small size of two bytes to be read
from the connection so that we can quickly parse an
incoming frame header. Then the stream indicates
whatever size must be read from the connection since
it knows the frame payload length.
It returns `False` if an error occurred at the
socket level or during the bytes processing. Otherwise,
it returns `True`.
"""
if self.terminated:
return False
try:
b = self.sock.recv(self.reading_buffer_size)
except (socket.error, OSError, pyOpenSSLError) as e:
self.unhandled_error(e)
return False
else:
if not self.process(b):
return False
return TrueThat fix appears to be good enough for the threaded client. |
|
I just spent the last few hours debugging this issue before realizing there was an open issue on it: I'll file a PR later. |
hyperair
added a commit
to hyperair/WebSocket-for-Python
that referenced
this issue
Jun 6, 2017
When Frame.parser is given more data than expected, the next frame's header + contents can end up as part of the payload, causing a UTF-8 validation error, or ends up being thrown away. This can happen in the SSL case. This commit fixes that by amending WebSocket.once() to only provide bytes <= reading_buffer_size to the stream parser and keeping the rest in a buffer, giving the Frame a chance to stop receiving data at a frame boundary. Fixes: Lawouach#218
hyperair
added a commit
to hyperair/WebSocket-for-Python
that referenced
this issue
Jun 6, 2017
When Frame.parser is given more data than expected, the next frame's header + contents can end up as part of the payload, causing a UTF-8 validation error, or ends up being thrown away. This can happen in the SSL case if we read slowly and allow the socket's buffer to fill up a little. This commit fixes that by amending WebSocket.once() to only provide bytes <= reading_buffer_size to the stream parser and keeping the rest in a buffer, giving the Frame a chance to stop receiving data at a frame boundary. Fixes: Lawouach#218
hyperair
added a commit
to hyperair/WebSocket-for-Python
that referenced
this issue
Jun 6, 2017
When Frame.parser is given more data than expected, the next frame's header + contents can end up as part of the payload, causing a UTF-8 validation error, or ends up being thrown away. This can happen in the SSL case if we read slowly and allow the socket's buffer to fill up a little. This commit fixes that by amending WebSocket.once() to only provide bytes <= reading_buffer_size to the stream parser and keeping the rest in a buffer, giving the Frame a chance to stop receiving data at a frame boundary. Fixes: Lawouach#218
tsavola
pushed a commit
to ninchat/ninchat-python
that referenced
this issue
Oct 30, 2017
- ws4py 0.4.2 has a known issue which breaks the client: Lawouach/WebSocket-for-Python#218 - gevent 1.1.x has an infinite recursion problem with ssl context initialization
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If the socket reads more data than requested (as in the SSL case), then stream parser can get more data than it requested.
This means that the
some_bytesvariable here can contain more than one frame's worth of data.The frame parser then correctly reads one frame out of the data, but the rest of the data is thrown away.
The text was updated successfully, but these errors were encountered: