Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed error detection check when sending data via socket #404

Merged
merged 1 commit into from
Dec 16, 2024

Conversation

dngrudin
Copy link
Contributor

Checking the size inequality of the sent data and the input data will not always correctly indicate problems. Sometimes, even for a blocking socket, the send function can return a size smaller than that passed to the function. And only if the function returned the value -1 will it indicate errors.

@CLAassistant
Copy link

CLAassistant commented Dec 13, 2024

CLA assistant check
All committers have signed the CLA.

@dngrudin dngrudin force-pushed the check_for_sending_errors branch 2 times, most recently from 64b0af9 to d09db36 Compare December 13, 2024 11:37
@@ -457,11 +457,12 @@ size_t SocketOutput::DoWrite(const void* data, size_t len) {
static const int flags = 0;
#endif

if (::send(s_, (const char*)data, (int)len, flags) != (int)len) {
const ssize_t ret = ::send(s_, (const char*)data, (int)len, flags);
Copy link
Collaborator

@Enmk Enmk Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is it possible that less data would be sent than requested? If yes, what will happen?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had this happen (it happens very rarely, so I can't give an example of the sequence of actions that would lead to it). I don't know the exact reasons, since this is not typical for blocking sockets. Perhaps this happens when the server is slow in reading data from the socket compared to writing to the data on the client. I also found an increase in the frequency of such situations when using the 'strace' utility (but perhaps this was due to something else).
If, after the size has returned smaller, resend what was not sent, then all the data will be written to the table.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, but it seems like resent is not implemented

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is implemented in this place:

void WireFormat::WriteAll(OutputStream& output, const void* buf, size_t len) {
    const size_t original_len = len;
    const uint8_t* p = static_cast<const uint8_t*>(buf);

    size_t written_previously = 1; // 1 to execute loop at least once
    while (len > 0 && written_previously) {
        written_previously = output.Write(p, len);

        p += written_previously;
        len -= written_previously;
    }

    if (len) {
        throw ProtocolError("Failed to write " + std::to_string(original_len)
                + " bytes, only written " + std::to_string(original_len - len));
    }
}

As I understand it, all data is sent through this call.

Copy link
Collaborator

@Enmk Enmk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Enmk Enmk merged commit 2a49a25 into ClickHouse:master Dec 16, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants