Feature Request: Limiting Incoming Stream Size in TIdHTTPServer

[dbcto]

Dear Indy development team,

I am using your great Indy library in Delphi for an application that receives HTTP requests through the TIdHTTPServer component. I am looking for a way to limit the size of incoming data streams, and stop receiving them when they exceed a certain size limit.

After looking through the documentation and the available events of the TIdHTTPServer component, it appears that there isn't currently a way to achieve this. The OnCommandGet event, for example, only gets triggered after the full request data stream has been received from the client. There doesn't seem to be an event that is called while the stream is still being received, which would allow me to stop the receiving process based on the size received so far.

Therefore, I would like to propose the addition of a new feature that allows users to set a maximum size for incoming data streams and abort the reception process if this size is exceeded. I believe that this would be a valuable feature that could help to prevent potential problems caused by clients sending overly large requests.

I am hopeful that you will consider this feature for future versions of the library or I would appreciate any guidance or pointers you could provide that would help us to implement this feature on our own. I also like to contribute to the Indy project and I believe that my work could benefit other users as well.

Thanks in advance and best regards,
Dennis

[rlebeau]

There are two different ways you can handle this in the current version:

• use the OnHeadersAvailable event to check the client's Content-Length request header, and if it is too large then set the VContinueProcessing parameter to False, and optionally use the OnHeadersBlocked event to customize the response if desired, ie by setting the VResponseNo parameter to 413 (Content Too Large). By default, it is 403 (Forbidden). Note that this approach will not work for "chunked" requests.

• use the OnCreatePostStream event to provide your own custom TStream object to receive the client's data (by default, a TMemoryStream is used). You can also look at the client's request headers in this event, too. If the Content-Length header is too large, or if too much data gets written to the stream, then you can raise an exception to terminate the request.

[dbcto]

Thank you very much! I will check this and get back to you.

[dbcto]

Hi @rlebeau.

I've checked the OnHeadersAvailable and OnHeadersBlocked approach.
This worked so far as I can check the headers and disrupt the communication if needed.

If I set VContinueProcessing to false the event OnHeadersBlocked is fired and I can set the VResponseNo etc. Unfortunately this information doesn't reach the calling client. I used postman for my test and there I only get "Could not get response" and "aborted".

Do I have to "send" the response in some way?

Thx!

[rlebeau]

No, the response is sent automatically after the event handlers return, as you can see in the source code:

  function HeadersCanContinue: Boolean;
  var
    LResponseNo: Integer;
    LResponseText, LContentText, S: String;
  begin
    // let the user decide if the request headers are acceptable
    // TODO pass the whole LRequestInfo object so the user has access
    // to the request method, too...
    Result := DoHeadersAvailable(AContext, LRequestInfo.URI, LRequestInfo.RawHeaders); // <-- FIRES THE OnHeadersAvailable EVENT
    if not Result then begin
      DoHeadersBlocked(AContext, LRequestInfo.RawHeaders, LResponseNo, LResponseText, LContentText); // <-- FIRES THE OnHeadersBlocked EVENT
      LResponseInfo.ResponseNo := LResponseNo;
      if Length(LResponseText) > 0 then begin
        LResponseInfo.ResponseText := LResponseText;
      end; 
      LResponseInfo.ContentText := LContentText;
      LResponseInfo.CharSet := 'utf-8'; {Do not localize}
      LResponseInfo.CloseConnection := True;
      LResponseInfo.WriteHeader; // <-- SENDS THE RESPONSE STATUS AND HEADERS
      if Length(LContentText) > 0 then begin
        LResponseInfo.WriteContent; // <-- SENDS THE RESPONSE BODY
      end;
      Exit;
    end;
    ...
  end;

[dbcto]

You are right. I tested it with Chrome and there I get the correct return code. Thank you very much.

[dbcto]

The limitation with the custom Stream and OnCreatePostStream also works great!
Do you know the normal chunk size that is written at once with http 1.1?

[rlebeau]

There is no "normal chunk size" in HTTP. If the client or server sends a chunked message, each chunk specifies its own size, so it can be whatever size the sender wants.

In this case, depending on buffering, the size written to the PostStream may not always be a full chunk at a time, it may be pieces of a chunk. The only requirement is that pieces are written to the PostStream in the correct order.