Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix body max limit issue #33

Open
tonysun83 opened this issue Oct 8, 2021 · 2 comments
Open

Fix body max limit issue #33

tonysun83 opened this issue Oct 8, 2021 · 2 comments

Comments

@tonysun83
Copy link

When ateles tries to index large documents, we get a :

"error":"error","reason":"{map_docs,{sel_conn_closed,[{ateles,'-map_docs/2-fun-2-',3,\n                                    [{file,\"src/ateles.erl\"},{line,86}]},\n                            {ateles,'-pmap_docs/2-fun-0-',3,\n                                    [{file,\"src/ateles.erl\"},{line,126}]}]}}","ref":1226203943}

We send the document here via ibrowse:
https://github.com/cloudant-labs/ateles/blob/master/src/ateles_util.erl#L123-L126

The http server closes the connection because the body (JSON document sent via HTTP) is too large.
https://github.com/cloudant-labs/ateles/blob/master/c_src/connection.cc#L112

Here are some potential ideas on how to fix this issue:
https://stackoverflow.com/questions/50348516/boost-beast-message-with-body-limit

@nickva
Copy link
Contributor

nickva commented Oct 8, 2021

The default is 1MB for received request body and we can have documents size just above that limit.

It looks like there isn't a simple config file or one-line method we can set, so might have to re-architect the parser a bit to use the parser interface vs the the basic message oriented currently in use.

The other escape hatch we have is to simply switch to back to couch_js if customers are upset and want their views to be built.

@iilyak
Copy link
Contributor

iilyak commented Feb 11, 2022

Just wanted to add some info for future generations.

https://www.boost.org/doc/libs/develop/libs/beast/doc/html/beast/ref/boost__beast__http__basic_parser/body_limit.html

Setting the limit after any body octets have been parsed results in undefined behavior. The default limit is 1MB for requests and 8MB for responses.

The hardcoded value is here

    static constexpr
    std::uint64_t
    default_body_limit(std::true_type)
    {
        // limit for requests
        return 1 * 1024 * 1024; // 1MB
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants