Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Db prune & pageSize api restrictions #19

Open
equ1miner opened this issue Dec 30, 2018 · 3 comments
Open

Db prune & pageSize api restrictions #19

equ1miner opened this issue Dec 30, 2018 · 3 comments

Comments

@equ1miner
Copy link

equ1miner commented Dec 30, 2018

Due to the high number of blocks a database pruning function and request limitations on pagesize values appears to be a good idea to prevent incremental api request floods on the master and maintain a healthier db size for all pools.

@equ1miner
Copy link
Author

equ1miner commented Dec 31, 2018

I'd like to point out if 200k aion blocks were mined on a pool the database can get requests of large pageSizes

Aion has 60x the number of blocks a day produced compared to traditional 10min blockchains and pools are likely to build databases much more quickly than other chains and are subject to master load exploints.

Pruning and api pagesize limits would reduce the cost of the of hosting the master and create improved stability and faster performance for all pool owners during incremental api request floods!

An example solution would be to limit the possible calls on pagesizes. I am sorry I don't have time to review the source code at this time!

For example only allow ?pageSize=10000 or ?pagesize=100
Caching the responses by nginx, cloudflaire results in minincore only receiving one forwarded request until the the cache expiry time is reached.

Requests of incremental pageSizes such as 99999 then 100000 flood as each request isn't cached thus can load the DB server up even as the DB grows with a rate limit of 5 requests/second.

@equ1miner equ1miner changed the title Db prune Db prune & pageSize api restrictions Dec 31, 2018
@dipeshnuco
Copy link

@easy-mine-crypto thanks for putting up this issue. I have added it to the backlog for the pool software and we will analyze this feature request in more detail

@equ1miner
Copy link
Author

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants