How to control concurrency?

[luzihang123]

demand:10W web pages rendered into Base64 upload OSS one day

[elacuesta]

First of all, thanks for taking an interest in this project.

Concurrency should be handled by the Scrapy Scheduler (which acts before the handler provided by this package), taking into account settings like CONCURRENT_REQUESTS, CONCURRENT_REQUESTS_PER_DOMAIN and CONCURRENT_REQUESTS_PER_IP. Could you elaborate a bit more on you use case, maybe with some minimal code to help me understand the scenario? Thanks again.