You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Ruby introduced non-blocking fibers and fiber schedulers lately, is there any way to make Curl::Easy#perform non-blocking in the same way?
As perform releases GIL under the hood, I expected it to work this way already, but it doesn't seem so:
async_crawler.rb
#!/usr/bin/env -S mise exec [email protected] -- rubygem'curb','1.0.9'require'curl'gem'awesome_print','1.9.2'require'awesome_print'gem'async','2.23.0'require'async'gem'fiber_scheduler','0.13.0'require'fiber_scheduler'gem'io-event','1.9.0'require'io/event'curls=Array.new(3){Curl::Easy.new}tasks_arr=curls.size.times.mapdo5.times.mapdo{url: 'https://httpbin.org/status/200',}endenddefcrawl(curl,url)curl.url=urlstart_time=Time.now.to_fcurl.perform{code: curl.response_code,start_time: start_time,response_time: Time.now.to_f}end##### crawl using fiber_scheduler ####Fiber.set_scheduler(FiberScheduler.new)curls.zip(tasks_arr).eachdo |curl,tasks|
Fiber.scheduledotasks.eachdo |task|
Fiber.scheduledotask.merge!(crawl(curl,task[:url]))endendendendFiber.set_scheduler(nil)#### end of fiber_scheduler usage ####aptasks_arr#### crawl using Async ####curls.zip(tasks_arr).eachdo |curl,tasks|
Asyncdotasks.eachdo |task|
Asyncdotask.merge!(crawl(curl,task[:url]))endendendend#### end of Async usage ####aptasks_arr
The text was updated successfully, but these errors were encountered:
Ruby introduced non-blocking fibers and fiber schedulers lately, is there any way to make
Curl::Easy#perform
non-blocking in the same way?As
perform
releasesGIL
under the hood, I expected it to work this way already, but it doesn't seem so:async_crawler.rb
The text was updated successfully, but these errors were encountered: