I am wondering how i can go about opening multiple concurrent connections using open-uri? i THINK I need to use threading or fibers some how but i'm not sure.
Example code:
def get_doc(url)
begin
Nokogiri::HTML(open(url).read)
rescue Exception => ex
puts "Failed at #{Time.now}"
puts "Error: #{ex}"
end
end
array_of_urls_to_process = [......]
# How can I iterate over items in the array in parallel (instead of one at a time?)
array_of_urls_to_process.each do |url|
x = get_doc(url)
do_something(x)
end
A simple method using threads:
threads = []
[1, 2, 3].each do |i|
threads << Thread.new { puts i }
end
threads.each(&:join)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With