How to run multiple webdriver python programs at the same time?

Question

I have 4 Python scripts (3 web drivers and a main script). I would like to open these 3 web drivers at the same time when I run the mainscript.py. I used multiprocessing but you can use whatever you want.

Now it opens bot_1.py and then bot_2.py and then bot_3.py.

bot_1.py

from selenium import webdriver

driver = webdriver.Chrome(executable_path="C:\\Users\Andrei\Downloads\chromedriver_win32\chromedriver.exe")
links=['https://ro.wikipedia.org/wiki/Emil_Constantinescu','https://ro.wikipedia.org/wiki/Traian_B%C4%83sescu','https://ro.wikipedia.org/wiki/Napoleon_I']
for i in range(len(links)):
    driver.get(links[i])

bot_2.py

from selenium import webdriver

driver = webdriver.Chrome(executable_path="C:\\Users\Andrei\Downloads\chromedriver_win32\chromedriver.exe")

links=['https://ro.wikipedia.org/wiki/Abraham_Lincoln','https://ro.wikipedia.org/wiki/Winston_Churchill','https://ro.wikipedia.org/wiki/Mihail_Gorbaciov']
for i in range(len(links)):
    driver.get(links[i])

bot_3.py

from selenium import webdriver

driver = webdriver.Chrome(executable_path="C:\\Users\Andrei\Downloads\chromedriver_win32\chromedriver.exe")

links = ['https://ro.wikipedia.org/wiki/Gabriela_Firea', 'https://ro.wikipedia.org/wiki/Ion_Iliescu',
         'https://ro.wikipedia.org/wiki/Mihai_Eminescu']
for i in range(len(links)):
    driver.get(links[i])

mainscript.py

import bot_1, bot_2, bot_3
import multiprocessing

for bot in ('bot_1', 'bot_2','bot_3'):
    p = multiprocessing.Process(target=lambda: __import__(bot))
    p.start()

Answer 1

PyTest with the xdist extension is one option: https://docs.pytest.org/en/3.0.0/xdist.html

pip install pytest
pip install pytest-xdist

Then you run pytest -n NUM, where NUM is the number of processes (or in your case - webdriver instances) you want to run. I can't exactly remember now, but I think the command above runs all .py files in your current folder.

You can also use Behave + behave-parallel.

https://github.com/hugeinc/behave-parallel