Multiprocessing: How to use pool.map on a list and function with arguments?

Question

I have some misunderstandings with multiprocessing and map function.

I'll try to describe briefly:

Firstly, I have an list, for instance:

INPUT_MAGIC_DATA_STRUCTURE = [
    ['https://github.com', 'Owner', 'Repo', '', '', '0', '0'],
    ['https://github.com', 'Owner', 'Repo', '', '', '0', '0'],
    ['https://github.com', 'Owner', 'Repo', '', '', '0', '0'],
    ['https://github.com', 'Owner', 'Repo', '', '', '0', '0'],
    ['https://github.com', 'Owner', 'Repo', '', '', '0', '0'],
    ['https://github.com', 'Owner', 'Repo', '', '', '0', '0'],
]

Also I have method, which currently parsing this list using specific internal logic:

def parse(api_client1, api_client2):
     for row in INPUT_MAGIC_DATA_STRUCTURE: 
         parsed_repo_row = ... (some logic with row)
         OUTPUT_MAGIC_DATA_STRUCTURE.append(parsed_repo_row)

Finally, I've red that there is some variants to make it async instead of for.

from multiprocessing import Pool
    pool = Pool(10)
    pool.map(<???>, INPUT_MAGIC_STRUCTURE)

??? – I cannot understand how to transfer my parse() from for row in INPUT_MAGIC_DATA_STRUCTURE as a first argument to pool.map() and transfer all its arguments — api_client1, api_client2.

Could you help me?

Thanks in advance.

UPD:

I've already made:

pool = Pool(10)
pool.map(parse(magic_parser, magic_staff), INPUT_MAGIC_DATA_STRUCTURE)

Anyway, when interpreter comes to the second line it stops and makes only one instance of parse() method (I see the logging output of parsed rows: 1 , 2 , 3 , 4 , 5 – one by one).

Answer 1

Put (some logic with row) in a function:

def row_logic(row):
    return result

Pass the function to Pool.map:

pool = Pool(10)
pool.map(row_logic, INPUT_MAGIC_DATA_STRUCTURE)