How can I determine sensible thread number in python?

Question

I am writing a simple script which should do big amount of checks. Every check is independent so I decided to put it into multiple threads. However I don't know how fast will be machine on which the script will be set. I've already found quite nice util to check basic parameters of the target machine but I am wondering if there's any way to determine what is the max sensible amount of threads (I mean the moment when new thread starts slowing the process instead of speeding it)?

Answer 1

You can find out the number of cores your target machine has with

import multiprocessing

multiprocessing.cpu_count()

If you choose multiprocessing to manage your tasks then you can set the Pool size, or number of worker threads dependent on system load and .cpu_count().

As to what is a good number for you to choose for your program, you will have to decide for yourself :-)