how to tell if a semaphore is full in python

Question

I have a bounded semaphore object that ensures my program doesn't download more than a certain number of files at a time. Each worker thread acquires the semaphore when it starts downloading and releases it when done.

I have another thread that would like to run code when nothing is being downloaded. I would like a method for locking until the semaphore is completely available. How can I do this in Python?

Answer 1

A pool of workers sounds like a good solution for this, so you might consider this question and its answers. You can create the pool of workers, submit all the jobs, close the pool, and then join it to wait for things to finish before handing off to the code you want to run when the files are done downloading.

Answer 2

You should definitely not try to peek into the counter value of the semaphore. Doing this breaks the abstraction of the semaphore. Moreover the value that you read may not even be the correct value because, one - there might be another thread that could change the value of the count before you can actually make use of the read value (depends on the scheduling policy); two - your read is not atomic. So hacking around might work but is not guaranteed to work all the time.

A better thing to do would be to use a counter variable in your program. But care should be taken to synchronize the access to this counter.