Python - how exactly Queue works?

Question

Regarding example from documentation: https://docs.python.org/2/library/queue.html#Queue.Queue.get

def worker():
    while True:
        item = q.get()
        do_work(item)
        q.task_done()

How actually the worker would know all work is done, the queue is empty and we can exit? I don't understand it...

Answer 1

Your worker hangs in a while True: loop which means that the function/thread will never return.

The "magic" lies in the code you didn't show:

 t = Thread(target=worker)
 t.daemon = True
 t.start()

The daemon parameter controls when the uppper thread can exit

The entire Python program exits when no alive non-daemon threads are left.

Which means, that the program will exit, because the main thread exists. The worker thread thenically lives, but will be destroyed when the main thread ends (because "there are no non-daemon threads left").

The main thread exit condition is

q.join()

The documentatino for join will show, when it stops blocking the execution.

[...] When the count of unfinished tasks drops to zero, join() unblocks.

Answer 2

I'll keep it simple. Queue is basically collections of items like list for instance, difference it doesn't allow random access of elements. You insert and delete items in certain manner. The default type of queue is FIFO( first in first out). As you can figure from its name, its like a normal queue you see at any supermarket(or any place) the first person who entered the line will leave first.

There are three types of queue:

1. FIFO
2. LIFO
3. PRIORITY

FIFO like i said has rule of first in first out:

import queue #importing the library

q=queue.Queue() #create a queue object

for i in range(5):
    print(q.put(i)) #adding elements into our queue

while not q.empty():
    print(q.get()) #to delete item and printing it

LIFO works on the principle of last in first out:

import queue #importing the library

q=queue.LifoQueue() #create a queue object

for i in range(5):
    print(q.put(i)) #adding elements into our queue

while not q.empty():
    print(q.get()) #to delete item and printing it

PRIORTY queue gives out data in ascending order, as in, the smallest one will exit the queue first.

import queue #importing the library

q=queue.LifoQueue() #create a queue object


q.put(3) 
q.put(7) 
q.put(2) 
q.put(7) 
q.put(1) 


while not q.empty():
    print(q.get()) #to delete item and printing it

To answer your last question, as you can see in example, you can use q.empty() to check if your queue is empty or not.

If you have any further doubt feel free to ask.