Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python queue & multiprocessing queue: how they behave?

Tags:

python

queue

This sample code works (I can write something in the file):

from multiprocessing import Process, Queue  queue = Queue() def _printer(self, queue):     queue.put("hello world!!")  def _cmdDisp(self, queue):     f = file("Cmd.log", "w")     print >> f, queue.get()     f.close() 

instead this other sample not: (errormsg: 'module' object is not callable)

import Queue  queue = Queue() def _printer(self, queue):     queue.put("hello world!!")  def _cmdDisp(self, queue):     f = file("Cmd.log", "w")     print >> f, queue.get()     f.close() 

this other sample not (I cannot write something in the file):

import Queue  queue = Queue.Queue() def _printer(self, queue):     queue.put("hello world!!")  def _cmdDisp(self, queue):     f = file("Cmd.log", "w")     print >> f, queue.get()     f.close() 

Can someone explain the differences? and the right to do?

like image 568
DrFalk3n Avatar asked May 29 '09 09:05

DrFalk3n


People also ask

Does Python have queue?

Queue is built-in module of Python which is used to implement a queue. queue. Queue(maxsize) initializes a variable to a maximum size of maxsize. A maxsize of zero '0' means a infinite queue.

How do you access queue elements in Python?

To obtain the first element and the last element in the queue, the most straightforward way is to use indices. The first element in the queue has an index of 0. For the last item in the queue, we can use the -1 index. The minus sign indicates to Python to start counting items backward from the end of the queue.

What is import queue in Python?

So you must not write name of the module, just q = Queue(maxsize=100) . But if you want use classes with name of module: q = queue. Queue(maxsize=100) you mast write another import string: import queue , this is mean that you import all module with all functions not only all functions that in first case.


1 Answers

For your second example, you already gave the explanation yourself---Queue is a module, which cannot be called.

For the third example: I assume that you use Queue.Queue together with multiprocessing. A Queue.Queue will not be shared between processes. If the Queue.Queue is declared before the processes then each process will receive a copy of it which is then independent of every other process. Items placed in the Queue.Queue by the parent before starting the children will be available to each child. Items placed in the Queue.Queue by the parent after starting the child will only be available to the parent. Queue.Queue is made for data interchange between different threads inside the same process (using the threading module). The multiprocessing queues are for data interchange between different Python processes. While the API looks similar (it's designed to be that way), the underlying mechanisms are fundamentally different.

  • multiprocessing queues exchange data by pickling (serializing) objects and sending them through pipes.
  • Queue.Queue uses a data structure that is shared between threads and locks/mutexes for correct behaviour.
like image 70
Torsten Marek Avatar answered Sep 24 '22 18:09

Torsten Marek