Create a buffer in python

Question

I do not have much experience with Python and need to figure out a few things regarding buffering:

I want to generate a list in which I can allocate a certain amount of "buffer" space (do not know the specifics on this), to generate a list of integers. If the 'buffer' gets full I'm assuming it needs a flush command? Or how would you clear the buffer to continue putting things into that buffer?

Here is a sample of my code:

for i in range(0,500):
    randoms = random.randint(0,100000)
    looplist.append(randoms)

What I would want is in looplist, to be a sort of buffer I assume? in which if the maximum buffer space in looplist gets full, it would need to clear out (is there a pause during that time? or what happens) the list to continue re-generating integers to that list/ buffer.

Part 2 question: Would like explanation as simple of how a buffer could work for python? or does memory management of python just disable need to allocate own buffers? (can we still do it if we want too?)

I will edit my question if it seems to broad scope, trying to be as descriptive as I know how.

Answer 1

In Python, you don't have to care about memory management. You don't need to reserve "buffers", just assign the variables you want, and use them.

If you assign too many (for your RAM), you might run into MemoryErrors, indicating you don't have enough memory.

You could use a Queue as some kind of buffer, though, e.g.

import random
from Queue import Queue, Full

q = Queue(100)

for i in range(10):
    #fill queue
    while True:
        try:
            q.put_nowait(random.randint(1,10))
        except Full:
            break
    #take some values from queue
    print "Round", i,
    number_of_values_to_get = random.randint(0,20)
    print "getting %i values." % number_of_values_to_get
    for j in range(number_of_values_to_get):
        value = q.get()
        print "  got value", value

I create a Queue of size 100, i.e., a maximum of 100 entries can be stored in it. If you try to put on a full queue, it will raise the corresponding exception, so you better catch it. (This is only True if you use put_nowait or put with a timeout, just look at the docs for more details.)

In this example, for ten rounds, I fill a "buffer" (queue) of 100 random integers. Then, I pick between 0 and 20 values from this buffer and print them.

I hope this helps. The main use-case for queues is multithreaded program execution, they are thread-safe. So maybe you'll want to fill the queue from one thread and read it from another. If you don't want multithreading, collections.deque might be a faster alternative, but they are not thread-safe.