I am trying to understand when does it make sense to catch MemoryError
in Python, I got two scenarios:
Scenario 1: MemoryError is successfully caught.
import numpy as np
try:
a = np.ones(100000000000)
except MemoryError:
print 'got memory error, plan B'
a = np.ones(10) # this gets created
Scenario 2: My program freezes
silly = []
c = 0
try:
while True:
silly.append((str(c))) # just increasing the list
c += 1
if c % 1000000 == 0:
print 'counter : {}'.format(c)
except MemoryError:
print 'oops' # never get here
silly.append('silly')
My guess is in the first case, python "knows" how much memory would need to be allocated, and thus raises a MemoryError
exception. While in the second case python does not "know" how big I intent silly
to be. However, list
is a dynamic array; therefore, python should know that extending this array by a certain amount would cause a MemoryError
, why is the exception not raised then?
I have looked at this question, and the relevant paragraph from the docs is:
exception MemoryError
Raised when an operation runs out of memory but the situation may still be rescued (by deleting some objects). The associated value is a string indicating what kind of (internal) operation ran out of memory. Note that because of the underlying memory management architecture (C’s malloc() function), the interpreter may not always be able to completely recover from this situation; it nevertheless raises an exception so that a stack traceback can be printed, in case a run-away program was the cause.
Though this helped me, I am still not very clear as to what is going on, and I did not get an exception as the docs seem to indicate.
My questions: Is my guess right on scenario 1? Why is MemoryException
not raised in scenario 2?
I am using Python 2.7.5+, and I am on ubuntu 13.10
A MemoryError means that the interpreter has run out of memory to allocate to your Python program. This may be due to an issue in the setup of the Python environment or it may be a concern with the code itself loading too much data at the same time.
The try and except block in Python is used to catch and handle exceptions. Python executes code following the try statement as a “normal” part of the program. The code that follows the except statement is the program's response to any exceptions in the preceding try clause.
In Python list is a linked list so the memory Exception MemoryError
will be raised only when the system can't allocate additional memory.
This scenario occurs only when your RAM (main memory)
is utilized to its maximum level most of the OS handles this scenario by moving less priority data to Hard Disk
hence you don't get MemoryError
in new OS but its better to handle this because the legacy OS
don't have the above mentioned mechanism so the programer needs to handle it.
The MemoryError
is raised by OS not by Python.
Legacy OS raise MemoryError immediately when it finds that memory is full, Where as Most of latest OS try to move the content to Hard Disk as it find that Main Memory is full and no room for new request, if its still can't be accommodated in Allocated space of Hard Disk or the current request size exceeds limit of data storage space allocated for this particular process in RAM(Main Memory) then it raise's MemoryError.
The MemoryError
raised by OS
can be captured by your python program using try exception
.
The MemoryError
is also raised by OS
when you make a request for memory that exceeds limit of Main Memory
to store data.
Example: If you are using 512MB RAM and your OS allocated 200MB in RAM for data and rest for code, In that 200MB for your process 50MB Data space is allocated by OS and you make a request to store 100MB which exceeds limit hence OS raises MemoryError in this scenario too.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With