I am implementing Kosaraju's Strong Connected Component(SCC) graph search algorithm in Python.
The program runs great on small data set, but when I run it on a super-large graph (more than 800,000 nodes), it says "Segmentation Fault".
What might be the cause of it? Thank you!
Additional Info: First I got this Error when running on the super-large data set:
"RuntimeError: maximum recursion depth exceeded in cmp"
Then I reset the recursion limit using
sys.setrecursionlimit(50000)
but got a 'Segmentation fault'
Believe me it's not a infinite loop, it runs correct on relatively smaller data. It is possible the program exhausted the resources?
The following are some typical causes of a segmentation fault: Attempting to access a nonexistent memory address (outside process's address space) Attempting to access memory the program does not have rights to (such as kernel structures in process context) Attempting to write read-only memory (such as code segment)
It can be resolved by having a base condition to return from the recursive function. A pointer must point to valid memory before accessing it.
This happens when a python extension (written in C) tries to access a memory beyond reach.
You can trace it in following ways.
sys.settrace
at the very first line of the code.Use gdb
as described by Mark in this answer.. At the command prompt
gdb python (gdb) run /path/to/script.py ## wait for segfault ## (gdb) backtrace ## stack trace of the c code
I understand you've solved your issue, but for others reading this thread, here is the answer: you have to increase the stack that your operating system allocates for the python process.
The way to do it, is operating system dependant. In linux, you can check with the command ulimit -s
your current value and you can increase it with ulimit -s <new_value>
Try doubling the previous value and continue doubling if it does not work, until you find one that does or run out of memory.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With