Consequences of Ruby's fiber 4kB stack size

Question

Fibers are a relatively new concept to me. I'm aware that each fiber's stack size is limited to 4kB and I keep reading that I should "beware" of this. What exactly are the real world consequences of this limit?

Edit:

It seems that this 4kB limitation isn't such a hindrance after all and it takes a good number of local variables (4,045) within the fiber itself to cause a SystemStackError to be raised.

count = 0
loop do
  count += 1
  puts count
  varlist = String.new
  count.times do |i|
    varlist += "a#{i} = 1\n"
  end
  s = "fiber = Fiber.new do \n #{varlist} \n end \n fiber.resume"
  eval(s)
end

Not the most elegant code but it does seem to demonstrate the limitations of a fiber's stack. It seems as though it's only return values, local variables (all of which contain a reference to an object on the heap) and method calls get put on the stack. I haven't tested whether the local variables etc in methods that are called from a fiber are part of the fiber's stack.

Edit 2:

Modified the above code. It does appear that variables etc in called methods become part of the fiber's stack. If this is the case, then the call depth (even without recursion) could be more of an issue as methods themselves are likely to require more space on the stack than variables (which seem to be transparent references to objects on the heap).

The following code fails on the 4,031st iteration and indicates variables in called methods become part of the fiber's stack:

count = 0
loop do
  count += 1
  puts count
  varlist = String.new
  count.times do |i|
    varlist += "a#{i} = 1\n"
  end
  m = "def meth\n #{varlist} \n end"
  eval(m)
  fiber = Fiber.new do
    meth
  end
  fiber.resume
end

Edit 3:

Just tried running the initial code example on Rubinius 2.0. Its fibers don't seem to have a 4kB stack limit, although beyond about 3,500th iteration it becomes increasingly and noticeably slow, and at around the 5,000th iteration it's averaging about one iteration a second. I don't know if there is a limit with RBX because I quit execution at just over 5,100 iterations. RBX is also using several times more memory than MRI 1.9.3.

JRuby 1.7 also doesn't seem to have a 4kB stack size for fibers and if fibers have a max stack size it's unknown to me. I completed 5,000 iterations of the first code example without problems, although as can be expected, the JVM chewed through a few hundred MB of RAM.

Answer 1

As Anton mentioned in his answer, you memory intensive code within a fiber. Examples of stuff that could (potentially) eat up a lot of memory:

• Large strings (ie: a string containing a decent sized HTTP response)
• Recursive functions (Stack Level Too Deep!)
• Streams or stream like objects: be VERY careful about stream buffers; if they get near or exceed 4k you'll start seeing some very strange behavior

Answer 2

The consequences of that are that you must pay more attention to the memory of your Fiber code, because you might have memory leaking.

Some recursive functions might give you problems