Running a test suite with over a million test cases

Question

We have a distributed test environment with the robotremoteserver starting a bunch of other applications and working with them as part of the test. The test that I am trying to run requires me to run over a million test cases in a single suite. The issue here is, when the pybot starts running, it blocks up 8 GB of RAM and results in slow performance. After a while it freezes up.

To overcome this issue I was planning to create separate suites with less than 100 cases in each. But in this case I am not able to use the keywords from the other remoteservers initiated from other suites. The only way I can use it is by disconnecting from the remoteserver and reconnecting it in each suite - which would defeat the purpose of the test.

I am not sure if anybody has come across this scenario. I would appreciate if anybody can think of a solution to this issue.

Additional information

Another thing that helps solve this: Is it is possible to call a keyword from a library (with state) that is initiated by another suite?

Can I use get library instance when using RIDE? I am importing the library in one suite, then try to get the library instance in other suite. Is this possible?

I keep getting the following error:

AttributeError: Remote instance has no attribute 'replace'

In Suite 1 this is what I am doing:

Import Library    Remote    ${verifix_xmlrpc_url}    WITH NAME    Verifix ${lib}=    BuiltIn.Get Library Instance    Verifix Set Global Variable    ${lib} 

In Suite 2:

${lib}.remove messages    ${VenueNSDQ} 

Answer 1

I don't know any specifics about Python, but you might want to try spinning up each test case as a separate process. This will allow you to recover the memory from each process as it completes, and may allow you to run your million test cases safely on one machine.

Generally, when a program needs more memory from the OS, it takes it, but it cannot give it back until it exits. This is likely why your process falls over.

Quick processes that you can spin up and then kill, will alleviate this memory problem.... and then it is a pretty short step from there to running each of these test cases through rsh on a different machine entirely.

I hope that helps a little.