Can Pipelines run concurrently in the same Runspace?

Question

How to run two cmdlets parallely in same Runspace. I am using C#.

InitialSessionState iss = InitialSessionState.CreateDefault();
iss.AuthorizationManager = new AuthorizationManager("MyShellId");
iss.ImportPSModule(new string[] { "MSOnline" });
Runspace powerShellRunspace = RunspaceFactory.CreateRunspace(iss);

In two threads, i will run cmdlets using the same runspace.

Pipeline pipeLine = powerShellRunspace.CreatePipeline();
pipeLine.Commands.Add(shellCommand);
pipeLine.Input.Close();
pipeLine.Invoke();
pipeLine.Output.DataReady += new EventHandler(processData);    //processData is a method which processes data emitted by pipeline as and when it comes to avoid out of memory
if (pipeLine.Error != null && pipeLine.Error.Count > 0) {
    Collection<Object> errors = (Collection<Object>)(pipeLine.Error.ReadToEnd());
    //process those errors
}

But when two threads simultaneously using the same runspace to run cmdlets. I am getting exception, "Pipeline not executed because a pipeline is already executing. Pipelines cannot be executed concurrently."

I need to use same runspace for performance reasons. How to achieve my objective?

Answer 1

Have you looked at the System.Management.Automation.Runspaces.RunspacePool class? Using it and the InitialSessionState can assist with eliminating overhead from module imports as it is only done once per pool and not per runspace. If you are looking for asynchronous execution of powershell commands here is a very basic, not production-ready example: (note I am not at a computer with Visual Studio but this should be right)

InitialSessionState iss = InitialSessionState.CreateDefault();
iss.AuthorizationManager = new AuthorizationManager("MyShellId");
iss.ImportPSModule(new string[] { "MSOnline" });
#set commands we want to run concurrently
string[] commands = new string[4] {
    "Start-Sleep -Seconds 5; 'Hi from #1'",
    "Start-Sleep -Seconds 7; 'Hi from #2'",
    "Start-Sleep -Seconds 3; 'Hi from #3'",
    "throw 'Danger Will Robinson'"
};
Dictionary<PowerShell, IAsyncResult> dict = new Dictionary<PowerShell, IAsyncResult>();
//this loads the InitialStateSession for all instances
//Note you can set the minimum and maximum number of runspaces as well
using(RunspacePool rsp = RunspaceFactory.CreateRunspacePool(iss))
{
    rsp.SetMinRunspaces(5);
    rsp.SetMaxRunspaces(10);
    rsp.Open();
    foreach(string cmd in commands)
    {
        PowerShell ps = PowerShell.Create();
        ps.AddScript(cmd);
        ps.RunspacePool = rsp;
        //Add parameters if needed with ps.AddParameter or ps.AddArgument
        dict.Add(ps,ps.BeginInvoke());            
    }
    do{
        List<PowerShell> toBeRemoved = new List<PowerShell>();
        foreach(KeyValuePair<PowerShell, IAsyncResult> kvp in dict)
        {
            if(kvp.Value.IsCompleted)
            {
                try
                {
                    PSDataCollection<PSObject> objs = kvp.Key.EndInvoke(kvp.Value);
                    foreach(PSObject obj in objs)
                    {
                        Console.WriteLine(obj);
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine(e);
                }
                finally
                {
                    toBeRemoved.Add(kvp.Key);
                }
            }
        }
        foreach(PowerShell item in toBeRemoved)
        {
            dict.Remove(item);
        }
        //Wait before we check again
        Thread.Sleep(200);
    } while (dict.Count > 0)
    rsp.Close();
}
//Added to keep console open
Console.Read();

This should give:

Hi from #3
Hi from #1
Hi from #2