"Object reference not set to an instance of an object" when calling method on NoteProperty added in parallel processing

Question

I'm trying to add an object of a class as a NoteProperty to another class inside a ForEach-Object -Parallel block in PowerShell 7. However, when I try to call a method on that added NoteProperty, I get an "Object reference not set to an instance of an object" error.

Here's a simplified version of my code:

class test {}

$test = [test]::New()

1..2 | ForEach-Object -Parallel { 
    class test2 {
        testmethod () {
            Write-Host "test method called"
        }
    }
    Add-Member -InputObject $using:test -MemberType NoteProperty -Name "property$_" -Value ([test2]::New())
}

$test.property1.testmethod() 

If I do a Get-Member on $test.property1 I get the method testmethod in the output.

I understand that the issue might be related to how objects and properties are scoped within the parallel block. How can I correctly add the NoteProperty and ensure that the method call works outside the parallel block?

The error I get:

Get-Member on $test.property1:

It works with simple ForEach-Object without -Parallel.

Answer 1

Redefining a custom class (in your case class test2 { ... } inside every iteration of your parallel foreach) has some known issues - see for example https://github.com/PowerShell/PowerShell/issues/8767 and https://github.com/PowerShell/PowerShell/issues/20893.

It's likely this is another symptom of the same underlying problem.

One workaround would be to declare your class in the parent scope - if you do this you'll find there's an immediate issue in that the parallel foreach can't access that type in its scope and you get a Unable to find type [test2] error:

# broken code - don't use (see next code sample for a workaround)

class test {}

class test2 {
    testmethod () {
        Write-Host "test method called"
    }
}

$test = [test]::New()

1..2 | ForEach-Object -Parallel { 
    Add-Member -InputObject $using:test -MemberType NoteProperty -Name "property$_" -Value ([test2]::New())
}

# Unable to find type [test2].

But a follow-up workaround is to capture a reference to the type in a variable and then use the $using scope modifier to access it:

class test {}

class test2 {
    testmethod () {
        Write-Host "test method called"
    }
}
$test2 = [test2]
# ^^^^^^^^^^^^^^

$test = [test]::New()

1..2 | ForEach-Object -Parallel { 
    Add-Member -InputObject $using:test -MemberType NoteProperty -Name "property$_" -Value (($using:test2)::New())
    # >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>    ^^^^^^^^^^^^^^^^^^^^^^^
}

$test.property1.testmethod()

which gives the output

test method called

---

Update

It wasn't part of your original question, but be aware that you'll need to ensure your parallel code is thread-safe - if multiple foreach -parallel iterations try to run Add-Member concurrently you may get unpredictable results.

@SantiagoSquarzon has a good answer outlining some options for coordinating across multiple threads in PowerShell here - https://stackoverflow.com/a/75252238/3156906

Answer 2

To add to mclayton's helpful answer:

Using ForEach-Object -Parallel (available in PowerShell (Core) 7 only) comes with several challenges:

• Each parallel thread runs in a separate runspace, which, in essence, is an independent PowerShell session that knows nothing about the caller's state.

  • The latter is the reason that any of the caller's variable (values) must be referenced via the $using: scope.

• Due to parallel execution, you may have to perform explicit synchronization between the runspaces (threads) to avoid race conditions or state corruption.

  • Due to the latter's complexity - see this answer for a discussion and solutions - it is generally best to avoid cross-runspace (cross-thread) modification of data.

• An additional challenge comes with cross-runspace use of PowerShell custom classes, as in your case:

  • PowerShell class instances by default have runspace affinity (i.e., they're tied to a specific runspace) and trying to invoke their members from a different runspace can lead to state corruption, with symptoms ranging from subtle bugs to outright errors, such as in your case.

    • [scriptblock] literals ({ ... }) too have runspace affinity, and in their case ForEach-Object -Parallel explicitly prevents their cross-runspace use;^[1] e.g.,
      $sb = { 'hi!' }; % -Parallel { & $using:sb } triggers the following error:
      A ForEach-Object -Parallel using variable cannot be a script block [...]

---

PowerShell 7.4+ offers a way to create classes without runspace affinity, which can then safely be used from any runspace, namely via the [NoRunspaceAffinity()] attribute.

The following solution:

• demonstrates use of [NoRunspaceAffinity()]
• avoids the need for synchronization by handling the ETS property decoration via (Add-Member) outside the ForEach-Object -Parallel block, in an additional, regular ForEach-Object call where no concurrency issues can arise.

Note:

• This is largely for illustration, as in real life you wouldn't be defining a class inside a ForEach-Object -Parallel call, because not only is a class then defined in each parallel runspace, all resulting classes are then distinct .NET types.

class test {}

$test = [test]::New()

1..2 | ForEach-Object -Parallel {
    # Define the class without runspace affinity, so its instance
    # can safely be used in other runspaces, notably the caller's.
    [NoRunspaceAffinity()]
    class test2 {
        testmethod () {
            Write-Host "test method called"
        }
    }
    # Output an custom object with the target property name
    # and an instance of the [test2] class
    [pscustomobject] @{
        Name = "Property$_"
        Value = [test2]::New()
    }
} | ForEach-Object {
    # Attach an ETS NoteProperty member based on each runspace's output.
    Add-Member -InputObject $test -MemberType NoteProperty -Name $_.Name -Value $_.Value
}

# OK - the [test2] instances stored as property values can now
# be safely accessed.
$test.property1.testmethod()
$test.property2.testmethod()

---

^{[1] By contrast, creating a script block from a string, using [scriptblock]::Create(), creates an unbound [scriptblock] instance, which runs in whatever runspace it is called from; however, ForEach-Object -Parallel's refusal to use a script block via $using: is categorical.}