Index out of Range exception with null returned from a static function

I get a strange exception on calling a static function that return a null object:

Index was out of range. Must be non-negative and less than the size of the
collection.
Parameter name: index
At H:\test\issue_nullret.ps1:12 char:5
+     [CmdUtils]::RunCommand($cmd)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], ArgumentOutOfRangeExce
   ption
    + FullyQualifiedErrorId : System.ArgumentOutOfRangeException

The script is:

class CmdUtils {
  static [object]RunCommand($cmd)
  {
      $res = .$cmd
      #### if ($res -eq $null) { $res = $null } ### uncommenting this fixes it, but why ??
      return $res
  }
}

function RunCmd($cmd)
{
    [CmdUtils]::RunCommand($cmd) ### this errors out at the [ ] brackets
}

RunCmd {write-host 'Hi'}

I am using Powershell 5.1 on Windows 7. It'd be great if someone could offer an explanation of what happens here, or if this behaviour pertains on PS 7.4 and is just a known bug.

The behavior you're observing is a bug in Windows PowerShell 5.1 that has since been fixed in newer versions of PowerShell. An easy way to reproduce this error:

class Test { static [object] Testing() { return & { } } }
[Test]::Testing()

From here we can can check the exception's StackTrace:

PS ..\> $Error[0].Exception.StackTrace

   at System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument, ExceptionResource resource)
   at System.Management.Automation.ScriptBlock.InvokeAsMemberFunctionT[T](Object instance, Object[] args)
   at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite site, T0 arg0)
   at System.Management.Automation.Interpreter.DynamicInstruction`2.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)

Right after throwing the out of range exception we can see a call for InvokeAsMemberFunctionT[T](Object instance, Object[] args), and if we check the source code for this method in PowerShell 7+ (scriptblock.cs#L543-L548) we can see the following:

// This is needed only for the case where the
// method returns [object]. If the argument to 'return'
// is a pipeline that emits nothing then result.Count will
// be zero so we catch that and "convert" it to null. Note that
// the return statement is still required in the method, it
// just receives nothing from it's argument.
if (result.Count == 0)
{
    return default(T);
}

So they have essentially fixed the bug by doing what you have found by yourself as a workaround :)

There is also reference GitHub issue for this bug: Fix for #7128 Methods with return type [object] should return null for an empty result #7138.

In summary, this error occurs on methods having a return type of [object] where the argument is a script block, which, when invoked returns AutomationNull.Value.

Personally, to solve this specific issue, I would use ScriptBlock.Invoke(), this method ensures return type will be Collection<PSObject> from there the collection gets enumerated when outputted from your function without issues. But note, the use of .Invoke() is not always recommended, see this answer for more details.

class CmdUtils {
    static [object] RunCommand([scriptblock] $cmd) {
        return $cmd.Invoke()
    }
}

function RunCmd([scriptblock] $cmd) {
    [CmdUtils]::RunCommand($cmd)
}

RunCmd { Write-Host 'Hi' }