Azure DevOps - conditional package restoration

A solution that is built via Azure DevOps pipeline has some projects that use conditional package references such as:

<Choose>
    <When Condition="'$(Configuration)'=='Debug'">
      <ItemGroup>
         <PackageReference Include="Mock.MyPackage" Version="1.0.0" />
      </ItemGroup>
    </When>
    <Otherwise>
      <ItemGroup>
         <PackageReference Include="MyPackage" Version="1.2.0" />
      </ItemGroup>
    </Otherwise>
</Choose>

The package source is a private NuGet feed.

The dotnet restore task does not respect the conditional package selection. (it restores Mock.MyPackage)
Question:
how can I conditionally restore packages (based on a $Configuration) ?

Remarks:
I have also tried restoring during Visual Studio Build task by specifying an MsBuild argument: /t:restore.
In that case it fails with a message: Failed to retrieve information about XX from remote source. If this command can restore packages how can I specify authorization args for the private feed ?

There is an issue on Github: https://github.com/NuGet/Home/issues/5895 where such issue is mentioned at the end.

I wanted minimal change in deployment configuration, so the following was acceptable.

1. An empty project is created and all packages which are restored conditionally are referenced there (without any conditions).

2. Additional argument to msbuild task is added: '/t:restore'

During build the following will happen: 'dotnet restore' will restore all the packages thanks to 1).
Since it does not know a selected configuration it may pick a wrong package (Mock.MyPackage instead of MyPackage). That is where 2) comes, where msbuild task will restore packages from a local cache made by 'dotnet restore'.

Remarks: As @(Leo Liu-MSFT) wrote, dotnet restore can authenticate but it does not know configuration, and msbuild knows configuration but it can not authenticate, so package restoration in AzDevOps from private feeds is tricky.