c++visual-studiogoogletesttest-explorer

How to set path at test discovery in visual studio?


How can I set the path to my external binaries during test discovery in visual studio's Test Explorer? After that how to make sure, it uses the correct paths?

I use windows 10 and VS 2019. I have a solution that builds some binaries and some tests into different folders. Also, I have some 3rd party dependencies, each in its own folder.

Something like:

solutionDir/
-ownBinaries/
-testBinaries/
-externalBinaries/

I'd like to use the Test Explorer to run my tests. For this purpose, I use a .runsettings file. I installed Google Test adapter via NuGet (later it will run on CI, so this is the only option). The automatic runsetting discovery is disabled, and this file is selected as the runsettings file. It overrides the workingDir to my ownBinaries folder, and extend the PATH enviroment variable with the externalBinaries. The relevant parts are:

<SolutionSettings>
    <Settings>
        <AdditionalTestExecutionParam>-testdirectory=$(SolutionDir)</AdditionalTestExecutionParam>
        <WorkingDir>$(SolutionDir)ownBinaries</WorkingDir>
        <PathExtension>$(SolutionDir)externalBinaries</PathExtension>
    </Settings>
</SolutionSettings>

This is works fine, after my tests are discovered, but I have problems when it tries to discover my tests.

I use google test and c++, so the test discovery tries to run those tests with the --gtest-list-tests argument, then populate the view with the test name, case, etc. The binaries are just fine, builds without error, I can run them from the debugger, and they produce the output I want.

But the test explorer won't show them, because it doesn't set the externalBinaries path.

This is what lead me to this situation.

Google Test Adapter: Test discovery starting...
Failed to run test executable 'D:\MySolution\testBinaries\SBCUnitTest.exe': One or more errors occurred.
Check out Google Test Adapter's trouble shooting section at https://github.com/csoltenborn/GoogleTestAdapter#trouble_shooting
In particular: launch command prompt, change into directory '..\ownBinaries', and execute the following command to make sure your tests can be run in general.
D:\MySolution\testBinaries\SBCUnitTest.exe --gtest_list_tests -testdirectory=
Found 0 tests in executable D:\MySolution\testBinaries\SBCUnitTest.exe
Test discovery completed, overall duration: 00:00:00.3022924

Have you noticed that -testDirectory= is empty despite it is set in the runsettings file?

I'm completely lost how I can proceed with it. This workaround is quite heavy to copy all files, then delete all but the test binaries each time when I start VS.

Here is the link for the Troubleshooting section mentioned in the error message. I've read through the readme file on github, also the runsetting docs on Microsoft's website.

Edit

I made progress with the VsTest.console.exe, I can successfully run all my tests with the proper arguments as below:

& "VSTest.console.exe" *_uTest.exe /Settings:..\MySolution.gta.runsettings /TestAdapterPath:"..\packages\GoogleTestAdapter.0.18.0\build\_common\"

I use the same *.runsettings and *.gta_settings_helper files. Those files are used to get absolute paths for the dependencies. I could run this from different folders, but then I had to adjust the arguments (test discovery pattern, relative path to runsettings, and relative path to GTA).

Great news, that it successfully runs on Azure (it uses vstest.console).

Edit 2

Tried to merge the workingDir and pathExtension nodes, so only one needed (the pathExtension). No success.

Tried to install Test adapter for google test in the VS installer, delete the runsetting file, and set the properties in VS->Tools->Options then Test adapter for google test. Even the example pathExtension didn't worked for me.

Found the extended logs under %AppData%/Local/Temp/TestAdapter/someNumber/*.txt and in that log I've found one line as the runsettings file. I paste here the formatted version of the log

<RunSettings>
    <GoogleTestAdapterSettings>
        <SolutionSettings>
            <Settings>
                <WorkingDir>$(SolutionDir)</WorkingDir>
                                <PathExtension>$(SolutionDir)externalBinaries</PathExtension>
            </Settings>
        </SolutionSettings>
        <ProjectSettings>
        </ProjectSettings>
               <GoogleTestAdapterSettings>
        <SolutionSettings>
            <Settings>
            </Settings>
        </SolutionSettings>
        <ProjectSettings>
        </ProjectSettings>
        </GoogleTestAdapterSettings>
    </GoogleTestAdapterSettings>
</RunSettings>

Does anybody know why is there an empty google test adapter setting? Where does it comes from? I think this is overwrites my settings.


Solution

  • It turned out, before first run the relative paths are not known.

    Trivial solution
    Add the full path to the PATH Extension under Visual Studio -> Options -> Test Adapter for Google Test settings. Meanwhile the custom *.runsetting file is not selected.

    Using this method all my tests are discoverable, but it is a manual setting for each repo cloned.