Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 7, 2025

Plan for adding VSTest console integration tests for AppDomain scenario:

  • Extract vstest.console.exe locator helper from CLITestBase.e2e.cs to a shared VSTestConsoleLocator class
  • Update CLITestBase.e2e.cs to use the new shared VSTestConsoleLocator class
  • Update MSTest.Acceptance.IntegrationTests project to reference the VSTestConsoleLocator
  • Expand AppDomainTests.cs to include tests that use vstest.console.exe directly
  • Fix CommandLine usage with proper command line formatting
  • Improve error messages to include stderr for better diagnostics
  • Refactor to reduce code duplication with helper methods
  • Add null check in GetTestPlatformVersion to prevent null reference exceptions
  • Fix nullable reference warnings for environment variables
  • Fix formatting issues (remove trailing whitespace)
  • Remove unnecessary package reference
  • Address code review feedback
  • Build and test the changes (blocked by network issues, will be tested in CI)

Summary

This PR addresses the issue where AppDomain should be enabled by default when running VSTest. The changes include:

  1. Created VSTestConsoleLocator class (test/Utilities/Automation.CLI/VSTestConsoleLocator.cs):

    • Extracted the vstest.console.exe location logic from CLITestBase.e2e.cs
    • Made it reusable across different test projects
    • Provides a static method GetConsoleRunnerPath() to locate vstest.console.exe
    • Includes proper null checks and handles nullable environment variables correctly
  2. Updated CLITestBase.e2e.cs:

    • Removed duplicate vstest.console.exe locator code
    • Now uses the shared VSTestConsoleLocator class
  3. Expanded AppDomainTests.cs:

    • Added two new test methods: RunTests_With_VSTestConsole_Directly and DiscoverTests_With_VSTestConsole_Directly
    • These tests run vstest.console.exe directly (not through dotnet test)
    • Tests validate AppDomain behavior with different DisableAppDomain settings (true, false, null)
    • Tests build the test project, locate the DLL, and run vstest.console.exe with optional runsettings
    • Added helper methods to reduce code duplication
  4. Updated MSTest.Acceptance.IntegrationTests.csproj:

    • Added link to VSTestConsoleLocator.cs for shared access

The new tests will expose any issues with AppDomain configuration when running under vstest.console.exe directly (as opposed to testhost.exe), which was the original problem described in the issue.

Original prompt

This section details on the original issue you should resolve

<issue_title>Appdomain should be enabled by default when running VSTest</issue_title>
<issue_description>

// HACK: When running VSTest, and VSTest didn't create TestHostAppDomain (default behavior), we must be enabling appdomain in MSTest.
// Otherwise, we will not merge app.config properly, nor we will have correct BaseDirectory of current domain.
#if NETFRAMEWORK
if (AppDomain.CurrentDomain.Id == 1 &&
AppDomain.CurrentDomain.FriendlyName.StartsWith("testhost.", StringComparison.Ordinal) &&
AppDomain.CurrentDomain.FriendlyName.EndsWith(".exe", StringComparison.Ordinal))
{
disableAppDomain = false;
}
#endif

This hack to enable appdomain by default in VSTest isn't enough. It only works when running in isolation (i.e, under testhost.exe). However, if we run under vstest.console directly, we don't get the expected behavior.</issue_description>

<agent_instructions>Write an acceptance integration test for this scenario. The scenario must use vstest.console.exe directly instead of dotnet test.

Note: There are already tests in test/IntegrationTests/MSTest.Acceptance.IntegrationTests/AppDomainTests.cs. They should only be expanded to run with both dotnet test and also with vstest.console.exe directly.

Also note that test/Utilities/Automation.CLI/CLITestBase.e2e.cs already contains a helper to locate vstest.console.exe. Consider extracting the helper to a VSTestConsoleLocator helper class that is shared in the two projects.</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Add integration test for appdomain behavior in VSTest Add integration tests for AppDomain behavior with vstest.console.exe Nov 7, 2025
Copilot AI requested a review from Youssef1313 November 7, 2025 10:14
Copilot finished work on behalf of Youssef1313 November 7, 2025 10:14
@Youssef1313
Copy link
Member

@copilot

test/Utilities/Automation.CLI/VSTestConsoleLocator.cs(46,42): error CS8600: (NETCORE_ENGINEERING_TELEMETRY=Build) Converting null literal or possible null value to non-nullable type.

Copy link
Contributor Author

Copilot AI commented Nov 7, 2025

@copilot

test/Utilities/Automation.CLI/VSTestConsoleLocator.cs(46,42): error CS8600: (NETCORE_ENGINEERING_TELEMETRY=Build) Converting null literal or possible null value to non-nullable type.
...

Fixed in 327f5c9. Changed string to string? for environment variable return values and added a null check for USERPROFILE with a descriptive exception message.

Copilot finished work on behalf of Youssef1313 November 7, 2025 10:27
@Youssef1313
Copy link
Member

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/AppDomainTests.cs(173,1): error IDE0055: (NETCORE_ENGINEERING_TELEMETRY=Build) Fix formatting (https://learn.microsoft.com/dotnet/fundamentals/code-analysis/style-rules/ide0055)

@copilot

Copy link
Contributor Author

Copilot AI commented Nov 7, 2025

test/IntegrationTests/MSTest.Acceptance.IntegrationTests/AppDomainTests.cs(173,1): error IDE0055: (NETCORE_ENGINEERING_TELEMETRY=Build) Fix formatting (https://learn.microsoft.com/dotnet/fundamental...

Fixed in 19a8212. Removed trailing whitespace on lines 173 and 221.

Copilot finished work on behalf of Youssef1313 November 7, 2025 11:06
Copilot AI requested a review from Youssef1313 November 7, 2025 11:12
Copilot finished work on behalf of Youssef1313 November 7, 2025 11:12
@Youssef1313 Youssef1313 changed the title Add integration tests for AppDomain behavior with vstest.console.exe Fix DisableAppDomain default value when not running in isolation under VSTest Nov 7, 2025
@Youssef1313 Youssef1313 marked this pull request as ready for review November 7, 2025 14:37
@Youssef1313
Copy link
Member

/backport to rel/4.0

@github-actions
Copy link
Contributor

github-actions bot commented Nov 7, 2025

Started backporting to rel/4.0: https://github.com/microsoft/testfx/actions/runs/19171676553

Evangelink pushed a commit that referenced this pull request Nov 8, 2025
…r VSTest by @Copilot in #6900 (backport to rel/4.0) (#6909)

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: Youssef1313 <[email protected]>
Co-authored-by: Youssef Victor <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Appdomain should be enabled by default when running VSTest

2 participants