Fails to create test run when Test Plan is in a different Azure DevOps project (Build {buildId} cannot be found)

[ombacke]

Summary
When using Cross-Platform Test Result Publisher, publishing fails if the pipeline runs in one Azure DevOps project and the Test Plan is in another project.

The task can successfully match test cases in the target project, but fails during test run creation.

Error
[error]Build {buildId} cannot be found.

Impact
Test results are not published to the target Test Plan, even though test case matching succeeds.

Current Setup

• Pipeline project: Project A
• Test Plan project: Project B
• Task: Cross-Platform Test Result Publisher@1
• testResultFormat: jUnit
• testResultFiles: **/test-results/junit.xml
• ProjectName: Project B
• TestPlan: Automation Suite
• testCaseMatchStrategy: Regex
• testCaseRegex: TC(\d+)_

Observed Behavior

• JUnit result file is detected and parsed.
• Test cases are matched in the target project (Project B).
• CreateTestRun fails with Build {buildId} cannot be found.

Expected Behavior
If ProjectName points to a different project and test case matching works there, then CreateTestRun should:

• Resolve a valid build reference in the target project context, or
• Allow creating the test run without requiring a build from the target project, or
• Expose a configuration option to control build association for cross-project publishing.

Why This Appears to Be a Bug
The extension already works against the target project for test case matching, so project access and mapping appear correct. The failure seems isolated to build lookup during CreateTestRun when projects differ.

Request
Please confirm whether cross-project Test Plan publishing is officially supported.
If supported, please provide:

• A fix for the Build not found failure, or
• A documented workaround/configuration for cross-project CreateTestRun.

[bryanbcook]

At the moment, the task picks up the build from the actively running pipeline and defaults to $(Build.BuildId). There is similar logic for detecting if the task is running in a Release pipeline context.

The error originates in the ADO Test API as it assumes the value provided is within the scope of the Test Plan's project. I'll do some research to investigate if we can enrich the build reference with the originating project name. I believe like work item ids, build ids are unique for an entire organization, so I'm surprised the system is failing to associate. This could potentially be a bug in the Test API.

In the meantime, if this is a blocking issue, you might be able to override the build id at the task level by providing any build id from the target project. Obviously, tracking will be wrong, but this might get you past the lookup issue:

- task: PublishTestPlanResults@1
  inputs:
    ....
  env:
     BUILD_BUILDID: <build-id-from-project-b>

No guarantees this will work as BuildId is a system variable and may be read-only.

[ombacke]

Hi Bryan,
As you mentioned it looks like a bug in the Test API. For now I will ask our teams to keep the Test Plan in the same project as the Pipeline until a solution is available.
I will also try to override the BuildID to see if it even works.

[bryanbcook]

Having your team adjust the design of their TestPlan by moving it to another Project sounds like a big inconvenience. Especially since I feel this should be a solvable problem on my end.

I'll take a look at this over the weekend. If I can't find an immediate resolution, I can create a temporary feature flag that you can use to bypass the issue (This feature was added in #45, it can be turned off)

[ombacke]

That would be great if you could find a solution to this. I tried forcing the buildid to a buildid in project b but the task still failed.
Let me know if you have a solution.

[bryanbcook]

I've published a new release that includes a feature flag to disable the build association, which will get you around the issue you're experiencing.

To use, Feature flags are just pipeline variables, you can set them at the variable group, stage, job or at the task level (env):

- task: PublishTestPlanResults@1
  inputs:
    ...
  env:
    PUBLISHTESTPLANRESULTS_DISABLEBUILDASSOCIATION: true

[ombacke]

Great I will get the new version installed and test it out.

Thank you