surefire version 3.5.4 broke maven-surefire-junit5-tree-reporter

[fabriciorby]

Affected version

3.5.4

Bug description

Hello,

I've developed this plugin extension some years ago and it's kind of totally dependant on the test "source" field from the ReportEntry to have the full nested classes paths

From version 3.5.4 on, there's no full path received by the ConsoleReporter.

From version 3.5.3, I got it filled like this

I suspect that it has to do with the new bug fix described in the Release Notes:
[SUREFIRE-2298] - fix xml output with junit 5 nested classes (fix integration with Cucumber and Archunit) (#828) @olamy

This is absolutely necessary for the plugin to work, I took a quicklook in the MR that is linked in the bug fix but I couldn't relate it to the source field. If someone can point me towards where this info is being retrieved/consumed and if this can behavior can be rollbacked it would be helpful so I can open a PR myself.

Link to the plugin: https://github.com/fabriciorby/maven-surefire-junit5-tree-reporter

[fabriciorby]

It seems there are some broken tests that were not detected by the CI/CD pipeline btw

When running for 3.5.3 it's all green

pipeline logs:

[INFO] --- surefire:3.5.2:test (default-test) @ surefire-junit-platform ---
[INFO] Surefire report directory: /home/runner/work/maven-surefire/maven-surefire/surefire-providers/surefire-junit-platform/target/surefire-reports
[INFO] Using configured provider org.apache.maven.shadefire.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[olamy]

please provide logs those screenshots are not usable.

[fabriciorby]

sure, now I know better (and have the project properly set up)

This is about the testing part, not sure why it does not work, I needed to screenshot because I managed to make it run on IntelliJ

╭╴󰀵 // …/maven-surefire/surefire-providers/surefire-junit-platform //  [master] 
mvn clean test

[INFO] Scanning for projects...
[INFO] 
[INFO] ---------< org.apache.maven.surefire:surefire-junit-platform >----------
[INFO] Building Surefire JUnit Platform Runner 3.5.5-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- clean:3.4.1:clean (default-clean) @ surefire-junit-platform ---
[INFO] Deleting /Users/fabriciorby/Desktop/projects/maven-surefire/surefire-providers/surefire-junit-platform/target
[INFO] 
[INFO] --- enforcer:3.5.0:enforce (enforce-bytecode-version) @ surefire-junit-platform ---
[INFO] Rule 0: org.codehaus.mojo.extraenforcer.dependencies.EnforceBytecodeVersion passed
[INFO] 
[INFO] --- enforcer:3.5.0:enforce (enforce-maven-version) @ surefire-junit-platform ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] 
[INFO] --- enforcer:3.5.0:enforce (enforce-java-version) @ surefire-junit-platform ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO] 
[INFO] --- jacoco:0.8.13:prepare-agent (jacoco-agent) @ surefire-junit-platform ---
[INFO] jacoco.agent set to -javaagent:/Users/fabriciorby/.m2/repository/org/jacoco/org.jacoco.agent/0.8.13/org.jacoco.agent-0.8.13-runtime.jar=destfile=/Users/fabriciorby/Desktop/projects/maven-surefire/surefire-providers/surefire-junit-platform/target/jacoco.exec,includes=**/failsafe/*:**/failsafe/**/*:**/surefire/*:**/surefire/**/*,excludes=**/HelpMojo.class:**/shadefire/**/*:org/jacoco/**/*:com/vladium/emma/rt/*
[INFO] 
[INFO] --- checkstyle:3.6.0:check (checkstyle-check) @ surefire-junit-platform ---
[INFO] You have 0 Checkstyle violations.
[INFO] 
[INFO] --- spotless:2.44.3:apply (default) @ surefire-junit-platform ---
[INFO] Index file does not exist. Fallback to an empty index
[INFO] Spotless.Java is keeping 13 files clean - 0 were changed to be clean, 13 were already clean, 0 were skipped because caching determined they were already clean
[INFO] Spotless.Pom is keeping 1 files clean - 0 were changed to be clean, 1 were already clean, 0 were skipped because caching determined they were already clean
[INFO] 
[INFO] --- remote-resources:3.3.0:process (process-resource-bundles) @ surefire-junit-platform ---
[INFO] Preparing remote bundle org.apache.apache.resources:apache-jar-resource-bundle:1.7
[INFO] Copying 3 resources from 1 bundle.
[INFO] 
[INFO] --- resources:3.3.1:resources (default-resources) @ surefire-junit-platform ---
[INFO] Copying 1 resource from src/main/resources to target/classes
[INFO] Copying 3 resources from target/maven-shared-archive-resources to target/classes
[INFO] 
[INFO] --- apache-rat:0.16.1:check (rat-check) @ surefire-junit-platform ---
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 14 licenses.
[INFO] 
[INFO] --- compiler:3.14.0:compile (default-compile) @ surefire-junit-platform ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 8 source files with javac [forked debug deprecation release 8] to target/classes
[INFO] 
[INFO] --- resources:3.3.1:testResources (default-testResources) @ surefire-junit-platform ---
[INFO] skip non existing resourceDirectory /Users/fabriciorby/Desktop/projects/maven-surefire/surefire-providers/surefire-junit-platform/src/test/resources
[INFO] Copying 3 resources from target/maven-shared-archive-resources to target/test-classes
[INFO] 
[INFO] --- compiler:3.14.0:testCompile (default-testCompile) @ surefire-junit-platform ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 5 source files with javac [forked debug deprecation release 8] to target/test-classes
[INFO] 
[INFO] --- surefire:3.5.2:test (default-test) @ surefire-junit-platform ---
[INFO] Using configured provider org.apache.maven.shadefire.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.865 s
[INFO] Finished at: 2025-10-14T00:57:59+02:00
[INFO] ------------------------------------------------------------------------

But about my main issue, is that there's no inner class path in the source of nested tests; this explicitly breaks one of the tests, which is this one (log from IntelliJ execution)

org.opentest4j.AssertionFailedError: expected: <org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest$NestingTest$Level1NestedTest> but was: <org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest$NestingTest>
Expected :org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest$NestingTest$Level1NestedTest
Actual   :org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest$NestingTest
<Click to see difference>


	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:177)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1145)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest.runNestingTest(JUnitPlatformProviderTest.java:442)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

test path: org.apache.maven.surefire.junitplatform.JUnitPlatformProviderTest.runNestingTest(JUnitPlatformProviderTest.java:442)

[fabriciorby]

and to be more precise, it happens here:

maven-surefire/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/RunListenerAdapter.java

Line 225 in 2b16f5d

String className = classMethodName.getClassName();

It will always get the class name without the inner class

[olamy]

correct this change is on purpose to fix issue with parallel test with junit5.
the choice has been made to use the source class name only. Ideally, adding another field with this value could help.
This will be easier once the big refactoring is finished from #3179
currently you only need to setup pluginManagement section to stick with previous version of surefire before the change

[fabriciorby]

that's indeed a big refactoring! if you could add this extra field in the SimpleReportEntry it would be very helpful.

meanwhile I'll keep recommending people to use surefire version 3.5.3, thank you for your reply