This problem occurs if the POM configures multiple testing plug-ins which all output to the same directory and the previous results are still present in the reports directory when the build starts. The SurefireArchiver remembers all existing result files when processing the first test MOJO execution, which misleads it to ignoring updated results produced by subsequent test MOJO executions in the same build.

One solution would be to record both the existing file and its last modified timestamp and on processing subsequent test MOJOs only reject the same file if its timestamp is unchanged.

I have prepared a fix, for which I am awaiting legal approval to submit a Pull Request.