How to reproduce:
In either a Freestyle or Pipeline job,
- Specify 2 sourceDataFiles, say JMeter csv files: a.csv;b.csv
- Run Publisher (this will create serialized versions of the parsed files).
- Run once more for a relative comparison.
- Run once again but with files in different order: b.csv;a.csv
This will produce output similar to the following.
One build will have a relative comparison for just one file's samples, whereas the other build will have no comparison (as it tries to compare a.csv with b.csv), as below.