It is look so, that also in case that this checkbox is unchecked, only stable builds are used as reference build.
What I need achieve is: If result of current build is so, that there is more warnings that previous build, commiters of code (it is continues builds connected to SVN server) get mail that that they commit code with more warnings.
I setup plugin so, that "Compute new warnings (based on reference build)" is turned ON and "Status thresholds (New warnings)" have number 1 in "All priorities" - unstable (process is so that PMD plugin turn build to unstable state and Email-Ext plugin send mail when build is unstable).
Checkbox "Only use stable builds as reference" is unchecked but when builds turn to unstable, plugin use always as reference build last stable build (in opposite that I need him to use always previous build). I see this on build page where it write which build is used as reference.
I look in code and this is only guess, but in method BuildHistory.getReferenceAction is called getAction(true, useStableBuildAsReference) and this first true parameter (isStatusRelevant) cause that not stable build are not count as reference build.