JenkinsRule (or the hpi plugin) and not loading the optional dependencies correctly for dependant plugins
I think all you are seeing is that Maven does not traverse transitive dependencies past <optional>true</optional>.
Anyway I am not sure I am following what issue you are seeing.
newer optional dependencies than provided by Jenkins (ie previously bundled)
Are you referring to split plugins, à la ClassicPluginStrategy.DETACHED_LIST? Or what? It is up to the person defining tests to ensure that the plugins available in Maven’s test classpath are mutually compatible; if something is too old, you are obliged to either add <exclusions> for inappropriate dependency trails, or add explicit dependencies on a newer version. Now we may want to provide a friendlier system in the future, but for now that is how it is.