As far as I can tell this has never worked in 2.X. In 1.X all of the plugins bundled in the war get installed, so the detached plugins are added to new installations without problem. I tested various version combos with the first version before 2.16 (including 1.651), and the second after 2.18, but bouncycastle-api was never installed (I tried installing plugins, creating jobs, and changing config.xml before the upgrade but bouncycastle-api was never installed).
After attaching a debugger and doing an upgrade, I realized PluginManager#loadDetachedPlugins is called before InstallUtil#getDefaultInstallState, so the upgrade branch will never be taken. Looking through Jenkins' constructor, the call to Jenkins#executeReactor (which will execute the 'Loading bundled plugins' task that calls PluginManager#loadDetachedPlugins) does indeed happen a few lines before the call to InstallUtil#proceedToNext (which will update the install state to UPGRADE if an upgrade occurs). As far as I can tell the call to executeReactor does not return until all tasks are complete (though the tasks run in parallel by default), so there isn't even a race condition that would allow this to work occasionally.
I think the easiest fix here is to check InstallUtil.getLastExecVersion() against Jenkins.VERSION directly in PluginManager#loadDetachedPlugins since Jenkins.installState hasn't been initialized.