When a not-logged-in user tries to view the start page, a big StackTrace indicating that they don't have access to some job appears. (This is actually true)
For an user with access to all plugins, everything is shown properly.
This issue appeared immediately after I updated my Build Pipeline Plugin from 1.4.1 to 1.4.2 and the plugin also appears in the StackTrace.

This is the StackTrace I get: http://pastebin.com/jNz8E4qw
This is what the User interface looks like for not-logged-in users: http://screencloud.net/img/screenshots/a45ce4757a1b6ef7dfe3006e08900557.png
This is what is looks like for a user with access to all jobs: http://screencloud.net//img/screenshots/ba953d7c92a35ec2dd10dbfb72734537.png