Jenkins currently supports a discover permission, which when granted to the anonymous user, means if you visit a job and you don't have permission to see it, it will redirect you to the login page. Without this, the page just 404s, as if the job didn't exist. But what if you don't want to grant anonymous discover permission? The user experience when you're not logged in and you visit a job that you know exists, eg by clicking a link in GitHub or an email, is terrible, there's not even a link to click log in on the 404 page, you have to manually edit the URL bar to go to a page that does exist, click log in, log in, then click back in your browser until you get back to the screen that you wanted to view, and then click refresh to re-request that page.
Behaving the same way is if the job was not found is a good security practice, however returning 404 in both circumstances is not the only way to implement this security practice, in fact in my opinion it's a terrible way, given the terrible experience it yields. A better implementation would be, if the user is not logged in, redirect to the login page in both circumstances. Then, if the user does log in, the user can be redirected back to the page they tried to access, and either a 404 page can be rendered (this would happen both if it didn't exist, or if the user didn't have read permission on the job), or the job can be rendered if they do have permission. This way, an anonymous user is still unable to discover the existence of a project, since the same behaviour occurs in either situation, but users clicking links from emails/github etc who aren't logged in are not frustrated by the current terrible experience.
This I think is also a more semantic approach - if you 404 when the user doesn't have permission to view the project, you're lying to the user. If you redirect to login (effectively 401), you're saying "you're not allowed to find out whether this exists or not", which is exactly the intent, you don't have discover permission, so before you get a 404 to be told that it doesn't exist, you have to log in.