Manage Jenkins --> Restrict project naming --> Strategy: Pattern --> Name Pattern: <your regex>
When trying to save the configuration, the regex should be checked for correctness (e.g. matching parens)
Alternatively this could even be done in real time on the configuration page.
[Workaround: any regex errors will appear - and need to be fixed immediately - when you try to configure/create a job]