We use a declarative pipeline that is stored on our Bitbucket server and fetch on every run (default Jenkins functionality).

Every first PR build fails. There is unfortunately no further information on why it failed. We do see the following on every first PR build:
 

fatal: bad object 97264b302405d60800f1fe3724963fa18afea158

Any further build within the same PR does not include that message anymore.

Switching the jenkins-master durability from MAX_SURVIVABILITY to PERFORMANCE_OPTIMIZED solves the issue for us. The "fatal bad object" message remains in the logs.