I can't duplicate the problem you're reporting. I enabled CSRF protection using the default crumb issuer with Jenkins 1.643.3, git client plugin 1.19.6 and git plugin 2.4.4, then confirmed that I was still able to use the notifyCommit URL (http://localhost:8080/notifyCommit?url=my-url-to-git-repo) to trigger new builds.
The git plugin wiki page mentions "Push notification from repository" and then describes how the notifyCommit URL is used to start builds without requiring that Jenkins poll the remote repository. That's what I tested and confirmed is working as I expected.
Is there a proxy between your Jenkins server and the server that is generation the HTML requests to the notifyCommit URL? If so, then you may need to check the proxy support box on the CSRF configuration.
I also don't understand your comment:
Yes, i know this is quite old however the issue is with the gitplugin which is not that much newer (currently 2.3.5)
The latest version of the git plugin is 2.4.4. Any idea why you're not seeing more recent versions of the plugin? Are you using a private update center, or some other technique that prevents you from seeing the latest plugins?