Currently the SSH-Agent plugin doesn't support SSH certificates (e.g. RSA-CERT). It can only load private keys. For example below my ssh-add -l command:2048 

2048 SHA256:FINGERPRINT COMMENT1 (RSA)
2048 SHA256:FINGERPRINT COMMENT2 (RSA)
256 SHA256:FINGERPRINT COMMENT3 (ED25519)
2048 SHA256:FINGERPRINT COMMENT4 (RSA-CERT)

It would be nice to load a certificate from the credential manager and add it to SSH-agent.

At this location: https://github.com/jenkinsci/ssh-agent-plugin/blob/878b53c895119c248eac9dd07da155a47aa99c88/src/main/java/com/cloudbees/jenkins/plugins/sshagent/exec/ExecRemoteAgent.java#L92C33-L92C51 a "*-cert.pub" file should be created besides the private key file so ssh-agent will also load the certificate.

When more information is required please ask!