If you remove all of the .p12 certificates, the engine-setup process will recreate them.
However, the Engine will lose SSH access to the hosts for tasks that require SSH
connectivity.
A better approach is to re-sign the certificates as mentioned above.