We had it happen again, but this time on the other host. This started happening not long after I tried to update the engine, and I think I screwed that up. The instructions I was working off of was outdated and a lot of the links on the ovirt site are broken. My theory is that the engines were not "synced" since a copy runs on each host. This would cause the engine on one host to come online and think the other is not responding, at this point it forces the other host offline using iDRAC (This is in the logs how it reboots the other hsot) and forces all VMs over.
So what I did was remove one host from the cluster, re-install CentOS7, and re-add it to the cluster. This way the engine config would be copied from the primary host and keep them synced. If this does not work the next step is to convert the hosted engine into a dedicated engine. Are there any good guides for making this conversion?