From: "Neil" <nwilson123(a)gmail.com>
To: "Alon Bar-Lev" <alonbl(a)redhat.com>
Cc: users(a)ovirt.org
Sent: Wednesday, May 28, 2014 10:04:00 AM
Subject: Re: [ovirt-users] Can't Install/Upgrade host
Hi Alon,
Thanks for the reply, below is the output.
Something changed the file attributes of ca.pem (two places) to be incorrect.
[root@engine01 ovirt-engine]# ls -lR /etc/pki/ovirt-engine/
/etc/pki/ovirt-engine/:
total 80
lrwxrwxrwx. 1 root root 6 May 16 13:56 apache-ca.pem -> ca.pem
-rw-r--r--. 1 root root 570 May 16 13:56 cacert.conf
-rw-r--r--. 1 root root 519 May 16 13:56 cacert.template
-rw-r--r--. 1 root root 384 Mar 24 12:47 cacert.template.in
-rw-r--r--. 1 root root 482 May 16 13:56 cacert.template.rpmnew
-rwxr-x---. 1 root root 3362 May 16 13:56 ca.pem
-rw-r--r--. 1 root root 585 May 16 13:56 cert.conf
drwxr-xr-x. 2 ovirt ovirt 4096 Mar 24 12:47 certs
-rw-r--r--. 1 root root 572 May 16 13:56 cert.template
-rw-r--r--. 1 root root 483 Mar 24 12:47 cert.template.in
-rw-r--r--. 1 root root 534 May 16 13:56 cert.template.rpmnew
-rw-r--r--. 1 ovirt ovirt 950 May 22 20:07 database.txt
-rw-r--r--. 1 ovirt ovirt 20 May 22 20:07 database.txt.attr
-rw-r--r--. 1 ovirt ovirt 20 May 16 13:56 database.txt.attr.old
-rw-r--r--. 1 ovirt ovirt 885 May 16 13:56 database.txt.old
drwxr-xr-x. 2 root root 4096 Mar 24 12:47 keys
-rw-r--r--. 1 root root 548 Mar 24 12:47 openssl.conf
drwxr-x---. 2 ovirt ovirt 4096 Mar 24 12:47 private
drwxr-xr-x. 2 ovirt ovirt 4096 May 27 13:16 requests
-rw-r--r--. 1 ovirt ovirt 3 May 22 20:07 serial.txt
-rw-r--r--. 1 ovirt ovirt 3 May 16 13:56 serial.txt.old
/etc/pki/ovirt-engine/certs:
total 100
-rw-r--r--. 1 root root 3362 May 16 13:56 01.pem
-rw-r--r--. 1 root root 3509 May 16 13:56 02.pem
-rw-r--r--. 1 root root 3466 May 16 13:56 03.pem
-rw-r--r--. 1 root root 3466 May 16 13:56 04.pem
-rw-r--r--. 1 root root 3362 May 16 13:56 05.pem
-rw-r--r--. 1 root root 3509 May 16 13:56 06.pem
-rw-r--r--. 1 root root 3362 May 16 13:56 07.pem
-rw-r--r--. 1 root root 3509 May 16 13:56 08.pem
-rw-r--r--. 1 root root 3466 May 16 13:56 09.pem
-rw-r--r--. 1 root root 3467 May 16 13:56 0A.pem
-rw-r--r--. 1 root root 3467 May 16 13:56 0B.pem
-rw-r--r--. 1 root root 3467 May 16 13:56 0C.pem
-rw-r--r--. 1 root root 3467 May 16 13:56 0D.pem
-rw-r--r--. 1 root root 3070 May 16 13:56 0E.pem
-rw-r--r--. 1 root root 3070 May 16 13:56 0F.pem
-rw-r--r--. 1 root root 3070 May 16 13:56 10.251.193.8cert.pem
-rw-r--r--. 1 root root 3070 May 16 13:56 10.251.193.9cert.pem
these two are strange as I expect to be owned by ovirt user as engine created.
-rw-r--r--. 1 root root 4267 May 22 20:07 10.pem
-rw-r-----. 1 root root 3509 May 16 13:56 apache.cer
-rw-r--r--. 1 root root 763 May 16 13:56 ca.der
-rw-r--r--. 1 root root 3509 May 16 13:56 engine.cer
-rw-r--r--. 1 root root 784 May 16 13:56 engine.der
-rw-r--r--. 1 root root 4267 May 22 20:07 websocket-proxy.cer
/etc/pki/ovirt-engine/keys:
total 36
-rw-r-----. 1 root root 916 May 16 13:56 apache.key.nopass
-rw-r-----. 1 root root 2786 May 16 13:56 apache.p12
-rw-------. 1 root root 1054 May 22 20:07 engine_id_rsa
-rw-------. 1 root root 916 May 16 13:56 engine_id_rsa.20140522200739
-rw-------. 1 root root 912 May 16 13:56 engine_id_rsa.old
-rw-r-----. 1 ovirt ovirt 2786 May 16 13:56 engine.p12
-rw-r--r--. 1 root root 220 May 16 13:56 engine.ssh.key.txt
-rw-------. 1 ovirt ovirt 1832 May 22 20:07 websocket-proxy.key.nopass
-rw-------. 1 root root 2517 May 22 20:07 websocket-proxy.p12
/etc/pki/ovirt-engine/private:
total 4
-rwxr-x---. 1 root root 887 May 16 13:56 ca.pem
/etc/pki/ovirt-engine/requests:
total 24
-rw-r--r--. 1 root root 862 May 16 13:56 10.251.193.8req.pem
-rw-r--r--. 1 ovirt ovirt 862 May 27 17:35 10.251.193.9.req
-rw-r--r--. 1 root root 862 May 16 13:56 10.251.193.9req.pem
-rw-r--r--. 1 root root 603 May 16 13:56 ca.csr
-rw-r--r--. 1 root root 597 May 16 13:56 engine.req
-rw-r--r--. 1 root root 863 May 22 20:07 websocket-proxy.req
On Wed, May 28, 2014 at 8:19 AM, Alon Bar-Lev <alonbl(a)redhat.com> wrote:
> Please send the output of:
>
> # ls -lR /etc/pki/ovirt-engine/
>
> ----- Original Message -----
>> From: "Neil" <nwilson123(a)gmail.com>
>> To: users(a)ovirt.org
>> Sent: Wednesday, May 28, 2014 9:04:57 AM
>> Subject: [ovirt-users] Can't Install/Upgrade host
>>
>> Hi guys,
>>
>> I'm trying to upgrade/re-install a host running Centos 6.5, but even
>> after removing the host completely and trying to re-add it, I keep
>> getting a "Certificate enrollment failed" error. The full error below
>> is taken from my engine.log...
>>
>> 2014-05-27 10:38:33,729 ERROR
>> [org.ovirt.engine.core.utils.servlet.ServletUtils]
>> (ajp--127.0.0.1-8702-4) Can't read file
>> "/var/lib/ovirt-engine/reports.xml" for request
>> "/ovirt-engine/services/reports-ui", will send a 404 error response.
>> 2014-05-27 11:10:49,343 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (VdsDeploy) Error during deploy dialog: java.io.IOException:
>> Unexpected connection termination
>> 2014-05-27 11:10:49,344 ERROR
>> [org.ovirt.engine.core.utils.ssh.SSHDialog]
>> (org.ovirt.thread.pool-6-thread-31) SSH error running command
>> root@10.251.193.9:'umask 0077; MYTMP="$(mktemp -t
ovirt-XXXXXXXXXX)";
>> trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm
-fr
>> \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr
"${MYTMP}" && mkdir
>> "${MYTMP}" && tar --warning=no-timestamp -C
"${MYTMP}" -x &&
>> "${MYTMP}"/setup DIALOG/dialect=str:machine
>> DIALOG/customization=bool:True':
>> javax.naming.TimeLimitExceededException: SSH session hard timeout host
>> 'root(a)10.251.193.9'
>> 2014-05-27 11:10:49,369 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-31) [26c21342] Timeout during host
>> 10.251.193.9 install: javax.naming.TimeLimitExceededException: SSH
>> session hard timeout host 'root(a)10.251.193.9'
>> 2014-05-27 11:10:49,377 ERROR
>> [org.ovirt.engine.core.bll.InstallerMessages]
>> (org.ovirt.thread.pool-6-thread-31) [26c21342] Installation
>> 10.251.193.9: Processing stopped due to timeout
>> 2014-05-27 11:10:49,434 ERROR
>> [org.ovirt.engine.core.bll.InstallVdsCommand]
>> (org.ovirt.thread.pool-6-thread-31) [26c21342] Host installation
>> failed for host 322cbee8-16e6-11e2-9d38-6388c61dd004,
>> node02.blabla.gov.za.: javax.naming.TimeLimitExceededException: SSH
>> session hard timeout host 'root(a)10.251.193.9'
>> 2014-05-27 12:44:36,200 ERROR
>> [org.ovirt.engine.core.utils.servlet.ServletUtils]
>> (ajp--127.0.0.1-8702-1) Can't read file
>> "/var/lib/ovirt-engine/reports.xml" for request
>> "/ovirt-engine/services/reports-ui", will send a 404 error response.
>> 2014-05-27 13:16:21,679 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request failed with exit code 1
>> 2014-05-27 13:16:21,680 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request script errors:
>> Error opening Certificate ca.pem
>> 140249235597128:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('ca.pem','r')
>> 140249235597128:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> Error opening CA private key private/ca.pem
>> 140630029801288:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('private/ca.pem','r')
>> 140630029801288:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> 2014-05-27 13:16:21,684 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (VdsDeploy) Error during deploy dialog: java.lang.RuntimeException:
>> Certificate enrollment failed
>> 2014-05-27 13:16:21,689 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-21) [1a930dd7] Error during host
>> 10.251.193.9 install: java.lang.RuntimeException: Certificate
>> enrollment failed
>> 2014-05-27 13:16:21,694 ERROR
>> [org.ovirt.engine.core.bll.InstallerMessages]
>> (org.ovirt.thread.pool-6-thread-21) [1a930dd7] Installation
>> 10.251.193.9: Certificate enrollment failed
>> 2014-05-27 13:16:21,740 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-21) [1a930dd7] Error during host
>> 10.251.193.9 install, prefering first exception:
>> java.lang.RuntimeException: Certificate enrollment failed
>> 2014-05-27 13:16:21,744 ERROR
>> [org.ovirt.engine.core.bll.InstallVdsCommand]
>> (org.ovirt.thread.pool-6-thread-21) [1a930dd7] Host installation
>> failed for host 322cbee8-16e6-11e2-9d38-6388c61dd004,
>> node02.blabla.gov.za.: java.lang.RuntimeException: Certificate
>> enrollment failed
>> 2014-05-27 14:31:12,192 ERROR
>> [org.ovirt.engine.core.utils.servlet.ServletUtils]
>> (ajp--127.0.0.1-8702-2) Can't read file
>> "/var/lib/ovirt-engine/reports.xml" for request
>> "/ovirt-engine/services/reports-ui", will send a 404 error response.
>> 2014-05-27 14:32:58,669 ERROR
>> [org.ovirt.engine.core.utils.servlet.ServletUtils]
>> (ajp--127.0.0.1-8702-7) Can't read file
>> "/var/lib/ovirt-engine/reports.xml" for request
>> "/ovirt-engine/services/reports-ui", will send a 404 error response.
>> 2014-05-27 14:36:33,523 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request failed with exit code 1
>> 2014-05-27 14:36:33,524 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request script errors:
>> Error opening Certificate ca.pem
>> 140189576382280:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('ca.pem','r')
>> 140189576382280:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> Error opening CA private key private/ca.pem
>> 140632037402440:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('private/ca.pem','r')
>> 140632037402440:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> 2014-05-27 14:36:33,528 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (VdsDeploy) Error during deploy dialog: java.lang.RuntimeException:
>> Certificate enrollment failed
>> 2014-05-27 14:36:33,534 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-33) [5537b7c] Error during host
>> 10.251.193.9 install: java.lang.RuntimeException: Certificate
>> enrollment failed
>> 2014-05-27 14:36:33,545 ERROR
>> [org.ovirt.engine.core.bll.InstallerMessages]
>> (org.ovirt.thread.pool-6-thread-33) [5537b7c] Installation
>> 10.251.193.9: Certificate enrollment failed
>> 2014-05-27 14:36:33,572 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-33) [5537b7c] Error during host
>> 10.251.193.9 install, prefering first exception:
>> java.lang.RuntimeException: Certificate enrollment failed
>> 2014-05-27 14:36:33,576 ERROR
>> [org.ovirt.engine.core.bll.InstallVdsCommand]
>> (org.ovirt.thread.pool-6-thread-33) [5537b7c] Host installation failed
>> for host 322cbee8-16e6-11e2-9d38-6388c61dd004, node02.blabla.gov.za.:
>> java.lang.RuntimeException: Certificate enrollment failed
>> 2014-05-27 14:40:26,630 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request failed with exit code 1
>> 2014-05-27 14:40:26,631 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request script errors:
>> Error opening Certificate ca.pem
>> 139666318882632:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('ca.pem','r')
>> 139666318882632:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> Error opening CA private key private/ca.pem
>> 139701081003848:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('private/ca.pem','r')
>> 139701081003848:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> 2014-05-27 14:40:26,633 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (VdsDeploy) Error during deploy dialog: java.lang.RuntimeException:
>> Certificate enrollment failed
>> 2014-05-27 14:40:26,637 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-11) [7f68b0e2] Error during host
>> 10.251.193.9 install: java.lang.RuntimeException: Certificate
>> enrollment failed
>> 2014-05-27 14:40:26,639 ERROR
>> [org.ovirt.engine.core.bll.InstallerMessages]
>> (org.ovirt.thread.pool-6-thread-11) [7f68b0e2] Installation
>> 10.251.193.9: Certificate enrollment failed
>> 2014-05-27 14:40:26,709 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-11) [7f68b0e2] Error during host
>> 10.251.193.9 install, prefering first exception:
>> java.lang.RuntimeException: Certificate enrollment failed
>> 2014-05-27 14:40:26,711 ERROR
>> [org.ovirt.engine.core.bll.InstallVdsCommand]
>> (org.ovirt.thread.pool-6-thread-11) [7f68b0e2] Host installation
>> failed for host 322cbee8-16e6-11e2-9d38-6388c61dd004,
>> node02.blabla.gov.za.: java.lang.RuntimeException: Certificate
>> enrollment failed
>> 2014-05-27 15:04:24,260 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request failed with exit code 1
>> 2014-05-27 15:04:24,261 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request script errors:
>> Error opening Certificate ca.pem
>> 140668006123336:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('ca.pem','r')
>> 140668006123336:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> Error opening CA private key private/ca.pem
>> 140106430207816:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('private/ca.pem','r')
>> 140106430207816:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> 2014-05-27 15:04:24,265 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (VdsDeploy) Error during deploy dialog: java.lang.RuntimeException:
>> Certificate enrollment failed
>> 2014-05-27 15:04:24,270 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-34) [797b7d7a] Error during host
>> 10.251.193.9 install: java.lang.RuntimeException: Certificate
>> enrollment failed
>> 2014-05-27 15:04:24,277 ERROR
>> [org.ovirt.engine.core.bll.InstallerMessages]
>> (org.ovirt.thread.pool-6-thread-34) [797b7d7a] Installation
>> 10.251.193.9: Certificate enrollment failed
>> 2014-05-27 15:04:24,348 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-34) [797b7d7a] Error during host
>> 10.251.193.9 install, prefering first exception:
>> java.lang.RuntimeException: Certificate enrollment failed
>> 2014-05-27 15:04:24,352 ERROR
>> [org.ovirt.engine.core.bll.InstallVdsCommand]
>> (org.ovirt.thread.pool-6-thread-34) [797b7d7a] Host installation
>> failed for host 322cbee8-16e6-11e2-9d38-6388c61dd004,
>> node02.blabla.gov.za.: java.lang.RuntimeException: Certificate
>> enrollment failed
>> 2014-05-27 16:48:49,075 ERROR
>> [org.ovirt.engine.core.utils.servlet.ServletUtils]
>> (ajp--127.0.0.1-8702-4) Can't read file
>> "/var/lib/ovirt-engine/reports.xml" for request
>> "/ovirt-engine/services/reports-ui", will send a 404 error response.
>> 2014-05-27 17:03:10,817 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request failed with exit code 1
>> 2014-05-27 17:03:10,817 ERROR
>> [org.ovirt.engine.core.utils.hostinstall.OpenSslCAWrapper] (VdsDeploy)
>> Sign Certificate request script errors:
>> Error opening Certificate ca.pem
>> 140117678909256:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('ca.pem','r')
>> 140117678909256:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> Error opening CA private key private/ca.pem
>> 140049924028232:error:0200100D:system library:fopen:Permission
>> denied:bss_file.c:398:fopen('private/ca.pem','r')
>> 140049924028232:error:20074002:BIO routines:FILE_CTRL:system
>> lib:bss_file.c:400:
>> 2014-05-27 17:03:10,821 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (VdsDeploy) Error during deploy dialog: java.lang.RuntimeException:
>> Certificate enrollment failed
>> 2014-05-27 17:03:10,828 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-18) [2bb26823] Error during host
>> 10.251.193.9 install: java.lang.RuntimeException: Certificate
>> enrollment failed
>> 2014-05-27 17:03:10,839 ERROR
>> [org.ovirt.engine.core.bll.InstallerMessages]
>> (org.ovirt.thread.pool-6-thread-18) [2bb26823] Installation
>> 10.251.193.9: Certificate enrollment failed
>> 2014-05-27 17:03:10,891 ERROR [org.ovirt.engine.core.bll.VdsDeploy]
>> (org.ovirt.thread.pool-6-thread-18) [2bb26823] Error during host
>> 10.251.193.9 install, prefering first exception:
>> java.lang.RuntimeException: Certificate enrollment failed
>> 2014-05-27 17:03:10,895 ERROR
>> [org.ovirt.engine.core.bll.InstallVdsCommand]
>> (org.ovirt.thread.pool-6-thread-18) [2bb26823] Host installation
>> failed for host d2debdfe-76e7-40cf-a7fd-78a0f50f14d4,
>> node02.blabla.gov.za.: java.lang.RuntimeException: Certificate
>> enrollment failed
>>
>> I've looked around quite a bit and can't seem to find much.
>>
>> Please could someone assist.
>>
>> Thank you.
>>
>> Regards,
>>
>> Neil Wilson.
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>>
http://lists.ovirt.org/mailman/listinfo/users
>>