I've found that this issue is related to: https://bugzilla.redhat.com/show_bug.cgi?id=1648190

But i've no idea how fix it.... Il 24/06/2019 18:19, Stefano Danzi ha scritto: > I've just upgraded my test environment from ovirt 4.2 to 4.3.4.

> System has only one host (Centos 7.6.1810) and run a self hosted engine. > > After upgrade I'm not able to run vdsmd (and so hosted engine....) > > Above the error in log: > > journalctl -xe > > -- L'unità libvirtd.service ha iniziato la fase di avvio. > giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 > 16:09:17.006+0000: 8176: info : libvirt version: 4.5.0, package: > 10.el7_6.12 (CentOS BuildSystem <http://bugs.centos.org>;, > 2019-06-20-15:01:15, x86-01.bsys. > giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 > 16:09:17.006+0000: 8176: info : hostname: ovirt01.hawai.lan > giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 > 16:09:17.006+0000: 8176: error : virNetTLSContextLoadCertFromFile:513 > : Unable to import server certificate /etc/pki/vdsm/certs/vdsmcert.pem

> giu 24 18:09:17 ovirt01.hawai.lan systemd[1]: libvirtd.service: main > process exited, code=exited, status=6/NOTCONFIGURED > giu 24 18:09:17 ovirt01.hawai.lan systemd[1]: Failed to start > Virtualization daemon. > -- Subject: L'unità libvirtd.service è fallita > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/MAP4TPH7UAG... _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ADFJRSR4BDG...

Il 25/06/2019 08:27, Yedidyah Bar David ha scritto: > On Mon, Jun 24, 2019 at 7:56 PM Stefano Danzi <s.danzi(a)hawai.it&gt; wrote: >> I've found that this issue is related to: >> >> https://bugzilla.redhat.com/show_bug.cgi?id=1648190 > Are you sure? > > That bug is about an old cert, generated by an old version, likely > before we fixed bug 1210486 (even though it's not mentioned in above > bug). Yes! Malformed "Not Before" date/time in certs >> But i've no idea how fix it.... >> >> Il 24/06/2019 18:19, Stefano Danzi ha scritto: >>> I've just upgraded my test environment from ovirt 4.2 to 4.3.4. > Was it installed as 4.2, or upgraded? From which first version? I don't remember the first installed version. Maybe 4.0... I always upgraded the original installation. >>> System has only one host (Centos 7.6.1810) and run a self hosted engine. >>> >>> After upgrade I'm not able to run vdsmd (and so hosted engine....) >>> >>> Above the error in log: >>> >>> journalctl -xe >>> >>> -- L'unità libvirtd.service ha iniziato la fase di avvio. >>> giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 >>> 16:09:17.006+0000: 8176: info : libvirt version: 4.5.0, package: >>> 10.el7_6.12 (CentOS BuildSystem <http://bugs.centos.org>;, >>> 2019-06-20-15:01:15, x86-01.bsys. >>> giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 >>> 16:09:17.006+0000: 8176: info : hostname: ovirt01.hawai.lan >>> giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 >>> 16:09:17.006+0000: 8176: error : virNetTLSContextLoadCertFromFile:513 >>> : Unable to import server certificate /etc/pki/vdsm/certs/vdsmcert.pem > Did you check this file? Does it exist? > > ls -l /etc/pki/vdsm/certs/vdsmcert.pem > > Can vdsm user read it? > > su - vdsm -s /bin/bash -c 'cat /etc/pki/vdsm/certs/vdsmcert.pem > /dev/null' > > Please check/share output of: > > openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -text > > Thanks and best regards, vdsm can read vdsmcert. The problem is "Not Before" date: [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -text' Certificate: Data: Version: 3 (0x2) Serial Number: 4102 (0x1006) Signature Algorithm: sha1WithRSAEncryption Issuer: C=US, O=hawai.lan, CN=ovirtbk-sheng.hawai.lan.63272 Validity Not Before: Feb 4 08:36:07 2015 Not After : Feb 4 08:36:07 2020 GMT [CUT] [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in /etc/pki/vdsm/certs/cacert.pem -text' Certificate: Data: Version: 3 (0x2) Serial Number: 4096 (0x1000) Signature Algorithm: sha1WithRSAEncryption Issuer: C=US, O=hawai.lan, CN=ovirtbk-sheng.hawai.lan.63272 Validity Not Before: Feb 4 00:06:25 2015 Not After : Feb 2 00:06:25 2025 GMT

Il 25/06/2019 10:08, Yedidyah Bar David ha scritto: > On Tue, Jun 25, 2019 at 10:26 AM Stefano Danzi <s.danzi(a)hawai.it&gt; wrote: >> >> >> Il 25/06/2019 08:27, Yedidyah Bar David ha scritto: >>> On Mon, Jun 24, 2019 at 7:56 PM Stefano Danzi <s.danzi(a)hawai.it&gt; wrote: >>>> I've found that this issue is related to: >>>> >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1648190 >>> Are you sure? >>> >>> That bug is about an old cert, generated by an old version, likely >>> before we fixed bug 1210486 (even though it's not mentioned in above >>> bug). >> Yes! Malformed "Not Before" date/time in certs >> >>>> But i've no idea how fix it.... >>>> >>>> Il 24/06/2019 18:19, Stefano Danzi ha scritto: >>>>> I've just upgraded my test environment from ovirt 4.2 to 4.3.4. >>> Was it installed as 4.2, or upgraded? From which first version? >> I don't remember the first installed version. Maybe 4.0... I always >> upgraded the original installation. >> >>>>> System has only one host (Centos 7.6.1810) and run a self hosted engine. >>>>> >>>>> After upgrade I'm not able to run vdsmd (and so hosted engine....) >>>>> >>>>> Above the error in log: >>>>> >>>>> journalctl -xe >>>>> >>>>> -- L'unità libvirtd.service ha iniziato la fase di avvio. >>>>> giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 >>>>> 16:09:17.006+0000: 8176: info : libvirt version: 4.5.0, package: >>>>> 10.el7_6.12 (CentOS BuildSystem <http://bugs.centos.org>;, >>>>> 2019-06-20-15:01:15, x86-01.bsys. >>>>> giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 >>>>> 16:09:17.006+0000: 8176: info : hostname: ovirt01.hawai.lan >>>>> giu 24 18:09:17 ovirt01.hawai.lan libvirtd[8176]: 2019-06-24 >>>>> 16:09:17.006+0000: 8176: error : virNetTLSContextLoadCertFromFile:513 >>>>> : Unable to import server certificate /etc/pki/vdsm/certs/vdsmcert.pem >>> Did you check this file? Does it exist? >>> >>> ls -l /etc/pki/vdsm/certs/vdsmcert.pem >>> >>> Can vdsm user read it? >>> >>> su - vdsm -s /bin/bash -c 'cat /etc/pki/vdsm/certs/vdsmcert.pem > /dev/null' >>> >>> Please check/share output of: >>> >>> openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem -text >>> >>> Thanks and best regards, >> vdsm can read vdsmcert. The problem is "Not Before" date: >> >> [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in >> /etc/pki/vdsm/certs/vdsmcert.pem -text' >> Certificate: >> Data: >> Version: 3 (0x2) >> Serial Number: 4102 (0x1006) >> Signature Algorithm: sha1WithRSAEncryption >> Issuer: C=US, O=hawai.lan, CN=ovirtbk-sheng.hawai.lan.63272 >> Validity >> Not Before: Feb 4 08:36:07 2015 >> Not After : Feb 4 08:36:07 2020 GMT >> [CUT] >> >> >> [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in >> /etc/pki/vdsm/certs/cacert.pem -text' >> Certificate: >> Data: >> Version: 3 (0x2) >> Serial Number: 4096 (0x1000) >> Signature Algorithm: sha1WithRSAEncryption >> Issuer: C=US, O=hawai.lan, CN=ovirtbk-sheng.hawai.lan.63272 >> Validity >> Not Before: Feb 4 00:06:25 2015 >> Not After : Feb 2 00:06:25 2025 GMT >> > OK :-( > > So it will be rather difficult to fix. > > You should have been prompted by engine-setup long ago to renew PKI, > weren't you? And when you did, didn't you have to reinstall (or Re- > Enroll Certificates, in later versions) all hosts? I don't remember to ever seen a question about this during engine-setup, but it could be. In /etc/pki/vdsm/certs/ I can see an old cert and ca with subjet: [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in /etc/pki/vdsm/certs/cacert.pem.20150205093608 -text' Certificate: Data: Version: 3 (0x2) Serial Number: 1423056193 (0x54d21d41) Signature Algorithm: sha256WithRSAEncryption Issuer: CN=VDSM Certificate Authority Validity Not Before: Feb 4 13:23:13 2015 GMT Not After : Feb 4 13:23:13 2016 GMT Subject: CN=VDSM Certificate Authority Subject Public Key Info: [CUT] [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem.20150205093609 -text' Certificate: Data: Version: 3 (0x2) Serial Number: 1423056193 (0x54d21d41) Signature Algorithm: sha256WithRSAEncryption Issuer: CN=VDSM Certificate Authority Validity Not Before: Feb 4 13:23:13 2015 GMT Not After : Feb 4 13:23:13 2016 GMT Subject: CN=ovirt01.hawai.lan, O=VDSM Certificate Subject Public Key Info: Public Key Algorithm: rsaEncryption I think that was certs made during first hosted engine installation. Could it work if I manually create certs like this? Just to start libvirtd, vdsm and hosted-engine.

>> I don't remember to ever seen a question about this during >> engine-setup, >> but it could be. >> In /etc/pki/vdsm/certs/ I can see an old cert and ca with subjet: >> >> [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in >> /etc/pki/vdsm/certs/cacert.pem.20150205093608 -text' >> Certificate: >>       Data: >>           Version: 3 (0x2) >>           Serial Number: 1423056193 (0x54d21d41) >>       Signature Algorithm: sha256WithRSAEncryption >>           Issuer: CN=VDSM Certificate Authority >>           Validity >>               Not Before: Feb  4 13:23:13 2015 GMT >>               Not After : Feb  4 13:23:13 2016 GMT >>           Subject: CN=VDSM Certificate Authority >>           Subject Public Key Info: >> >> [CUT] >> >> [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in >> /etc/pki/vdsm/certs/vdsmcert.pem.20150205093609 -text' >> Certificate: >>       Data: >>           Version: 3 (0x2) >>           Serial Number: 1423056193 (0x54d21d41) >>       Signature Algorithm: sha256WithRSAEncryption >>           Issuer: CN=VDSM Certificate Authority >>           Validity >>               Not Before: Feb  4 13:23:13 2015 GMT >>               Not After : Feb  4 13:23:13 2016 GMT >>           Subject: CN=ovirt01.hawai.lan, O=VDSM Certificate >>           Subject Public Key Info: >>               Public Key Algorithm: rsaEncryption >> >> >> I think that was certs made during first hosted engine installation. >> Could it work if I manually create certs like this? >> Just to start libvirtd, vdsm and hosted-engine. > I think it's worth a try. Just create a self-signed CA, a keypair > signed by it, and place them correctly, should work. > > The engine won't be able to talk with the host, but you can then more > easily reinstall/re-enroll-certs. > > Good luck, This workaround works! I have hosted engine running! So I have to find how reinstall/re-enroll-certs on host. From engine UI host status is "NonResponsive" and I can't do nothing.... _______________________________________________

On Tue, Jun 25, 2019 at 8:37 PM Stefano Danzi <s.danzi(a)hawai.it&gt; wrote: > Il 25/06/2019 14:26, Stefano Danzi ha scritto: >>>> I don't remember to ever seen a question about this during >>>> engine-setup, >>>> but it could be. >>>> In /etc/pki/vdsm/certs/ I can see an old cert and ca with subjet: >>>> >>>> [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in >>>> /etc/pki/vdsm/certs/cacert.pem.20150205093608 -text' >>>> Certificate: >>>> Data: >>>> Version: 3 (0x2) >>>> Serial Number: 1423056193 (0x54d21d41) >>>> Signature Algorithm: sha256WithRSAEncryption >>>> Issuer: CN=VDSM Certificate Authority >>>> Validity >>>> Not Before: Feb 4 13:23:13 2015 GMT >>>> Not After : Feb 4 13:23:13 2016 GMT >>>> Subject: CN=VDSM Certificate Authority >>>> Subject Public Key Info: >>>> >>>> [CUT] >>>> >>>> [root@ovirt01 ~]# su - vdsm -s /bin/bash -c 'openssl x509 -in >>>> /etc/pki/vdsm/certs/vdsmcert.pem.20150205093609 -text' >>>> Certificate: >>>> Data: >>>> Version: 3 (0x2) >>>> Serial Number: 1423056193 (0x54d21d41) >>>> Signature Algorithm: sha256WithRSAEncryption >>>> Issuer: CN=VDSM Certificate Authority >>>> Validity >>>> Not Before: Feb 4 13:23:13 2015 GMT >>>> Not After : Feb 4 13:23:13 2016 GMT >>>> Subject: CN=ovirt01.hawai.lan, O=VDSM Certificate >>>> Subject Public Key Info: >>>> Public Key Algorithm: rsaEncryption >>>> >>>> >>>> I think that was certs made during first hosted engine installation. >>>> Could it work if I manually create certs like this? >>>> Just to start libvirtd, vdsm and hosted-engine. >>> I think it's worth a try. Just create a self-signed CA, a keypair >>> signed by it, and place them correctly, should work. >>> >>> The engine won't be able to talk with the host, but you can then more >>> easily reinstall/re-enroll-certs. >>> >>> Good luck, >> This workaround works! >> I have hosted engine running! >> >> So I have to find how reinstall/re-enroll-certs on host. From engine >> UI host status is "NonResponsive" and I can't do nothing.... >> _______________________________________________ > Status: > > now Host status is "Unassiged". Engine can't reach host for "General > SSLEngine problem" and It's ok because certs are "home made". > I can't switch host to maintenance because it's not operational. > I can't enroll certificate because is not in maintenance status. You can try to remove it. I think we do not support "force-remove" despite being asked about this occasionally, because generally-speaking, this is very unsafe. If you insist, you can try using the sql function DeleteVds to delete it from the database. > hou I can enroll host cert manually? You can try following what I wrote in "2. Try to manually fix" before. Create a CSR on the host (with whatever private key you want), copy it to engine, pki-enroll-request, copy the cert to host. Good luck and best regards,