Re: [ovirt-users] Ovirt 4.0 and EL 7.4

12 Oct 2017


      This is a multi-part message in MIME format.
--------------B9F58142C47930896CD55C26
Content-Type: text/plain;
	charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Fixed it by downgrading libvirt=2C I had the libvirt latest repo=20
=22http=3A//mirror=2Ecentos=2Eorg/centos-7/7=2E3=2E1611/virt/x86=5F64/libvi=
rt-latest/=22=20
configured on our Katello

Working packages=3A

    libvirt-daemon-driver-network-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-config-nwfilter-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-driver-nodedev-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-driver-nwfilter-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-driver-qemu-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-driver-lxc-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-driver-interface-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=

    libvirt-daemon-driver-secret-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-lock-sanlock-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-kvm-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-client-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-driver-storage-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-daemon-config-network-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64
    libvirt-python-2=2E0=2E0-2=2Eel7=2Ex86=5F64

On 10/10/2017 07=3A41 PM=2C Jorick Astrego wrote=3A
=3E
=3E Hi=2C
=3E
=3E I=27ve redeployed a node with 7=2E3 to fix this issue but got the same=
=20
=3E errors with ovirt 4=2E0=2E
=3E
=3E     MainThread=3A=3ADEBUG=3A=3A2017-10-10
=3E     18=3A30=3A30=2C945=3A=3Aupgrade=3A=3A90=3A=3Aupgrade=3A=3A=28apply=
=5Fupgrade=29 Running
=3E     upgrade upgrade-unified-persistence
=3E     MainThread=3A=3ADEBUG=3A=3A2017-10-10
=3E     18=3A30=3A30=2C951=3A=3Alibvirtconnection=3A=3A160=3A=3Aroot=3A=3A=
=28get=29 trying to
=3E     connect libvirt
=3E     MainThread=3A=3AERROR=3A=3A2017-10-10
=3E     18=3A30=3A41=2C125=3A=3Aupgrade=3A=3A94=3A=3Aupgrade=3A=3A=28apply=
=5Fupgrade=29 Failed to run
=3E     upgrade-unified-persistence
=3E     Traceback =28most recent call last=29=3A
=3E       File =22/usr/lib/python2=2E7/site-packages/vdsm/tool/upgrade=2Epy=
=22=2C
=3E     line 92=2C in apply=5Fupgrade
=3E         upgrade=2Erun=28ns=2C args=29
=3E       File
=3E     =22/usr/lib/python2=2E7/site-packages/vdsm/tool/unified=5Fpersisten=
ce=2Epy=22=2C
=3E     line 195=2C in run
=3E         run=28=29
=3E       File
=3E     =22/usr/lib/python2=2E7/site-packages/vdsm/tool/unified=5Fpersisten=
ce=2Epy=22=2C
=3E     line 46=2C in run
=3E         networks=2C bondings =3D =5FgetNetInfo=28=29
=3E       File
=3E     =22/usr/lib/python2=2E7/site-packages/vdsm/tool/unified=5Fpersisten=
ce=2Epy=22=2C
=3E     line 132=2C in =5FgetNetInfo
=3E         netinfo =3D NetInfo=28netswitch=2Enetinfo=28=29=29
=3E       File
=3E     =22/usr/lib/python2=2E7/site-packages/vdsm/network/netswitch=2Epy=
=22=2C line
=3E     298=2C in netinfo
=3E         =5Fnetinfo =3D netinfo=5Fget=28compatibility=3Dcompatibility=29=

=3E       File
=3E     =22/usr/lib/python2=2E7/site-packages/vdsm/network/netinfo/cache=2E=
py=22=2C
=3E     line 109=2C in get
=3E         return =5Fget=28vdsmnets=29
=3E       File
=3E     =22/usr/lib/python2=2E7/site-packages/vdsm/network/netinfo/cache=2E=
py=22=2C
=3E     line 70=2C in =5Fget
=3E         libvirt=5Fnets =3D libvirt=2Enetworks=28=29
=3E       File =22/usr/lib/python2=2E7/site-packages/vdsm/network/libvirt=
=2Epy=22=2C
=3E     line 113=2C in networks
=3E         conn =3D libvirtconnection=2Eget=28=29
=3E       File
=3E     =22/usr/lib/python2=2E7/site-packages/vdsm/libvirtconnection=2Epy=
=22=2C line
=3E     163=2C in get
=3E         password=29
=3E       File
=3E     =22/usr/lib/python2=2E7/site-packages/vdsm/libvirtconnection=2Epy=
=22=2C line
=3E     99=2C in open=5Fconnection
=3E         return utils=2Eretry=28libvirtOpen=2C timeout=3D10=2C sleep=3D0=
=2E2=29
=3E       File =22/usr/lib/python2=2E7/site-packages/vdsm/utils=2Epy=22=2C=
 line 547=2C
=3E     in retry
=3E         return func=28=29
=3E       File =22/usr/lib64/python2=2E7/site-packages/libvirt=2Epy=22=2C l=
ine 105=2C
=3E     in openAuth
=3E         if ret is None=3Araise libvirtError=28=27virConnectOpenAuth=28=
=29 failed=27=29
=3E     libvirtError=3A authentication failed=3A authentication failed
=3E
=3E
=3E     Oct 10 19=3A35=3A55 host1 sasldblistusers2=3A =5Fsasldb=5Fgetkeyhan=
dle has
=3E     failed
=3E
=3E     Oct 10 19=3A36=3A20 host1 libvirtd=3A 2017-10-10 17=3A36=3A20=2E002=
+0000=3A
=3E     13660=3A error =3A virNetSASLSessionListMechanisms=3A390 =3A intern=
al
=3E     error=3A cannot list SASL mechanisms -4 =28SASL=28-4=29=3A no mecha=
nism
=3E     available=3A Internal Error -4 in server=2Ec near line 1757=29
=3E     Oct 10 19=3A36=3A20 host1 libvirtd=3A 2017-10-10 17=3A36=3A20=2E002=
+0000=3A
=3E     13660=3A error =3A remoteDispatchAuthSaslInit=3A3411 =3A authentica=
tion
=3E     failed=3A authentication failed
=3E     Oct 10 19=3A36=3A20 host1 libvirtd=3A 2017-10-10 17=3A36=3A20=2E002=
+0000=3A
=3E     13650=3A error =3A virNetSocketReadWire=3A1808 =3A End of file whil=
e
=3E     reading data=3A Input/output error
=3E     Oct 10 19=3A36=3A20 host1 vdsm-tool=3A libvirt=3A XML-RPC error=20=
=3A
=3E     authentication failed=3A authentication failed
=3E     Oct 10 19=3A36=3A20 host1 systemd=3A vdsm-network=2Eservice=3A cont=
rol
=3E     process exited=2C code=3Dexited status=3D1
=3E     Oct 10 19=3A36=3A20 host1 systemd=3A Failed to start Virtual Deskto=
p
=3E     Server Manager network restoration=2E
=3E     Oct 10 19=3A36=3A20 host1 systemd=3A Dependency failed for Virtual=

=3E     Desktop Server Manager=2E
=3E     Oct 10 19=3A36=3A20 host1 systemd=3A Dependency failed for MOM inst=
ance
=3E     configured for VDSM purposes=2E
=3E     Oct 10 19=3A36=3A20 host1 systemd=3A Job mom-vdsm=2Eservice/start f=
ailed
=3E     with result =27dependency=27=2E
=3E     Oct 10 19=3A36=3A20 host1 systemd=3A Job vdsmd=2Eservice/start fail=
ed with
=3E     result =27dependency=27=2E
=3E     Oct 10 19=3A36=3A20 host1 systemd=3A Unit vdsm-network=2Eservice en=
tered
=3E     failed state=2E
=3E     Oct 10 19=3A36=3A20 host1 systemd=3A vdsm-network=2Eservice failed=
=2E
=3E
=3E
=3E cat /etc/redhat-release
=3E CentOS Linux release 7=2E3=2E1611 =28Core=29
=3E
=3E cat /etc/libvirt/passwd=2Edb
=3E cat=3A /etc/libvirt/passwd=2Edb=3A No such file or directory
=3E
=3E     vdsm-4=2E18=2E21-1=2Eel7=2Ecentos=2Ex86=5F64
=3E     vdsm-api-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch
=3E     vdsm-xmlrpc-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch
=3E     vdsm-hook-vmfex-dev-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch
=3E     vdsm-cli-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch
=3E     vdsm-python-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch
=3E     vdsm-yajsonrpc-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch
=3E     vdsm-infra-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch
=3E     vdsm-jsonrpc-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch
=3E
=3E     libvirt-daemon-driver-storage-scsi-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-storage-rbd-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-nodedev-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-client-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-python-2=2E0=2E0-2=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-network-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-storage-mpath-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-storage-iscsi-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-storage-logical-3=2E2=2E0-1=2Eel7=2Ex86=5F64=

=3E     libvirt-daemon-driver-storage-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-secret-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-interface-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-kvm-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-libs-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-storage-core-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-qemu-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-config-nwfilter-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-storage-disk-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-storage-gluster-3=2E2=2E0-1=2Eel7=2Ex86=5F64=

=3E     libvirt-lock-sanlock-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E     libvirt-daemon-driver-nwfilter-3=2E2=2E0-1=2Eel7=2Ex86=5F64
=3E
=3E     ovirt-imageio-common-0=2E4=2E0-1=2Eel7=2Enoarch
=3E     ovirt-release40-4=2E0=2E6-2=2Eel7=2Ecentos=2Enoarch
=3E     ovirt-vmconsole-1=2E0=2E4-1=2Eel7=2Ecentos=2Enoarch
=3E     ovirt-imageio-daemon-0=2E4=2E0-1=2Eel7=2Enoarch
=3E     ovirt-vmconsole-host-1=2E0=2E4-1=2Eel7=2Ecentos=2Enoarch
=3E
=3E Also tried with =22mech=5Flist=3A digest-md5=22
=3E
=3E     cat /etc/sasl2/libvirt=2Econf =7Cgrep mech=5Flist
=3E     =23mech=5Flist=3A gssapi
=3E     mech=5Flist=3A digest-md5
=3E     =23mech=5Flist=3A scram-sha-1
=3E     =23mech=5Flist=3A scram-sha-1 gssapi
=3E
=3E
=3E On 10/05/2017 01=3A26 PM=2C Pavel Gashev wrote=3A
=3E=3E Full /etc/sasl2/libvirt=2Econf=3A
=3E=3E mech=5Flist=3A digest-md5
=3E=3E sasldb=5Fpath=3A /etc/libvirt/passwd=2Edb
=3E=3E
=3E=3E Also note that VDSM has to be patched to work on 7=2E4 with no issue=
s=2E oVirt 3=2E6 and 4=2E1 have required fixes=2C but oVirt 4=2E0 doesn=
=E2=80=99t=2E
=3E=3E
=3E=3E On 04/10/2017=2C 18=3A44=2C=22users-bounces=40ovirt=2Eorg on behalf=
 of Alan Griffiths=22  =3Cusers-bounces=40ovirt=2Eorg on behalf of apgriffi=
ths79=40gmail=2Ecom=3E  wrote=3A
=3E=3E
=3E=3E      That didn=27t seem to make any difference=2E
=3E=3E     =20
=3E=3E      I can make it work by disabling authentication
=3E=3E     =20
=3E=3E      auth=5Funix=5Frw=3D=22none=22  in /etc/libvirt/libvirtd=2Econf=

=3E=3E     =20
=3E=3E      On 4 October 2017 at 15=3A05=2C VONDRA Alain=3CAVONDRA=40unicef=
=2Efr=3E  wrote=3A
=3E=3E      =3E Hi=2C
=3E=3E      =3E Did you modify your /etc/sasl2/libvirt=2Econf=2C because th=
e update has modify the way to authenticate from md5 to gssapi=2E
=3E=3E      =3E
=3E=3E      =3E If not just change this line =3A
=3E=3E      =3E mech=5Flist=3A gssapi
=3E=3E      =3E to
=3E=3E      =3E mech=5Flist=3A digest-md5
=3E=3E      =3E
=3E=3E      =3E And restart services
=3E=3E      =3E
=3E=3E      =3E As mentioned in the libvirt=2Econf file =3A
=3E=3E      =3E
=3E=3E      =3E =23 NB=2C previously DIGEST-MD5 was set as the default mech=
anism for
=3E=3E      =3E =23 libvirt=2E Per RFC 6331 this is vulnerable to many seri=
ous security
=3E=3E      =3E =23 flaws and should no longer be used=2E Thus GSSAPI is no=
w the default=2E
=3E=3E      =3E =23
=3E=3E      =3E =23 To use GSSAPI requires that a libvirtd service principa=
l is
=3E=3E      =3E =23 added to the Kerberos server for each host running libv=
irtd=2E
=3E=3E      =3E =23 This principal needs to be exported to the keytab file=
 listed below
=3E=3E      =3E
=3E=3E      =3E Alain
=3E=3E      =3E
=3E=3E      =3E
=3E=3E      =3E
=3E=3E      =3E Alain VONDRA
=3E=3E      =3E
=3E=3E      =3E Charg=C3=A9 d=27Exploitation et de S=C3=A9curit=C3=A9 des S=
yst=C3=A8mes d=27Information
=3E=3E      =3E Direction Administrative et Financi=C3=A8re
=3E=3E      =3E +33 1 44 39 77 76
=3E=3E      =3E
=3E=3E      =3E UNICEF France
=3E=3E      =3E 3 rue Duguay Trouin  75006
=3E=3E      =3E PARIS
=3E=3E      =3Ewww=2Eunicef=2Efr
=3E=3E      =3E -----Message d=27origine-----
=3E=3E      =3E De =3Ausers-bounces=40ovirt=2Eorg  =5Bmailto=3Ausers-bounce=
s=40ovirt=2Eorg=5D De la part de Alan Griffiths
=3E=3E      =3E Envoy=C3=A9 =3A mercredi 4 octobre 2017 15=3A50
=3E=3E      =3E =C3=80 =3A Ovirt Users=3Cusers=40ovirt=2Eorg=3E
=3E=3E      =3E Objet =3A =5Bovirt-users=5D Ovirt 4=2E0 and EL 7=2E4
=3E=3E      =3E
=3E=3E      =3E Hi=2C
=3E=3E      =3E
=3E=3E      =3E Is 4=2E0 supported/known to work on CentOS 7=2E4=3F
=3E=3E      =3E
=3E=3E      =3E I=27ve just tried to upgrade one of the hosts in my lab fro=
m 7=2E3 to 7=2E4 and now vdsm-network fails to start with
=3E=3E      =3E
=3E=3E      =3E vdsm-tool=3A libvirt=3A XML-RPC error =3A authentication fa=
iled=3A authentication failed
=3E=3E      =3E
=3E=3E      =3E To even get this far I had to exclude gluster packages as 7=
=2E4 introduces 3=2E8 but ovirt 4=2E0 repo is still on 3=2E7=2E
=3E=3E      =3E
=3E=3E      =3E So=2C more generally=2E If I=27m on ovirt 4=2E0=2C gluster=
 3=2E7 and EL 7=2E3=2E What is the best ordering for getting to ovirt 4=2E1=
 and EL 7=2E4=3F
=3E=3E      =3E
=3E=3E      =3E Thanks=2C
=3E=3E      =3E
=3E=3E      =3E Alan
=3E=3E      =3E =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F
=3E=3E      =3E Users mailing list
=3E=3E      =3EUsers=40ovirt=2Eorg
=3E=3E      =3Ehttp=3A//lists=2Eovirt=2Eorg/mailman/listinfo/users
=3E=3E      =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F
=3E=3E      Users mailing list
=3E=3E      Users=40ovirt=2Eorg
=3E=3E      http=3A//lists=2Eovirt=2Eorg/mailman/listinfo/users
=3E=3E     =20
=3E=3E
=3E=3E =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=

=3E=3E Users mailing list
=3E=3E Users=40ovirt=2Eorg
=3E=3E http=3A//lists=2Eovirt=2Eorg/mailman/listinfo/users
=3E


Met vriendelijke groet=2C With kind regards=2C

Jorick Astrego

Netbulae Virtualization Experts=20

----------------

=09Tel=3A 053 20 30 270 =09info=40netbulae=2Eeu =09Staalsteden 4-3A =09KvK=
 08198180
 =09Fax=3A 053 20 30 271 =09www=2Enetbulae=2Eeu =097547 TA Enschede =09BTW=
 NL821234584B01

----------------


--------------B9F58142C47930896CD55C26
Content-Type: text/html;
	charset="utf-8"
Content-Transfer-Encoding: quoted-printable

=3Chtml=3E
  =3Chead=3E
    =3Cmeta content=3D=22text/html=3B charset=3Dutf-8=22 http-equiv=3D=22Co=
ntent-Type=22=3E
  =3C/head=3E
  =3Cbody text=3D=22=23000000=22 bgcolor=3D=22=23FFFFFF=22=3E
    =3Cp=3EFixed it by downgrading libvirt=2C I had the libvirt latest repo=

      =3Ca class=3D=22moz-txt-link-rfc2396E=22 href=3D=22http=3A//mirror=2E=
centos=2Eorg/centos-7/7=2E3=2E1611/virt/x86=5F64/libvirt-latest/=22=3E=22ht=
tp=3A//mirror=2Ecentos=2Eorg/centos-7/7=2E3=2E1611/virt/x86=5F64/libvirt-la=
test/=22=3C/a=3E
      configured on our Katello=3C/p=3E
    Working packages=3A=3Cbr=3E
    =3Cbr=3E
    =3Cblockquote=3Elibvirt-daemon-driver-network-2=2E0=2E0-10=2Eel7=5F3=2E=
9=2Ex86=5F64=3Cbr=3E
      libvirt-daemon-config-nwfilter-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=
=3Cbr=3E
      libvirt-daemon-driver-nodedev-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=
=3Cbr=3E
      libvirt-daemon-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=3Cbr=3E
      libvirt-daemon-driver-nwfilter-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=
=3Cbr=3E
      libvirt-daemon-driver-qemu-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=3Cbr=
=3E
      libvirt-daemon-driver-lxc-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=3Cbr=
=3E
      libvirt-daemon-driver-interface-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=
=3Cbr=3E
      libvirt-daemon-driver-secret-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=3C=
br=3E
      libvirt-lock-sanlock-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=3Cbr=3E
      libvirt-daemon-kvm-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=3Cbr=3E
      libvirt-client-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=3Cbr=3E
      libvirt-daemon-driver-storage-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=
=3Cbr=3E
      libvirt-daemon-config-network-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=
=3Cbr=3E
      libvirt-2=2E0=2E0-10=2Eel7=5F3=2E9=2Ex86=5F64=3Cbr=3E
      libvirt-python-2=2E0=2E0-2=2Eel7=2Ex86=5F64=3Cbr=3E
      =3Cbr=3E
    =3C/blockquote=3E
    =3Cdiv class=3D=22moz-cite-prefix=22=3EOn 10/10/2017 07=3A41 PM=2C Jori=
ck Astrego
      wrote=3A=3Cbr=3E
    =3C/div=3E
    =3Cblockquote
      cite=3D=22mid=3Afafe11d8-458c-9ae6-8660-5ebb5aebda52=40netbulae=2Eeu=
=22
      type=3D=22cite=22=3E
      =3Cmeta content=3D=22text/html=3B charset=3Dutf-8=22 http-equiv=3D=22=
Content-Type=22=3E
      =3Cp=3EHi=2C=3C/p=3E
      =3Cp=3EI=27ve redeployed a node with 7=2E3 to fix this issue but got=
 the
        same errors with ovirt 4=2E0=2E =3Cbr=3E
      =3C/p=3E
      =3Cblockquote=3E
        =3Cp=3EMainThread=3A=3ADEBUG=3A=3A2017-10-10
          18=3A30=3A30=2C945=3A=3Aupgrade=3A=3A90=3A=3Aupgrade=3A=3A=28appl=
y=5Fupgrade=29 Running
          upgrade upgrade-unified-persistence=3Cbr=3E
          MainThread=3A=3ADEBUG=3A=3A2017-10-10
          18=3A30=3A30=2C951=3A=3Alibvirtconnection=3A=3A160=3A=3Aroot=3A=
=3A=28get=29 trying to
          connect libvirt=3Cbr=3E
          MainThread=3A=3AERROR=3A=3A2017-10-10
          18=3A30=3A41=2C125=3A=3Aupgrade=3A=3A94=3A=3Aupgrade=3A=3A=28appl=
y=5Fupgrade=29 Failed to
          run upgrade-unified-persistence=3Cbr=3E
          Traceback =28most recent call last=29=3A=3Cbr=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/tool/upgrade=2Epy=22=
=2C line
          92=2C in apply=5Fupgrade=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 upgrade=2Erun=28ns=2C args=29=3Cbr=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/tool/unified=5Fpersist=
ence=2Epy=22=2C
          line 195=2C in run=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 run=28=29=3Cbr=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/tool/unified=5Fpersist=
ence=2Epy=22=2C
          line 46=2C in run=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 networks=2C bondings =3D =5FgetNetInfo=28=29=
=3Cbr=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/tool/unified=5Fpersist=
ence=2Epy=22=2C
          line 132=2C in =5FgetNetInfo=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 netinfo =3D NetInfo=28netswitch=2Enetinfo=28=
=29=29=3Cbr=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/network/netswitch=2Epy=
=22=2C
          line 298=2C in netinfo=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 =5Fnetinfo =3D netinfo=5Fget=28compatibility=
=3Dcompatibility=29=3Cbr=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/network/netinfo/cache=
=2Epy=22=2C
          line 109=2C in get=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 return =5Fget=28vdsmnets=29=3Cbr=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/network/netinfo/cache=
=2Epy=22=2C
          line 70=2C in =5Fget=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 libvirt=5Fnets =3D libvirt=2Enetworks=28=29=3C=
br=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/network/libvirt=2Epy=
=22=2C
          line 113=2C in networks=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 conn =3D libvirtconnection=2Eget=28=29=3Cbr=3E=

          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/libvirtconnection=2Epy=
=22=2C
          line 163=2C in get=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 password=29=3Cbr=3E
          =C2=A0 File
          =22/usr/lib/python2=2E7/site-packages/vdsm/libvirtconnection=2Epy=
=22=2C
          line 99=2C in open=5Fconnection=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 return utils=2Eretry=28libvirtOpen=2C timeout=
=3D10=2C sleep=3D0=2E2=29=3Cbr=3E
          =C2=A0 File =22/usr/lib/python2=2E7/site-packages/vdsm/utils=2Epy=
=22=2C line
          547=2C in retry=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 return func=28=29=3Cbr=3E
          =C2=A0 File =22/usr/lib64/python2=2E7/site-packages/libvirt=2Epy=
=22=2C line
          105=2C in openAuth=3Cbr=3E
          =C2=A0=C2=A0=C2=A0 if ret is None=3Araise libvirtError=28=27virCo=
nnectOpenAuth=28=29
          failed=27=29=3Cbr=3E
          libvirtError=3A authentication failed=3A authentication failed=3C=
br=3E
        =3C/p=3E
        =3Cp=3E=3Cbr=3E
        =3C/p=3E
        =3Cp=3EOct 10 19=3A35=3A55 host1 sasldblistusers2=3A =5Fsasldb=5Fge=
tkeyhandle
          has failed=3Cbr=3E
        =3C/p=3E
        =3Cp=3EOct 10 19=3A36=3A20 host1 libvirtd=3A 2017-10-10 17=3A36=3A2=
0=2E002+0000=3A
          13660=3A error =3A virNetSASLSessionListMechanisms=3A390 =3A inte=
rnal
          error=3A cannot list SASL mechanisms -4 =28SASL=28-4=29=3A no mec=
hanism
          available=3A Internal Error -4 in server=2Ec near line 1757=29=3C=
br=3E
          Oct 10 19=3A36=3A20 host1 libvirtd=3A 2017-10-10 17=3A36=3A20=2E0=
02+0000=3A
          13660=3A error =3A remoteDispatchAuthSaslInit=3A3411 =3A
          authentication failed=3A authentication failed=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 libvirtd=3A 2017-10-10 17=3A36=3A20=2E0=
02+0000=3A
          13650=3A error =3A virNetSocketReadWire=3A1808 =3A End of file wh=
ile
          reading data=3A Input/output error=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 vdsm-tool=3A libvirt=3A XML-RPC error=
 =3A
          authentication failed=3A authentication failed=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 systemd=3A vdsm-network=2Eservice=3A co=
ntrol
          process exited=2C code=3Dexited status=3D1=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 systemd=3A Failed to start Virtual Desk=
top
          Server Manager network restoration=2E=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 systemd=3A Dependency failed for Virtua=
l
          Desktop Server Manager=2E=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 systemd=3A Dependency failed for MOM
          instance configured for VDSM purposes=2E=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 systemd=3A Job mom-vdsm=2Eservice/start=

          failed with result =27dependency=27=2E=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 systemd=3A Job vdsmd=2Eservice/start fa=
iled
          with result =27dependency=27=2E=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 systemd=3A Unit vdsm-network=2Eservice=

          entered failed state=2E=3Cbr=3E
          Oct 10 19=3A36=3A20 host1 systemd=3A vdsm-network=2Eservice faile=
d=2E=3Cbr=3E
        =3C/p=3E
      =3C/blockquote=3E
      =3Cp=3E=3Cbr=3E
      =3C/p=3E
      =3Cp=3Ecat /etc/redhat-release =3Cbr=3E
        CentOS Linux release 7=2E3=2E1611 =28Core=29 =3Cbr=3E
      =3C/p=3E
      =3Cp=3Ecat /etc/libvirt/passwd=2Edb=3Cbr=3E
        cat=3A /etc/libvirt/passwd=2Edb=3A No such file or directory=3Cbr=
=3E
      =3C/p=3E
      =3Cblockquote=3E
        =3Cp=3Evdsm-4=2E18=2E21-1=2Eel7=2Ecentos=2Ex86=5F64=3Cbr=3E
          vdsm-api-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E
          vdsm-xmlrpc-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E
          vdsm-hook-vmfex-dev-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E=

          vdsm-cli-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E
          vdsm-python-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E
          vdsm-yajsonrpc-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E
          vdsm-infra-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E
          vdsm-jsonrpc-4=2E18=2E21-1=2Eel7=2Ecentos=2Enoarch=3C/p=3E
        =3Cp=3Elibvirt-daemon-driver-storage-scsi-3=2E2=2E0-1=2Eel7=2Ex86=
=5F64=3Cbr=3E
          libvirt-daemon-driver-storage-rbd-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3C=
br=3E
          libvirt-daemon-driver-nodedev-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=
=3E
          libvirt-client-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=3E
          libvirt-python-2=2E0=2E0-2=2Eel7=2Ex86=5F64=3Cbr=3E
          libvirt-daemon-driver-network-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=
=3E
          libvirt-daemon-driver-storage-mpath-3=2E2=2E0-1=2Eel7=2Ex86=5F64=
=3Cbr=3E
          libvirt-daemon-driver-storage-iscsi-3=2E2=2E0-1=2Eel7=2Ex86=5F64=
=3Cbr=3E
          libvirt-daemon-driver-storage-logical-3=2E2=2E0-1=2Eel7=2Ex86=5F6=
4=3Cbr=3E
          libvirt-daemon-driver-storage-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=
=3E
          libvirt-daemon-driver-secret-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=3E=

          libvirt-daemon-driver-interface-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=
=3E
          libvirt-daemon-kvm-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=3E
          libvirt-libs-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=3E
          libvirt-daemon-driver-storage-core-3=2E2=2E0-1=2Eel7=2Ex86=5F64=
=3Cbr=3E
          libvirt-daemon-driver-qemu-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=3E=

          libvirt-daemon-config-nwfilter-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=
=3E
          libvirt-daemon-driver-storage-disk-3=2E2=2E0-1=2Eel7=2Ex86=5F64=
=3Cbr=3E
          libvirt-daemon-driver-storage-gluster-3=2E2=2E0-1=2Eel7=2Ex86=5F6=
4=3Cbr=3E
          libvirt-lock-sanlock-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=3E
          libvirt-daemon-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=3E
          libvirt-daemon-driver-nwfilter-3=2E2=2E0-1=2Eel7=2Ex86=5F64=3Cbr=
=3E
        =3C/p=3E
        =3Cp=3Eovirt-imageio-common-0=2E4=2E0-1=2Eel7=2Enoarch=3Cbr=3E
          ovirt-release40-4=2E0=2E6-2=2Eel7=2Ecentos=2Enoarch=3Cbr=3E
          ovirt-vmconsole-1=2E0=2E4-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E
          ovirt-imageio-daemon-0=2E4=2E0-1=2Eel7=2Enoarch=3Cbr=3E
          ovirt-vmconsole-host-1=2E0=2E4-1=2Eel7=2Ecentos=2Enoarch=3Cbr=3E=

          =3Cbr=3E
        =3C/p=3E
      =3C/blockquote=3E
      =3Cp=3EAlso tried with =22mech=5Flist=3A digest-md5=22=3Cbr=3E
      =3C/p=3E
      =3Cblockquote=3E
        =3Cp=3Ecat /etc/sasl2/libvirt=2Econf =7Cgrep mech=5Flist=3Cbr=3E
          =23mech=5Flist=3A gssapi=3Cbr=3E
          mech=5Flist=3A digest-md5=3Cbr=3E
          =23mech=5Flist=3A scram-sha-1=3Cbr=3E
          =23mech=5Flist=3A scram-sha-1 gssapi=3Cbr=3E
          =3Cbr=3E
        =3C/p=3E
      =3C/blockquote=3E
      =3Cp=3E=3Cbr=3E
      =3C/p=3E
      =3Cblockquote=3E =3C/blockquote=3E
      =3Cdiv class=3D=22moz-cite-prefix=22=3EOn 10/05/2017 01=3A26 PM=2C Pa=
vel Gashev
        wrote=3A=3Cbr=3E
      =3C/div=3E
      =3Cblockquote
        cite=3D=22mid=3A762B546B-3999-4786-893A-B57F3CE8A312=40acronis=2Eco=
m=22
        type=3D=22cite=22=3E
        =3Cpre wrap=3D=22=22=3EFull /etc/sasl2/libvirt=2Econf=3A
mech=5Flist=3A digest-md5
sasldb=5Fpath=3A /etc/libvirt/passwd=2Edb

Also note that VDSM has to be patched to work on 7=2E4 with no issues=2E oV=
irt 3=2E6 and 4=2E1 have required fixes=2C but oVirt 4=2E0 doesn=E2=80=99t=
=2E=20

On 04/10/2017=2C 18=3A44=2C =3Ca moz-do-not-send=3D=22true=22 class=3D=22mo=
z-txt-link-rfc2396E=22 href=3D=22mailto=3Ausers-bounces=40ovirt=2Eorgonbeha=
lfofAlanGriffiths=22=3E=22users-bounces=40ovirt=2Eorg on behalf of Alan Gri=
ffiths=22=3C/a=3E =3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-link=
-rfc2396E=22 href=3D=22mailto=3Ausers-bounces=40ovirt=2Eorgonbehalfofapgrif=
fiths79=40gmail=2Ecom=22=3E=26lt=3Busers-bounces=40ovirt=2Eorg on behalf of=
 apgriffiths79=40gmail=2Ecom=26gt=3B=3C/a=3E wrote=3A

    That didn=27t seem to make any difference=2E
   =20
    I can make it work by disabling authentication
   =20
    auth=5Funix=5Frw=3D=22none=22  in /etc/libvirt/libvirtd=2Econf
   =20
    On 4 October 2017 at 15=3A05=2C VONDRA Alain =3Ca moz-do-not-send=3D=22=
true=22 class=3D=22moz-txt-link-rfc2396E=22 href=3D=22mailto=3AAVONDRA=40un=
icef=2Efr=22=3E=26lt=3BAVONDRA=40unicef=2Efr=26gt=3B=3C/a=3E wrote=3A
    =26gt=3B Hi=2C
    =26gt=3B Did you modify your /etc/sasl2/libvirt=2Econf=2C because the u=
pdate has modify the way to authenticate from md5 to gssapi=2E
    =26gt=3B
    =26gt=3B If not just change this line =3A
    =26gt=3B mech=5Flist=3A gssapi
    =26gt=3B to
    =26gt=3B mech=5Flist=3A digest-md5
    =26gt=3B
    =26gt=3B And restart services
    =26gt=3B
    =26gt=3B As mentioned in the libvirt=2Econf file =3A
    =26gt=3B
    =26gt=3B =23 NB=2C previously DIGEST-MD5 was set as the default mechani=
sm for
    =26gt=3B =23 libvirt=2E Per RFC 6331 this is vulnerable to many serious=
 security
    =26gt=3B =23 flaws and should no longer be used=2E Thus GSSAPI is now t=
he default=2E
    =26gt=3B =23
    =26gt=3B =23 To use GSSAPI requires that a libvirtd service principal i=
s
    =26gt=3B =23 added to the Kerberos server for each host running libvirt=
d=2E
    =26gt=3B =23 This principal needs to be exported to the keytab file lis=
ted below
    =26gt=3B
    =26gt=3B Alain
    =26gt=3B
    =26gt=3B
    =26gt=3B
    =26gt=3B Alain VONDRA
    =26gt=3B
    =26gt=3B Charg=C3=A9 d=27Exploitation et de S=C3=A9curit=C3=A9 des Syst=
=C3=A8mes d=27Information
    =26gt=3B Direction Administrative et Financi=C3=A8re
    =26gt=3B +33 1 44 39 77 76
    =26gt=3B
    =26gt=3B UNICEF France
    =26gt=3B 3 rue Duguay Trouin  75006
    =26gt=3B PARIS
    =26gt=3B =3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-link-abbr=
eviated=22 href=3D=22http=3A//www=2Eunicef=2Efr=22=3Ewww=2Eunicef=2Efr=3C/a=
=3E
    =26gt=3B -----Message d=27origine-----
    =26gt=3B De =3A =3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-li=
nk-abbreviated=22 href=3D=22mailto=3Ausers-bounces=40ovirt=2Eorg=22=3Eusers=
-bounces=40ovirt=2Eorg=3C/a=3E =5B=3Ca moz-do-not-send=3D=22true=22 class=
=3D=22moz-txt-link-freetext=22 href=3D=22mailto=3Ausers-bounces=40ovirt=2Eo=
rg=22=3Emailto=3Ausers-bounces=40ovirt=2Eorg=3C/a=3E=5D De la part de Alan=
 Griffiths
    =26gt=3B Envoy=C3=A9 =3A mercredi 4 octobre 2017 15=3A50
    =26gt=3B =C3=80 =3A Ovirt Users =3Ca moz-do-not-send=3D=22true=22 class=
=3D=22moz-txt-link-rfc2396E=22 href=3D=22mailto=3Ausers=40ovirt=2Eorg=22=3E=
=26lt=3Busers=40ovirt=2Eorg=26gt=3B=3C/a=3E
    =26gt=3B Objet =3A =5Bovirt-users=5D Ovirt 4=2E0 and EL 7=2E4
    =26gt=3B
    =26gt=3B Hi=2C
    =26gt=3B
    =26gt=3B Is 4=2E0 supported/known to work on CentOS 7=2E4=3F
    =26gt=3B
    =26gt=3B I=27ve just tried to upgrade one of the hosts in my lab from 7=
=2E3 to 7=2E4 and now vdsm-network fails to start with
    =26gt=3B
    =26gt=3B vdsm-tool=3A libvirt=3A XML-RPC error =3A authentication faile=
d=3A authentication failed
    =26gt=3B
    =26gt=3B To even get this far I had to exclude gluster packages as 7=2E=
4 introduces 3=2E8 but ovirt 4=2E0 repo is still on 3=2E7=2E
    =26gt=3B
    =26gt=3B So=2C more generally=2E If I=27m on ovirt 4=2E0=2C gluster 3=
=2E7 and EL 7=2E3=2E What is the best ordering for getting to ovirt 4=2E1 a=
nd EL 7=2E4=3F
    =26gt=3B
    =26gt=3B Thanks=2C
    =26gt=3B
    =26gt=3B Alan
    =26gt=3B =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F
    =26gt=3B Users mailing list
    =26gt=3B =3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-link-abbr=
eviated=22 href=3D=22mailto=3AUsers=40ovirt=2Eorg=22=3EUsers=40ovirt=2Eorg=
=3C/a=3E
    =26gt=3B =3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-link-free=
text=22 href=3D=22http=3A//lists=2Eovirt=2Eorg/mailman/listinfo/users=22=3E=
http=3A//lists=2Eovirt=2Eorg/mailman/listinfo/users=3C/a=3E
    =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
    Users mailing list
    =3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-link-abbreviated=
=22 href=3D=22mailto=3AUsers=40ovirt=2Eorg=22=3EUsers=40ovirt=2Eorg=3C/a=3E=

    =3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-link-freetext=22 h=
ref=3D=22http=3A//lists=2Eovirt=2Eorg/mailman/listinfo/users=22=3Ehttp=3A//=
lists=2Eovirt=2Eorg/mailman/listinfo/users=3C/a=3E
   =20

=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Users mailing list
=3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-link-abbreviated=22 hr=
ef=3D=22mailto=3AUsers=40ovirt=2Eorg=22=3EUsers=40ovirt=2Eorg=3C/a=3E
=3Ca moz-do-not-send=3D=22true=22 class=3D=22moz-txt-link-freetext=22 href=
=3D=22http=3A//lists=2Eovirt=2Eorg/mailman/listinfo/users=22=3Ehttp=3A//lis=
ts=2Eovirt=2Eorg/mailman/listinfo/users=3C/a=3E
=3C/pre=3E
      =3C/blockquote=3E
      =3Cbr=3E
    =3C/blockquote=3E
    =3Cbr=3E
 =20=
=3CBR /=3E
=3CBR /=3E
=3Cb style=3D=22color=3A=23604c78=22=3E=3C/b=3E=3Cbr=3E=3Cbr=3E=3Cspan styl=
e=3D=22color=3A=23604c78=3B=22=3E=3Cfont color=3D=22000000=22=3E=3Cspan sty=
le=3D=22mso-fareast-language=3Aen-gb=3B=22 lang=3D=22NL=22=3EMet vriendelij=
ke groet=2C With kind regards=2C=3Cbr=3E=3Cbr=3EJorick Astrego=3Cbr=3E=3C/s=
pan=3E=3C/font=3E=3C/span=3E=3Cb style=3D=22color=3A=23604c78=22=3E=3Cbr=3E=
Netbulae Virtualization Experts =3C/b=3E=3Cbr=3E=3Chr style=3D=22border=3An=
one=3Bborder-top=3A1px solid =23ccc=3B=22=3E=3Ctable style=3D=22width=3A 52=
2px=22=3E=3Ctbody=3E=3Ctr=3E=3Ctd style=3D=22width=3A 130px=3Bfont-size=3A=
 10px=22=3ETel=3A  053 20 30 270=3C/td=3E    =3Ctd style=3D=22width=3A 130p=
x=3Bfont-size=3A 10px=22=3Einfo=40netbulae=2Eeu=3C/td=3E    =3Ctd style=3D=
=22width=3A 130px=3Bfont-size=3A 10px=22=3EStaalsteden 4-3A=3C/td=3E   =20=
=3Ctd style=3D=22width=3A 130px=3Bfont-size=3A 10px=22=3EKvK 08198180=3C/td=
=3E=3C/tr=3E=3Ctr=3E    =3Ctd style=3D=22width=3A 130px=3Bfont-size=3A 10px=
=22=3EFax=3A 053 20 30 271=3C/td=3E    =3Ctd style=3D=22width=3A 130px=3Bfo=
nt-size=3A 10px=22=3Ewww=2Enetbulae=2Eeu=3C/td=3E    =3Ctd style=3D=22width=
=3A 130px=3Bfont-size=3A 10px=22=3E7547 TA Enschede=3C/td=3E    =3Ctd style=
=3D=22width=3A 130px=3Bfont-size=3A 10px=22=3EBTW NL821234584B01=3C/td=3E=
=3C/tr=3E=3C/tbody=3E=3C/table=3E=3Cbr=3E=3Chr style=3D=22border=3Anone=3Bb=
order-top=3A1px solid =23ccc=3B=22=3E=3CBR /=3E
=3C/body=3E
=3C/html=3E


--------------B9F58142C47930896CD55C26--