[ovirt-users] oVirt Hosted Engine Setup fails
Simone Tiraboschi
stirabos at redhat.com
Wed May 3 14:09:39 UTC 2017
On Wed, May 3, 2017 at 11:30 AM, Manuel Luis Aznar <
manuel.luis.aznar at gmail.com> wrote:
> Hello Simone and all others,
>
> I have attached to the mail the requested files. If you have any other
> inquiry just say it, The failed installation drive would be keep safe until
> solving this problem.
>
> Thanks for all in advance
> Manuel
>
The issue is here:
May 1 11:47:45 host1 journal: libvirt version: 2.0.0, package: 10.el7_3.5
(CentOS BuildSystem <http://bugs.centos.org>, 2017-03-03-02:09:45,
c1bm.rdu2.centos.org)
May 1 11:47:45 host1 journal: hostname: host1.bajada.es
May 1 11:47:45 host1 journal: Falló al conectar con el socket de monitor:
No existe el proceso
May 1 11:47:45 host1 journal: internal error: process exited while
connecting to monitor: /dev/random -device
virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x7 -msg
timestamp=on#012Could not access KVM kernel module: Permission
denied#012failed to initialize KVM: Permission denied
May 1 11:47:45 host1 journal: libvirt version: 2.0.0, package: 10.el7_3.5
(CentOS BuildSystem <http://bugs.centos.org>, 2017-03-03-02:09:45,
c1bm.rdu2.centos.org)
May 1 11:47:45 host1 journal: hostname: host1.bajada.es
May 1 11:47:45 host1 journal: Fin de archivo al leer datos: Error de
entrada/salida
May 1 11:47:45 host1 journal: Fin de archivo al leer datos: Error de
entrada/salida
could you please also check the permission on /dev/random and SELinux audit
logs?
>
> 2017-05-02 10:55 GMT+01:00 Simone Tiraboschi <stirabos at redhat.com>:
>
>> Sure, but first we need to understand what it's happening: in our CI
>> process everything is fine so I think it's something specific to your env.
>> Could you please share your:
>> /var/log/libvirt/qemu/HostedEngine.log
>> /var/log/messages
>>
>> thanks,
>> Simone
>>
>>
>> On Tue, May 2, 2017 at 11:38 AM, Manuel Luis Aznar <
>> manuel.luis.aznar at gmail.com> wrote:
>>
>>> Ok thankyou.
>>>
>>> Suppose that this problem probably would be solve in a future release.
>>>
>>> Thanks,
>>> Manuel
>>>
>>> 2017-05-02 10:35 GMT+01:00 Simone Tiraboschi <stirabos at redhat.com>:
>>>
>>>>
>>>>
>>>> On Tue, May 2, 2017 at 11:30 AM, Manuel Luis Aznar <
>>>> manuel.luis.aznar at gmail.com> wrote:
>>>>
>>>>> Hello there again,
>>>>>
>>>>> Yes as I say, I have done several clean installations and the VM
>>>>> engine sometimes starts without any problem. So Simone any recommendation
>>>>> to make the engine VM starts properly?¿
>>>>>
>>>>> While is installing the HA agent and HA broker are down, would I get
>>>>> good result by starting the services myself?¿
>>>>>
>>>>> Any help from Simone or somebody would be appreciated
>>>>> Thanks for all in advance
>>>>> Manuel Luis Aznar
>>>>>
>>>>
>>>> I suggest to check libvirt logs.
>>>>
>>>>
>>>>>
>>>>> 2017-05-02 7:54 GMT+01:00 Simone Tiraboschi <stirabos at redhat.com>:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, May 1, 2017 at 3:14 PM, Manuel Luis Aznar <
>>>>>> manuel.luis.aznar at gmail.com> wrote:
>>>>>>
>>>>>>> Hello there,
>>>>>>>
>>>>>>> I have been looking in the internet using google why my installation
>>>>>>> of ovirt-hosted-engine is failing.
>>>>>>>
>>>>>>> I have found this link:
>>>>>>>
>>>>>>> https://www.mail-archive.com/users@ovirt.org/msg40864.html
>>>>>>> (Hosted engine install failed; vdsm upset about broker)
>>>>>>>
>>>>>>> It seems to be the same error...
>>>>>>>
>>>>>>> So to knarra and Jamie Lawrence my question is:
>>>>>>>
>>>>>>> Did you manage to discover the problem?? In my instalation I am
>>>>>>> using nfs and not gluster...
>>>>>>>
>>>>>>> I have read the error and is the same error "BrokerConnectionError:
>>>>>>> ...". The ovirt-ha-agent and ovirt-ha-broker did not start when the
>>>>>>> installation is creating the engine VM...
>>>>>>>
>>>>>>
>>>>>> This is just a false positive: the HA agent and the HA broker are
>>>>>> still down so vdsm is complaining but at that point it's absolutely fine by
>>>>>> itself since the engine VM still doesn't exists.
>>>>>> We already have an open bug to reduce the impact of that message.
>>>>>>
>>>>>> The real issue is that for some reason the engine VM could not start
>>>>>> on your system.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> As I have said before any help would be very appreciated...no matter
>>>>>>> whom will give it
>>>>>>> Thanks for all in advance
>>>>>>> Manuel Luis Aznar
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2017-05-01 12:21 GMT+01:00 Manuel Luis Aznar <
>>>>>>> manuel.luis.aznar at gmail.com>:
>>>>>>>
>>>>>>>> Hello Simone and all the comunity,
>>>>>>>>
>>>>>>>> I have been doing the instalation of ovirt hosted engine again and
>>>>>>>> it fails, libvirtd and vdsmd services are failing. They are failing with
>>>>>>>> the following errors:
>>>>>>>>
>>>>>>>>
>>>>>>>> libvirt daemon
>>>>>>>>
>>>>>>>> libvirtd.service - Virtualization daemon
>>>>>>>> Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
>>>>>>>> enabled; vendor preset: enabled)
>>>>>>>> Drop-In: /etc/systemd/system/libvirtd.service.d
>>>>>>>> ââunlimited-core.conf
>>>>>>>> Active: active (running) since lun 2017-05-01 11:43:49 WEST;
>>>>>>>> 14min ago
>>>>>>>> Docs: man:libvirtd(8)
>>>>>>>> http://libvirt.org
>>>>>>>> Main PID: 21993 (libvirtd)
>>>>>>>> CGroup: /system.slice/libvirtd.service
>>>>>>>> ââ21993 /usr/sbin/libvirtd --listen
>>>>>>>>
>>>>>>>> may 01 11:43:49 host1.bajada.es systemd[1]: Starting
>>>>>>>> Virtualization daemon...
>>>>>>>> may 01 11:43:49 host1.bajada.es systemd[1]: Started Virtualization
>>>>>>>> daemon.
>>>>>>>> may 01 11:47:45 host1.bajada.es libvirtd[21993]: libvirt version:
>>>>>>>> 2.0.0, package: 10.el7_3.5 (CentOS BuildSystem <
>>>>>>>> http://bugs.centos.org>, 2017-03-03-02:09:45, c1bm.rdu2.centos.org)
>>>>>>>> may 01 11:47:45 host1.bajada.es libvirtd[21993]: hostname:
>>>>>>>> host1.bajada.es
>>>>>>>> may 01 11:47:45 host1.bajada.es libvirtd[21993]: Falló al
>>>>>>>> conectar con el socket de monitor: No existe el proceso
>>>>>>>> may 01 11:47:45 host1.bajada.es libvirtd[21993]: internal error:
>>>>>>>> process exited while connecting to monitor: /dev/random -device
>>>>>>>> virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x7 -msg
>>>>>>>> timestamp=on
>>>>>>>> Could not access
>>>>>>>> KVM kernel module: Permission denied
>>>>>>>> failed to
>>>>>>>> initialize KVM: Permission denied
>>>>>>>>
>>>>>>>> vdsm daemon
>>>>>>>>
>>>>>>>> vdsmd.service - Virtual Desktop Server Manager
>>>>>>>> Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled;
>>>>>>>> vendor preset: enabled)
>>>>>>>> Active: active (running) since lun 2017-05-01 11:43:51 WEST;
>>>>>>>> 15min ago
>>>>>>>> Main PID: 22119 (vdsm)
>>>>>>>> CGroup: /system.slice/vdsmd.service
>>>>>>>> ââ22119 /usr/bin/python2 /usr/share/vdsm/vdsm
>>>>>>>> ââ22612 /usr/libexec/ioprocess --read-pipe-fd 68
>>>>>>>> --write-pipe-fd 67 --max-threads 10 --max-queued-requests 10
>>>>>>>> ââ22630 /usr/libexec/ioprocess --read-pipe-fd 76
>>>>>>>> --write-pipe-fd 75 --max-threads 10 --max-queued-requests 10
>>>>>>>> ââ22887 /usr/libexec/ioprocess --read-pipe-fd 44
>>>>>>>> --write-pipe-fd 43 --max-threads 10 --max-queued-requests 10
>>>>>>>> ââ22893 /usr/libexec/ioprocess --read-pipe-fd 52
>>>>>>>> --write-pipe-fd 50 --max-threads 10 --max-queued-requests 10
>>>>>>>>
>>>>>>>> may 01 11:58:37 host1.bajada.es vdsm[22119]: vdsm
>>>>>>>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to
>>>>>>>> connect to broker, the number of errors has exceeded the limit (1)
>>>>>>>> may 01 11:58:37 host1.bajada.es vdsm[22119]: vdsm root ERROR
>>>>>>>> failed to retrieve Hosted Engine HA info
>>>>>>>> Traceback (most recent
>>>>>>>> call last):
>>>>>>>> File
>>>>>>>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in
>>>>>>>> _getHaInfo
>>>>>>>> stats =
>>>>>>>> instance.get_all_stats()
>>>>>>>> File
>>>>>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>>>>>>>> line 102, in get_all_stats
>>>>>>>> with
>>>>>>>> broker.connection(self._retries, self._wait):
>>>>>>>> File
>>>>>>>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>>>>>>>> return
>>>>>>>> self.gen.next()
>>>>>>>> File
>>>>>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>>>>>> line 99, in connection
>>>>>>>>
>>>>>>>> self.connect(retries, wait)
>>>>>>>> File
>>>>>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>>>>>> line 78, in connect
>>>>>>>> raise
>>>>>>>> BrokerConnectionError(error_msg)
>>>>>>>> BrokerConnectionError:
>>>>>>>> Failed to connect to broker, the number of errors has exceeded the limit (1)
>>>>>>>> may 01 11:58:52 host1.bajada.es vdsm[22119]: vdsm
>>>>>>>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to
>>>>>>>> connect to broker, the number of errors has exceeded the limit (1)
>>>>>>>>
>>>>>>>> I have been looking to the ovirt mailing list (and also internet
>>>>>>>> looking up in google) but I dont get what is the problem.
>>>>>>>>
>>>>>>>> I have attached to the mail the vdsm, ovirt-hosted-engine-setup and
>>>>>>>> the answer of the installation. In the vdsm.log I got the following error:
>>>>>>>>
>>>>>>>> libvirtError: internal error: process exited while connecting to
>>>>>>>> monitor: /dev/random -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x7
>>>>>>>> -msg timestamp=on
>>>>>>>> Could not access KVM kernel module: Permission denied
>>>>>>>> failed to initialize KVM: Permission denied
>>>>>>>>
>>>>>>>> I have been looking for that error but I dont get anything clear,
>>>>>>>> so I will greatly appreciate the help of somebody...
>>>>>>>>
>>>>>>>> The KVM modules are loaded because if I fired up this "lsmod | grep
>>>>>>>> kvm" I get the following:
>>>>>>>>
>>>>>>>> kvm_intel 170181 0
>>>>>>>> kvm 554609 1 kvm_intel
>>>>>>>> irqbypass 13503 1 kvm
>>>>>>>>
>>>>>>>> Also the group owner of /dev/kvm is:
>>>>>>>>
>>>>>>>> crw-rw-rw-+ 1 root kvm 10, 232 may 1 01:26 /dev/kvm
>>>>>>>>
>>>>>>>> Hope somebody could help
>>>>>>>> Thanks for all in advance
>>>>>>>> Manuel Luis Aznar
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2017-03-15 11:58 GMT+00:00 Manuel Luis Aznar <
>>>>>>>> manuel.luis.aznar at gmail.com>:
>>>>>>>>
>>>>>>>>> Hello there again,
>>>>>>>>>
>>>>>>>>> Yes that is correct. I interrupted the setup with Ctrl+C. That was
>>>>>>>>> because while I was answering, at the same time, I was looking at this
>>>>>>>>> file, and I saw this:
>>>>>>>>>
>>>>>>>>> FAILED: conflicting vdsm and libvirt-qemu tls configuration.
>>>>>>>>> vdsm.conf with ssl=True requires the following changes:
>>>>>>>>> libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1
>>>>>>>>> qemu.conf: spice_tls=1.
>>>>>>>>>
>>>>>>>>> So I decided to interrupt the installation, because of the error,
>>>>>>>>> edit the files (vdsm.conf and qemu.conf) and then I executed the
>>>>>>>>> installation again and it was successful. It seams that the change of the
>>>>>>>>> values in that files, in my case, produced a successful instalation.
>>>>>>>>>
>>>>>>>>> Sorry if my english is hard to understand, now you understand what
>>>>>>>>> I did.
>>>>>>>>>
>>>>>>>>> Any question, remark just go ahead
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Manuel
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2017-03-15 11:22 GMT+00:00 Simone Tiraboschi <stirabos at redhat.com>
>>>>>>>>> :
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 15, 2017 at 12:17 PM, Manuel Luis Aznar <
>>>>>>>>>> manuel.luis.aznar at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Simone,
>>>>>>>>>>>
>>>>>>>>>>> The quoted lines on your last message are on lines 1238-1245 on
>>>>>>>>>>> the attached log ovirt-hosted-engine-setup file.
>>>>>>>>>>>
>>>>>>>>>>> That file is the first hosted-engine setup. But this log file is
>>>>>>>>>>> not the result of a host-engine-setup file. I start with the installation
>>>>>>>>>>> and while I was answering the question I was looking to this log file for
>>>>>>>>>>> errors and when I found this errors I stopped and done this:
>>>>>>>>>>>
>>>>>>>>>>> FAILED: conflicting vdsm and libvirt-qemu tls configuration.
>>>>>>>>>>> vdsm.conf with ssl=True requires the following changes:
>>>>>>>>>>> libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1
>>>>>>>>>>> qemu.conf: spice_tls=1.
>>>>>>>>>>>
>>>>>>>>>>> Previously to this installation I have done several
>>>>>>>>>>> installations without revising this log file and always getting failed
>>>>>>>>>>> installations.
>>>>>>>>>>>
>>>>>>>>>>> Please its important to note that:
>>>>>>>>>>>
>>>>>>>>>>> The currently setup which we are talking about was using
>>>>>>>>>>> repo "ovirt-release41-pre.rpm". After correcting that two files
>>>>>>>>>>> I do the installation and in the end it was completed successfully.
>>>>>>>>>>>
>>>>>>>>>>> When I have some time I will try to install again using the
>>>>>>>>>>> realease repo "ovirt-release41.rpm"
>>>>>>>>>>>
>>>>>>>>>>> If you have any explanation, question or remark, please go
>>>>>>>>>>> ahead...
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> From the attached logs it seams that you voluntary interrupted
>>>>>>>>>> the setup from keyboard here:
>>>>>>>>>> 2017-03-07 11:23:17 DEBUG otopi.plugins.otopi.dialog.human
>>>>>>>>>> dialog.__logString:204 DIALOG:SEND iptables was detected on
>>>>>>>>>> your computer, do you wish setup to configure it? (Yes, No)[Yes]:
>>>>>>>>>> 2017-03-07 12:06:15 DEBUG otopi.context
>>>>>>>>>> context._executeMethod:142 method exception
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line
>>>>>>>>>> 132, in _executeMethod
>>>>>>>>>> method['method']()
>>>>>>>>>> File "/usr/share/ovirt-hosted-engin
>>>>>>>>>> e-setup/scripts/../plugins/gr-he-setup/network/firewall_manager.py",
>>>>>>>>>> line 157, in _customization
>>>>>>>>>> default=_('Yes'),
>>>>>>>>>> File "/usr/share/otopi/plugins/otopi/dialog/human.py", line
>>>>>>>>>> 177, in queryString
>>>>>>>>>> value = self._readline(hidden=hidden)
>>>>>>>>>> File "/usr/lib/python2.7/site-packages/otopi/dialog.py", line
>>>>>>>>>> 246, in _readline
>>>>>>>>>> value = self.__input.readline()
>>>>>>>>>> File "/usr/lib/python2.7/site-packages/otopi/main.py", line
>>>>>>>>>> 53, in _signal
>>>>>>>>>> raise RuntimeError("SIG%s" % signum)
>>>>>>>>>> RuntimeError: SIG2
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I will report back.
>>>>>>>>>>> Thanks for all in advance
>>>>>>>>>>> Manuel
>>>>>>>>>>>
>>>>>>>>>>> 2017-03-13 17:29 GMT+00:00 Simone Tiraboschi <
>>>>>>>>>>> stirabos at redhat.com>:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Mar 13, 2017 at 4:08 PM, Manuel Luis Aznar <
>>>>>>>>>>>> manuel.luis.aznar at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello to all there again,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was having some troubles while installing ovirt Hosted
>>>>>>>>>>>>> Engine, I took some look at the hosted engine setup logs while I was
>>>>>>>>>>>>> running the hosted-engine --deploy and I found the following in the ovirt
>>>>>>>>>>>>> hosted engine setup logs:
>>>>>>>>>>>>>
>>>>>>>>>>>>> lvm requires configuration
>>>>>>>>>>>>> libvirt is not configured for vdsm yet
>>>>>>>>>>>>> FAILED: conflicting vdsm and libvirt-qemu tls configuration.
>>>>>>>>>>>>> vdsm.conf with ssl=True requires the following changes:
>>>>>>>>>>>>> libvirtd.conf: listen_tcp=0, auth_tcp="sasl", listen_tls=1
>>>>>>>>>>>>> qemu.conf: spice_tls=1.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> hosted-engine setup is already running vdsm-tool configure
>>>>>>>>>>>> --force so it should configure libvirt and qemu for you, not sure why it
>>>>>>>>>>>> failed.
>>>>>>>>>>>> Could you please attach the logs from the failed
>>>>>>>>>>>> hosted-engine-setup run?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> When I saw this I stopped the setup and edited this two files
>>>>>>>>>>>>> (vdsm.conf and qemu.conf) set the stated configurations and run the deploy
>>>>>>>>>>>>> again. All was fine and I dont have any trouble the installation finished
>>>>>>>>>>>>> successfully. This was using ovirt-release41-pre.rpm repo.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I will be trying the same installation with
>>>>>>>>>>>>> ovirt-release41.rpm (when I have time) and I will report what happened.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for all
>>>>>>>>>>>>> Manuel Luis Aznar
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-03-06 1:31 GMT+00:00 Manuel Luis Aznar <
>>>>>>>>>>>>> manuel.luis.aznar at gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey there,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have been loking around, of course as of now the following
>>>>>>>>>>>>>> I am going to say I suppose is not anything new to you:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is the status of libvirtd:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> â libvirtd.service - Virtualization daemon
>>>>>>>>>>>>>> Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
>>>>>>>>>>>>>> enabled; vendor preset: enabled)
>>>>>>>>>>>>>> Drop-In: /etc/systemd/system/libvirtd.service.d
>>>>>>>>>>>>>> ââunlimited-core.conf
>>>>>>>>>>>>>> Active: active (running) since lun 2017-03-06 01:25:05
>>>>>>>>>>>>>> WET; 1min 37s ago
>>>>>>>>>>>>>> Docs: man:libvirtd(8)
>>>>>>>>>>>>>> http://libvirt.org
>>>>>>>>>>>>>> Main PID: 24350 (libvirtd)
>>>>>>>>>>>>>> CGroup: /system.slice/libvirtd.service
>>>>>>>>>>>>>> ââ24350 /usr/sbin/libvirtd --listen
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mar 06 01:25:05 host1.bajada.es systemd[1]: Starting
>>>>>>>>>>>>>> Virtualization daemon...
>>>>>>>>>>>>>> mar 06 01:25:05 host1.bajada.es systemd[1]: Started
>>>>>>>>>>>>>> Virtualization daemon.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> After looking at the state I fire up the VM engine with the
>>>>>>>>>>>>>> command "hosted-engine --vm-start" and I got the following:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> VM exists and is down, destroying it
>>>>>>>>>>>>>> Machine destroyed
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ed786811-0321-431e-be4b-2d03764c1b02
>>>>>>>>>>>>>> Status = WaitForLaunch
>>>>>>>>>>>>>> nicModel = rtl8139,pv
>>>>>>>>>>>>>> statusTime = 4374100040 <(437)%20410-0040>
>>>>>>>>>>>>>> emulatedMachine = pc
>>>>>>>>>>>>>> pid = 0
>>>>>>>>>>>>>> vmName = HostedEngine
>>>>>>>>>>>>>> devices = [{'index': '2', 'iface': 'ide',
>>>>>>>>>>>>>> 'specParams': {}, 'readonly': 'true', 'deviceId':
>>>>>>>>>>>>>> '506df4eb-e783-4451-a8a6-993fa4dbb381', 'address': {'bus':
>>>>>>>>>>>>>> '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'},
>>>>>>>>>>>>>> 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'},
>>>>>>>>>>>>>> {'index': '0', 'iface': 'virtio', 'format': 'raw', 'bootOrder': '1',
>>>>>>>>>>>>>> 'poolID': '00000000-0000-0000-0000-000000000000',
>>>>>>>>>>>>>> 'volumeID': '2bc39472-1a4b-4c7d-8ef9-1212182ad802',
>>>>>>>>>>>>>> 'imageID': '08288fcf-6b12-4bd1-84d3-259992e7aa6d',
>>>>>>>>>>>>>> 'specParams': {}, 'readonly': 'false', 'domainID':
>>>>>>>>>>>>>> 'f44afe8d-56f9-4e1e-beee-4daa548dbad8', 'optional': 'false',
>>>>>>>>>>>>>> 'deviceId': '08288fcf-6b12-4bd1-84d3-259992e7aa6d',
>>>>>>>>>>>>>> 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type':
>>>>>>>>>>>>>> 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'exclusive',
>>>>>>>>>>>>>> 'propagateErrors': 'off', 'type': 'disk'}, {'device': 'scsi', 'model':
>>>>>>>>>>>>>> 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr':
>>>>>>>>>>>>>> '00:16:3e:65:a6:4e', 'linkActive': 'true', 'network': 'ovirtmgmt',
>>>>>>>>>>>>>> 'specParams': {}, 'deviceId': '84b82c6c-bcca-4983-82d5-8d1e3ab3811a',
>>>>>>>>>>>>>> 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type':
>>>>>>>>>>>>>> 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'},
>>>>>>>>>>>>>> {'device': 'console', 'specParams': {}, 'type': 'console', 'deviceId':
>>>>>>>>>>>>>> '6236af73-8dab-4d14-b950-fb4ad01d4420', 'alias':
>>>>>>>>>>>>>> 'console0'}, {'device': 'vga', 'alias': 'video0', 'type': 'video'},
>>>>>>>>>>>>>> {'device': 'virtio', 'specParams': {'source': 'random'}, 'model': 'virtio',
>>>>>>>>>>>>>> 'type': 'rng'}]
>>>>>>>>>>>>>> guestDiskMapping = {}
>>>>>>>>>>>>>> vmType = kvm
>>>>>>>>>>>>>> clientIp =
>>>>>>>>>>>>>> displaySecurePort = -1
>>>>>>>>>>>>>> memSize = 4096
>>>>>>>>>>>>>> displayPort = -1
>>>>>>>>>>>>>> cpuType = Broadwell
>>>>>>>>>>>>>> spiceSecureChannels = smain,sdisplay,sinputs,scursor
>>>>>>>>>>>>>> ,splayback,srecord,ssmartcard,susbredir
>>>>>>>>>>>>>> smp = 2
>>>>>>>>>>>>>> displayIp = 0
>>>>>>>>>>>>>> display = vnc
>>>>>>>>>>>>>> maxVCpus = 6
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> After that if I look again at the status of libvirtd I obtain:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> â libvirtd.service - Virtualization daemon
>>>>>>>>>>>>>> Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
>>>>>>>>>>>>>> enabled; vendor preset: enabled)
>>>>>>>>>>>>>> Drop-In: /etc/systemd/system/libvirtd.service.d
>>>>>>>>>>>>>> ââunlimited-core.conf
>>>>>>>>>>>>>> Active: active (running) since lun 2017-03-06 01:25:05
>>>>>>>>>>>>>> WET; 5min ago
>>>>>>>>>>>>>> Docs: man:libvirtd(8)
>>>>>>>>>>>>>> http://libvirt.org
>>>>>>>>>>>>>> Main PID: 24350 (libvirtd)
>>>>>>>>>>>>>> CGroup: /system.slice/libvirtd.service
>>>>>>>>>>>>>> ââ24350 /usr/sbin/libvirtd --listen
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mar 06 01:25:05 host1.bajada.es systemd[1]: Starting
>>>>>>>>>>>>>> Virtualization daemon...
>>>>>>>>>>>>>> mar 06 01:25:05 host1.bajada.es systemd[1]: Started
>>>>>>>>>>>>>> Virtualization daemon.
>>>>>>>>>>>>>> mar 06 01:29:39 host1.bajada.es libvirtd[24350]: libvirt
>>>>>>>>>>>>>> version: 2.0.0, package: 10.el7_3.5 (CentOS BuildSystem <
>>>>>>>>>>>>>> http://bugs.centos.org>, 2017-03-03-02:09:45,
>>>>>>>>>>>>>> c1bm.rdu2.centos.org)
>>>>>>>>>>>>>> mar 06 01:29:39 host1.bajada.es libvirtd[24350]: hostname:
>>>>>>>>>>>>>> host1.bajada.es
>>>>>>>>>>>>>> mar 06 01:29:39 host1.bajada.es libvirtd[24350]: Falló al
>>>>>>>>>>>>>> conectar con el socket de monitor: No existe el proceso
>>>>>>>>>>>>>> mar 06 01:29:39 host1.bajada.es libvirtd[24350]: internal
>>>>>>>>>>>>>> error: process exited while connecting to monitor: Could not access KVM
>>>>>>>>>>>>>> kernel module: Permission denied
>>>>>>>>>>>>>> failed to
>>>>>>>>>>>>>> initialize KVM: Permission denied
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So the libvirtd is the problem, as i said this is nothing new
>>>>>>>>>>>>>> to you of course...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks again for any help
>>>>>>>>>>>>>> Manuel
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-03-05 18:51 GMT+00:00 Manuel Luis Aznar <
>>>>>>>>>>>>>> manuel.luis.aznar at gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hey there again,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you check if you have KVM modules loaded?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In order to check that I fire up the following command:
>>>>>>>>>>>>>>> "lsmod | grep kvm"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Result was:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> kvm_intel 170181 0
>>>>>>>>>>>>>>> kvm 554609 1 kvm_intel
>>>>>>>>>>>>>>> irqbypass 13503 1 kvm
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also check group owner for "/dev/kvm". I fire this: "ls -la
>>>>>>>>>>>>>>> /dev/kvm". The result was:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> crw-rw-rw-+ 1 root kvm 10, 232 mar 5 03:35 /dev/kvm
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also I check if there were some remain packages pending to
>>>>>>>>>>>>>>> install for kvm and qemu and I got:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> yum install \*kvm\*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The result is, that the system need to install the following:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Instalando:
>>>>>>>>>>>>>>> centos-release-qemu-ev noarch
>>>>>>>>>>>>>>> 1.0-1.el7 extras
>>>>>>>>>>>>>>> 11 k
>>>>>>>>>>>>>>> qemu-guest-agent x86_64
>>>>>>>>>>>>>>> 10:2.5.0-3.el7 base
>>>>>>>>>>>>>>> 133 k
>>>>>>>>>>>>>>> qemu-kvm-ev-debuginfo x86_64
>>>>>>>>>>>>>>> 10:2.6.0-28.el7_3.3.1 ovirt-4.0 12 M
>>>>>>>>>>>>>>> vdsm-hook-faqemu noarch
>>>>>>>>>>>>>>> 4.18.21-1.el7.centos ovirt-4.0
>>>>>>>>>>>>>>> 15 k
>>>>>>>>>>>>>>> vdsm-hook-qemucmdline noarch
>>>>>>>>>>>>>>> 4.18.21-1.el7.centos ovirt-4.0
>>>>>>>>>>>>>>> 11 k
>>>>>>>>>>>>>>> Instalando para las dependencias:
>>>>>>>>>>>>>>> centos-release-virt-common noarch
>>>>>>>>>>>>>>> 1-1.el7.centos extras
>>>>>>>>>>>>>>> 4.5 k
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Checking libvirtd service status I got:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> libvirtd.service - Virtualization daemon
>>>>>>>>>>>>>>> Loaded: loaded (/usr/lib/systemd/system/libvirtd.service;
>>>>>>>>>>>>>>> enabled; vendor preset: enabled)
>>>>>>>>>>>>>>> Drop-In: /etc/systemd/system/libvirtd.service.d
>>>>>>>>>>>>>>> ââunlimited-core.conf
>>>>>>>>>>>>>>> Active: active (running) since dom 2017-03-05 15:56:11
>>>>>>>>>>>>>>> WET; 2h 51min ago
>>>>>>>>>>>>>>> Docs: man:libvirtd(8)
>>>>>>>>>>>>>>> http://libvirt.org
>>>>>>>>>>>>>>> Main PID: 19415 (libvirtd)
>>>>>>>>>>>>>>> CGroup: /system.slice/libvirtd.service
>>>>>>>>>>>>>>> 19415 /usr/sbin/libvirtd --listen
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> mar 05 15:56:10 host1.bajada.es systemd[1]: Starting
>>>>>>>>>>>>>>> Virtualization daemon...
>>>>>>>>>>>>>>> mar 05 15:56:11 host1.bajada.es systemd[1]: Started
>>>>>>>>>>>>>>> Virtualization daemon.
>>>>>>>>>>>>>>> mar 05 16:00:04 host1.bajada.es libvirtd[19415]: libvirt
>>>>>>>>>>>>>>> version: 2.0.0, package: 10.el7_3.5 (CentOS BuildSystem <
>>>>>>>>>>>>>>> http://bugs.centos.org>, 2017-03-03-02:09:45,
>>>>>>>>>>>>>>> c1bm.rdu2.centos.org)
>>>>>>>>>>>>>>> mar 05 16:00:04 host1.bajada.es libvirtd[19415]: hostname:
>>>>>>>>>>>>>>> host1.bajada.es
>>>>>>>>>>>>>>> mar 05 16:00:04 host1.bajada.es libvirtd[19415]: Failed to
>>>>>>>>>>>>>>> connect to the socket monitor: process does not exits
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (Fallo al conectar con el socket de monitor: No existe el
>>>>>>>>>>>>>>> proceso)
>>>>>>>>>>>>>>> mar 05 16:00:04 host1.bajada.es libvirtd[19415]: internal
>>>>>>>>>>>>>>> error: process exited while connecting to monitor: Could not access KVM
>>>>>>>>>>>>>>> kernel module: Permission denied
>>>>>>>>>>>>>>> failed to
>>>>>>>>>>>>>>> initialize KVM: Permission denied
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for all in advance
>>>>>>>>>>>>>>> I will be waiting for you. Any help appreciated
>>>>>>>>>>>>>>> Manuel
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-03-05 17:33 GMT+00:00 Artyom Lukianov <
>>>>>>>>>>>>>>> alukiano at redhat.com>:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I found this one under the vdsm log:
>>>>>>>>>>>>>>>> libvirtError: internal error: process exited while
>>>>>>>>>>>>>>>> connecting to monitor: Could not access KVM kernel module: Permission denied
>>>>>>>>>>>>>>>> failed to initialize KVM: Permission denied
>>>>>>>>>>>>>>>> Thread-70::INFO::2017-03-05 16:00:04,325::vm::1330::virt.vm::(setDownStatus)
>>>>>>>>>>>>>>>> vmId=`ed786811-0321-431e-be4b-2d03764c1b02`::Changed state
>>>>>>>>>>>>>>>> to Down: internal error: process exited while connecting to monitor: Could
>>>>>>>>>>>>>>>> not access KVM kernel module: Permission denied
>>>>>>>>>>>>>>>> failed to initialize KVM: Permission denied (code=1)
>>>>>>>>>>>>>>>> Thread-70::INFO::2017-03-05 16:00:04,325::guestagent::430::virt.vm::(stop)
>>>>>>>>>>>>>>>> vmId=`ed786811-0321-431e-be4b-2d03764c1b02`::Stopping
>>>>>>>>>>>>>>>> connection
>>>>>>>>>>>>>>>> Thread-70::DEBUG::2017-03-05 16:00:04,325::vmchannels::238::vds::(unregister)
>>>>>>>>>>>>>>>> Delete fileno 52 from listener.
>>>>>>>>>>>>>>>> Thread-70::DEBUG::2017-03-05 16:00:04,325::vmchannels::66::vds::(_unregister_fd)
>>>>>>>>>>>>>>>> Failed to unregister FD from epoll (ENOENT): 52
>>>>>>>>>>>>>>>> Thread-70::DEBUG::2017-03-05 16:00:04,326::__init__::209::jsonrpc.Notification::(emit)
>>>>>>>>>>>>>>>> Sending event {"params": {"ed786811-0321-431e-be4b-2d03764c1b02":
>>>>>>>>>>>>>>>> {"status": "Down", "exitReason": 1, "exitMessage": "internal error: process
>>>>>>>>>>>>>>>> exited while connecting to monitor: Could not access KVM kernel module:
>>>>>>>>>>>>>>>> Permission denied\nfailed to initialize KVM: Permission denied",
>>>>>>>>>>>>>>>> "exitCode": 1}, "notify_time": 4339924730}, "jsonrpc": "2.0", "method":
>>>>>>>>>>>>>>>> "|virt|VM_status|ed786811-0321-431e-be4b-2d03764c1b02"}
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you check if you have KVM modules loaded? Also, check
>>>>>>>>>>>>>>>> group owner for "/dev/kvm".
>>>>>>>>>>>>>>>> Best Regards
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sat, Mar 4, 2017 at 4:24 PM, Manuel Luis Aznar <
>>>>>>>>>>>>>>>> manuel.luis.aznar at gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello there again,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The error on the first email was using the repo
>>>>>>>>>>>>>>>>> ovirt-release41.rpm (http://resources.ovirt.org/pu
>>>>>>>>>>>>>>>>> b/yum-repo/ovirt-release41.rpm), so as I were getting the
>>>>>>>>>>>>>>>>> same error again and again I am currently trying with
>>>>>>>>>>>>>>>>> ovirt-release41-snapshot.rpm (
>>>>>>>>>>>>>>>>> http://resources.ovirt.org/pu
>>>>>>>>>>>>>>>>> b/yum-repo/ovirt-release41-snapshot.rpm) and the result
>>>>>>>>>>>>>>>>> is nearly the same.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> After creating the VM on the installation I got the same
>>>>>>>>>>>>>>>>> error with the command "systemctl status vdsmd":
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mar 04 14:10:19 host1.bajada.es vdsm[20443]: vdsm root
>>>>>>>>>>>>>>>>> ERROR failed to retrieve Hosted Engine HA info
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/vdsm/host/api.py",
>>>>>>>>>>>>>>>>> line 231, in _getHaInfo
>>>>>>>>>>>>>>>>> stats = instance.get_all_stats()
>>>>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>>>>>>>>>> ges/ovirt_hosted_engine_ha/client/client.py", line 102,
>>>>>>>>>>>>>>>>> in get_all_stats
>>>>>>>>>>>>>>>>> with broker.connection(self._retries,
>>>>>>>>>>>>>>>>> self._wait):
>>>>>>>>>>>>>>>>> File "/usr/lib64/python2.7/contextlib.py",
>>>>>>>>>>>>>>>>> line 17, in __enter__
>>>>>>>>>>>>>>>>> return self.gen.next()
>>>>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>>>>>>>>>> ges/ovirt_hosted_engine_ha/lib/brokerlink.py", line 99,
>>>>>>>>>>>>>>>>> in connection
>>>>>>>>>>>>>>>>> self.connect(retries, wait)
>>>>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>>>>>>>>>> ges/ovirt_hosted_engine_ha/lib/brokerlink.py", line 78,
>>>>>>>>>>>>>>>>> in connect
>>>>>>>>>>>>>>>>> raise BrokerConnectionError(error_msg)
>>>>>>>>>>>>>>>>> BrokerConnectionError: Failed to connect to
>>>>>>>>>>>>>>>>> broker, the number of errors has exceeded the limit (1)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> mar 04 14:10:34 host1.bajada.es vdsm[20443]: vdsm
>>>>>>>>>>>>>>>>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR
>>>>>>>>>>>>>>>>> Failed to connect to broker, the number of errors has exceeded the limit (1)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have noticed that the ovirt-ha-agent and ovirt-ha-broker
>>>>>>>>>>>>>>>>> services was not running. I guess if this have something to do with the
>>>>>>>>>>>>>>>>> error in vsmd service log.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But in this case the ovirt-hosted-engine-installation
>>>>>>>>>>>>>>>>> prints the vnc connection and I can connect to the engine VM.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks for all in advance
>>>>>>>>>>>>>>>>> Any help would be appreciated
>>>>>>>>>>>>>>>>> Manuel Luis Aznar
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2017-03-03 21:48 GMT+00:00 Manuel Luis Aznar <
>>>>>>>>>>>>>>>>> manuel.luis.aznar at gmail.com>:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello there,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I am having some trouble when deploying an oVirt 4.1
>>>>>>>>>>>>>>>>>> hosted engine installation.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> When I m just to end the installation and the hosted
>>>>>>>>>>>>>>>>>> engine setup script is about to start the Vm engine (appliance) it fails
>>>>>>>>>>>>>>>>>> saying "The VM is not powring up".
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If I double check the service vdsmd i get this error all
>>>>>>>>>>>>>>>>>> the time:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> vdsm root ERROR failed to retrieve Hosted Engine HA info
>>>>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/vdsm/host/api.py",
>>>>>>>>>>>>>>>>>> line 231, in _getHaInfo
>>>>>>>>>>>>>>>>>> stats = instance.get_all_stats()
>>>>>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>>>>>>>>>>> ges/ovirt_hosted_engine_ha/client/client.py", line 102,
>>>>>>>>>>>>>>>>>> in get_all_stats
>>>>>>>>>>>>>>>>>> with broker.connection(self._retries,
>>>>>>>>>>>>>>>>>> self._wait):
>>>>>>>>>>>>>>>>>> File "/usr/lib64/python2.7/contextlib.py", line 17,
>>>>>>>>>>>>>>>>>> in __enter__
>>>>>>>>>>>>>>>>>> return self.gen.next()
>>>>>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>>>>>>>>>>> ges/ovirt_hosted_engine_ha/lib/brokerlink.py", line 99,
>>>>>>>>>>>>>>>>>> in connection
>>>>>>>>>>>>>>>>>> self.connect(retries, wait)
>>>>>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packa
>>>>>>>>>>>>>>>>>> ges/ovirt_hosted_engine_ha/lib/brokerlink.py", line 78,
>>>>>>>>>>>>>>>>>> in connect
>>>>>>>>>>>>>>>>>> raise BrokerConnectionError(error_msg)
>>>>>>>>>>>>>>>>>> BrokerConnectionError: Failed to connect to broker, the
>>>>>>>>>>>>>>>>>> number of errors has exceeded the limit (1)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Did anyone have experimented the same problem?¿? Any hint
>>>>>>>>>>>>>>>>>> on How to solved it?¿? I have tried several times with clean installations
>>>>>>>>>>>>>>>>>> and always getting the same...
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The host where I am trying to do the installation have
>>>>>>>>>>>>>>>>>> CentOS 7...
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks for all in advance
>>>>>>>>>>>>>>>>>> Will be waiting for any hint to see what I am doing
>>>>>>>>>>>>>>>>>> wrong...
>>>>>>>>>>>>>>>>>> Manuel Luis Aznar
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>>>>>> Users at ovirt.org
>>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>> Users at ovirt.org
>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170503/d239f5ca/attachment-0001.html>
More information about the Users
mailing list