- Users - oVirt List Archives

RHEL 8 Template Seal failed
by Vinícius Ferrão 21 May '19

21 May '19

Hello, I’m trying to seal a RHEL8 template but the operation is failing. Here’s the relevant information from engine.log: 2019-05-17 01:30:31,153-03 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHostJobsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [91e1acd6-efc5-411b-8c76-970def4ebbbe] FINISH, GetHostJobsVDSCommand, return: {b80c0bbd-25b8-4007-9b91-376cb0a18e30=HostJobInfo:{id='b80c0bbd-25b8-4007-9b91-376cb0a18e30', type='virt', description='seal_vm', status='failed', progress='null', error='VDSError:{code='GeneralException', message='General Exception: ('Command [\'/usr/bin/virt-sysprep\', \'-a\', u\'/rhev/data-center/mnt/192.168.10.6:_mnt_pool0_ovirt_vm/d19456e4-0051-456e-b33c-57348a78c2e0/images/1ecdfbfc-1c22-452f-9a53-2159701549c8/f9de3eae-f475-451b-b587-f6a1405036e8\'] failed with rc=1 out=\'[ 0.0] Examining the guest ...\\nvirt-sysprep: warning: mount_options: mount exited with status 32: mount: \\nwrong fs type, bad option, bad superblock on /dev/mapper/rhel_rhel8-root,\\n missing codepage or helper program, or other error\\n\\n In some cases useful info is found in syslog - try\\n dmesg | tail or so. (ignored)\\nvirt-sysprep: warning: mount_options: mount: /boot: mount point is not a \\ndirectory (ignored)\\nvirt-sysprep: warning: mount_options: mount: /boot/efi: mount point is not \\na directory (ignored)\\n[ 17.9] Performing "abrt-data" ...\\n\' err="virt-sysprep: error: libguestfs error: glob_expand: glob_expand_stub: you \\nmust call \'mount\' first to mount the root filesystem\\n\\nIf reporting bugs, run virt-sysprep with debugging enabled and include the \\ncomplete output:\\n\\n virt-sysprep -v -x [...]\\n"',)'}'}}, log id: 1bbb34bf I’m not shure what’s wrong or missing. The VM image is using UEFI with Secure Boot, so standard UEFI partition is in place. Ive found something on bugzilla but does not seem to be related: https://bugzilla.redhat.com/show_bug.cgi?id=1671895 Thanks,

2 1

VM pools broken in 4.3
by Rik Theys 21 May '19

21 May '19

Hi, It seems VM pools are completely broken since our upgrade to 4.3. Is anybody else also experiencing this issue? Only a single instance from a pool can be used. Afterwards the pool becomes unusable due to a lock not being released. Once ovirt-engine is restarted, another (single) VM from a pool can be used. I've added my findings to bug 1462236, but I'm no longer sure the issue is the same as the one initially reported. When the first VM of a pool is started: 2019-05-14 13:26:46,058+02 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='d8a99676-d520-425e-9974-1b1efe6da8a5'}), log id: 2fb4f7f5 2019-05-14 13:26:46,058+02 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 2fb4f7f5 2019-05-14 13:26:46,208+02 INFO [org.ovirt.engine.core.bll.VmPoolHandler] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Lock Acquired to object 'EngineLock:{exclusiveLocks='[d8a99676-d520-425e-9974-1b1efe6da8a5=VM]', sharedLocks=''}' -> it has acquired a lock (lock1) 2019-05-14 13:26:46,247+02 INFO [org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Lock Acquired to object 'EngineLock:{exclusiveLocks='[a5bed59c-d2fe-4fe4-bff7-52efe089ebd6=USER_VM_POOL]', sharedLocks=''}' -> it has acquired another lock (lock2) 2019-05-14 13:26:46,352+02 INFO [org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Running command: AttachUserToVmFromPoolAndRunCommand internal: false. Entities affected : ID: 4c622213-e5f4-4032-8639-643174b698cc Type: VmPoolAction group VM_POOL_BASIC_OPERATIONS with role type USER 2019-05-14 13:26:46,393+02 INFO [org.ovirt.engine.core.bll.AddPermissionCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Running command: AddPermissionCommand internal: true. Entities affected : ID: d8a99676-d520-425e-9974-1b1efe6da8a5 Type: VMAction group MANIPULATE_PERMISSIONS with role type USER 2019-05-14 13:26:46,433+02 INFO [org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Succeeded giving user 'a5bed59c-d2fe-4fe4-bff7-52efe089ebd6' permission to Vm 'd8a99676-d520-425e-9974-1b1efe6da8a5' 2019-05-14 13:26:46,608+02 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='d8a99676-d520-425e-9974-1b1efe6da8a5'}), log id: 67acc561 2019-05-14 13:26:46,608+02 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 67acc561 2019-05-14 13:26:46,719+02 INFO [org.ovirt.engine.core.bll.RunVmCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Running command:RunVmCommand internal: true. Entities affected : ID: d8a99676-d520-425e-9974-1b1efe6da8a5 Type: VMAction group RUN_VM with role type USER 2019-05-14 13:26:46,791+02 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] START, UpdateVmDynamicDataVDSCommand( UpdateVmDynamicDataVDSCommandParameters:{hostId='null', vmId='d8a99676-d520-425e-9974-1b1efe6da8a5', vmDynamic='org.ovirt.engine.core.common.businessentities.VmDynamic@6db8c94d'}), log id: 2c110e4 2019-05-14 13:26:46,795+02 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] FINISH, UpdateVmDynamicDataVDSCommand, return: , log id: 2c110e4 2019-05-14 13:26:46,804+02 INFO [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] START,CreateVDSCommand( CreateVDSCommandParameters:{hostId='eec7ec2b-cae1-4bb9-b933-4dff47a70bdb', vmId='d8a99676-d520-425e-9974-1b1efe6da8a5', vm='VM [stud-c7-1]'}), log id: 71d599f2 2019-05-14 13:26:46,809+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] START, CreateBrokerVDSCommand(HostName = studvirt1, CreateVDSCommandParameters:{hostId='eec7ec2b-cae1-4bb9-b933-4dff47a70bdb', vmId='d8a99676-d520-425e-9974-1b1efe6da8a5', vm='VM [stud-c7-1]'}), log id: 3aa6b5ff 2019-05-14 13:26:46,836+02 INFO [org.ovirt.engine.core.vdsbroker.builder.vminfo.VmInfoBuildUtils] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Kernel FIPS - Guid: eec7ec2b-cae1-4bb9-b933-4dff47a70bdb fips: false 2019-05-14 13:26:46,903+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] VM <?xml version="1.0" encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> [domain xml stripped] 2019-05-14 13:26:46,928+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] FINISH, CreateBrokerVDSCommand, return: , log id: 3aa6b5ff 2019-05-14 13:26:46,932+02 INFO [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] FINISH, CreateVDSCommand, return: WaitForLaunch, log id: 71d599f2 2019-05-14 13:26:46,932+02 INFO [org.ovirt.engine.core.bll.RunVmCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Lock freed to object 'EngineLock:{exclusiveLocks='[a5bed59c-d2fe-4fe4-bff7-52efe089ebd6=USER_VM_POOL]', sharedLocks=''}' -> it has released lock2 2019-05-14 13:26:47,004+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] EVENT_ID: VDS_INITIATED_RUN_VM(506), Trying to restart VM stud-c7-1 on Host studvirt1 2019-05-14 13:26:47,094+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] EVENT_ID: USER_ATTACH_USER_TO_VM_FROM_POOL(316), Attaching User u0045469 to VM stud-c7-1 in VM Pool stud-c7-? was initiated by u0045469(a)esat.kuleuven.be-authz <mailto:u0045469@esat.kuleuven.be-authz>. 2019-05-14 13:26:47,098+02 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Trying to release exclusive lock which does not exist, lock key: 'a5bed59c-d2fe-4fe4-bff7-52efe089ebd6USER_VM_POOL' -> it's trying to release the same lock2 as above and failing 2019-05-14 13:26:47,098+02 INFO [org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand] (default task-6) [e3c5745c-e593-4aed-ba67-b173808140e8] Lock freed to object 'EngineLock:{exclusiveLocks='[a5bed59c-d2fe-4fe4-bff7-52efe089ebd6=USER_VM_POOL]', sharedLocks=''}' -> now it's indicating that it released/freed the lock (lock2) 2019-05-14 13:26:48,518+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [e3c5745c-e593-4aed-ba67-b173808140e8] Command 'AttachUserToVmFromPoolAndRun' id: '0148c91d-b053-4dc9-960c-f10bf0d3f343' child commands '[0470802d-09fa-4579-b95b-3d9603feff7b, 47dbfc58-3bae-4229-96eb-d1fc08911237]' executions were completed, status 'SUCCEEDED' 2019-05-14 13:26:49,584+02 INFO [org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [e3c5745c-e593-4aed-ba67-b173808140e8] Ending command 'org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand' successfully. 2019-05-14 13:26:49,650+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [e3c5745c-e593-4aed-ba67-b173808140e8] EVENT_ID: USER_ATTACH_USER_TO_VM_FROM_POOL_FINISHED_SUCCESS(318), User u0045469 successfully attached to VM stud-c7-1 in VM Pool stud-c7-?. 2019-05-14 13:26:50,584+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-2) [] EVENT_ID: VM_CONSOLE_DISCONNECTED(168), User <UNKNOWN> got disconnected from VM stud-c7-1. 2019-05-14 13:26:50,585+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-2) [] VM 'd8a99676-d520-425e-9974-1b1efe6da8a5'(stud-c7-1) moved from 'WaitForLaunch' --> 'PoweringUp' The first lock (on the pool itself) is never released. It seems this lock should also be released? Maybe it's this lock that should be released the second time (instead of the second lock twice)? When we start to launch another instance of the pool it fails: 2019-05-14 13:49:32,656+02 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-11) [55cc0796-4f53-49cd-8739-3b7e7dd2d95b] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='d8a99676-d520-425e-9974-1b1efe6da8a5'}), log id: 7db2f4fc 2019-05-14 13:49:32,656+02 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-11) [55cc0796-4f53-49cd-8739-3b7e7dd2d95b] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 7db2f4fc 2019-05-14 13:49:32,688+02 INFO [org.ovirt.engine.core.bll.VmPoolHandler] (default task-11) [55cc0796-4f53-49cd-8739-3b7e7dd2d95b] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[d8a99676-d520-425e-9974-1b1efe6da8a5=VM]', sharedLocks=''}' 2019-05-14 13:49:32,700+02 INFO [org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand] (default task-11) [55cc0796-4f53-49cd-8739-3b7e7dd2d95b] Lock Acquired to object 'EngineLock:{exclusiveLocks='[a5bed59c-d2fe-4fe4-bff7-52efe089ebd6=USER_VM_POOL]', sharedLocks=''}' 2019-05-14 13:49:32,700+02 WARN [org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand] (default task-11) [55cc0796-4f53-49cd-8739-3b7e7dd2d95b] Validation of action 'AttachUserToVmFromPoolAndRun' failed for user u0045469(a)esat.kuleuven.be-authz <mailto:u0045469@esat.kuleuven.be-authz>. Reasons: VAR__ACTION__ALLOCATE_AND_RUN,VAR__TYPE__VM_FROM_VM_POOL,ACTION_TYPE_FAILED_NO_AVAILABLE_POOL_VMS 2019-05-14 13:49:32,700+02 INFO [org.ovirt.engine.core.bll.AttachUserToVmFromPoolAndRunCommand] (default task-11) [55cc0796-4f53-49cd-8739-3b7e7dd2d95b] Lock freed to object 'EngineLock:{exclusiveLocks='[a5bed59c-d2fe-4fe4-bff7-52efe089ebd6=USER_VM_POOL]', sharedLocks=''}' 2019-05-14 13:49:32,706+02 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-11) [] Operation Failed: [Cannot allocate and run VM from VM-Pool. There are no available VMs in the VM-Pool.] Regards, Rik -- Rik Theys System Engineer KU Leuven - Dept. Elektrotechniek (ESAT) Kasteelpark Arenberg 10 bus 2440 - B-3001 Leuven-Heverlee +32(0)16/32.11.07 ---------------------------------------------------------------- <<Any errors in spelling, tact or fact are transmission errors>>

3 7

Re: [ovirt-announce] Re: [ANN] oVirt 4.3.4 First Release Candidate is now available
by Strahil 21 May '19

21 May '19

On this one https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/h… We should have the following options: performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.stat-prefetch=off performance.low-prio-threads=32 network.remote-dio=enable cluster.eager-lock=enable cluster.quorum-type=auto cluster.server-quorum-type=server cluster.data-self-heal-algorithm=full cluster.locking-scheme=granular cluster.shd-max-threads=8 cluster.shd-wait-qlength=10000 features.shard=on user.cifs=off By the way the 'virt' gluster group disables 'cluster.choose-local' and I think it wasn't like that. Any reasons behind that , as I use it to speedup my reads, as local storage is faster than the network? Best Regards, Strahil NikolovOn May 19, 2019 09:36, Strahil <hunter86_bg(a)yahoo.com> wrote: > > OK, > > Can we summarize it: > 1. VDO must 'emulate512=true' > 2. 'network.remote-dio' should be off ? > > As per this: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3/htm… > > We should have these: > > quick-read=off > read-ahead=off > io-cache=off > stat-prefetch=off > eager-lock=enable > remote-dio=on > quorum-type=auto > server-quorum-type=server > > I'm a little bit confused here. > > Best Regards, > Strahil Nikolov > > On May 19, 2019 07:44, Sahina Bose <sabose(a)redhat.com> wrote: >> >> >> >> On Sun, 19 May 2019 at 12:21 AM, Nir Soffer <nsoffer(a)redhat.com> wrote: >>> >>> On Fri, May 17, 2019 at 7:54 AM Gobinda Das <godas(a)redhat.com> wrote: >>>> >>>> From RHHI side default we are setting below volume options: >>>> >>>> { group: 'virt', >>>> storage.owner-uid: '36', >>>> storage.owner-gid: '36', >>>> network.ping-timeout: '30', >>>> performance.strict-o-direct: 'on', >>>> network.remote-dio: 'off' >>> >>> >>> According to the user reports, this configuration is not compatible with oVirt. >>> >>> Was this tested? >> >> >> Yes, this is set by default in all test configuration. We’re checking on the bug, but the error is likely when the underlying device does not support 512b writes. >> With network.remote-dio off gluster will ensure o-direct writes >>> >>> >>>> } >>>> >>>> >>>> On Fri, May 17, 2019 at 2:31 AM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: >>>>> >>>>> Ok, setting 'gluster volume set data_fast4 network.remote-dio on' allowed me to create the storage domain without any issues. >>>>> I set it on all 4 new gluster volumes and the storage domains were successfully created. >>>>> >>>>> I have created bug for that: >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1711060 >>>>> >>>>> If someone else already opened - please ping me to mark this one as duplicate. >>>>> >>>>> Best Regards, >>>>> Strahil Nikolov >>>>> >>>>> >>>>> В четвъртък, 16 май 2019 г., 22:27:01 ч. Гринуич+3, Darrell Budic <budic(a)onholyground.com> написа: >>>>> >>>>> >>>>> On May 16, 2019, at 1:41 PM, Nir Soffer <nsoffer(a)redhat.com> wrote: >>>>> >>>>>> >>>>>> On Thu, May 16, 2019 at 8:38 PM Darrell Budic <budic(a)onholyground.com> wrote: >>>>>>> >>>>>>> I tried adding a new storage domain on my hyper converged test cluster running Ovirt 4.3.3.7 and gluster 6.1. I was able to create the new gluster volume fine, but it’s not able to add the gluster storage domain (as either a managed gluster volume or directly entering values). The created gluster volume mounts and looks fine from the CLI. Errors in VDSM log: >>>>>>> >>>>>> ... >>>>>>> >>>>>>> 2019-05-16 10:25:09,584-0500 ERROR (jsonrpc/5) [storage.fileSD] Underlying file system doesn't supportdirect IO (fileSD:110) >>>>>>> 2019-05-16 10:25:09,584-0500 INFO (jsonrpc/5) [vdsm.api] FINISH createStorageDomain error=Storage Domain target is unsupported: () from=::ffff:10.100.90.5,44732, flow_id=31d993dd, task_id=ecea28f3-60d4-476d-9ba8-b753b7c9940d (api:52) >>>>>> >>>>>> >>>>>> The direct I/O check has failed. >>>>>> >>>>>> >>>>>> So something is wrong in the files system. >>>>>> >>>>>> To confirm, you can try to do: >>>>>> >>>>>> dd if=/dev/zero of=/path/to/mountoint/test bs=4096 count=1 oflag=direct >>>>>> >>>>>> This will probably fail with: >>>>>> dd: failed to open '/path/to/mountoint/test': Invalid argument >>>>>> >>>>>> If it succeeds, but oVirt fail to connect to this domain, file a bug and we will investigate. >>>>>> >>>>>> Nir >>>>> >>>>> >>>>> Yep, it fails as expected. Just to check, it is working on pre-existing volumes, so I poked around at gluster settings for the new volume. It has network.remote-dio=off set on the new volume, but enabled on old volumes. After enabling it, I’m able to run the dd test: >>>>> >>>>> [root@boneyard mnt]# gluster vol set test network.remote-dio enable >>>>> volume set: success >>>>> [root@boneyard mnt]# dd if=/dev/zero of=testfile bs=4096 count=1 oflag=direct >>>>> 1+0 records in >>>>> 1+0 records out >>>>> 4096 bytes (4.1 kB) copied, 0.0018285 s, 2.2 MB/s >>>>> >>>>> I’m also able to add the storage domain in ovirt now. >>>>> >>>>> I see network.remote-dio=enable is part of the gluster virt group, so apparently it’s not getting set by ovirt duding the volume creation/optimze for storage? >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Users mailing list -- users(a)ovirt.org >>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: >>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/OPBXHYOHZA4XR… >>>>> >>>>> _______________________________________________ >>>>> Users mailing list -- users(a)ovirt.org >>>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/B7K24XYG3M43C… >>>> >>>> >>>> >>>> -- >>>> >>>> >>>> Thanks, >>>> Gobinda

3 2

Re: [ovirt-announce] Re: Re: [ANN] oVirt 4.3.4 First Release Candidate is now available
by Strahil 21 May '19

21 May '19

On May 21, 2019 06:00, Satheesaran Sundaramoorthi <sasundar(a)redhat.com> wrote: > > > On Fri, May 17, 2019 at 1:12 AM Nir Soffer <nsoffer(a)redhat.com> wrote: >> >> On Thu, May 16, 2019 at 10:12 PM Darrell Budic <budic(a)onholyground.com> wrote: >>> >>> On May 16, 2019, at 1:41 PM, Nir Soffer <nsoffer(a)redhat.com> wrote: >>>> >>>> >>>> On Thu, May 16, 2019 at 8:38 PM Darrell Budic <budic(a)onholyground.com> wrote: >>>>> >>>>> I tried adding a new storage domain on my hyper converged test cluster running Ovirt 4.3.3.7 and gluster 6.1. I was able to create the new gluster volume fine, but it’s not able to add the gluster storage domain (as either a managed gluster volume or directly entering values). The created gluster volume mounts and looks fine from the CLI. Errors in VDSM log: >>>>> >>>> ... >>>>> >>>>> 2019-05-16 10:25:09,584-0500 ERROR (jsonrpc/5) [storage.fileSD] Underlying file system doesn't supportdirect IO (fileSD:110) >>>>> 2019-05-16 10:25:09,584-0500 INFO (jsonrpc/5) [vdsm.api] FINISH createStorageDomain error=Storage Domain target is unsupported: () from=::ffff:10.100.90.5,44732, flow_id=31d993dd, task_id=ecea28f3-60d4-476d-9ba8-b753b7c9940d (api:52) >>>> >>>> >>>> The direct I/O check has failed. >>>> >>>> >>>> So something is wrong in the files system. >>>> >>>> To confirm, you can try to do: >>>> >>>> dd if=/dev/zero of=/path/to/mountoint/test bs=4096 count=1 oflag=direct >>>> >>>> This will probably fail with: >>>> dd: failed to open '/path/to/mountoint/test': Invalid argument >>>> >>>> If it succeeds, but oVirt fail to connect to this domain, file a bug and we will investigate. >>>> >>>> Nir >>> >>> >>> Yep, it fails as expected. Just to check, it is working on pre-existing volumes, so I poked around at gluster settings for the new volume. It has network.remote-dio=off set on the new volume, but enabled on old volumes. After enabling it, I’m able to run the dd test: >>> >>> [root@boneyard mnt]# gluster vol set test network.remote-dio enable >>> volume set: success >>> [root@boneyard mnt]# dd if=/dev/zero of=testfile bs=4096 count=1 oflag=direct >>> 1+0 records in >>> 1+0 records out >>> 4096 bytes (4.1 kB) copied, 0.0018285 s, 2.2 MB/s >>> >>> I’m also able to add the storage domain in ovirt now. >>> >>> I see network.remote-dio=enable is part of the gluster virt group, so apparently it’s not getting set by ovirt duding the volume creation/optimze for storage? >> >> >> I'm not sure who is responsible for changing these settings. oVirt always required directio, and we >> never had to change anything in gluster. >> >> Sahina, maybe gluster changed the defaults? >> >> Darrell, please file a bug, probably for RHHI. > > > Hello Darrell & Nir, > > Do we have a bug available now for this issue ? > I just need to make sure performance.strict-o-direct=on is enabled on that volume. > > > Satheesaran Sundaramoorthi > > Senior Quality Engineer, RHHI-V QE > > Red Hat APAC > Please check https://bugzilla.redhat.com/show_bug.cgi?id=1711054 Best Regards, Strahil Nikolov

1 0

Re: [ANN] oVirt 4.3.4 First Release Candidate is now available
by Strahil 21 May '19

21 May '19

Hi Sandro, Thanks for the update. I have just upgraded to RC1 (using gluster v6 here) and the issue I detected in 4.3.3.7 - where gluster Storage domain fails creation - is still present. Can you check if the 'dd' command executed during creation has been recently modified ? I've received update from Darrell (also gluster v6) , but haven't received an update from anyone who is using gluster v5 -> thus I haven't opened a bug yet. Best Regards, Strahil NikolovOn May 16, 2019 11:21, Sandro Bonazzola <sbonazzo(a)redhat.com> wrote: > > The oVirt Project is pleased to announce the availability of the oVirt 4.3.4 First Release Candidate, as of May 16th, 2019. > > This update is a release candidate of the fourth in a series of stabilization updates to the 4.3 series. > This is pre-release software. This pre-release should not to be used inproduction. > > This release is available now on x86_64 architecture for: > * Red Hat Enterprise Linux 7.6 or later > * CentOS Linux (or similar) 7.6 or later > > This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: > * Red Hat Enterprise Linux 7.6 or later > * CentOS Linux (or similar) 7.6 or later > * oVirt Node 4.3 (available for x86_64 only) > > Experimental tech preview for x86_64 and s390x architectures for Fedora 28 is also included. > > See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. > > Notes: > - oVirt Appliance is already available > - oVirt Node is already available[2] > > Additional Resources: > * Read more about the oVirt 4.3.4 release highlights:http://www.ovirt.org/release/4.3.4/ > * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt > * Check out the latest project news on the oVirt blog:http://www.ovirt.org/blog/ > > [1] http://www.ovirt.org/release/4.3.4/ > [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA > > sbonazzo(a)redhat.com

8 17

Virtual office, how?
by andres.b.dev＠gmail.com 21 May '19

21 May '19

I'm trying to be able to create different virtual LANs, where, for example, I have 2 groups of pcs A and B belongs to network N1 C and D belongs to network N2 N1 and N2 with his own public IP. For example A: Local ip: 192.168.122.100 B: Local ip: 192.168.122.101 C: Local ip: 192.168.122.102 D: Local ip: 192.168.122.103 Where A and B has the same public ip, and C and D has the same public ip. Now, I want that A can ssh on B, but not on C or D. The same goes for C, where C can access to D via ssh but not to A or B I'm not sure if OVS solve this problem or not, or if this is not possible. Is this possible? How?

2 2

Every now and then Backup question
by Markus Schaufler 21 May '19

21 May '19

Hi, looking for a backup solution compareable to vmware/veeam - ie. features like agent-less, incremental, lan-free backups with dedup and single item restore. As I understand the main construction site for all commercial backup solutions available for kvm/rhev was the underlying qemu with its CBT implementation. The main functions should now be implemented as there are solutions like vprotect which state that they support incremental backups for rhev. But as I read at https://www.openvirtualization.pro/agent-less-backup-strategies-for-ovirt-r… there might still be some drawbacks. Are there ressources or a roadmap covering the state of the backup/recovery process, ie. what is possible now and whats in progress? I'm sure a big showstopper for open source virtualization projects are is uncertainty about backup/recovery and especially disaster recovery processes. Follow up questions to the users with bigger environments: Whats your current backup and DR strategy? Does anybody have experience with commercially availbe backup solutions like Bacula, SEP, Commvault, etc.?

2 1

newbie questions about installation using synology NAS
by bmillett＠gmail.com 20 May '19

20 May '19

I'm starting fairly fresh. Ovirt engine 4.3.3.7 on a host 2 nodes installed with ovirt-release-host-node-4.3.3.1 synology DS418 with 5 TB disks I've configured the master engine and am ready to add the nodes. My questions have to do with the order or steps . Do I need to mount /volume1/data/images/rhev from the DS418 on each of the nodes before I add them or do I just add the nodes, then define a domain using the NFS mounts from the DS418? Thanks.

3 2

Re: newbie questions about installation using synology NAS
by Strahil 20 May '19

20 May '19

Define the nodes and when you create your NFS storage domain - it will be automatically mounted on all hosts in the DC. Best Regards, Strahil NikolovOn May 20, 2019 00:03, bmillett(a)gmail.com wrote: > > I'm starting fairly fresh. > Ovirt engine 4.3.3.7 on a host > 2 nodes installed with ovirt-release-host-node-4.3.3.1 > synology DS418 with 5 TB disks > > I've configured the master engine and am ready to add the nodes. > My questions have to do with the order or steps . > Do I need to mount /volume1/data/images/rhev from the DS418 on each of the nodes before I add them or > do I just add the nodes, then define a domain using the NFS mounts from the DS418? > > Thanks. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/IXCF3CZGQDDOF…

2 1

Re: oVirt upgrade version from 4.2 to 4.3
by Strahil 20 May '19

20 May '19

I would recommend you to postpone your upgrade if you use gluster (without the API) , as creation of virtual disks via UI on gluster is having issues - only preallocated can be created. Best Regards, Strahil NikolovOn May 19, 2019 09:53, Yedidyah Bar David <didi(a)redhat.com> wrote: > > On Thu, May 16, 2019 at 3:40 PM <dmarini(a)it.iliad.com> wrote: > > > > I cannot find an official upgrade procedure from 4.2 to 4.3 oVirt version on this page: https://www.ovirt.org/documentation/upgrade-guide/upgrade-guide.html > > > > Can you help me? > > As others noted, the above should be sufficient, for general upgrade > instructions, even though it does require some updates. > > You probably want to read also: > > https://ovirt.org/release/4.3.0/ > > as well as all the other relevant pages in: > > https://ovirt.org/release/ > > Best regards, > > > > > Thanks > > _______________________________________________ > > Users mailing list -- users(a)ovirt.org > > To unsubscribe send an email to users-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WG2EI6HL3S2AT… > > > > -- > Didi > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KAJGM3URCFSNN…

3 2

oVirt upgrade version from 4.2 to 4.3
by dmarini＠it.iliad.com 19 May '19

19 May '19

I cannot find an official upgrade procedure from 4.2 to 4.3 oVirt version on this page: https://www.ovirt.org/documentation/upgrade-guide/upgrade-guide.html Can you help me? Thanks

5 6

Re: [ovirt-announce] Re: [ANN] oVirt 4.3.4 First Release Candidate is now available
by Strahil 19 May '19

19 May '19

OK, Can we summarize it: 1. VDO must 'emulate512=true' 2. 'network.remote-dio' should be off ? As per this: https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3/htm… We should have these: quick-read=off read-ahead=off io-cache=off stat-prefetch=off eager-lock=enable remote-dio=on quorum-type=auto server-quorum-type=server I'm a little bit confused here. Best Regards, Strahil NikolovOn May 19, 2019 07:44, Sahina Bose <sabose(a)redhat.com> wrote: > > > > On Sun, 19 May 2019 at 12:21 AM, Nir Soffer <nsoffer(a)redhat.com> wrote: >> >> On Fri, May 17, 2019 at 7:54 AM Gobinda Das <godas(a)redhat.com> wrote: >>> >>> From RHHI side default we are setting below volume options: >>> >>> { group: 'virt', >>> storage.owner-uid: '36', >>> storage.owner-gid: '36', >>> network.ping-timeout: '30', >>> performance.strict-o-direct: 'on', >>> network.remote-dio: 'off' >> >> >> According to the user reports, this configuration is not compatible with oVirt. >> >> Was this tested? > > > Yes, this is set by default in all test configuration. We’re checking on the bug, but the error is likely when the underlying device does not support 512b writes. > With network.remote-dio off gluster will ensure o-direct writes >> >> >>> } >>> >>> >>> On Fri, May 17, 2019 at 2:31 AM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: >>>> >>>> Ok, setting 'gluster volume set data_fast4 network.remote-dio on' allowed me to create the storage domain without any issues. >>>> I set it on all 4 new gluster volumes and the storage domains were successfully created. >>>> >>>> I have created bug for that: >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1711060 >>>> >>>> If someone else already opened - please ping me to mark this one as duplicate. >>>> >>>> Best Regards, >>>> Strahil Nikolov >>>> >>>> >>>> В четвъртък, 16 май 2019 г., 22:27:01 ч. Гринуич+3, Darrell Budic <budic(a)onholyground.com> написа: >>>> >>>> >>>> On May 16, 2019, at 1:41 PM, Nir Soffer <nsoffer(a)redhat.com> wrote: >>>> >>>>> >>>>> On Thu, May 16, 2019 at 8:38 PM Darrell Budic <budic(a)onholyground.com> wrote: >>>>>> >>>>>> I tried adding a new storage domain on my hyper converged test cluster running Ovirt 4.3.3.7 and gluster 6.1. I was able to create the new gluster volume fine, but it’s not able to add the gluster storage domain (as either a managed gluster volume or directly entering values). The created gluster volume mounts and looks fine from the CLI. Errors in VDSM log: >>>>>> >>>>> ... >>>>>> >>>>>> 2019-05-16 10:25:09,584-0500 ERROR (jsonrpc/5) [storage.fileSD] Underlying file system doesn't supportdirect IO (fileSD:110) >>>>>> 2019-05-16 10:25:09,584-0500 INFO (jsonrpc/5) [vdsm.api] FINISH createStorageDomain error=Storage Domain target is unsupported: () from=::ffff:10.100.90.5,44732, flow_id=31d993dd, task_id=ecea28f3-60d4-476d-9ba8-b753b7c9940d (api:52) >>>>> >>>>> >>>>> The direct I/O check has failed. >>>>> >>>>> >>>>> So something is wrong in the files system. >>>>> >>>>> To confirm, you can try to do: >>>>> >>>>> dd if=/dev/zero of=/path/to/mountoint/test bs=4096 count=1 oflag=direct >>>>> >>>>> This will probably fail with: >>>>> dd: failed to open '/path/to/mountoint/test': Invalid argument >>>>> >>>>> If it succeeds, but oVirt fail to connect to this domain, file a bug and we will investigate. >>>>> >>>>> Nir >>>> >>>> >>>> Yep, it fails as expected. Just to check, it is working on pre-existing volumes, so I poked around at gluster settings for the new volume. It has network.remote-dio=off set on the new volume, but enabled on old volumes. After enabling it, I’m able to run the dd test: >>>> >>>> [root@boneyard mnt]# gluster vol set test network.remote-dio enable >>>> volume set: success >>>> [root@boneyard mnt]# dd if=/dev/zero of=testfile bs=4096 count=1 oflag=direct >>>> 1+0 records in >>>> 1+0 records out >>>> 4096 bytes (4.1 kB) copied, 0.0018285 s, 2.2 MB/s >>>> >>>> I’m also able to add the storage domain in ovirt now. >>>> >>>> I see network.remote-dio=enable is part of the gluster virt group, so apparently it’s not getting set by ovirt duding the volume creation/optimze for storage? >>>> >>>> >>>> >>>> _______________________________________________ >>>> Users mailing list -- users(a)ovirt.org >>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/OPBXHYOHZA4XR… >>>> >>>> _______________________________________________ >>>> Users mailing list -- users(a)ovirt.org >>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/B7K24XYG3M43C… >>> >>> >>> >>> -- >>> >>> >>> Thanks, >>> Gobinda

1 0

Re: [ovirt-announce] Re: [ANN] oVirt 4.3.4 First Release Candidate is now available
by Strahil 19 May '19

19 May '19

This is my previous e-mail: On May 16, 2019 15:23, Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: It seems that the issue is within the 'dd' command as it stays waiting for input: [root@ovirt1 mnt]# /usr/bin/dd iflag=fullblock of=file oflag=direct,seek_bytes seek=1048576 bs=256512 count=1 conv=notrunc,nocreat,fsync ^C0+0 records in 0+0 records out 0 bytes (0 B) copied, 19.3282 s, 0.0 kB/s Changing the dd command works and shows that the gluster is working: [root@ovirt1 mnt]# cat /dev/urandom | /usr/bin/dd of=file oflag=direct,seek_bytes seek=1048576 bs=256512 count=1 conv=notrunc,nocreat,fsync 0+1 records in 0+1 records out 131072 bytes (131 kB) copied, 0.00705081 s, 18.6 MB/s Best Regards, Strahil Nikolov ----- Препратено съобщение ----- От: Strahil Nikolov <hunter86_bg(a)yahoo.com> До: Users <users(a)ovirt.org> Изпратено: четвъртък, 16 май 2019 г., 5:56:44 ч. Гринуич-4 Тема: ovirt 4.3.3.7 cannot create a gluster storage domain Hey guys, I have recently updated (yesterday) my platform to latest available (v4.3.3.7) and upgraded to gluster v6.1 .The setup is hyperconverged 3 node cluster with ovirt1/gluster1 & ovirt2/gluster2 as replica nodes (glusterX is for gluster communication) while ovirt3 is the arbiter. Today I have tried to add new domain storages but they fail with the following: 2019-05-16 10:15:21,296+0300 INFO (jsonrpc/2) [vdsm.api] FINISH createStorageDomain error=Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.0138582 s, 0.0 kB/s\n" from=::ffff:192.168.1.2,43864, flow_id=4a54578a, task_id=d2535d0f-c7f7-4f31-a10f-704923ce1790 (api:52) 2019-05-16 10:15:21,296+0300 ERROR (jsonrpc/2) [storage.TaskManager.Task] (Task='d2535d0f-c7f7-4f31-a10f-704923ce1790') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in createStorageDomain File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2614, in createStorageDomain storageType, domVersion, block_size, alignment) File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 106, in create block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 466, in _prepareMetadata cls.format_external_leases(sdUUID, xleases_path) File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 1255, in format_external_leases xlease.format_index(lockspace, backend) File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 681, in format_index index.dump(file) File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 843, in dump file.pwrite(INDEX_BASE, self._buf) File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 1076, in pwr It seems that the 'dd' is having trouble checking the new gluster volume. The output is from the RC1 , but as you see Darell's situation is maybe the same.On May 16, 2019 21:41, Nir Soffer <nsoffer(a)redhat.com> wrote: > > On Thu, May 16, 2019 at 8:38 PM Darrell Budic <budic(a)onholyground.com> wrote: >> >> I tried adding a new storage domain on my hyper converged test cluster running Ovirt 4.3.3.7 and gluster 6.1. I was able to create the new gluster volume fine, but it’s not able to add the gluster storage domain (as either a managed gluster volume or directly entering values). The created gluster volume mounts and looks fine from the CLI. Errors in VDSM log: >> > ... >> >> 2019-05-16 10:25:09,584-0500 ERROR (jsonrpc/5) [storage.fileSD] Underlying file system doesn't supportdirect IO (fileSD:110) >> 2019-05-16 10:25:09,584-0500 INFO (jsonrpc/5) [vdsm.api] FINISH createStorageDomain error=Storage Domain target is unsupported: () from=::ffff:10.100.90.5,44732, flow_id=31d993dd, task_id=ecea28f3-60d4-476d-9ba8-b753b7c9940d (api:52) > > > The direct I/O check has failed. > > This is the code doing the check: > > 98 def validateFileSystemFeatures(sdUUID, mountDir): > 99 try: > 100 # Don't unlink this file, we don't have the cluster lock yet as it > 101 # requires direct IO which is what we are trying to test for. This > 102 # means that unlinking the file might cause a race. Since we don't > 103 # care what the content of the file is, just that we managed to > 104 # open it O_DIRECT. > 105 testFilePath = os.path.join(mountDir, "__DIRECT_IO_TEST__") > 106 oop.getProcessPool(sdUUID).directTouch(testFilePath) > 107 except OSError as e: > 108 if e.errno == errno.EINVAL: > 109 log = logging.getLogger("storage.fileSD") > 110 log.error("Underlying file system doesn't support"

4 3

Re: Gluster VM image Resync Time
by Strahil 18 May '19

18 May '19

By default ovirt uses 'sharding' which splits the files into logical chunks. This greatly reduces healing time, as VM's disk is not always completely overwritten and only the shards that are different will be healed. Maybe you should change the default shard size. Best Regards, Strahil NikolovOn Mar 27, 2019 08:24, Indivar Nair <indivar.nair(a)techterra.in> wrote: > > Hi All, > > We are planning a 2 + 1 arbitrated mirrored Gluster setup. > We would have around 50 - 60 VMs, with an average 500GB disk size. > > Now in case one of the Gluster Nodes go completely out of sync, roughly, how long would it take to resync? (as per your experience) > Will it impact the working of VMs in any way? > Is there anything to be taken care of, in advance, to prepare for such a situation? > > Regards, > > > Indivar Nair >

4 7

Any plans for oVirt and SBD fencing (a.k.a. poison pill)
by hunter86_bg＠yahoo.com 18 May '19

18 May '19

Hello Community, Do you have any idea if SBD fencing mechanism will be implemented in the nearest future? It will be nice to be able to resset a host just like corosync/pacemaker cluster implementation.

4 3

HostedEngine Deployment fails activating NFS Storage Domain hosted_storage via GUI Step 4
by Ralf Schenk 17 May '19

17 May '19

Hello, I'm deploying HostedEngine to a NFS Storage. HostedEngineLocal ist setup and running already. But Step 4 (Moving to hosted_storage Domain on NFS) fails. The Host ist Node-NG 4.3.3.1 based. The intended NFS Domain gets mounted in the host but activation (I think via EngineAPI fails): [ INFO ] TASK [ovirt.hosted_engine_setup : Activate storage domain] [ ERROR ] Error: Fault reason is "Operation Failed". Fault detail is "[]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[]\". HTTP response code is 400."} mount in host shows: storage.rxmgmt.databay.de:/ovirt/hosted_storage on /rhev/data-center/mnt/storage.rxmgmt.databay.de:_ovirt_hosted__storage type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,soft,nosharecache,proto=tcp,timeo=600,retrans=6,sec=sys,clientaddr=172.16.252.231,local_lock=none,addr=172.16.252.3) I also sshd into the locally running engine vi 192.168.122.XX and the VM can mount the storage domain, too: [root@engine01 ~]# mount storage.rxmgmt.databay.de:/ovirt/hosted_storage /mnt/ -o vers=4.1 [root@engine01 ~]# mount | grep nfs sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) storage.rxmgmt.databay.de:/ovirt/hosted_storage on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.122.71,local_lock=none,addr=172.16.252.3) [root@engine01 ~]# ls -al /mnt/ total 18 drwxrwxr-x. 3 vdsm kvm 4 Apr 30 12:59 . dr-xr-xr-x. 17 root root 224 Apr 16 14:31 .. drwxr-xr-x. 4 vdsm kvm 4 Apr 30 12:40 4dc42146-b3fb-47ec-bf06-8d9bf7cdf893 -rwxr-xr-x. 1 vdsm kvm 0 Apr 30 12:55 __DIRECT_IO_TEST__ Anything I can do ? Log-Extract of ovirt-hosted-engine-setup-ansible-create_storage_domain included. -- *Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs(a)databay.de* <mailto:rs@databay.de> *Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de> Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Wilhelm Dohmen ------------------------------------------------------------------------

3 9

Re: Host needs to be reinstalled after configuring power management
by Michael Watters 17 May '19

17 May '19

Same here. The ssh host keys on our host nodes never change but there must be something in the ovirt database that needs to be updated. On 5/16/19 6:20 PM, Andrew DeMaria wrote: > That did the trick, thank you! That is really strange behavior however > considering the fingerprint did not change.. > > On Thu, May 16, 2019 at 7:14 AM Michael Watters <wattersm(a)watters.ws > <mailto:wattersm@watters.ws>> wrote: > > Had the same message on our cluster. The solution was to click > edit on each host and refetch the ssh host key. I'm not sure why > this is necessary in the first place however. > > > On 5/14/19 3:15 PM, Andrew DeMaria wrote: >> Hi, >> >> I am running ovirt 4.3 and have found the following action item >> immediately after configuring power management for a host: >> >> Host needs to be reinstalled as important configuration changes >> were applied on it. >> >> >> The thing is - I've just freshly installed this host and it seems >> strange that I need to reinstall it. >> Is there a better way to install a host and configure power >> management without having to reinstall it after? >> >> >> Thanks, >> >> Andrew >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org> >> To unsubscribe send an email to users-leave(a)ovirt.org <mailto:users-leave@ovirt.org> >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/5HMTDLIYVJJEP… > _______________________________________________ > Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org> > To unsubscribe send an email to users-leave(a)ovirt.org > <mailto:users-leave@ovirt.org> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/P6NHGE2EJ7PYH… >

1 0

Re: Dropped RX Packets
by Karli Sjöberg 17 May '19

17 May '19

1 0

Re: seal a rhel8 vm for templating fails
by Strahil 16 May '19

16 May '19

Can you try on a linux host to seal the VM via virt-sysprep ? Maybe virt-sysprep is not EL 8 ready... Best Regards, Strahil NikolovOn May 16, 2019 19:15, Nathanaël Blanchet <blanchet(a)abes.fr> wrote: > > Hi, > > I was used to successfully seal some el7 vms when templating, but with > rhel8, it always fails with that logs: > > 2019-05-16 15:45:03,499+02 ERROR > [org.ovirt.engine.core.bll.AddVmTemplateCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-55) > [db02e602-4b9d-4935-908c-e9e8c90a808b] Ending command > 'org.ovirt.engine.core.bll.AddVmTemplateCommand' with failure. > 2019-05-16 15:45:03,533+02 INFO > [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-55) > [db02e602-4b9d-4935-908c-e9e8c90a808b] START, SetVmStatusVDSCommand( > SetVmStatusVDSCommandParameters:{vmId='b79d8d62-212a-4f62-b236-3be6f1ed251e', > status='Down', exitStatus='Normal'}), log id: 7e657121 > 2019-05-16 15:45:03,538+02 INFO > [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-55) > [db02e602-4b9d-4935-908c-e9e8c90a808b] FINISH, SetVmStatusVDSCommand, > log id: 7e657121 > 2019-05-16 15:45:03,546+02 INFO > [org.ovirt.engine.core.bll.AddVmTemplateCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-55) > [db02e602-4b9d-4935-908c-e9e8c90a808b] Lock freed to object > 'EngineLock:{exclusiveLocks='[rhel8.0=TEMPLATE_NAME, > 8fb93ef2-d8a1-4d95-afed-c37131312462=TEMPLATE, > 3ae1ad71-193b-4222-9727-159b402fef49=DISK]', > sharedLocks='[b79d8d62-212a-4f62-b236-3be6f1ed251e=VM]'}' > 2019-05-16 15:45:03,560+02 ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (EE-ManagedThreadFactory-engineScheduled-Thread-55) > [db02e602-4b9d-4935-908c-e9e8c90a808b] EVENT_ID: > USER_ADD_VM_TEMPLATE_SEAL_FAILURE(1,324), Failed to seal Template > rhel8.0 (VM: thym-rhel8). > > But I successfully make a el8 template without sealing, but the issue is > that subscription-manager doesn't activate properly when creating new > vms from it. > > ovirt 3.3.3 > > -- > Nathanaël Blanchet > > Supervision réseau > Pôle Infrastrutures Informatiques > 227 avenue Professeur-Jean-Louis-Viala > 34193 MONTPELLIER CEDEX 5 > Tél. 33 (0)4 67 54 84 55 > Fax 33 (0)4 67 54 84 14 > blanchet(a)abes.fr > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OO7EGM25KWGV5…

2 1

Dropped RX Packets
by Magnus Isaksson 16 May '19

16 May '19

Hello all! I'm having quite some trouble with VMs that have a large amount of dropped packets on RX. This, plus customers complain about short dropped connections, for example one customer has a SQL server and an other serevr connecting to it, and it is randomly dropping connections. Before they moved their VM:s to us they did not have any of these issues. Does anyone have an idea of what this can be due to? And how can i fix it? It is starting to be a deal breaker for our customers on whether they will stay with us or not. I was thinking of reinstalling the nodes with oVirt Node, instead of the full CentOS, would this perhaps fix the issue? The enviroment is: Huawei x6000 with 4 nodes Each node having Intel X722 network card and connecting with 10G (fiber) to a Juniper EX 4600. Storage via FC to a IBM FS900. Each node is running a full CentOS 7.6 connecting to a Engine 4.2.8.2 Regards Magnus

4 4

seal a rhel8 vm for templating fails
by Nathanaël Blanchet 16 May '19

16 May '19

Hi, I was used to successfully seal some el7 vms when templating, but with rhel8, it always fails with that logs: 2019-05-16 15:45:03,499+02 ERROR [org.ovirt.engine.core.bll.AddVmTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [db02e602-4b9d-4935-908c-e9e8c90a808b] Ending command 'org.ovirt.engine.core.bll.AddVmTemplateCommand' with failure. 2019-05-16 15:45:03,533+02 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [db02e602-4b9d-4935-908c-e9e8c90a808b] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{vmId='b79d8d62-212a-4f62-b236-3be6f1ed251e', status='Down', exitStatus='Normal'}), log id: 7e657121 2019-05-16 15:45:03,538+02 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [db02e602-4b9d-4935-908c-e9e8c90a808b] FINISH, SetVmStatusVDSCommand, log id: 7e657121 2019-05-16 15:45:03,546+02 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [db02e602-4b9d-4935-908c-e9e8c90a808b] Lock freed to object 'EngineLock:{exclusiveLocks='[rhel8.0=TEMPLATE_NAME, 8fb93ef2-d8a1-4d95-afed-c37131312462=TEMPLATE, 3ae1ad71-193b-4222-9727-159b402fef49=DISK]', sharedLocks='[b79d8d62-212a-4f62-b236-3be6f1ed251e=VM]'}' 2019-05-16 15:45:03,560+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [db02e602-4b9d-4935-908c-e9e8c90a808b] EVENT_ID: USER_ADD_VM_TEMPLATE_SEAL_FAILURE(1,324), Failed to seal Template rhel8.0 (VM: thym-rhel8). But I successfully make a el8 template without sealing, but the issue is that subscription-manager doesn't activate properly when creating new vms from it. ovirt 3.3.3 -- Nathanaël Blanchet Supervision réseau Pôle Infrastrutures Informatiques 227 avenue Professeur-Jean-Louis-Viala 34193 MONTPELLIER CEDEX 5 Tél. 33 (0)4 67 54 84 55 Fax 33 (0)4 67 54 84 14 blanchet(a)abes.fr

1 0

Re: oVirt Open Source Backup solution?
by Strahil 16 May '19

16 May '19

Derek, That's risky. Just read lock the DB, create the lvm snapshot and release the lock. Otherwise you risk a transaction to be interrupted. Best Regards, Strahil NikolovOn May 13, 2019 16:47, Derek Atkins <derek(a)ihtfp.com> wrote: > > Strahil <hunter86_bg(a)yahoo.com> writes: > > > Another option is to create a snapshot, backup the snapahot and merge > > the disks (delete the snapshot actually). > > Sadly that option doesn't work with Databases, as you might inyerrupt > > a transaction and leave the DB in inconsistent state. > > Yet another reason to do it from inside the VM. > > What I do (on systems that have a running database) is to run a "flush" > operation to sync the database to disk, and then from within the flush > operation I create an LVM snapshot, and then I backup off the snapshot. > If I'm not running a database, then I just create the snapshot directly. > > > Best Regards, > > Strahil Nikolov > > -derek > -- > Derek Atkins 617-623-3745 > derek(a)ihtfp.com www.ihtfp.com > Computer and Internet Security Consultant > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/JS6YVB3S33VYL…

4 6

oVirt Open Source Backup solution?
by michael＠wanderingmad.com 16 May '19

16 May '19

Is there a good low to no-cost solution to backup oVirt and the virtual machines? I've been unabel to find something that will do a direct VM backup instead of a backup agent installed on VM

6 6

[ANN] oVirt 4.3.4 First Release Candidate is now available
by Sandro Bonazzola 16 May '19

16 May '19

The oVirt Project is pleased to announce the availability of the oVirt 4.3.4 First Release Candidate, as of May 16th, 2019. This update is a release candidate of the fourth in a series of stabilization updates to the 4.3 series. This is pre-release software. This pre-release should not to be used inproduction. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.6 or later * CentOS Linux (or similar) 7.6 or later This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.6 or later * CentOS Linux (or similar) 7.6 or later * oVirt Node 4.3 (available for x86_64 only) Experimental tech preview for x86_64 and s390x architectures for Fedora 28 is also included. See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available[2] Additional Resources: * Read more about the oVirt 4.3.4 release highlights: http://www.ovirt.org/release/4.3.4/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.4/ [2] http://resources.ovirt.org/pub/ovirt-4.3-pre/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig> <https://redhat.com/summit>

1 0

Cluster Un-stable since power outage
by Alan G 16 May '19

16 May '19

Hi, We have a dev cluster running 4.2. It had to be powered down as the building was going to loose power. Since we've brought it back up it has been massively un-stable (Hosts constantly switching state, VMs migrating all the time). I now have one host running (with HE) and all others in maintenance mode. When I try activate another host i see storage errors in vdsm.log 2019-05-07 09:41:00,114+0000 ERROR (monitor/a98c0b4) [storage.Monitor] Error checking domain a98c0b42-47b9-4632-8b54-0ff3bd80d4c2 (monitor:424) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 416, in _checkDomainStatus masterStats = self.domain.validateMaster() File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 941, in validateMaster if not self.validateMasterMount(): File "/usr/lib/python2.7/site-packages/vdsm/storage/blockSD.py", line 1377, in validateMasterMount return mount.isMounted(self.getMasterDir()) File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 161, in isMounted getMountFromTarget(target) File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 173, in getMountFromTarget for rec in _iterMountRecords(): File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 143, in _iterMountRecords for rec in _iterKnownMounts(): File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 139, in _iterKnownMounts yield _parseFstabLine(line) File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 81, in _parseFstabLine fs_spec = fileUtils.normalize_path(_unescape_spaces(fs_spec)) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileUtils.py", line 94, in normalize_path host, tail = address.hosttail_split(path) File "/usr/lib/python2.7/site-packages/vdsm/common/network/address.py", line 43, in hosttail_split raise HosttailError('%s is not a valid hosttail address:' % hosttail) HosttailError: :/ is not a valid hosttail address: Not sure if it's related but since the restart the hosted_storage domain has been elected the master domain. I'm a bit stuck at the moment. My only idea is to remove HE and switch to a standalone Engine VM running outside the cluster. Thanks, Alan

2 2