- Users - oVirt List Archives

Fwd: Issues with Gluster Domain
by C Williams 18 Jun '20

18 Jun '20

Resending to deal with possible email issues Thank You For Your Help !! ---------- Forwarded message --------- From: C Williams <cwilliams3320(a)gmail.com> Date: Thu, Jun 18, 2020 at 2:03 PM Subject: Re: [ovirt-users] Issues with Gluster Domain To: Strahil Nikolov <hunter86_bg(a)yahoo.com> Strahil, Here you go -- Thank You For Your Help ! BTW -- I can write a test file to gluster and it replicates properly. Thinking something about the oVirt Storage Domain ? [root@ov08 ~]# gluster pool list UUID Hostname State 5b40c659-d9ab-43c3-9af8-18b074ea0b83 ov06 Connected 36ce5a00-6f65-4926-8438-696944ebadb5 ov07.ntc.srcle.com Connected c7e7abdb-a8f4-4842-924c-e227f0db1b29 localhost Connected [root@ov08 ~]# gluster volume list images3 On Thu, Jun 18, 2020 at 1:13 PM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: > Log to the oVirt cluster and provide the output of: > gluster pool list > gluster volume list > for i in $(gluster volume list); do echo $i;echo; gluster volume info > $i; echo;echo;gluster volume status $i;echo;echo;echo;done > > ls -l /rhev/data-center/mnt/glusterSD/ > > Best Regards, > Strahil Nikolov > > > На 18 юни 2020 г. 19:17:46 GMT+03:00, C Williams <cwilliams3320(a)gmail.com> > написа: > >Hello, > > > >I recently added 6 hosts to an existing oVirt compute/gluster cluster. > > > >Prior to this attempted addition, my cluster had 3 Hypervisor hosts and > >3 > >gluster bricks which made up a single gluster volume (replica 3 volume) > >. I > >added the additional hosts and made a brick on 3 of the new hosts and > >attempted to make a new replica 3 volume. I had difficulty creating > >the > >new volume. So, I decided that I would make a new compute/gluster > >cluster > >for each set of 3 new hosts. > > > >I removed the 6 new hosts from the existing oVirt Compute/Gluster > >Cluster > >leaving the 3 original hosts in place with their bricks. At that point > >my > >original bricks went down and came back up . The volume showed entries > >that > >needed healing. At that point I ran gluster volume heal images3 full, > >etc. > >The volume shows no unhealed entries. I also corrected some peer > >errors. > > > >However, I am unable to copy disks, move disks to another domain, > >export > >disks, etc. It appears that the engine cannot locate disks properly and > >I > >get storage I/O errors. > > > >I have detached and removed the oVirt Storage Domain. I reimported the > >domain and imported 2 VMs, But the VM disks exhibit the same behaviour > >and > >won't run from the hard disk. > > > > > >I get errors such as this > > > >VDSM ov05 command HSMGetAllTasksStatusesVDS failed: low level Image > >copy > >failed: ("Command ['/usr/bin/qemu-img', 'convert', '-p', '-t', 'none', > >'-T', 'none', '-f', 'raw', > >u'/rhev/data-center/mnt/glusterSD/192.168.24.18: > _images3/5fe3ad3f-2d21-404c-832e-4dc7318ca10d/images/3ea5afbd-0fe0-4c09-8d39-e556c66a8b3d/fe6eab63-3b22-4815-bfe6-4a0ade292510', > >'-O', 'raw', > >u'/rhev/data-center/mnt/192.168.24.13: > _stor_import1/1ab89386-a2ba-448b-90ab-bc816f55a328/images/f707a218-9db7-4e23-8bbd-9b12972012b6/d6591ec5-3ede-443d-bd40-93119ca7c7d5'] > >failed with rc=1 out='' err=bytearray(b'qemu-img: error while reading > >sector 135168: Transport endpoint is not connected\\nqemu-img: error > >while > >reading sector 131072: Transport endpoint is not connected\\nqemu-img: > >error while reading sector 139264: Transport endpoint is not > >connected\\nqemu-img: error while reading sector 143360: Transport > >endpoint > >is not connected\\nqemu-img: error while reading sector 147456: > >Transport > >endpoint is not connected\\nqemu-img: error while reading sector > >155648: > >Transport endpoint is not connected\\nqemu-img: error while reading > >sector > >151552: Transport endpoint is not connected\\nqemu-img: error while > >reading > >sector 159744: Transport endpoint is not connected\\n')",) > > > >oVirt version is 4.3.82-1.el7 > >OS CentOS Linux release 7.7.1908 (Core) > > > >The Gluster Cluster has been working very well until this incident. > > > >Please help. > > > >Thank You > > > >Charles Williams >

1 0

Issues with Gluster Domain
by C Williams 18 Jun '20

18 Jun '20

Hello, I recently added 6 hosts to an existing oVirt compute/gluster cluster. Prior to this attempted addition, my cluster had 3 Hypervisor hosts and 3 gluster bricks which made up a single gluster volume (replica 3 volume) . I added the additional hosts and made a brick on 3 of the new hosts and attempted to make a new replica 3 volume. I had difficulty creating the new volume. So, I decided that I would make a new compute/gluster cluster for each set of 3 new hosts. I removed the 6 new hosts from the existing oVirt Compute/Gluster Cluster leaving the 3 original hosts in place with their bricks. At that point my original bricks went down and came back up . The volume showed entries that needed healing. At that point I ran gluster volume heal images3 full, etc. The volume shows no unhealed entries. I also corrected some peer errors. However, I am unable to copy disks, move disks to another domain, export disks, etc. It appears that the engine cannot locate disks properly and I get storage I/O errors. I have detached and removed the oVirt Storage Domain. I reimported the domain and imported 2 VMs, But the VM disks exhibit the same behaviour and won't run from the hard disk. I get errors such as this VDSM ov05 command HSMGetAllTasksStatusesVDS failed: low level Image copy failed: ("Command ['/usr/bin/qemu-img', 'convert', '-p', '-t', 'none', '-T', 'none', '-f', 'raw', u'/rhev/data-center/mnt/glusterSD/192.168.24.18:_images3/5fe3ad3f-2d21-404c-832e-4dc7318ca10d/images/3ea5afbd-0fe0-4c09-8d39-e556c66a8b3d/fe6eab63-3b22-4815-bfe6-4a0ade292510', '-O', 'raw', u'/rhev/data-center/mnt/192.168.24.13:_stor_import1/1ab89386-a2ba-448b-90ab-bc816f55a328/images/f707a218-9db7-4e23-8bbd-9b12972012b6/d6591ec5-3ede-443d-bd40-93119ca7c7d5'] failed with rc=1 out='' err=bytearray(b'qemu-img: error while reading sector 135168: Transport endpoint is not connected\\nqemu-img: error while reading sector 131072: Transport endpoint is not connected\\nqemu-img: error while reading sector 139264: Transport endpoint is not connected\\nqemu-img: error while reading sector 143360: Transport endpoint is not connected\\nqemu-img: error while reading sector 147456: Transport endpoint is not connected\\nqemu-img: error while reading sector 155648: Transport endpoint is not connected\\nqemu-img: error while reading sector 151552: Transport endpoint is not connected\\nqemu-img: error while reading sector 159744: Transport endpoint is not connected\\n')",) oVirt version is 4.3.82-1.el7 OS CentOS Linux release 7.7.1908 (Core) The Gluster Cluster has been working very well until this incident. Please help. Thank You Charles Williams

2 1

Ovirt fails to retrieve iSCSI targets during installation
by Ricardo Alonso 18 Jun '20

18 Jun '20

Trying to connect to a an iSCSI target (no chap/secrets) is failing with the message: 2020-06-17 05:25:40,050-0300 ERROR ansible failed { "ansible_host": "localhost", "ansible_playbook": "/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml", "ansible_result": { "_ansible_no_log": false, "changed": false, "connection": "close", "content": "{\n \"detail\" : \"For correct usage, see: https://ovirt.<removed>/ovirt-engine/apidoc#services/host/methods/iscsi_discover\",\n \"reason\" : \"Request syntactically incorrect.\"\n}", "content_encoding": "identity", "content_type": "application/json", "correlation_id": "a5ba94a7-b22b-4ed6-9a47-87a87186c341", "date": "Wed, 17 Jun 2020 08:25:38 GMT", "elapsed": 0, "invocation": { "module_args": { "attributes": null, "backup": null, "body": "{\"iscsi\": {\"address\": \"192.168.6.1\", \"port\": \"3260,3260\", \"username\": null, \"password\": \"\"}}", "body_format": "json", "client_cert": null, "client_key": null, "content": null, "creates": null, "delimiter": null, "dest": null, "directory_mode": null, "follow": false, "follow_redirects": "safe", "force": false, "force_basic_auth": false, "group": null, "headers": { "Accept": "application/json", "Authorization": "Basic YWRtaW5AaW50ZXJuYWw6cGQyMDAx", "Content-Type": "application/json" }, "http_agent": "ansible-httpget", "method": "POST", "mode": null, "owner": null, "regexp": null, "remote_src": null, "removes": null, "return_content": true, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": null, "status_code": [ "200" ], "timeout": 30, "unix_socket": null, "unsafe_writes": null, "url": "https://ovirt.<removed>/ovirt-engine/api/hosts/2c173b7f-9f9e-4046-890a-ab16d1babc35/iscsidiscover", "url_password": null, "url_username": null, "use_proxy": true, "validate_certs": false } }, "json": { "detail": "For correct usage, see: https://ovirt.<removed>/ovirt-engine/apidoc#services/host/methods/iscsi_discover", "reason": "Request syntactically incorrect." }, "msg": "Status code was 400 and not [200]: HTTP Error 400: Bad Request", "redirected": false, "server": "Apache/2.4.37 (centos) OpenSSL/1.1.1c mod_wsgi/4.6.4 Python/3.6", "status": 400, "transfer_encoding": "chunked", "url": "https://ovirt.<removed>/ovirt-engine/api/hosts/2c173b7f-9f9e-4046-890a-ab16d1babc35/iscsidiscover" }, "ansible_task": "iSCSI discover with REST API", "ansible_type": "task", "status": "FAILED", "task_duration": 4 } I found this old bug, but doesn't seams to have a resolution. Any clue/tips?

3 4

Re: Cannot authenticate user Invalid scopes: ovirt-app-api
by Anton Louw 18 Jun '20

18 Jun '20

So I think I have narrowed it down to the OVN settings. The only problem now is, is that when I want to update the OVN settings, it fails with “Failed to communicate with External Provider. See logs for details” When checking the logs, I see an error stating the “root hostname does not match” (In the OVN settings via the WebUI, I see that it also points to the old hostname) A bit of background on this, when the engine was initially built, it was configured with a different hostname, which was then changed, but somehow it is still referencing the old hostname. When I run the change hostname scripts (/usr/share/ovirt-engine/setup/bin/ovirt-engine-rename) it runs through everything, until it needs to modify the certs. (I have attached the screenshot) I am really not sure where to go from here, and I believe that most of this has to do with the certs (And I am just grasping at straws here) I am starting to think that it would just be easier to deploy everything from scratch, but if anybody has any ideas, I would appreciate it. Thank you Anton Louw Cloud Engineer: Storage and Virtualization ______________________________________ D: 087 805 1572 | M: N/A A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg anton.louw(a)voxtelecom.co.za www.vox.co.za From: Anton Louw via Users <users(a)ovirt.org> Sent: 18 June 2020 12:39 To: users(a)ovirt.org Subject: [ovirt-users] Cannot authenticate user Invalid scopes: ovirt-app-api Hi All, A new issue 😊 We have configured oVirt to use KeyCloak for authentication. This all works, I can log into the WebUI etc, but as soon as I need to talk to the API, it gives me the “invalid scopes” error. I have double checked KeyCloak, and the scopes are added. I went through the logs, but there is nothing telling me exactly what the actual cause is. I get the below when trying to get a token from the engine: “{"error_code":"access_denied","error":"Cannot authenticate user Invalid scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access."}” Does anybody have any idea where this is going wrong? Thanks Anton Louw Cloud Engineer: Storage and Virtualization at Vox ________________________________ T: 087 805 0000 | D: 087 805 1572 M: N/A E: anton.louw(a)voxtelecom.co.za<mailto:anton.louw@voxtelecom.co.za> A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg www.vox.co.za<http://www.vox.co.za> [F]<https://www.facebook.com/voxtelecomZA> [T]<https://www.twitter.com/voxtelecom> [I]<https://www.instagram.com/voxtelecomza> [L]<https://www.linkedin.com/company/voxtelecom> [Y]<https://www.youtube.com/user/VoxTelecom> [#VoxBrand]<https://www.vox.co.za/fibre/fibre-to-the-home/?prod=HOME> Disclaimer The contents of this email are confidential to the sender and the intended recipient. Unless the contents are clearly and entirely of a personal nature, they are subject to copyright in favour of the holding company of the Vox group of companies. Any recipient who receives this email in error should immediately report the error to the sender and permanently delete this email from all storage devices. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here<https://www.voxtelecom.co.za/security/mimecast/?prod=Enterprise>.

1 0

Cannot authenticate user Invalid scopes: ovirt-app-api
by Anton Louw 18 Jun '20

18 Jun '20

Hi All, A new issue 😊 We have configured oVirt to use KeyCloak for authentication. This all works, I can log into the WebUI etc, but as soon as I need to talk to the API, it gives me the “invalid scopes” error. I have double checked KeyCloak, and the scopes are added. I went through the logs, but there is nothing telling me exactly what the actual cause is. I get the below when trying to get a token from the engine: “{"error_code":"access_denied","error":"Cannot authenticate user Invalid scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access."}” Does anybody have any idea where this is going wrong? Thanks Anton Louw Cloud Engineer: Storage and Virtualization ______________________________________ D: 087 805 1572 | M: N/A A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg anton.louw(a)voxtelecom.co.za www.vox.co.za

1 0

Shutting be down kvm host does what to VMs
by Louis Bohm 18 Jun '20

18 Jun '20

I have a single ovirt host running 2 VMs. If I do a shutdown -h now on the kvm host will ovirt do a similar shutdown command on the VMs? And wait for them to shutdown before finishing it's shutdown? Please do not tell me I should have 2 physical hosts. I know but it's not in the clients budget. Thanks, Louis

2 1

cant edit ISO domain in any way
by Philip Brown 18 Jun '20

18 Jun '20

oVirt 4.3: Okay, I found documentation that I cant have more than one "ISO" type storage domain. I can kinda understand that. But, I cant even edit or delete the existing one? Even when logged in to the oVirt GUI as admin@internal? Is this some kind of bug or something?

2 3

Information requested on oVirt + RDO/OpenStack
by Glenn Marcy 18 Jun '20

18 Jun '20

I am hoping to try out adding RDO to oVirt after things with CentOS 8.2 settle down. I've done deployments of RDO on KVM before without any issues (TripleO Quickstart), but this would be my first time trying it on oVirt. Since oVirt recently updated to the openstack-java 3.2.9 packages, is there a specific version of OpenStack (Train or Ussuri) that runs on CentOS 8.x that I need? Are there any issues with the fact that oVirt already uses and/or provides some OpenStack services? Does the fact that OpenStack deployments are now more "containerized" present any issues I should be aware of? Are there initiatives for further convergence between oVirt, RDO/OpenStack and OKD/OCP that are changing this space where it would make sense to follow and/or participate in those efforts? I'm starting out with oVirt to get an understanding of the capabilities that it brings to the table, but interested in the convergence of the other technologies that would benefit from having oVirt as, or a part of, the foundation. Best Regards, Glenn

2 1

Ansible ovirt_disk module: how to attach disk without activation
by Gianluca Cecchi 17 Jun '20

17 Jun '20

Hello, root problem in a 4.3.8 environment is that sometimes when I hot add a floating disk to a VM in web admin gui, it attaches and also activates it by default (the flag is checked by default, but you can uncheck it) and this two-phases action fails as a single step. What I get is: VDSM ovhost command HotPlugDiskVDS failed: Requested operation is not valid: Domain already contains a disk with that address And then the VM event regarding hot add disk failure. I'm going to investigate more: it should be a past bug already solved in theory by my libvirt sw version (I've to crosscheck its id)... but I still get the error sometimes. What programmatically works is to attach the disk without activating it (this step always completes with success) and then activate the disk as a separate operation (this one typically fails the first time when I have the "problematic" situation, but then the second time succeeds). So I would like to use it as a workaround in the mean time. The problem reflects using the ansible module ovirt_disk, that by default activates the disk. So I have tried to specify "activate: no" (introduced in 2.8, readings the docs) when I attach it, but it seems not to work as expected. I'm testing using awx and its info says: AWX 9.0.1.0 Ansible 2.8.5 I tried this on a VM that (as usual when you want to reproduce... ;-) doesn't show the problem: ovirt_disk: auth: "{{ ovirt_auth }}" state: attached activate: no name: "{{ tobe_sw_disk_name }}" vm_name: "{{ ansible_hostname }}" size: "{{ current_sw_disk_size }}" interface: virtio_scsi delegate_to: localhost register: disk_attach_info But going to see registered variable it contains "active: true": "disk_activate_info": { "changed": false, "id": "6f7cdf02-cf8b-4fa8-ac00-6b47f6e0c827", "diskattachment": { "href": "/ovirt-engine/api/vms/b5c67c93-bd5d-42b6-a873-05f69cece2f1/diskattachments/6f7cdf02-cf8b-4fa8-ac00-6b47f6e0c827", "id": "6f7cdf02-cf8b-4fa8-ac00-6b47f6e0c827", "active": true, "bootable": false, "disk": { .... In fact I followed with the step to activate the disk: ovirt_disk: auth: "{{ ovirt_auth }}" state: present activate: yes name: "{{ tobe_sw_disk_name }}" vm_name: "{{ ansible_hostname }}" size: "{{ current_sw_disk_size }}" interface: virtio_scsi delegate_to: localhost register: disk_activate_info But in playbook run I got: ok: [target_vm -> localhost] And inside registered variable "changed: false": "disk_activate_info": { "changed": false, "id": "6f7cdf02-cf8b-4fa8-ac00-6b47f6e0c827", "diskattachment": { "href": "/ovirt-engine/api/vms/b5c67c93-bd5d-42b6-a873-05f69cece2f1/diskattachments/6f7cdf02-cf8b-4fa8-ac00-6b47f6e0c827", "id": "6f7cdf02-cf8b-4fa8-ac00-6b47f6e0c827", "active": true, ... Any tip to realize what needed: task to only attach disk without activating it and then task to activate the disk? Thanks in advance, Gianluca

2 3

Re: New VM from Template and Storage size
by Eyal Shenitzky 17 Jun '20

17 Jun '20

Hi Erez, This sounds like a nice RFE. Can you please submit it to Bugzilla - https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-engine ? In the title add in the beginning - [RFE] ... Thanks. On Wed, 17 Jun 2020 at 08:24, Erez Zarum <erezz(a)nanosek.com> wrote: > It is reasonable flow that you inherit the configuration from the > template, but you can change most of the settings during the creation of > the VM, I just don't understand why not to be able to change the disk size > and name. > Why disk size? You can say the same thing with any other settings, so just > make a template for any different setting you need for different VMs based > on that same OS. > This flow of changing the disk size is supported in any other platforms > take a few, OpenStack, VMWare, Xen and any other Cloud providers. > I don't see a reason why to do this in two API calls, one for VM creation > and then another one for just expanding the disk size. > I do understand if you create a VM based on a thin template which is just > a link that there's no other way. > Another "issue" is that it creates the new of the new disk for the VM > based on the template name instead of "vmname_Disk1" etc... > I have a customer with many different disk sizes, varying from 50GB up to > 200GB, If I go and create a template for an OS based on that, I will > probably have 10 templates, it's not scalable if you need to just modify > something in the template as well. > > So the only option right now is to use OpenStack Glance for saving images > or create a new feature in Foreman/Satelite for that. > > On 16/06/2020, 20:53, "Nir Soffer" <nsoffer(a)redhat.com> wrote: > > On Tue, Jun 16, 2020 at 5:32 PM Erez Zarum <erezz(a)nanosek.com> wrote: > > > > Hey, > > > > I have created a template that the disk is a thick (raw) with a size > of 15GB. > > > > I am trying to create a VM from that template but having the disk > with a much larger size, both template and the VM are set to use thick and > I create the new VM with the clone option, so there’s no link between the > new VM and the template. > > > > So far the only solution I managed to find is to adjust the size > after creating the VM. > > This looks like reasonable flow. You create the vm from a template, so > you inherit the > disk size from the template. Then you modify the disk to the required > size. > > > For example, this situation causes a problematic step when using > Foreman/Satelite as trying to provision a VM based on a template but with > different size for the OS disk. > > Why is this problematic? > > > I assume that one of the ways to solve it is to convert the template > somehow into an “image”? like using OpenStack Glance as a place to store > “templates” as images? Any other way besides using Glance? > > If you always need to resize the template, why not create another > templated with the right > size based on this template? > > Nir > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/46IFRJCTZNZZH… > -- Regards, Eyal Shenitzky

1 0