- Users - oVirt List Archives

Re: Single instance scaleup.
by Strahil 26 May '19

26 May '19

Hi Leo, Gluster is quite smart, but in order to provide any hints , can you provide output of 'gluster volume info <glustervol>'. If you have 2 more systems , keep in mind that it is best to mirror the storage on the second replica (2 disks on 1 machine -> 2 disks on the new machine), while for the arbiter this is not neccessary. What is your network and NICs ? Based on my experience , I can recommend at least 10 gbit/s interfase(s). Best Regards, Strahil NikolovOn May 26, 2019 07:52, Leo David <leoalex(a)gmail.com> wrote: > > Hello Everyone, > Can someone help me to clarify this ? > I have a single-node 4.2.8 installation ( only two gluster storage domains - distributed single drive volumes ). Now I just got two identintical servers and I would like to go for a 3 nodes bundle. > Is it possible ( after joining the new nodes to the cluster ) to expand the existing volumes across the new nodes and change them to replica 3 arbitrated ? > If so, could you share with me what would it be the procedure ? > Thank you very much ! > > Leo

2 1

Active-Passive Disaster Recovery
by Paulo Silva 26 May '19

26 May '19

Hi, I want to setup an Active-Passive between 2 sites using oVirt 4.3. For now I have a primary site with an ovirt-engine, 2 ovirt hosts and 5 VMs in an iscsi storage. The backup site will have an ovirt-engine with 2 ovirt hosts and an iscsi storage that will will have the same LUNs of the primary replicated. I'm reading the DR Guide ( https://ovirt.org/documentation/disaster-recovery-guide/) in the backup site do I need to manually create the storage domains and the VM networks or will the ansible playbook take care of that? Thanks -- Paulo Silva <paulojjs(a)gmail.com>

3 2

Single instance scaleup.
by Leo David 26 May '19

26 May '19

Hello Everyone, Can someone help me to clarify this ? I have a single-node 4.2.8 installation ( only two gluster storage domains - distributed single drive volumes ). Now I just got two identintical servers and I would like to go for a 3 nodes bundle. Is it possible ( after joining the new nodes to the cluster ) to expand the existing volumes across the new nodes and change them to replica 3 arbitrated ? If so, could you share with me what would it be the procedure ? Thank you very much ! Leo

1 0

Unable to find OVF_STORE after recovery / upgrade
by Sam Cappello 26 May '19

26 May '19

This is a multi-part message in MIME format. --------------05F3D0780062C6D37C1619A7 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, so i was running a 3.4 hosted engine two node setup on centos 6, had some disk issues so i tried to upgrade to centos 7 and follow the path 3.4 > 3.5 > 3.6 > 4.0. i screwed up dig time somewhere between 3.6 and 4.0, so i wiped the drives, installed a fresh 4.0.3, then created the database and restored the 3.6 engine backup before running engine-setup as per the docs. things seemed to work, but i have the the following issues / symptoms: - ovirt-ha-agent running 100% CPU on both nodes - messages in the UI that the Hosted Engine storage Domain isn't active and Failed to import the Hosted Engine Storage Domain - hosted engine is not visible in the UI and the following repeating in the agent.log: MainThread::INFO::2016-10-03 12:38:27,718::hosted_engine::461::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400) MainThread::INFO::2016-10-03 12:38:27,720::hosted_engine::466::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host vmhost1.oracool.net (id: 1, score: 3400) MainThread::INFO::2016-10-03 12:38:37,979::states::421::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost MainThread::INFO::2016-10-03 12:38:37,985::hosted_engine::612::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM MainThread::INFO::2016-10-03 12:38:45,645::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2016-10-03 12:38:45,647::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2016-10-03 12:39:00,543::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2016-10-03 12:39:00,562::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain MainThread::INFO::2016-10-03 12:39:01,235::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images MainThread::INFO::2016-10-03 12:39:01,236::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images MainThread::INFO::2016-10-03 12:39:09,295::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Reloading vm.conf from the shared storage domain MainThread::INFO::2016-10-03 12:39:09,296::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Trying to get a fresher copy of vm configuration from the OVF_STORE MainThread::WARNING::2016-10-03 12:39:16,928::ovf_store::107::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE MainThread::ERROR::2016-10-03 12:39:16,934::config::235::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf I have searched a bit and not really found a solution, and have come to the conclusion that i have made a mess of things, and am wondering if the best solution is to export the VMs, and reinstall everything then import them back? i am using remote NFS storage. if i try and add the hosted engine storage domain it says it is already registered. i have also upgraded and am now running oVirt Engine Version: 4.0.4.4-1.el7.centos hosts were installed using ovirt-node. currently at 3.10.0-327.28.3.el7.x86_64 if a fresh install is best, any advice / pointer to doc that explains best way to do this? i have not moved my most important server over to this cluster yet so i can take some downtime to reinstall. thanks! sam --------------05F3D0780062C6D37C1619A7 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> Hi, so i was running a 3.4 hosted engine two node setup on centos 6, had some disk issues so i tried to upgrade to centos 7 and follow the path 3.4 > 3.5 > 3.6 > 4.0. i screwed up dig time somewhere between 3.6 and 4.0, so i wiped the drives, installed a fresh 4.0.3, then created the database and restored the 3.6 engine backup before running engine-setup as per the docs. things seemed to work, but i have the the following issues / symptoms: - ovirt-ha-agent running 100% CPU on both nodes - messages in the UI that the Hosted Engine storage Domain isn't active and Failed to import the Hosted Engine Storage Domain - hosted engine is not visible in the UI and the following repeating in the agent.log: MainThread::INFO::2016-10-03 12:38:27,718::hosted_engine::461::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Current state EngineUp (score: 3400) MainThread::INFO::2016-10-03 12:38:27,720::hosted_engine::466::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Best remote host vmhost1.oracool.net (id: 1, score: 3400) MainThread::INFO::2016-10-03 12:38:37,979::states::421::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Engine vm running on localhost MainThread::INFO::2016-10-03 12:38:37,985::hosted_engine::612::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM MainThread::INFO::2016-10-03 12:38:45,645::hosted_engine::639::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2016-10-03 12:38:45,647::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2016-10-03 12:39:00,543::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2016-10-03 12:39:00,562::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain MainThread::INFO::2016-10-03 12:39:01,235::hosted_engine::666::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images MainThread::INFO::2016-10-03 12:39:01,236::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images MainThread::INFO::2016-10-03 12:39:09,295::hosted_engine::669::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Reloading vm.conf from the shared storage domain MainThread::INFO::2016-10-03 12:39:09,296::config::206::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Trying to get a fresher copy of vm configuration from the OVF_STORE MainThread::WARNING::2016-10-03 12:39:16,928::ovf_store::107::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE MainThread::ERROR::2016-10-03 12:39:16,934::config::235::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf I have searched a bit and not really found a solution, and have come to the conclusion that i have made a mess of things, and am wondering if the best solution is to export the VMs, and reinstall everything then import them back? i am using remote NFS storage. if i try and add the hosted engine storage domain it says it is already registered. i have also upgraded and am now running oVirt Engine Version: 4.0.4.4-1.el7.centos hosts were installed using ovirt-node. currently at 3.10.0-327.28.3.el7.x86_64 if a fresh install is best, any advice / pointer to doc that explains best way to do this? i have not moved my most important server over to this cluster yet so i can take some downtime to reinstall. thanks! sam </body> </html> --------------05F3D0780062C6D37C1619A7--

3 2

[Feedback Needed] oVirt metrics store
by Shirly Radco 23 May '19

23 May '19

Hi, In the latest oVirt 4.3.z and 4.2.z releases we have simplified the metrics store installation immensely and updated it to be based on OpenShift 3.11, with Elasticsearch and Kibana 5. The oVirt metrics store provides monitoring for the oVirt hosts and vms, for both metrics and logs with pre-built example dashboards and self service visualizations in Kibana. 5 steps for installing oVirt metrics store: 1. Update variables files 2. Run 1st ansible playbook -> creates the vms 3. Run 2nd ansible playbook -> deploys the metrics store on the created vms 4. Run 3rd ansible playbook -> configures the data collectors on the oVirt hosts and engine 4. Load example dashboards to Kibana In addition, we have replaced Fluentd with Rsyslog in 4.3.z hosts, for better performance. I invite you to try it out and give us feedback. - Installation guide - https://ovirt.org/documentation/metrics-install-guide/metrics_store_install… - User guide - https://ovirt.org/documentation/metrics-user-guide/metrics-user-guide.html - Short installation video <https://dms.licdn.com/playback/C5605AQHirxO3r-3OFw/14a815a1904945ac83774d7e…> Best regards, -- SHIRLY RADCO BI SENIOR SOFTWARE ENGINEER Red Hat Israel <https://www.redhat.com/> <https://red.ht/sig>

1 0

enabling vPMU for virtual machine
by Jarosław Prokopowski 23 May '19

23 May '19

Hi Guys, Is there a way to enable vPMU for VM in oVirt? I already enabled Nested Virtualization on hosts and installed vdsm-hook-nestedvt. In libvirt it is possible to set <cpu mode='host-passthrough'/> This blog post shows how to set it in libvirt xml vm config: https://stackoverflow.com/questions/46490857/how-to-enable-pmu-in-kvm-guest A clue from Red Hat: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/ht… Maybe something with cpuflags? https://mpolednik.github.io/2017/07/31/cpuflags-ovirt-hook-cpu-features/ Thanks Jarek

1 1

"Upgrade" from oVirt to RHV
by Vinícius Ferrão 23 May '19

23 May '19

Hello, I would like to know if there’s a supported path to move from oVirt to RHV. oVirt is running on version 4.3.0.4-1.el7. RHV would be version 4.3.3.7-0.1.el7. I was thinking in reinstalling a host with RHV 4.3 and adding it to the oVirt HE. Move all the VM’s to the RHV host and than do the same process. After it install a new RHVH-M to remove the oVirt Engine. Not sure if this would work or not. Thanks,

4 3

Unable to deploy Hyperconverged Engine Node - v4.3.3
by anonmix＠gmail.com 23 May '19

23 May '19

Hi everyone, I am trying a Gluster Hyperconvergence deployment where the Gluster part has been completed successfully. All hosts are Centos 7.6.1810 (fresh install) and two HP DL20 G9 (for VM's) and one HP 120 G7 (which hosts the Gluster arbiter volumes). Unfortunately I am unable to deploy the Engine, both CLI and GUI approaches fail with the error below. On first sight it looks similar to https://lists.ovirt.org/pipermail/users/2018-March/087802.html but I've configured a static IP (same subnet as the host), no DHCP. I also tried to force ipv4 with "/usr/sbin/ovirt-hosted-engine-setup --4" but the very same error was thrown in every case when trying to deploy the engine: [ ERROR ] fatal: [localhost]: FAILED! => {"ansible_facts": {"ovirt_hosts": [{"address": "sub.sub.domain.tld", "affinity_labels": [], "auto_numa_status": "unknown", "certificate": {"organization": "sub.domain.tld", "subject": "O=sub.domain.tld,CN=sub.sub.domain.tld"}, "cluster": {"href": "/ovirt-engine/api/clusters/f083f056-74fd-11e9-bba9-00163e522076", "id": "f083f056-74fd-11e9-bba9-00163e522076"}, "comment": "", "cpu": {"speed": 0.0, "topology": {}}, "device_passthrough": {"enabled": false}, "devices": [], "external_network_provider_configurations": [], "external_status": "ok", "hardware_information": {"supported_rng_sources": []}, "hooks": [], "href": "/ovirt-engine/api/hosts/dc4f5c15-4989-4454-ba46-3bd600796b69", "id": "dc4f5c15-4989-4454-ba46-3bd600796b69", "katello_errata": [], "kdump_status": "unknown", "ksm": {"enabled": false}, "max_scheduling_memory": 0, "memory": 0, "name": "sub.sub.domain.tld", "network_attachments": [], "nics": [], "numa_nodes": [], "numa_supported": fals e, "os": {"custom_kernel_cmdline": ""}, "permissions": [], "port": 54321, "power_management": {"automatic_pm_enabled": true, "enabled": false, "kdump_detection": true, "pm_proxies": []}, "protocol": "stomp", "se_linux": {}, "spm": {"priority": 5, "status": "none"}, "ssh": {"fingerprint": "SHA256:L8YyAMcxLFJEng+CoDympwkpMwoagcBafI4fpLP4Kk0", "port": 22}, "statistics": [], "status": "install_failed", "storage_connection_extensions": [], "summary": {"total": 0}, "tags": [], "transparent_huge_pages": {"enabled": false}, "type": "rhel", "unmanaged_networks": [], "update_available": false, "vgpu_placement": "consolidated"}]}, "attempts": 120, "changed": false} Unfortunately I don't really have an idea where to check for what considering the error message. The to be deployed engine VM gets listed as KVM VM, is accessible through the bridge and seems to be started up completely, I can even access the Engine web interface (engine01.sub.domain.tld/ovirt-engine). In /var/log/messages the following can be found ... "May 13 12:40:55 host ansible-async_wrapper.py: 15505 still running (86015) May 13 12:40:57 host python: ansible-ovirt_host_facts Invoked with all_content=False pattern=name=sub.sub.domain.tld fetch_nested=False nested_attributes=[] auth={'timeout': 0, 'url': 'https://engine01.sub.domain.tld/ovirt-engine/api', 'insecure': True, 'kerberos': False, 'compress': True, 'headers': None, 'token': '8s-vELzQqNTR6l7-KRuqnYLE3sVwVWU5NxiNWzc-s2CllaQG_5YZ32fCFkVsAgwEyLWjPIOxvyS-_4js-VYFFQ', 'ca_file': None}" ... and after 120 attempts Ansible stops and fails with a deployment error. When re-trying after removing the VM and ovirt-hosted-engine-cleanup the very same error is thrown. What is a bit weird is this entry in /var/log/ovirt-hosted-engine-setup/ ./engine-logs-2019-05-13T12:26:20Z/ovirt-engine/engine.log:2019-05-13 12:34:40,369Z ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (EE-ManagedThreadFactory-engine-Thread-1) [12746235] SSH error running command root(a)sub.sub.domain.tld:'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:True': RuntimeException: Unexpected error during execution: bash: /tmp/ovirt-pTVEEzlb8b/ovirt-host-deploy: Permission denied ./engine-logs-2019-05-13T12:26:20Z/ovirt-engine/engine.log:2019-05-13 12:34:40,406Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [12746235] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during installation of Host sub.sub.domain.tld: Unexpected error during execution: bash: /tmp/ovirt-pTVEEzlb8b/ovirt-host-deploy: Permission denied Could that be the cause and how can I fix it? What else do you guys need? Thanks in advance, Martin

4 6

Engine restore errors out on "Wait for OVF_STORE disk content"
by Andreas Elvers 22 May '19

22 May '19

Hello, when trying to deploy the engine using "hosted-engine --deploy --restore-from-file=myenginebackup" the ansible playbook errors out at [ INFO ] TASK [ovirt.hosted_engine_setup : Trigger hosted engine OVF update and enable the serial console] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Wait until OVF update finishes] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Parse OVF_STORE disk list] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Check OVF_STORE volume status] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Wait for OVF_STORE disk content] [ ERROR ] {u'_ansible_parsed': True, u'stderr_lines': [u'20+0 records in', u'20+0 records out', u'10240 bytes (10 kB) copied, 0.000141645 s, 72.3 MB/s', u'tar: ebb09b0e-2d03-40f0-8fa4-c40b18612a54.ovf: Not found in archive', u'tar: Exiting with failure status due to previous errors'], u'changed': True, u'end': u'2019-05-08 15:21:47.595195', u'_ansible_item_label': {u'image_id': u'65fd6c57-033c-4c95-87c1-b16c26e4bc98', u'name': u'OVF_STORE', u'id': u'9ff8b389-5e24-4166-9842-f1d6104b662b'}, u'stdout': u'', u'failed': True, u'_ansible_item_result': True, u'msg': u'non-zero return code', u'rc': 2, u'start': u'2019-05-08 15:21:46.906877', u'attempts': 12, u'cmd': u"vdsm-client Image prepare storagepoolID=597f329c-0296-03af-0369-000000000139 storagedomainID=f708ced4-e339-4d02-a07f-78f1a30fc2a8 imageID=9ff8b389-5e24-4166-9842-f1d6104b662b volumeID=65fd6c57-033c-4c95-87c1-b16c26e4bc98 | grep path | awk '{ print $2 }' | xargs -I{} sudo -u vdsm dd if={} | tar -tvf - ebb09b0e-2d03-40f0-8fa4-c40 b18612a54.ovf", u'item': {u'image_id': u'65fd6c57-033c-4c95-87c1-b16c26e4bc98', u'name': u'OVF_STORE', u'id': u'9ff8b389-5e24-4166-9842-f1d6104b662b'}, u'delta': u'0:00:00.688318', u'invocation': {u'module_args': {u'warn': False, u'executable': None, u'_uses_shell': True, u'_raw_params': u"vdsm-client Image prepare storagepoolID=597f329c-0296-03af-0369-000000000139 storagedomainID=f708ced4-e339-4d02-a07f-78f1a30fc2a8 imageID=9ff8b389-5e24-4166-9842-f1d6104b662b volumeID=65fd6c57-033c-4c95-87c1-b16c26e4bc98 | grep path | awk '{ print $2 }' | xargs -I{} sudo -u vdsm dd if={} | tar -tvf - ebb09b0e-2d03-40f0-8fa4-c40b18612a54.ovf", u'removes': None, u'argv': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'stdout_lines': [], u'stderr': u'20+0 records in\n20+0 records out\n10240 bytes (10 kB) copied, 0.000141645 s, 72.3 MB/s\ntar: ebb09b0e-2d03-40f0-8fa4-c40b18612a54.ovf: Not found in archive\ntar: Exiting with failure status due to previous errors', u'_ansible_no_log': False} [ ERROR ] {u'_ansible_parsed': True, u'stderr_lines': [u'20+0 records in', u'20+0 records out', u'10240 bytes (10 kB) copied, 0.000140541 s, 72.9 MB/s', u'tar: ebb09b0e-2d03-40f0-8fa4-c40b18612a54.ovf: Not found in archive', u'tar: Exiting with failure status due to previous errors'], u'changed': True, u'end': u'2019-05-08 15:24:01.387469', u'_ansible_item_label': {u'image_id': u'dacf9ad8-77b9-4205-8ca2-d6877627ad4a', u'name': u'OVF_STORE', u'id': u'8691076a-8e45-4429-a18a-5faebef866cc'}, u'stdout': u'', u'failed': True, u'_ansible_item_result': True, u'msg': u'non-zero return code', u'rc': 2, u'start': u'2019-05-08 15:24:00.660309', u'attempts': 12, u'cmd': u"vdsm-client Image prepare storagepoolID=597f329c-0296-03af-0369-000000000139 storagedomainID=f708ced4-e339-4d02-a07f-78f1a30fc2a8 imageID=8691076a-8e45-4429-a18a-5faebef866cc volumeID=dacf9ad8-77b9-4205-8ca2-d6877627ad4a | grep path | awk '{ print $2 }' | xargs -I{} sudo -u vdsm dd if={} | tar -tvf - ebb09b0e-2d03-40f0-8fa4-c40 b18612a54.ovf", u'item': {u'image_id': u'dacf9ad8-77b9-4205-8ca2-d6877627ad4a', u'name': u'OVF_STORE', u'id': u'8691076a-8e45-4429-a18a-5faebef866cc'}, u'delta': u'0:00:00.727160', u'invocation': {u'module_args': {u'warn': False, u'executable': None, u'_uses_shell': True, u'_raw_params': u"vdsm-client Image prepare storagepoolID=597f329c-0296-03af-0369-000000000139 storagedomainID=f708ced4-e339-4d02-a07f-78f1a30fc2a8 imageID=8691076a-8e45-4429-a18a-5faebef866cc volumeID=dacf9ad8-77b9-4205-8ca2-d6877627ad4a | grep path | awk '{ print $2 }' | xargs -I{} sudo -u vdsm dd if={} | tar -tvf - ebb09b0e-2d03-40f0-8fa4-c40b18612a54.ovf", u'removes': None, u'argv': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'stdout_lines': [], u'stderr': u'20+0 records in\n20+0 records out\n10240 bytes (10 kB) copied, 0.000140541 s, 72.9 MB/s\ntar: ebb09b0e-2d03-40f0-8fa4-c40b18612a54.ovf: Not found in archive\ntar: Exiting with failure status due to previous errors', u'_ansible_no_log': False} [ ERROR ] Failed to execute stage 'Closing up': Failed executing ansible-playbook I tried twice. Same result. Should I retry? Is it safe to use the local hosted engine for starting stopping vms? I'm kind of headless for some days :-) Best regards.

3 11

ovirt 4.3.3.7 cannot create a gluster storage domain
by Strahil Nikolov 22 May '19

22 May '19

Hey guys, I have recently updated (yesterday) my platform to latest available (v4.3.3.7) and upgraded to gluster v6.1 .The setup is hyperconverged 3 node cluster with ovirt1/gluster1 & ovirt2/gluster2 as replica nodes (glusterX is for gluster communication) while ovirt3 is the arbiter. Today I have tried to add new domain storages but they fail with the following: 2019-05-16 10:15:21,296+0300 INFO (jsonrpc/2) [vdsm.api] FINISH createStorageDomain error=Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.0138582 s, 0.0 kB/s\n" from=::ffff:192.168.1.2,43864, flow_id=4a54578a, task_id=d2535d0f-c7f7-4f31-a10f-704923ce1790 (api:52) 2019-05-16 10:15:21,296+0300 ERROR (jsonrpc/2) [storage.TaskManager.Task] (Task='d2535d0f-c7f7-4f31-a10f-704923ce1790') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in createStorageDomain File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2614, in createStorageDomain storageType, domVersion, block_size, alignment) File "/usr/lib/python2.7/site-packages/vdsm/storage/nfsSD.py", line 106, in create block_size) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 466, in _prepareMetadata cls.format_external_leases(sdUUID, xleases_path) File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line 1255, in format_external_leases xlease.format_index(lockspace, backend) File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 681, in format_index index.dump(file) File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 843, in dump file.pwrite(INDEX_BASE, self._buf) File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 1076, in pwrite self._run(args, data=buf[:]) File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py", line 1093, in _run raise cmdutils.Error(args, rc, "[suppressed]", err) Error: Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.0138582 s, 0.0 kB/s\n" 2019-05-16 10:15:21,296+0300 INFO (jsonrpc/2) [storage.TaskManager.Task] (Task='d2535d0f-c7f7-4f31-a10f-704923ce1790') aborting: Task is aborted: u'Command [\'/usr/bin/dd\', \'iflag=fullblock\', u\'of=/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases\', \'oflag=direct,seek_bytes\', \'seek=1048576\', \'bs=256512\', \'count=1\', \'conv=notrunc,nocreat,fsync\'] failed with rc=1 out=\'[suppressed]\' err="/usr/bin/dd: error writing \'/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases\': Invalid argument\\n1+0 records in\\n0+0 records out\\n0 bytes (0 B) copied, 0.0138582 s, 0.0 kB/s\\n"' - code 100 (task:1181) 2019-05-16 10:15:21,297+0300 ERROR (jsonrpc/2) [storage.Dispatcher] FINISH createStorageDomain error=Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.0138582 s, 0.0 kB/s\n" (dispatcher:87) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper result = ctask.prepare(func, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper return m(self, *a, **kw) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1189, in prepare raise self.error Error: Command ['/usr/bin/dd', 'iflag=fullblock', u'of=/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases', 'oflag=direct,seek_bytes', 'seek=1048576', 'bs=256512', 'count=1', 'conv=notrunc,nocreat,fsync'] failed with rc=1 out='[suppressed]' err="/usr/bin/dd: error writing '/rhev/data-center/mnt/glusterSD/gluster1:_data__fast2/591d9b61-5c7d-4388-a6b7-ab03181dff8a/dom_md/xleases': Invalid argument\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 0.0138582 s, 0.0 kB/s\n" 2019-05-16 10:15:21,297+0300 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call StorageDomain.create failed (error 351) in 0.45 seconds (__init__:312) 2019-05-16 10:15:22,068+0300 INFO (jsonrpc/1) [vdsm.api] START disconnectStorageServer(domType=7, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'mnt_options': u'backup-volfile-servers=gluster2:ovirt3', u'id': u'7442e9ab-dc54-4b9a-95d9-5d98a1e81b05', u'connection': u'gluster1:/data_fast2', u'iqn': u'', u'user': u'', u'tpgt': u'1', u'ipv6_enabled': u'false', u'vfs_type': u'glusterfs', u'password': '********', u'port': u''}], options=None) from=::ffff:192.168.1.2,43864, flow_id=33ced9b2-cdd5-4147-a223-d0eb398a2daf, task_id=a9a8f90a-1603-40c6-a959-3cbff29d1d7b (api:48) 2019-05-16 10:15:22,068+0300 INFO (jsonrpc/1) [storage.Mount] unmounting /rhev/data-center/mnt/glusterSD/gluster1:_data__fast2 (mount:212) I have tested manually mounting and trying it again: [root@ovirt1 logs]# mount -t glusterfs -o backupvolfile-server=gluster2:ovirt3 gluster1:/data_fast2 /mnt [root@ovirt1 logs]# cd /mnt/ [root@ovirt1 mnt]# ll total 0 [root@ovirt1 mnt]# dd if=/dev/zero of=file bs=4M status=progress count=250 939524096 bytes (940 MB) copied, 8.145447 s, 115 MB/s 250+0 records in 250+0 records out 1048576000 bytes (1.0 GB) copied, 9.08347 s, 115 MB/s [root@ovirt1 mnt]# /usr/bin/dd iflag=fullblock of=file oflag=direct,seek_bytes seek=1048576 bs=256512 count=1 conv=notrunc,nocreat,fsync status=progress ^C0+0 records in 0+0 records out 0 bytes (0 B) copied, 46.5877 s, 0.0 kB/s Can someone give a hint ? Maybe it's related to gluster v6 ? Can someone test with older version of Gluster ? Best Regards,Strahil Nikolov

3 8