- Users - oVirt List Archives

NFS based - incremental backups
by Leo David 27 Sep '19

27 Sep '19

Hello Everyone, Ive been struggling since a while to find out a propper solution to have a full backup scenario for a production environment. In the past, we have used Proxmox, and the scheduled incremenral nfs based full vm backups is a thing that we really miss. As far as i know, at this point the only way to have a backup in oVirt / rhev is by using gluster geo-replication feature. This is nice, but as far as i know it lacks some important features: - ability to have incremental backups to restore vms from - ability to backup vms placed on different storage domains ( only one storage domain can be geo-replicated !!! some vms have disks on ssd volume, some on hdd, some on both) - need to setup an external 3 nodes gluster cluster ( although a workaround would be to have single bricks based volumes for a single instance ) I know we can create snaps, but they will die with the platform in a fail scenario, and neither they can be scheduled. We have found bacchus project that looked promising, although it had a pretty hassled way to achieve backups ( create snap, create vm from the snap, export vm to export domain, delete vm, delete snap - all in a scheduled fashion ) As a mention, Proxmox is incrementaly creating a tar archive of the vm disk content, and places it to an external network storage like nfs. This allowed us to backup/reatore both linux and windows vms very easily. Now, I know this have been discussed before, but i would like to know if there are at least any future plans to implement this feature in the next releases. Personally, i consider this a major, and quite decent feature to have with the platform, without the need to pay for 3rd party solutions that may or may not achieve the goal while adding extra pieces to the stack. Geo-replication is a good and nice feature, but in my oppinion it is not what a "backup domain" would be. Have a nice day, Leo

1 0

not able to import ova file
by Crazy Ayansh 27 Sep '19

27 Sep '19

Hi Team, I am getting below error message when i am trying to import a vm from ova method. [image: image.png] could anyone please help me out here. Hosted engine version:-4.2 Please let me know if any other info required from me. Thanks

1 1

4.2.7 to 4.3.5
by Demeter Tibor 27 Sep '19

27 Sep '19

Dear list members, I would like to upgrade my Centos 7.6 based ovirt 4.2 to 4.3.5. How is safe the upgrade ? How is stable the actual branch ? It seems to me, in 4.3.x there are a lot of cases caused by live migration, snapshot deletion, etc. Are these really existing problems? So, I'm a little bit afraid of the upgrade. It is a productive system, I don't want to make an outage. Thanks in advance, Regards, Tibor

2 1

oVirt and CentOS Stream
by Sandro Bonazzola 26 Sep '19

26 Sep '19

Progress cannot be made without change. As technologists, we recognize this every day. Most of the time, these changes are iterative: progressive additions of features to projects like oVirt. Sometimes those changes are small, and sometimes not. And that’s, of course, just talking about our project. But one of the biggest strengths of our community’s software is that we are not alone, and because of that, changes to other projects have ripple effects that can affect our own, even in positive ways. This week, our collaborators in the CentOS Project have announced a change <https://lists.centos.org/pipermail/centos-announce/2019-September/023449.ht…> in the way their software is released moving forward. Beyond this week release of CentOS Linux 8, the CentOS team has announced CentOS Stream, a rolling release distribution that will be the "midstream" between Fedora and Red Hat Enterprise Linux. To be clear, Fedora will remain as the first upstream of RHEL. But when RHEL branches off, CentOS will be upstream for ongoing work on those RHEL versions. This change gives public visibility into ongoing work on released RHEL, and a place for developers and community projects like ours to collaborate at that level. CentOS has been heading in the direction of Stream for quite some time. CentOS SIGs — the special interest groups for virtualization, storage, config management, and so on — have given our project a way to provide input into the CentOS platform. Now, though, we can actually provide direct improvements or fixes to CentOS Stream platform that will be beneficial to our project. Long-term, those project-specific changes can find their way into the next release of RHEL, providing smoother transitions for those users of our downstream projects. We expect the positive effects to extend beyond our own project’s ecosystem. Through CentOS Stream, developers will have early access to new features and content that are being built into the upcoming RHEL version. This will help to allow next-generation applications to have compatibility with future versions of RHEL. The benefits of these changes are clear for ecosystem developers working on projects such as ours, writing hardware drivers, or extending protocols for RHEL. As we push the innovations that start in Fedora through the new CentOS Stream, the community will have a clear vision of the future of Red Hat Enterprise Linux. For users of oVirt, we expect CentOS Stream to be the preferred upstream platform on which oVirt should be run, especially with the capability for our users to now contribute changes to our software and the community-built platform on which oVirt runs. -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Re: Change hostname of physical hosts under an oVirt and Gluster combination
by Strahil 26 Sep '19

26 Sep '19

Unassigned in 99% of the cases means engine cannot communicate with vdsm.service on the host. Check that vdsm.service & glusterd.service are running. Check gluster volume status for each volume. If they are OK - from UI just select activate. Best Regards, Strahil NikolovOn Sep 25, 2019 22:24, TomK <tomkcpr(a)mdevsys.com> wrote: > > On 9/25/2019 8:49 AM, TomK wrote: > > On 9/24/2019 10:55 AM, Sahina Bose wrote: > >> > >> > >> On Tue, Sep 24, 2019 at 6:38 PM TomK <tomkcpr(a)mdevsys.com > >> <mailto:tomkcpr@mdevsys.com>> wrote: > >> > >> Hey Sahina, > >> > >> Thanks very much. > >> > >> I've taken a quick glance and it does mention I need a third host > >> which > >> I don't have. > >> > >> However, since I wrote, I removed everything. Had to force remove > >> hosts > >> however the gluster setup and data in it stayed (Or so I'm thinking). > >> > >> I'll read the article more thoroughly however is there a specific > >> procedure for adding existing oVirt gluster volumes back into a > >> cluster? > >> > >> One additional question. Is the sequence below correct? > >> > >> 1) Create Data Center > >> 2) Create Cluster within the DC > >> > >> > >> When you create the cluster - there's an Import cluster option, if I > >> remember correctly. This should discover the peers and volumes and add > >> it to the engine > > > > Trying that now. On a side note, the autodiscovery box is hard to edit. > > > > So in the meantime, trying to up the version to take advantage of some > > new features and ensure Gluster 6 is fully supported. Not having luck > > with the oVirt UI upgrade at the moment ( Not a self hosted engine. It's > > on a separate VM. ): > > > > --> Processing Conflict: > > ovirt-engine-setup-plugin-ovirt-engine-4.3.5.5-1.el7.noarch conflicts > > ovirt-engine < 4.2.6 > > --> Finished Dependency Resolution > > Error: Package: ovirt-engine-ui-extensions-1.0.6-1.el7.noarch (ovirt-4.3) > > Requires: ovirt-engine-webadmin-portal >= 4.3 > > Installed: > > ovirt-engine-webadmin-portal-4.2.1.7-1.el7.centos.noarch (@ovirt-4.2) > > ovirt-engine-webadmin-portal = 4.2.1.7-1.el7.centos > > Error: ovirt-engine-setup-plugin-ovirt-engine conflicts with > > ovirt-engine-4.2.1.7-1.el7.centos.noarch > > You could try using --skip-broken to work around the problem > > You could try running: rpm -Va --nofiles --nodigest > > > > > > > > Cheers, > > TK > > > > Just upgraded to 4.3.5. > > What do I do if I can't remove a host from a cluster? Maintenance > window options Active and Maintenance are grayed out. Host is currently > showing as Unassigned with an exclamation mark next to it. Items listed > include: > > "Power Management is not configured for this Host. Enable Power Management" > "Gluster status is disconnected for this host. Restart Glusterd service" > "Host has no default route." > > > I try to modify some of this thinking maybe it will nudge the server out > of it's slumber but get this when trying to fix Power Management: > > > "Error while executing action: > mdskvm-p01.nix.mds.xyz: > Cannot edit Host. Host parameters cannot be modified while Host is > operational. > Please switch Host to Maintenance mode first." > > > Of course I can't put it into maintenance since it's grayed out. > > Anyway to remove this host and start fresh? > > Cheers, > TK > > > >> > >> 3) Create a Domain within the Cluster > >> 4) Add hosts to DC > >> 5) Import Gluster Volume > >> > >> > >> 4 & 5 not required if you change step 2 to import. > >> You are running into errors below because the engine sees the addition > >> of new hosts as gluster peer probing to a new cluster > >> > >> > >> > >> Some issues. I create a DC, add a Cluster then add the Host to it. > >> Then > >> I try to add another the second node but it says It's already in a > >> cluster. There is no other cluster and the second host is not listed > >> anywhere. > >> > >> So I try to remove the first host to res

2 1

Outage Notification: Jenkins and Resources Are Not Accessible
by Anton Marchukov 26 Sep '19

26 Sep '19

Hello All. Please note that oVirt services hosted at PHX datacenter are not accessible. Main services running there are: jenkins.ovirt.org resources.ovirt.org This looks to be network related and we are working with the relevant teams to resolve it. Anton. -- Anton Marchukov Associate Manager - RHV DevOps - Red Hat

1 1

Re: ovirtmgmt STP disabled - any reason behind that
by Strahil 26 Sep '19

26 Sep '19

Did I miss someone's reply on this one ? Best Regards, Strahil NikolovOn Sep 24, 2019 00:46, Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: > > Hi All, > > I recently got an issue and due to lack of debug time , I'm still not sure if it was a loop on the network. > Can someone clarify why the ovirtmgmt bridge has STP disabled ? Any reason behind that ? > > Also, what is the proper way to enable STP on those bridges ? > > For now , I have set highest priority of the STP on my router and I don't expect the issue to reoccur (if it really was a loop), but I want to avoid future issues like that. > > > Best Regards, > Strahil Nikolov

1 0

Re: Urgent help needed / Snapshot deletion failure
by Strahil 26 Sep '19

26 Sep '19

Is it really down ? Have you tried to migrate it to another host. I found out yesterday that one of my VMs was constantly reporting errors (relayed to snapshot delete) and the errors stopped when the system got migrated to another host. Best Regards, Strahil NikolovOn Sep 25, 2019 15:27, smirta(a)gmx.net wrote: > > Dear all we are desperate > > We have tried to delete a Snapshot and it hangs at merging the snapshots. We've found out that this is a known bug because the merge process is being called with wrong parameters according to the bug report https://bugzilla.redhat.com/show_bug.cgi?id=1601212. The Snapshot's Disks are in illegal state and the VM is locked. Since it is a very important VM for our non-profit organization, we need to have this machine back online as soon as possible. Is there a way to fix this without updating to qemu-kvm-ev-2.12.0? At least back to the state before the deletion would be fantastic. > > Our vdsm.log: > 2019-09-25 14:01:41,283+0200 WARN (vdsm.Scheduler) [Executor] Worker blocked: <Worker name=jsonrpc/0 running <Task <JsonRpcTask {'params': {u'topVolUUID': u'8a9f190f-2725-4535-a69d-c74e4e57d372', u'vmID': u'1422899a-2151-4d5d-9d66-e74f19084542', u'drive': {u'imageID': u'eb2bce92-e758-4bea-93fa-02a56574b932', u'volumeID': u'8a9f190f-2725-4535-a69d-c74e4e57d372', u'domainID': u'022f39ee-eeb8-4b51-9549-9d7e3c88d4a8', u'poolID': u'00000001-0001-0001-0001-000000000307'}, u'bandwidth': u'0', u'jobUUID': u'c3fdc4a9-9d6d-424a-b9df-b96be5622e0a', u'baseVolUUID': u'3e15121b-0795-4056-bafe-448068c9ec71'}, 'jsonrpc': '2.0', 'method': u'VM.merge', 'id': u'9cd540b7-a32f-4f95-9fe2-9ce70d5b6478'} at 0x7f674fec5710> timeout=60, duration=6420 at 0x7f674fec58d0> task#=1896299 at 0x7f674c06ae90>, traceback: > File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap > self.__bootstrap_inner() > File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner > self.run() > File: "/usr/lib64/python2.7/threading.py", line 765, in run > self.__target(*self.__args, **self.__kwargs) > File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194, in run > ret = func(*args, **kwargs) > File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run > self._execute_task() > File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task > task() > File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__ > self._callable() > File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 523, in __call__ > self._handler(self._ctx, self._req) > File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 566, in _serveRequest > response = self._handle_request(req, ctx) > File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 606, in _handle_request > res = method(**params) > File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 197, in _dynamicMethod > result = fn(*methodArgs) > File: "<string>", line 2, in merge > File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method > ret = func(*args, **kwargs) > File: "<string>", line 2, in merge > File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 122, in method > ret = func(*args, **kwargs) > File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 739, in merge > drive, baseVolUUID, topVolUUID, bandwidth, jobUUID) > File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 6041, in merge > self.updateVmJobs() > File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5818, in updateVmJobs > self._vmJobs = self.queryBlockJobs() > File: "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5832, in queryBlockJobs > with self._jobsLock: > File: "/usr/lib/python2.7/site-packages/pthreading.py", line 60, in __enter__ > self.acquire() > File: "/usr/lib/python2.7/site-packages/pthreading.py", line 68, in acquire > rc = self.lock() if blocking else self.trylock() > File: "/usr/lib/python2.7/site-packages/pthread.py", line 96, in lock > return _libpthread.pthread_mutex_lock(self._mutex) (executor:363) > > > Kind regards > Simon > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LW7BNZBO5AAGN…

1 0

Change hostname of physical hosts under an oVirt and Gluster combination
by TomK 26 Sep '19

26 Sep '19

Hey All, Would like to change the hostname of hosts within a gluster volume that's been added to oVirt. I also need to readd the bricks to the gluster volume under the new hostname. What's the recommended approach to do so when all this is added to oVirt? -- Thx, TK.

2 7

Managed Block Storage: ceph detach_volume failing after migration
by Dan Poltawski 25 Sep '19

25 Sep '19

On ovirt 4.3.5 we are seeing various problems related to the rbd device staying mapped after a guest has been live migrated. This causes problems migrating the guest back, as well as rebooting the guest when it starts back up on the original host. The error returned is ‘nrbd: unmap failed: (16) Device or resource busy’. I’ve pasted the full vdsm log below. As far as I can tell this isn’t happening 100% of the time, and seems to be more prevalent on busy guests. (Not sure if I should create a bug for this, so thought I’d start here first) Thanks, Dan Sep 24 19:26:18 mario vdsm[5485]: ERROR FINISH detach_volume error=Managed Volume Helper failed.: ('Error executing helper: Command [\'/usr/libexec/vdsm/managedvolume-helper\', \'detach\'] failed with rc=1 out=\'\' err=\'oslo.privsep.daemon: Running privsep helper: [\\\'sudo\\\', \\\'privsep-helper\\\', \\\'--privsep_context\\\', \\\'os_brick.privileged.default\\\', \\\'--privsep_sock_path\\\', \\\'/tmp/tmptQzb10/privsep.sock\\\']\\noslo.privsep.daemon: Spawned new privsep daemon via rootwrap\\noslo.privsep.daemon: privsep daemon starting\\noslo.privsep.daemon: privsep process running with uid/gid: 0/0\\noslo.privsep.daemon: privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none\\noslo.privsep.daemon: privsep daemon running as pid 76076\\nTraceback (most recent call last):\\n File "/usr/libexec/vdsm/managedvolume-helper", line 154, in <module>\\n sys.exit(main(sys.argv[1:]))\\n File "/usr/libexec/vdsm/managedvolume-helper", line 77, in main\\n args.command(args)\\n File "/usr/libexec/vdsm/managedvolume-helper", line 149, in detach\\n ignore_errors=False)\\n File "/usr/lib/python2.7/site-packages/vdsm/storage/nos_brick.py", line 121, in disconnect_volume\\n run_as_root=True)\\n File "/usr/lib/python2.7/site-packages/os_brick/executor.py", line 52, in _execute\\n result = self.__execute(*args, **kwargs)\\n File "/usr/lib/python2.7/site-packages/os_brick/privileged/rootwrap.py", line 169, in execute\\n return execute_root(*cmd, **kwargs)\\n File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 241, in _wrap\\n return self.channel.remote_call(name, args, kwargs)\\n File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 203, in remote_call\\n raise exc_type(*result[2])\\noslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.\\nCommand: rbd unmap /dev/rbd/rbd/volume-0e8c1056-45d6-4740-934d-eb07a9f73160 --conf /tmp/brickrbd_LCKezP --id ovirt --mon_host 172.16.10.13:3300 --mon_host 172.16.10.14:3300 --mon_host 172.16.10.12:6789\\nExit code: 16\\nStdout: u\\\'\\\'\\nStderr: u\\\'rbd: sysfs write failed\\\\nrbd: unmap failed: (16) Device or resource busy\\\\n\\\'\\n\'',)#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 124, in method#012 ret = func(*args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1766, in detach_volume#012 return managedvolume.detach_volume(vol_id)#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/managedvolume.py", line 67, in wrapper#012 return func(*args, **kwargs)#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/managedvolume.py", line 135, in detach_volume#012 run_helper("detach", vol_info)#012 File "/usr/lib/python2.7/site-packages/vdsm/storage/managedvolume.py", line 179, in run_helper#012 sub_cmd, cmd_input=cmd_input)#012 File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012 return callMethod()#012 File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in managedvolume_run_helper#012 File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod#012 raise convert_to_error(kind, result)#012ManagedVolumeHelperFailed: Managed Volume Helper failed.: ('Error executing helper: Command [\'/usr/libexec/vdsm/managedvolume-helper\', \'detach\'] failed with rc=1 out=\'\' err=\'oslo.privsep.daemon: Running privsep helper: [\\\'sudo\\\', \\\'privsep-helper\\\', \\\'--privsep_context\\\', \\\'os_brick.privileged.default\\\', \\\'--privsep_sock_path\\\', \\\'/tmp/tmptQzb10/privsep.sock\\\']\\noslo.privsep.daemon: Spawned new privsep daemon via rootwrap\\noslo.privsep.daemon: privsep daemon starting\\noslo.privsep.daemon: privsep process running with uid/gid: 0/0\\noslo.privsep.daemon: privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none\\noslo.privsep.daemon: privsep daemon running as pid 76076\\nTraceback (most recent call last):\\n File "/usr/libexec/vdsm/managedvolume-helper", line 154, in <module>\\n sys.exit(main(sys.argv[1:]))\\n File "/usr/libexec/vdsm/managedvolume-helper", line 77, in main\\n args.command(args)\\n File "/usr/libexec/vdsm/managedvolume-helper", line 149, in detach\\n ignore_errors=False)\\n File "/usr/lib/python2.7/site-packages/vdsm/storage/nos_brick.py", line 121, in disconnect_volume\\n run_as_root=True)\\n File "/usr/lib/python2.7/site-packages/os_brick/executor.py", line 52, in _execute\\n result = self.__execute(*args, **kwargs)\\n File "/usr/lib/python2.7/site-packages/os_brick/privileged/rootwrap.py", line 169, in execute\\n return execute_root(*cmd, **kwargs)\\n File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 241, in _wrap\\n return self.channel.remote_call(name, args, kwargs)\\n File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 203, in remote_call\\n raise exc_type(*result[2])\\noslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.\\nCommand: rbd unmap /dev/rbd/rbd/volume-0e8c1056-45d6-4740-934d-eb07a9f73160 --conf /tmp/brickrbd_LCKezP --id ovirt --mon_host 172.16.10.13:3300 --mon_host 172.16.10.14:3300 --mon_host 172.16.10.12:6789\\nExit code: 16\\nStdout: u\\\'\\\'\\nStderr: u\\\'rbd: sysfs write failed\\\\nrbd: unmap failed: (16) Device or resource busy\\\\n\\\'\\n\'',) ________________________________ The Networking People (TNP) Limited. Registered office: Network House, Caton Rd, Lancaster, LA1 3PE. Registered in England & Wales with company number: 07667393 This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

4 4