April 2016 - Users - oVirt List Archives

Re: [ovirt-users] Unable to power off the vm
by Budur Nagaraju 29 Apr '16

29 Apr '16

Not able to access through console ,SSH, even the migration option is not getting highlighted unable to perform any actions. To reboot host I need to migrate remaining vms to other host , that is time consuming. Any commands to kill the process without rebooting the host? On Apr 28, 2016 5:42 PM, "Michal Skrivanek" <michal.skrivanek(a)redhat.com> wrote: On 28 Apr 2016, at 14:11, Budur Nagaraju <nbudoor(a)gmail.com> wrote: Earlier it was working ,now not able to power on/off shutdown. deploy in another host etc. I don’t mean in ovirt, I mean the guest itself. Can you get to the console? Can you ssh to that guest? Does it do anythign? if so it might be worth trying to save it (e.g. migrate), if not, just kill it from the host…or migrate everything else away and reboot the host On Apr 28, 2016 5:38 PM, "Michal Skrivanek" <michal.skrivanek(a)redhat.com> wrote: > > On 28 Apr 2016, at 14:01, Budur Nagaraju <nbudoor(a)gmail.com> wrote: > > ovirt node is having 50vms and one VM is having issues, by restarting > libvirt will the other vms get affect? And am not getting the option to > delete. > > > it will not affect the running VMs, they will keep running > again, does that one VM actually work? > > On Apr 28, 2016 5:26 PM, "Michal Skrivanek" <michal.skrivanek(a)redhat.com> > wrote: > >> >> > On 28 Apr 2016, at 13:49, Budur Nagaraju <nbudoor(a)gmail.com> wrote: >> > >> > Any commands to check the same ? >> >> so does the VM actually work? >> what’s the status of the process? >> >> if it works, restart libvirtd (that will induce a vdsm restart as well), >> and check if it makes any difference. If not then I guess you’re out of >> luck and you can try to kill the qemu process yourself…or reboot the box >> >> > On Apr 28, 2016 5:10 PM, "Michal Skrivanek" < >> michal.skrivanek(a)redhat.com> >> > wrote: >> > >> >> >> >>> On 28 Apr 2016, at 10:03, Budur Nagaraju <nbudoor(a)gmail.com> wrote: >> >>> >> >>> HI >> >>> >> >>> One of the vm is showing "?" and unable to perform any actions below >> >> are the logs ,let me know is there any ways to bring it back ? >> >> >> >> then it’s probably broken in lower layers. check/add vdsm.log from that >> >> period, but it is likely that libvirt lost control over the qemu >> process. >> >> You may want to check that particular qemu process if it is alright or >> .g >> >> >

2 2

audit_log table performance tuning
by Marina Kalinin 28 Apr '16

28 Apr '16

Hi, Any suggestions or maybe already available features in the pipeline for tuning the database, and specifically the audit_log table? The problem today is that with multiple applications accessing the engine through the RestAPI, especially deployments with CloudForms, create huge amount of login records in the audit_table. Which, in turns, consumes most of the available memory on the machine running the engine and the database and results in a terrible performance of engine and inaccessible Web UI. The solution today is to delete those records from the table [1]: => delete from audit_log where message like '%logged%'; Are there any current tunings we can apply to the database? And if not - do we have any RFEs on limiting the records entered to the database or a way to delete/filter those records somehow from the WebUI? All I could find was RFE#1120659 [2], but it does not describe the exact issue. -- Thanks, Marina. [1] https://access.redhat.com/solutions/2110011 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1120659

5 7

Re: [ovirt-users] vms in paused state
by Bill James 28 Apr '16

28 Apr '16

trying again with smaller attachments... On 04/28/2016 10:40 AM, Bill James wrote: > thank you for response. > I bold-ed the ones that are listed as "paused". > > > [root@ovirt1 test vdsm]# virsh -r list --all > Id Name State > ---------------------------------------------------- > * 2 puppet.test.j2noc.com running* > * 4 sftp2.test.j2noc.com running* > * 5 oct.test.j2noc.com running* > * 6 sftp2.dev.j2noc.com running* > * 10 darmaster1.test.j2noc.com running* > * 14 api1.test.j2noc.com running* > 25 ftp1.frb.test.j2noc.com running > 26 auto7.test.j2noc.com running > * 32 epaymv02.j2noc.com running* > 34 media2.frb.test.j2noc.com running > 36 auto2.j2noc.com running > 44 nfs.testhvy2.colo.j2noc.com running > * 53 billapp-zuma1.dev.j2noc.com running* > * 54 billing-ci.dev.j2noc.com running* > 60 log2.test.j2noc.com running > 63 log1.test.j2noc.com running > * 69 sonar.dev.j2noc.com running* > * 73 billapp-ui1.dev.j2noc.com running* > * 74 billappvm01.dev.j2noc.com running* > 75 db2.frb.test.j2noc.com running > 83 billapp-ui1.test.j2noc.com running > * 84 epayvm01.test.j2noc.com running* > * 87 billappvm01.test.j2noc.com running* > * 89 etapi1.test.j2noc.com running* > * 93 billapp-zuma2.test.j2noc.com running* > * 94 git.dev.j2noc.com running* > > Yes I did "systemctl restart libvirtd" which apparently also restart vdsm? > > > Looks like problem started around 2016-04-17 20:19:34,822, based on > engine.log attached. > There's a lot of vdsm logs! > > fyi, the storage domain for these Vms is a "local" nfs share, > 7e566f55-e060-47b7-bfa4-ac3c48d70dda. > > attached more logs. > > > On 04/28/2016 12:53 AM, Michal Skrivanek wrote: >>> On 27 Apr 2016, at 19:16, Bill James<bill.james(a)j2.com> wrote: >>> >>> virsh # list --all >>> error: failed to connect to the hypervisor >>> error: no valid connection >>> error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory >>> >> you need to run virsh in read-only mode >> virsh -r list —all >> >>> [root@ovirt1 test vdsm]# systemctl status libvirtd >>> ● libvirtd.service - Virtualization daemon >>> Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) >>> Drop-In: /etc/systemd/system/libvirtd.service.d >>> └─unlimited-core.conf >>> Active: active (running) since Thu 2016-04-21 16:00:03 PDT; 5 days ago >>> >>> >>> tried systemctl restart libvirtd. >>> No change. >>> >>> Attached vdsm.log and supervdsm.log. >>> >>> >>> [root@ovirt1 test vdsm]# systemctl status vdsmd >>> ● vdsmd.service - Virtual Desktop Server Manager >>> Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled) >>> Active: active (running) since Wed 2016-04-27 10:09:14 PDT; 3min 46s ago >>> >>> >>> vdsm-4.17.18-0.el7.centos.noarch >> the vdsm.log attach is good, but it’s too short interval, it only shows recovery(vdsm restart) phase when the VMs are identified as paused….can you add earlier logs? Did you restart vdsm yourself or did it crash? >> >> >>> libvirt-daemon-1.2.17-13.el7_2.4.x86_64 >>> >>> >>> Thanks. >>> >>> >>> On 04/26/2016 11:35 PM, Michal Skrivanek wrote: >>>>> On 27 Apr 2016, at 02:04, Nir Soffer<nsoffer(a)redhat.com> wrote: >>>>> >>>>> jjOn Wed, Apr 27, 2016 at 2:03 AM, Bill James<bill.james(a)j2.com> wrote: >>>>>> I have a hardware node that has 26 VMs. >>>>>> 9 are listed as "running", 17 are listed as "paused". >>>>>> >>>>>> In truth all VMs are up and running fine. >>>>>> >>>>>> I tried telling the db they are up: >>>>>> >>>>>> engine=> update vm_dynamic set status = 1 where vm_guid =(select >>>>>> vm_guid from vm_static where vm_name = 'api1.test.j2noc.com'); >>>>>> >>>>>> GUI then shows it up for a short while, >>>>>> >>>>>> then puts it back in paused state. >>>>>> >>>>>> 2016-04-26 15:16:46,095 INFO [org.ovirt.engine.core.vdsbroker.VmAnalyzer] >>>>>> (DefaultQuartzScheduler_Worker-16) [157cc21e] VM '242ca0af-4ab2-4dd6-b515-5 >>>>>> d435e6452c4'(api1.test.j2noc.com) moved from 'Up' --> 'Paused' >>>>>> 2016-04-26 15:16:46,221 INFO [org.ovirt.engine.core.dal.dbbroker.auditlogh >>>>>> andling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) [157cc21e] Cor >>>>>> relation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM api1. >>>>>> test.j2noc.com has been paused. >>>>>> >>>>>> >>>>>> Why does the engine think the VMs are paused? >>>>>> Attached engine.log. >>>>>> >>>>>> I can fix the problem by powering off the VM then starting it back up. >>>>>> But the VM is working fine! How do I get ovirt to realize that? >>>>> If this is an issue in engine, restarting engine may fix this. >>>>> but having this problem only with one node, I don't think this is the issue. >>>>> >>>>> If this is an issue in vdsm, restarting vdsm may fix this. >>>>> >>>>> If this does not help, maybe this is libvirt issue? did you try to check vm >>>>> status using virsh? >>>> this looks more likely as it seems such status is being reported >>>> logs would help, vdsm.log at the very least. >>>> >>>>> If virsh thinks that the vms are paused, you can try to restart libvirtd. >>>>> >>>>> Please file a bug about this in any case with engine and vdsm logs. >>>>> >>>>> Adding Michal in case he has better idea how to proceed. >>>>> >>>>> Nir >>> Cloud Services for Businesswww.j2.com >>> j2 | eFax | eVoice | FuseMail | Campaigner | KeepItSafe | Onebox >>> >>> >>> This email, its contents and attachments contain information from j2 Global, Inc. and/or its affiliates which may be privileged, confidential or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you are not an addressee, any disclosure, copy, distribution, or use of the contents of this message is prohibited. If you have received this email in error please notify the sender by reply e-mail and delete the original message and any copies. (c) 2015 j2 Global, Inc. All rights reserved. eFax, eVoice, Campaigner, FuseMail, KeepItSafe, and Onebox are registered trademarks of j2 Global, Inc. and its affiliates. >>> <supervdsm.log.gz><vdsm.log.gz>_______________________________________________ >>> Users mailing list >>> Users(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >

1 0

node unresponsive after reboot
by Campbell McLeay 28 Apr '16

28 Apr '16

Hi, I have a two node + engine ovirt setup, and I was having problems doing a live migration between nodes. I looked in the vdsm logs and noticed selinux errors, so I checked the selinux config, and both the ovirt-engine host and one of the nodes had selinux disabled. So I thought I would enable it on these two hosts, as it is officially supported anyway. I started with the node, and put it into maintenance mode, which interestingly, migrated the VMs off to the other node without issue. After modifying the selinux config, I then rebooted that node, which came back up. I then tried to activate the node but it fails and marks it as unresponsive. From the log: ----------8<---------- 2016-04-28 16:34:31,326 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [29acb18b] Connecting to kvm-ldn-02.ldn.framestore.com/172.16.75.189 2016-04-28 16:34:31,327 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) [ac322cb] Command 'GetCapabilitiesVDSCommand(HostName = kvm-ldn-02, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', hostId='b12c0b80-d64d-42fd-8a55-94f92b9ca3aa', vds='Host[kvm-ldn-02,b12c0b80-d64d-42fd-8a55-94f92b9ca3aa]'})' execution failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection failed 2016-04-28 16:34:31,327 ERROR [org.ovirt.engine.core.vdsbroker.HostMonitoring] (DefaultQuartzScheduler_Worker-32) [ac322cb] Failure to refresh Vds runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection failed 2016-04-28 16:34:31,327 ERROR [org.ovirt.engine.core.vdsbroker.HostMonitoring] (DefaultQuartzScheduler_Worker-32) [ac322cb] Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection failed at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:157) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:120) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65) [vdsbroker.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33) [dal.jar:] at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:467) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:652) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.HostMonitoring.refreshVdsRunTimeInfo(HostMonitoring.java:119) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.HostMonitoring.refresh(HostMonitoring.java:84) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:227) [vdsbroker.jar:] at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source) [:1.8.0_71] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_71] at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_71] at org.ovirt.engine.core.utils.timer.JobWrapper.invokeMethod(JobWrapper.java:81) [scheduler.jar:] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:52) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection failed at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:157) [vdsm-jsonrpc-java-client.jar:] at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:114) [vdsm-jsonrpc-java-client.jar:] at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:73) [vdsm-jsonrpc-java-client.jar:] at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:68) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:268) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:110) [vdsbroker.jar:] ... 14 more ----------8<---------- Any ideas? Thanks, Cam

1 0

Keyboard mapping VNC
by Jonas Israelsson 28 Apr '16

28 Apr '16

Greetings. We are under the impression the the keyboard layout for the VNC-server is missing from a vm's xml definition, hence the same default layout is always used. We therefore (just to test) created a small host hook adding a static keyboard layout, and with the hook, it seem to work as expected as the -k switch to kvm is now populated. vm started with our hook <graphics type='vnc' port='5904' autoport='yes' keymap='sv' passwd='***********' passwdValidTo='2016-04-11T09:21:09'> <listen type='network' address='192.168.42.113' network='vdsm-ovirtmgmt' fromConfig='0'/> </graphics> and one without <graphics type='vnc' port='5903' autoport='yes' passwd='*****' passwdValidTo='1970-01-01T00:00:01'> <listen type='network' address='192.168.42.113' network='vdsm-ovirtmgmt' fromConfig='0'/> </graphics> oVirt Engine Version: 3.6.3.4-1.el7.centos Node: oVirt Node Hypervisor release 3.6 (0.999.201512141021.el7.centos) Is this a known issue ? Brgds, Jonas

2 7

Max amount of datacenters per ovirt engine
by Joost＠familiealbers.nl 28 Apr '16

28 Apr '16

Hi all, i have a somewhat awkward requirement to deploy datacenters to around 500 satelite locations. Each datacenter will have 1 or 2 hypervisors and if 2 a cluster with glusterfs for shared storage will be deployed. I am expecting to run between 4 and 10 vms per data center. During operations it is expected 20% to 40% of the satelite locations will be down or have a very bad connection to the engine. I will still like to manage these datacenters from ovirt engine. Does anyone have any figures on how many datacenters i can add given what requirements for a single engine and perhaps some best practices for dealing with 'bad' connections between the hypervisors and the engine. Thanks a lot!

5 10

Unable to power off the vm
by Budur Nagaraju 28 Apr '16

28 Apr '16

HI One of the vm is showing "?" and unable to perform any actions below are the logs ,let me know is there any ways to bring it back ? 2016-04-28 13:31:10,501 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-39) [191cb679] Failed in DestroyVDS method 2016-04-28 13:31:10,502 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-39) [191cb679] Command org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=42, mMessage=Virtual machine destroy error]] 2016-04-28 13:31:10,502 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-39) [191cb679] HostName = cstkvm1 2016-04-28 13:31:10,503 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-39) [191cb679] Command DestroyVDSCommand(HostName = cstkvm1, HostId = 808e0118-32af-47d0-b9eb-43c34494c292, vmId=7212cdd2-8171-4882-977e-82b7c95f5e62, force=false, secondsToWait=0, gracefully=false, reason=) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine destroy error, code = 42 2016-04-28 13:31:10,503 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-39) [191cb679] FINISH, DestroyVDSCommand, log id: 5e35dcad 2016-04-28 13:31:10,504 ERROR [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-39) [191cb679] VDS::destroy Failed destroying vm 7212cdd2-8171-4882-977e-82b7c95f5e62 in vds = 808e0118-32af-47d0-b9eb-43c34494c292 : cstkvm1, error = org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine destroy error, code = 42 2016-04-28 13:31:10,513 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-39) [191cb679] FINISH, DestroyVmVDSCommand, log id: 6ff5ad79 2016-04-28 13:31:10,516 ERROR [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-39) [191cb679] Command org.ovirt.engine.core.bll.StopVmCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine destroy error, code = 42 (Failed with error destroyErr and code 42) 2016-04-28 13:31:10,531 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-39) [191cb679] Correlation ID: 191cb679, Job ID: b8b77109-e559-46b4-be1f-6402e85c0a62, Call Stack: null, Custom Event ID: -1, Message: Failed to power off VM raghpai-MOSS10 (Host: cstkvm1, User: admin@internal). Thanks, Nagaraju

2 7

[Attention needed] GlusterFS repository down - affects CI / Installations
by Nadav Goldin 28 Apr '16

28 Apr '16

Hi, The GlusterFS repository became unavailable this morning, as a result all Jenkins jobs that use the repository will fail, the common error would be: > > http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/epel-7/n…: > [Errno 14] HTTP Error 403 - Forbidden > Also, installations of oVirt will fail. We are working on a solution and will update asap. Nadav.

5 6

vms in paused state
by Bill James 28 Apr '16

28 Apr '16

I have a hardware node that has 26 VMs. 9 are listed as "running", 17 are listed as "paused". In truth all VMs are up and running fine. I tried telling the db they are up: engine=> update vm_dynamic set status = 1 where vm_guid =(select vm_guid from vm_static where vm_name = 'api1.test.j2noc.com'); GUI then shows it up for a short while, then puts it back in paused state. 2016-04-26 15:16:46,095 INFO [org.ovirt.engine.core.vdsbroker.VmAnalyzer] (DefaultQuartzScheduler_Worker-16) [157cc21e] VM '242ca0af-4ab2-4dd6-b515-5 d435e6452c4'(api1.test.j2noc.com) moved from 'Up' --> 'Paused' 2016-04-26 15:16:46,221 INFO [org.ovirt.engine.core.dal.dbbroker.auditlogh andling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) [157cc21e] Cor relation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM api1. test.j2noc.com has been paused. Why does the engine think the VMs are paused? Attached engine.log. I can fix the problem by powering off the VM then starting it back up. But the VM is working fine! How do I get ovirt to realize that? ovirt-engine-3.6.4.1-1.el7.centos.noarch

3 4

Cannot add Interface. Not enough MAC addresses left in MAC Address Pool.
by Bill James 27 Apr '16

27 Apr '16

How do you increase and display the settings of I saw some references to changing via /MaxMacsCountInPool & //MacPoolRanges but those don't seem to exist in ovirt3.6./ http://lists.ovirt.org/pipermail/users/2015-November/035694.html https://www.ovirt.org/develop/developer-guide/engine/engine-config-options/ [root@ovirt dev engine.conf.d]# engine-config -g MaxMacsCountInPool null [root@ovirt dev engine.conf.d]# engine-config -s "MacPoolRanges=00:1a:4a:ab:64:00-00:1a:4a:ab:64:ff,00:1A:4A:97:5F:00-00:1A:4A:97:5F:FF" Error setting MacPoolRanges's value. No such entry. 2016-04-27 15:06:06,807 INFO [org.ovirt.engine.core.bll.AddVmFromTemplateComm and] (default task-32) [31879003] Lock freed to object 'EngineLock:{exclusiveL ocks='[sr140vm1.dev.j2noc.com=<VM_NAME, ACTION_TYPE_FAILED_OBJECT_LOCKED>]', s haredLocks='[00000000-0000-0000-0000-000000000000=<TEMPLATE, ACTION_TYPE_FAILE D_TEMPLATE_IS_USED_FOR_CREATE_VM$VmName sr140vm1.dev.j2noc.com>]'}' 2016-04-27 15:06:08,289 WARN [org.ovirt.engine.core.bll.network.vm.AddVmInter faceCommand] (default task-48) [4e78dc27] CanDoAction of action 'AddVmInterfac e' failed for user bill.james@j2global.com@j2global.com. Reasons: VAR__TYPE__I NTERFACE,VAR__ACTION__ADD,MAC_POOL_NOT_ENOUGH_MAC_ADDRESSES Strange thing is I only have 143 VMs. ovirt-engine-3.6.4.1-1.el7.centos.noarch

2 2