<html><head></head><body><div>On Thu, 2016-08-18 at 17:45 +0200, Simone Tiraboschi wrote:</div><blockquote type="cite"><div>On Thu, Aug 18, 2016 at 5:43 PM, Carlos Rodrigues <<a href="mailto:cmar@eurotux.com">cmar@eurotux.com</a>> wrote:</div><blockquote type="cite"><div><br></div><div>I increase hosted_engine disk space to 160G. How do i force to create</div><div>OVF_STORE.</div></blockquote><div><br></div><div>I think that restarting the engine on the engine VM will trigger it</div><div>although I'm not sure that it was a size issue.</div><div><br></div></blockquote><div><br></div><div>I found to OVF_STORE on another storage domain with "Domain Type" "Data (Master)"</div><div><br></div><div><img src="cid:1471538309.2273.30.camel@eurotux.com"><br></div><div><br></div><div><br></div><blockquote type="cite"><blockquote type="cite"><div><br></div><div>Regards,</div><div>Carlos Rodrigues</div><div><br></div><div>On Thu, 2016-08-18 at 12:14 +0100, Carlos Rodrigues wrote:</div><blockquote type="cite"><div><br></div><div>On Thu, 2016-08-18 at 12:34 +0200, Simone Tiraboschi wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div>On Thu, Aug 18, 2016 at 12:11 PM, Carlos Rodrigues <cmar@eurotux.co</div><div>m></div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div>On Thu, 2016-08-18 at 11:53 +0200, Simone Tiraboschi wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>On Thu, Aug 18, 2016 at 11:50 AM, Carlos Rodrigues <cmar@eurotu</div><div>x.</div><div>com></div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div>On Thu, 2016-08-18 at 11:42 +0200, Simone Tiraboschi wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div>On Thu, Aug 18, 2016 at 11:25 AM, Carlos Rodrigues <cmar@eu</div><div>ro</div><div>tux.</div><div>com> wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div>On Thu, 2016-08-18 at 11:04 +0200, Simone Tiraboschi</div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div>On Thu, Aug 18, 2016 at 10:36 AM, Carlos Rodrigues</div><div><cmar@</div><div>euro</div><div>tux.com></div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>On Thu, 2016-08-18 at 10:27 +0200, Simone Tiraboschi</div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>On Thu, Aug 18, 2016 at 10:22 AM, Carlos Rodrigues</div><div><cmar@</div><div>eurotux.</div><div>com></div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>On Thu, 2016-08-18 at 08:54 +0200, Simone</div><div>Tiraboschi</div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>On Tue, Aug 16, 2016 at 12:53 PM, Carlos</div><div>Rodrigues <c</div><div><a href="mailto:mar@euro">mar@euro</a></div><div>tux.</div><div>com></div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>On Sun, 2016-08-14 at 14:22 +0300, Roy Golan</div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>On 12 August 2016 at 20:23, Carlos</div><div>Rodrigues</div><div><cma</div><div><a href="mailto:r@eurotu">r@eurotu</a></div><div>x.co</div><div>m></div><div>wrote:</div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>Hello,</div><div><br></div><div>I have one cluster with two hosts with</div><div>power</div><div>management</div><div>correctly</div><div>configured and one virtual machine with</div><div>HostedEngine</div><div>over</div><div>shared</div><div>storage with FiberChannel.</div><div><br></div><div>When i shutdown the network of host with</div><div>HostedEngine</div><div>VM, it</div><div>should be</div><div>possible the HostedEngine VM migrate</div><div>automatically to</div><div>another</div><div>host?</div><div><br></div></blockquote><div>migrate on which network?</div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>What is the expected behaviour on this HA</div><div>scenario?</div></blockquote><div><br></div><div>After a few minutes your vm will be</div><div>shutdown</div><div>by</div><div>the High</div><div>Availability</div><div>agent, as it can't see network, and started</div><div>on</div><div>another</div><div>host.</div></blockquote><div><br></div><div><br></div><div>I'm testing this scenario and after shutdown</div><div>network, it</div><div>should</div><div>be</div><div>expected that agent shutdown ha and started</div><div>on</div><div>another</div><div>host,</div><div>but</div><div>after</div><div>couple minutes nothing happens and on host</div><div>with</div><div>network we</div><div>getting</div><div>the</div><div>following messages:</div><div><br></div><div>Aug 16 11:44:08 ied-</div><div>blade11.install.eurotux.local</div><div>ovirt-ha-</div><div>agent[2779]:</div><div>ovirt-ha-agent</div><div>ovirt_hosted_engine_ha.agent.hosted_engine.Ho</div><div>st</div><div>edEn</div><div>gine.con</div><div>fig</div><div>ERROR</div><div>Unable to get vm.conf from OVF_STORE, falling</div><div>back</div><div>to</div><div>initial</div><div>vm.conf</div><div><br></div><div>I think the HA agent its trying to get vm</div><div>configuration but</div><div>some</div><div>how it</div><div>can't get vm.conf to start VM.</div></blockquote><div><br></div><div>No, this is a different issues.</div><div>In 3.6 we added a feature to let the engine</div><div>manage</div><div>also the</div><div>engine VM</div><div>itself; ovirt-ha-agent will pickup the latest</div><div>engine</div><div>VM</div><div>configuration</div><div>from the OVF_STORE which is managed by the</div><div>engine.</div><div>If something goes wrong, ovirt-ha-agent could</div><div>fallback to the</div><div>initial</div><div>(bootstrap time) vm.conf. This will normally</div><div>happen</div><div>till you</div><div>add</div><div>your</div><div>first regular storage domain and the engine</div><div>imports</div><div>the</div><div>engine</div><div>VM.</div></blockquote><div><br></div><div>But i already have my first storage domain and</div><div>storage</div><div>engine</div><div>domain</div><div>and already imported engine VM.</div><div><br></div><div>I'm using 4.0 version.</div></blockquote><div><br></div><div>This seams an issue, can you please share your</div><div>/var/log/ovirt-hosted-engine-ha/agent.log ?</div><div><br></div></blockquote><div><br></div><div>I sent it in attachment.</div></blockquote><div><br></div><div>Nothing strange here;</div><div>do you see a couple of disks with alias OVF_STORE on</div><div>the</div><div>hosted-</div><div>engine</div><div>storage domain if you check it from the engine?</div><div><br></div></blockquote><div><br></div><div>Do you mean any disk label?</div><div>I don't have it anyone:</div><div><br></div><div>[<a href="mailto:root@ied-blade11">root@ied-blade11</a> ~]# ls /dev/disk/by-label/</div><div>ls: cannot access /dev/disk/by-label/: No such file or</div><div>directory</div></blockquote><div><br></div><div>No I mean: go to the engine web-ui, select the hosted-</div><div>engine</div><div>storage</div><div>domain, check the disks there.</div></blockquote><div><br></div><div>No, the alias is virtio-disk0.</div><div><br></div></blockquote><div><br></div><div>And this is the engine VM disk, so the issue is why the engine</div><div>has</div><div>still to create the OVF_STORE.</div><div>Can you please share your engine.log from the engine VM?</div><div><br></div></blockquote><div><br></div><div>Go in attachment.</div></blockquote><div><br></div><div>The creation of the OVF_STORE disk failed but it's not that clear</div><div>why:</div><div><br></div><div>2016-08-17 08:43:33,538 ERROR</div><div>[org.ovirt.engine.core.bll.storage.ovfstore.CreateOvfVolumeForStora</div><div>ge</div><div>DomainCommand]</div><div>(DefaultQuartzScheduler6) [6f1f1fd4] Ending command</div><div>'org.ovirt.engine.core.bll.storage.ovfstore.CreateOvfVolumeForStora</div><div>ge</div><div>DomainCommand'</div><div>with failure.</div><div>2016-08-17 08:43:33,540 ERROR</div><div>[org.ovirt.engine.core.bll.storage.disk.AddDiskCommand]</div><div>(DefaultQuartzScheduler6) [6f1f1fd4] Ending command</div><div>'org.ovirt.engine.core.bll.storage.disk.AddDiskCommand' with</div><div>failure.</div><div>2016-08-17 08:43:33,541 WARN</div><div>[org.ovirt.engine.core.bll.storage.disk.AddDiskCommand]</div><div>(DefaultQuartzScheduler6) [6f1f1fd4] VmCommand::EndVmCommand: Vm is</div><div>null - not performing endAction on Vm</div><div>2016-08-17 08:43:33,553 ERROR</div><div>[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirect</div><div>or</div><div>]</div><div>(DefaultQuartzScheduler6) [6f1f1fd4] Correlation ID: 6f1f1fd4, Call</div><div>Stack: null, Custom Event ID: -1, Message: Add-Disk operation</div><div>failed</div><div>to complete.</div><div>2016-08-17 08:43:33,557 WARN</div><div>[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirect</div><div>or</div><div>]</div><div>(DefaultQuartzScheduler6) [] Correlation ID: 19ac5bda, Call Stack:</div><div>null, Custom Event ID: -1, Message: Failed to create OVF store disk</div><div>for Storage Domain hosted_storage.</div><div> OVF data won't be updated meanwhile for that domain.</div><div>2016-08-17 08:43:33,585 INFO</div><div>[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback]</div><div>(DefaultQuartzScheduler6) [5f5a8daf] Command</div><div>'ProcessOvfUpdateForStorageDomain' (id:</div><div>'71aaaafe-7b9e-45e8-a40c-6d33bdf646a0') waiting on child command</div><div>id:</div><div>'eb2e6f1a-c756-4ccd-85a1-60d97d6880de'</div><div>type:'CreateOvfVolumeForStorageDomain' to complete</div><div>2016-08-17 08:43:33,595 ERROR</div><div>[org.ovirt.engine.core.bll.storage.ovfstore.CreateOvfVolumeForStora</div><div>ge</div><div>DomainCommand]</div><div>(DefaultQuartzScheduler6) [5d314e49] Ending command</div><div>'org.ovirt.engine.core.bll.storage.ovfstore.CreateOvfVolumeForStora</div><div>ge</div><div>DomainCommand'</div><div>with failure.</div><div>2016-08-17 08:43:33,596 ERROR</div><div>[org.ovirt.engine.core.bll.storage.disk.AddDiskCommand]</div><div>(DefaultQuartzScheduler6) [5d314e49] Ending command</div><div>'org.ovirt.engine.core.bll.storage.disk.AddDiskCommand' with</div><div>failure.</div><div>2016-08-17 08:43:33,596 WARN</div><div>[org.ovirt.engine.core.bll.storage.disk.AddDiskCommand]</div><div>(DefaultQuartzScheduler6) [5d314e49] VmCommand::EndVmCommand: Vm is</div><div>null - not performing endAction on Vm</div><div>2016-08-17 08:43:33,602 ERROR</div><div>[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirect</div><div>or</div><div>]</div><div>(DefaultQuartzScheduler6) [5d314e49] Correlation ID: 5d314e49, Call</div><div>Stack: null, Custom Event ID: -1, Message: Add-Disk operation</div><div>failed</div><div>to complete.</div><div>2016-08-17 08:43:33,605 WARN</div><div>[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirect</div><div>or</div><div>]</div><div>(DefaultQuartzScheduler6) [] Correlation ID: 5f5a8daf, Call Stack:</div><div>null, Custom Event ID: -1, Message: Failed to create OVF store disk</div><div>for Storage Domain hosted_storage.</div><div> OVF data won't be updated meanwhile for that domain.</div><div>2016-08-17 08:43:36,460 INFO</div><div>[org.ovirt.engine.core.bll.scheduling.HaReservationHandling]</div><div>(DefaultQuartzScheduler7) [5d314e49] HA reservation status for</div><div>cluster</div><div>'Default' is 'OK'</div><div>2016-08-17 08:43:36,662 INFO</div><div>[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback]</div><div>(DefaultQuartzScheduler4) [5f5a8daf] Command</div><div>'ProcessOvfUpdateForStorageDomain' id:</div><div>'71aaaafe-7b9e-45e8-a40c-6d33bdf646a0' child commands</div><div>'[84959a4b-6a10-4d22-b37e-6c154e17a0da,</div><div>eb2e6f1a-c756-4ccd-85a1-60d97d6880de]' executions were completed,</div><div>status 'FAILED'</div><div>2016-08-17 08:43:37,691 ERROR</div><div>[org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStor</div><div>ag</div><div>eDomainCommand]</div><div>(DefaultQuartzScheduler6) [5f5a8daf] Ending command</div><div>'org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStor</div><div>ag</div><div>eDomainCommand'</div><div>with failure.</div><div><br></div><div>Can you please check vdsm logs for that time frame on the SPM host?</div><div><br></div></blockquote><div><br></div><div>I sent in attachment the vdsm logs from both hosts, but i think the</div><div>SPM</div><div>host on this time frame it was ied-blade13</div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div>It seams that you also have an issue in the SPM election procedure:</div><div><br></div><div>2016-08-17 18:04:31,053 ERROR</div><div>[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData]</div><div>(DefaultQuartzScheduler1) [] SPM Init: could not find reported vds</div><div>or</div><div>not up - pool: 'Default' vds_spm_id: '2'</div><div>2016-08-17 18:04:31,076 INFO</div><div>[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData]</div><div>(DefaultQuartzScheduler1) [] SPM selection - vds seems as spm</div><div>'hosted_engine_2'</div><div>2016-08-17 18:04:31,076 WARN</div><div>[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData]</div><div>(DefaultQuartzScheduler1) [] spm vds is non responsive, stopping</div><div>spm</div><div>selection.</div><div>2016-08-17 18:04:31,539 INFO</div><div>[org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher]</div><div>(DefaultQuartzScheduler7) [] Fetched 1 VMs from VDS</div><div>'06372186-572c-41ad-916f-7cbb0aba5302'</div><div><br></div><div>probably due to:</div><div>2016-08-17 18:02:33,569 ERROR</div><div>[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]</div><div>(DefaultQuartzScheduler6) [] Failure to refresh Vds runtime info:</div><div>VDSGenericException: VDSNetworkException: Message timeout which can</div><div>be</div><div>caused by communication issues</div><div>2016-08-17 18:02:33,569 ERROR</div><div>[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]</div><div>(DefaultQuartzScheduler6) [] Exception:</div><div>org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:</div><div>VDSGenericException: VDSNetworkException: Message timeout which can</div><div>be</div><div>caused by communication issues</div><div><br></div></blockquote><div><br></div><div>This messages maybe cause by connection issues from yesterday at 6pm.</div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div>can you please check if the engine VM could correctly resolve and</div><div>reach each host?</div></blockquote><div><br></div><div>Now i can read engine VM from both hosts</div><div><br></div><div>[<a href="mailto:root@ied-blade11">root@ied-blade11</a> ~]# ping ied-hosted-engine</div><div>PING ied-hosted-engine.install.eurotux.local (10.10.4.115) 56(84)</div><div>bytes</div><div>of data.</div><div>64 bytes from ied-hosted-engine.install.eurotux.local (10.10.4.115):</div><div>icmp_seq=1 ttl=64 time=0.179 ms</div><div>64 bytes from ied-hosted-engine.install.eurotux.local (10.10.4.115):</div><div>icmp_seq=2 ttl=64 time=0.141 ms</div><div>^C</div><div>--- ied-hosted-engine.install.eurotux.local ping statistics ---</div><div>2 packets transmitted, 2 received, 0% packet loss, time 999ms</div><div>rtt min/avg/max/mdev = 0.141/0.160/0.179/0.019 ms</div><div>[<a href="mailto:root@ied-blade13">root@ied-blade13</a> ~]# ping ied-hosted-engine</div><div>PING ied-hosted-engine.install.eurotux.local (10.10.4.115) 56(84)</div><div>bytes</div><div>of data.</div><div>64 bytes from ied-hosted-engine.install.eurotux.local (10.10.4.115):</div><div>icmp_seq=1 ttl=64 time=0.172 ms</div><div>64 bytes from ied-hosted-engine.install.eurotux.local (10.10.4.115):</div><div>icmp_seq=2 ttl=64 time=0.169 ms</div><div>^C</div><div>--- ied-hosted-engine.install.eurotux.local ping statistics ---</div><div>2 packets transmitted, 2 received, 0% packet loss, time 1000ms</div><div>rtt min/avg/max/mdev = 0.169/0.170/0.172/0.013 ms</div><div><br></div><div><br></div><div>I have a message critical of low disk space on hosted_storage domain.</div><div>I have 50G of disk and i created i VM with 40G. Do i need more space</div><div>of</div><div>OVF_STORAGE?</div><div>What is the minimum requirements of disk space for deploy engine VM?</div><div><br></div><div>Regards,</div><div>Carlos</div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div>Regards,</div><div>Carlos Rodrigues</div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><blockquote type="cite"><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>Regards,</div><div><br></div><div>--</div><div>Carlos Rodrigues</div><div><br></div><div>Engenheiro de Software Sénior</div><div><br></div><div>Eurotux Informática, S.A. | www.eurotux.c</div><div>om</div><div>(t) +351 253 680 300 (m) +351 911 926 110</div><div><br></div><div>_________________________________________</div><div>__</div><div>____</div><div>Users mailing list</div><div><a href="mailto:Users@ovirt.org">Users@ovirt.org</a></div><div>http://lists.ovirt.org/mailman/listinfo/u</div><div>se</div><div>rs</div><div><br></div></blockquote><div><br></div></blockquote><div>--</div><div>Carlos Rodrigues</div><div><br></div><div>Engenheiro de Software Sénior</div><div><br></div><div>Eurotux Informática, S.A. | www.eurotux.com</div><div>(t) +351 253 680 300 (m) +351 911 926 110</div><div><br></div><div>_____________________________________________</div><div>__</div><div>Users mailing list</div><div><a href="mailto:Users@ovirt.org">Users@ovirt.org</a></div><div>http://lists.ovirt.org/mailman/listinfo/users</div></blockquote></blockquote><div>--</div><div>Carlos Rodrigues</div><div><br></div><div>Engenheiro de Software Sénior</div><div><br></div><div>Eurotux Informática, S.A. | <a href="www.eurotux.com">www.eurotux.com</a></div><div>(t) +351 253 680 300 (m) +351 911 926 110</div><div><br></div></blockquote></blockquote><div>--</div><div>Carlos Rodrigues</div><div><br></div><div>Engenheiro de Software Sénior</div><div><br></div><div>Eurotux Informática, S.A. | <a href="www.eurotux.com">www.eurotux.com</a></div><div>(t) +351 253 680 300 (m) +351 911 926 110</div></blockquote></blockquote><div>--</div><div>Carlos Rodrigues</div><div><br></div><div>Engenheiro de Software Sénior</div><div><br></div><div>Eurotux Informática, S.A. | <a href="www.eurotux.com">www.eurotux.com</a></div><div>(t) +351 253 680 300 (m) +351 911 926 110</div><div><br></div></blockquote></blockquote><div>--</div><div>Carlos Rodrigues</div><div><br></div><div>Engenheiro de Software Sénior</div><div><br></div><div>Eurotux Informática, S.A. | <a href="www.eurotux.com">www.eurotux.com</a></div><div>(t) +351 253 680 300 (m) +351 911 926 110</div><div><br></div><div><br></div></blockquote><div><br></div></blockquote><div>--</div><div>Carlos Rodrigues</div><div><br></div><div>Engenheiro de Software Sénior</div><div><br></div><div>Eurotux Informática, S.A. | <a href="www.eurotux.com">www.eurotux.com</a></div><div>(t) +351 253 680 300 (m) +351 911 926 110</div></blockquote></blockquote><div>_______________________________________________</div><div>Users mailing list</div><div><a href="mailto:Users@ovirt.org">Users@ovirt.org</a></div><div><a href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a></div></blockquote><div>--</div><div>Carlos Rodrigues</div><div><br></div><div>Engenheiro de Software Sénior</div><div><br></div><div>Eurotux Informática, S.A. | <a href="www.eurotux.com">www.eurotux.com</a></div><div>(t) +351 253 680 300 (m) +351 911 926 110</div><div><br></div></blockquote></blockquote><div><span><pre>-- <br></pre><div>Carlos Rodrigues <br><br>Engenheiro de Software Sénior<br><br>Eurotux Informática, S.A. | <a href="http://www.eurotux.com/">www.eurotux.com</a><br>(t) +351 253 680 300 (m) +351 911 926 110<br><br></div></span></div></body></html>