Re: [ovirt-users] hosted-engine deploy fails at "Wait for the engine to come up on the target VM" step

On Mon, Mar 12, 2018 at 6:44 PM, Kristian Petersen <nesretep@chem.byu.edu> wrote:
I think I accidentally sent that reply before I was really finished with it. The error said the the VM mentioned in the conf file didn't exist. I included the log file as requested.
As far as I can see from the logs, the engine VM went up as expected now: 2018-03-12 11:15:00,183-0600 INFO (jsonrpc/5) [api.virt] FINISH getStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': '-1', 'ipAddress': '0', 'type': u'vnc', 'port': '5900'}], 'memUsage': '17', 'acpiEnable': 'true', 'guestFQDN': u' rhv-engine.cpms.byu.edu', 'vmId': 'cbe9b80f-9c18-409c-b7b1-54d95f4734ca', 'session': 'Unknown', 'displaySecurePort': '-1', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '16777216', 'balloon_min': '0', 'balloon_target': '16777216', 'balloon_cur': '16777216'}, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'/', u'total': '6565134336', u'fs': u'xfs', u'used': '1950617600'}, {u'path': u'/boot', u'total': '1063256064', u'fs': u'xfs', u'used': '170590208'}, {u'path': u'/home', u'total': '1063256064', u'fs': u'xfs', u'used': '33792000'}, {u'path': u'/var', u'total': '21464350720', u'fs': u'xfs', u'used': '396513280'}, {u'path': u'/var/log', u'total': '10726932480', u'fs': u'xfs', u'used': '42823680'}, {u'path': u'/tmp', u'total': '2136997888', u'fs': u'xfs', u'used': '34058240'}, {u'path': u'/var/log/audit', u'total': '1063256064', u'fs': u'xfs', u'used': '34586624'}], 'network': {'vnet0': {'macAddr': u'00:16:3e:54:f3:8e', 'rxDropped': '0', 'tx': '30742', 'rxErrors': '0', 'txDropped': '0', 'rx': '167904', 'txErrors': '0', 'state': 'unknown', 'sampleTime': 9385842.59, 'speed': '1000', 'name': 'vnet0'}}, 'vmJobs': {}, 'cpuUser': '7.43', 'elapsedTime': '81', 'memoryStats': {'swap_out': '0', 'majflt': '0', 'mem_cached': '452216', 'mem_free': '13404476', 'mem_buffers': '2104', 'swap_in': '0', 'pageflt': '418', 'mem_total': '16263704', 'mem_unused': '13404476'}, 'cpuSys': '1.67', 'appsList': (u'ovirt-guest-agent-common-1.0.14-1.el7', u'kernel-3.10.0-693.17.1.el7', u'cloud-init-0.7.9-9.el7.centos.2'), 'guestOs': u'3.10.0-693.17.1.el7.x86_64', 'vmName': 'HostedEngine', 'displayType': 'vnc', 'vcpuCount': '4', 'clientIp': '', 'hash': '-7630705381253994604', 'guestCPUCount': 4, 'vmType': 'kvm', 'displayIp': '0', 'cpuUsage': '9110000000', 'vcpuPeriod': 100000L, 'displayPort': '5900', 'guestTimezone': {u'zone': u'America/Denver', u'offset': -420}, 'vcpuQuota': '-1', 'statusTime': '9385842590', 'kvmEnable': 'true', 'disks': {'vda': {'readLatency': '387098', 'writtenBytes': '36851200', 'writeOps': '465', 'apparentsize': '125627793408', 'readOps': '15711', 'writeLatency': '1931806', 'imageID': u'ec964354-ac01-4799-9c20-4bc923d285d4', 'readBytes': '480176128', 'flushLatency': '237153', 'readRate': '545.769487017', 'truesize': '2503184384', 'writeRate': '76646.5023329'}, 'hdc': {'readLatency': '0', 'writtenBytes': '0', 'writeOps': '0', 'apparentsize': '0', 'readOps': '4', 'writeLatency': '0', 'readBytes': '152', 'flushLatency': '0', 'readRate': '0.0', 'truesize': '0', 'writeRate': '0.0'}}, 'monitorResponse': '0', 'guestOsInfo': {u'kernel': u'3.10.0-693.17.1.el7.x86_64', u'arch': u'x86_64', u'version': u'7.4.1708', u'distribution': u'CentOS Linux', u'type': u'linux', u'codename': u'Core'}, 'username': u'None', 'guestName': u'rhv-engine.cpms.byu.edu', 'status': 'Up', 'lastLogin': 1520874832.982759, 'guestIPs': u'192.168.1.22', 'guestContainers': [], 'netIfaces': [{u'inet6': [u'fe80::216:3eff:fe54:f38e'], u'hw': u'00:16:3e:54:f3:8e', u'inet': [u'192.168.1.22'], u'name': u'eth0'}]}]} from=::1,52334 (api:52)
On Mon, Mar 12, 2018 at 11:36 AM, Simone Tiraboschi <stirabos@redhat.com> wrote:
On Mon, Mar 12, 2018 at 6:27 PM, Kristian Petersen <nesretep@chem.byu.edu
wrote:
I tried using my customized vm.conf with the fix in the CPU name as you suggested. When I ran hosted-engine --vm-start --vm-conf=/root/myvm.conf and that failed.
This is fine if the VM doesn't exist. Can you please share your vdsm.log?
It said the vm didn't exist. It sounds like I might need to get the updated package from the ovirt-4.2-pre repo and try deploying again.
On Mon, Mar 12, 2018 at 10:31 AM, Simone Tiraboschi <stirabos@redhat.com
wrote:
On Mon, Mar 12, 2018 at 5:25 PM, Kristian Petersen < nesretep@chem.byu.edu> wrote:
I'm guessing that v2.2.10 is not in the oVirt repo yet. When I looked at vm.conf, the CPU name has a space in it like the one mentioned in the link you included. So replacing that space with an underscore should do the trick prehaps?
v2.2.12 is in -pre repo.
You should replace the space with a dash: Broadwell-IBRS
On Mon, Mar 12, 2018 at 10:00 AM, Kristian Petersen < nesretep@chem.byu.edu> wrote:
I have v2.2.9 of ovirt-hosted-engine-setup currently installed. I'll try out the other suggestion you made also. Thanks for the help.
On Fri, Mar 9, 2018 at 4:26 PM, Simone Tiraboschi < stirabos@redhat.com> wrote:
> > > On Fri, Mar 9, 2018 at 8:33 PM, Kristian Petersen < > nesretep@chem.byu.edu> wrote: > >> I have attached the relevant log files as requested. >> vdsm.log.1 >> <https://drive.google.com/a/chem.byu.edu/file/d/1ibJG_SEjK9NSEPft_HCkZzQO25dGgMVo/view?usp=drive_web> >> >> > > > The real issue is here: > > <cpu match="exact"> > <model>BroadwellIBRS</model> > </cpu> > <on_poweroff>destroy</on_poweroff><on_reboot>destroy</on_reb > oot><on_crash>destroy</on_crash></domain> (vm:2751) > 2018-03-08 08:04:13,757-0700 ERROR (vm/9a1e133d) [virt.vm] > (vmId='9a1e133d-13d8-4613-b1a5-fd3ca81ffcc3') The vm start process > failed (vm:927) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 856, in _startUnderlyingVm > self._run() > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line > 2756, in _run > dom.createWithFlags(flags) > File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", > line 130, in wrapper > ret = f(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", > line 92, in wrapper > return func(inst, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1069, > in createWithFlags > if ret == -1: raise libvirtError ('virDomainCreateWithFlags() > failed', dom=self) > libvirtError: internal error: Unknown CPU model BroadwellIBRS > > Indeed it should be Broadwell-IBRS > > Can you please report which rpm version of ovirt-hosted-engine-setup > did you used? > > You can fix it in this way: > copy /var/run/ovirt-hosted-engine-ha/vm.conf somewhere, edit it and > update the cpuType field. > > Then start the engine VM with your custom vm.conf with something > like: > hosted-engine --vm-start --vm-conf=/root/my_vm.conf > keep the engine up for at least one hour and it will generate the > OVF_STORE disks with the right configuration for the hosted-engine VM. > > It failed really at the end of the setup so anything else should be > fine. > > > >> >> On Fri, Mar 9, 2018 at 1:21 AM, Simone Tiraboschi < >> stirabos@redhat.com> wrote: >> >>> >>> >>> On Thu, Mar 8, 2018 at 7:28 PM, Kristian Petersen < >>> nesretep@chem.byu.edu> wrote: >>> >>>> I am trying to deploy oVirt with a self-hosted engine and the >>>> setup seems to go well until near the very end when the status message says: >>>> [ INFO ] TASK [Wait for the engine to come up on the target VM] >>>> >>>> [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, >>>> "changed": true, "cmd": ["hosted-engine", "--vm-status", "--json"], >>>> "delta": "0:0 >>>> 0:00.216412", "end": "2018-03-07 16:02:02.677478", "rc": 0, >>>> "start": "2018-03-07 16:02:02.461066", "stderr": "", "stderr_lines": [], >>>> "stdout >>>> ": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": >>>> true, \"extra\": \"metadata_parse_version=1\\nm >>>> etadata_feature_version=1\\ntim >>>> estamp=4679955 (Wed Mar 7 16:01:50 2018)\\nhost-id=1\\nscore=3400 >>>> \\nvm_conf_refresh_time=4679956 (Wed Mar 7 16:01:51 >>>> 2018)\\nconf_on_share >>>> d_storage=True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", >>>> \"hostname\": \"rhv1.cpms.byu.edu\", \"host-id\": 1, \"engin >>>> e-status\": {\"reason\": \"vm not running on this host\", >>>> \"health\": \"bad\", \"vm\": \"down\", \"detail\": \"unknown\"}, \"score\": >>>> 3400, >>>> \"stopped\": false, \"maintenance\": false, \"crc32\": >>>> \"d3a67cf7\", \"local_conf_timestamp\": 4679956, \"host-ts\": 4679955}, >>>> \"global_main >>>> tenance\": false}", "stdout_lines": ["{\"1\": >>>> {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": >>>> \"metadata_parse_version=1\ >>>> \nmetadata_feature_version=1\\ntimestamp=4679955 (Wed Mar 7 >>>> 16:01:50 2018)\\nhost-id=1\\nscore=3400 >>>> \\nvm_conf_refresh_time=4679956 (Wed Mar >>>> 7 16:01:51 2018)\\nconf_on_shared_storage >>>> =True\\nmaintenance=False\\nstate=EngineStarting\\nstopped=False\\n\", >>>> \"hostname\": \"rhv1.cpms. >>>> byu.edu\", \"host-id\": 1, \"engine-status\": {\"reason\": \"vm >>>> not running on this host\", \"health\": \"bad\", \"vm\": \"down\", \"detail\ >>>> ": \"unknown\"}, \"score\": 3400, \"stopped\": false, >>>> \"maintenance\": false, \"crc32\": \"d3a67cf7\", \"local_conf_timestamp\": >>>> 4679956, \" >>>> host-ts\": 4679955}, \"global_maintenance\": false}"]} >>>> [ ERROR ] Failed to execute stage 'Closing up': Failed executing >>>> ansible-playbook >>>> >>>> Any ideas that might help? >>>> >>> >>> >>> Hi Kristian, >>> {\"reason\": \"vm not running on this host\" sonds really bad. >>> I means that ovirt-ha-agent (in charge of restarting the engine >>> VM) think that another host took over but at that stage you should have >>> just one host. >>> >>> Could you please attach /var/log/ovirt-hosted-engine-ha/agent.log >>> and /var/log/vdsm/vdsm.log for the relevant time frame? >>> >>> >>>> >>>> >>>> -- >>>> Kristian Petersen >>>> System Administrator >>>> Dept. of Chemistry and Biochemistry >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users@ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/users >>>> >>>> >>> >> >> >> -- >> Kristian Petersen >> System Administrator >> Dept. of Chemistry and Biochemistry >> > >
-- Kristian Petersen System Administrator BYU Dept. of Chemistry and Biochemistry
-- Kristian Petersen System Administrator BYU Dept. of Chemistry and Biochemistry
-- Kristian Petersen System Administrator BYU Dept. of Chemistry and Biochemistry
-- Kristian Petersen System Administrator BYU Dept. of Chemistry and Biochemistry
participants (1)
-
Simone Tiraboschi