Correct workflow for modifying engine vm.conf file
Hello, I would like to know a way to interact and modify in a "supported"/working way the vm.conf file on shared storage that contains details for the engine vm and I think is the base to dynamically generate the /var/run/ovirt-hosted-engine-ha/vm.conf In the past it was possible to temporarily modify the file under /var/run (or copy to another location) and start engine with it, but now it seems it doesn't work and that the file is continuously updated/refreshed Eg on engine storage domain on a lab with oVirt 4.4. beta4 I have the tar file [root@novirt2 ~]# tar tvf /rhev/data-center/mnt/glusterSD/novirt2st.storage.local\:_engine/7c9397f5-859b-4fba-bcc5-4ecbd454a189/images/530490b0-935e-46a8-84f6-6cd6618c6f0d/2e0f414c-2589-4545-a0a4-3bfed24ef3fc -rw-r--r-- root/root 181 2020-04-19 15:04 broker.conf -rw-r--r-- root/root 2777 2020-04-19 15:04 fhanswers.conf -rw-r--r-- root/root 1076 2020-04-19 15:04 hosted-engine.conf -rw-r--r-- root/root 6 2020-04-19 15:04 version -rw-r--r-- root/root 1359 2020-04-19 15:04 vm.conf [root@novirt2 ~]# And its vm.conf has: cpuType=Cascadelake-Server,+md-clear,+mds-no,-hle,-rtm,+tsx-ctrl,+arch-capabilities I would like to modify it and remove the "+tsx-ctrl" flag. Is there anything in the database that generates modification in the file on shared storage and then on the directory under /var/run? Or how is all the workflow managed to generate at the end the xml file to use to start the engine vm? What is that initially determines how to populate the cpuType in vm.conf at all? Thanks, Gianluca
On April 20, 2020 10:25:42 AM GMT+03:00, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
Hello, I would like to know a way to interact and modify in a "supported"/working way the vm.conf file on shared storage that contains details for the engine vm and I think is the base to dynamically generate the /var/run/ovirt-hosted-engine-ha/vm.conf
In the past it was possible to temporarily modify the file under /var/run (or copy to another location) and start engine with it, but now it seems it doesn't work and that the file is continuously updated/refreshed
Eg on engine storage domain on a lab with oVirt 4.4. beta4 I have the tar file
[root@novirt2 ~]# tar tvf /rhev/data-center/mnt/glusterSD/novirt2st.storage.local\:_engine/7c9397f5-859b-4fba-bcc5-4ecbd454a189/images/530490b0-935e-46a8-84f6-6cd6618c6f0d/2e0f414c-2589-4545-a0a4-3bfed24ef3fc -rw-r--r-- root/root 181 2020-04-19 15:04 broker.conf -rw-r--r-- root/root 2777 2020-04-19 15:04 fhanswers.conf -rw-r--r-- root/root 1076 2020-04-19 15:04 hosted-engine.conf -rw-r--r-- root/root 6 2020-04-19 15:04 version -rw-r--r-- root/root 1359 2020-04-19 15:04 vm.conf [root@novirt2 ~]#
And its vm.conf has:
cpuType=Cascadelake-Server,+md-clear,+mds-no,-hle,-rtm,+tsx-ctrl,+arch-capabilities
I would like to modify it and remove the "+tsx-ctrl" flag.
Is there anything in the database that generates modification in the file on shared storage and then on the directory under /var/run? Or how is all the workflow managed to generate at the end the xml file to use to start the engine vm? What is that initially determines how to populate the cpuType in vm.conf at all?
Thanks, Gianluca
Hey Gianluca, You can use 'hosted-engine' do define your modified config file and once the engine is up and running - it should update the OVF on the shared storage. Also, there is a workaround - you can define the HE manually using virsh , but it requires more work - like creating the vdsm network, create symbolic links on the node you want to power it up, etc. What is the reason behind your will to modify the engine ? Best Regards, Strahil Nikolov
On Mon, Apr 20, 2020 at 1:26 PM Strahil Nikolov <hunter86_bg@yahoo.com> wrote:
Hey Gianluca,
You can use 'hosted-engine' do define your modified config file and once the engine is up and running - it should update the OVF on the shared storage.
Ok, how?
Also, there is a workaround - you can define the HE manually using virsh , but it requires more work - like creating the vdsm network, create symbolic links on the node you want to power it up, etc.
I'd prefer the other outlined approach above.
What is the reason behind your will to modify the engine ?
Because I'm testing oVirt 4.4. beta4 and for some reason the engine vm takes a cpu feature tsx-ctrl that is not recongnized, so I would like to try eliminating it and see if it can correctly start. See also this thread of mine of yesterday: https://lists.ovirt.org/archives/list/users@ovirt.org/thread/CR6XYIEYHT4N75U... Thanks, Gianluca
Try with 'hosted-engine --vm-conf' options. В понеделник, 20 април 2020 г., 14:48:29 Гринуич+3, Gianluca Cecchi <gianluca.cecchi@gmail.com> написа: On Mon, Apr 20, 2020 at 1:26 PM Strahil Nikolov <hunter86_bg@yahoo.com> wrote:
Hey Gianluca,
You can use 'hosted-engine' do define your modified config file and once the engine is up and running - it should update the OVF on the shared storage.
Ok, how?
Also, there is a workaround - you can define the HE manually using virsh , but it requires more work - like creating the vdsm network, create symbolic links on the node you want to power it up, etc.
I'd prefer the other outlined approach above.
What is the reason behind your will to modify the engine ?
Because I'm testing oVirt 4.4. beta4 and for some reason the engine vm takes a cpu feature tsx-ctrl that is not recongnized, so I would like to try eliminating it and see if it can correctly start. See also this thread of mine of yesterday: https://lists.ovirt.org/archives/list/users@ovirt.org/thread/CR6XYIEYHT4N75U... Thanks, Gianluca
On Mon, Apr 20, 2020 at 4:57 PM Strahil Nikolov <hunter86_bg@yahoo.com> wrote:
Try with 'hosted-engine --vm-conf' options.
Is it just a guess or did you try and worked with this parameter? [root@novirt2 ovirt-hosted-engine]# hosted-engine --vm-start --help Usage: /sbin/hosted-engine --vm-start Start the engine VM on this host. Available only after deployment has completed. --vm-conf=<file> Load an alternative vm.conf file as a recovery action. [root@novirt2 ovirt-hosted-engine]# But hwo to give my vm.conf? Eg if I untar the vm.conf on shared storage # tar -C /tmp/q -xvf 2e0f414c-2589-4545-a0a4-3bfed24ef3fc and then modify vm.conf removing the +tsx-ctrl # hosted-engine --vm-start --vm-conf=/tmp/q/vm.conf VM exists and is Down, cleaning up and restarting Command VM.create with args {'vmID': '22082519-70bb-476f-a373-45c2d402a870', 'vmParams': {'vmId': '22082519-70bb-476f-a373-45c2d402a870', 'memSize': '16384', 'display': 'vnc', 'vmName': 'HostedEngine', 'spiceSecureChannels': 'smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir', 'smp': '2', 'maxVCpus': '8', 'cpuType': 'Cascadelake-Server,+md-clear,+mds-no,-hle,-rtm,+arch-capabilities', 'emulatedMachine': '', 'devices': [{'index': '2', 'iface': 'ide', 'address': {'controller': '0', 'target': '0', 'unit': '0', 'bus': '1', 'type': 'drive'}, 'specParams': {}, 'readonly': 'true', 'deviceId': '', 'path': '', 'device': 'cdrom', 'shared': 'false', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'format': 'raw', 'poolID': '00000000-0000-0000-0000-000000000000', 'volumeID': '15f606c1-757f-44a8-adc7-85e75ff02944', 'imageID': '8c2b8355-ba2d-4ffa-8055-0034d56f0dd7', 'specParams': {}, 'readonly': 'false', 'domainID': '7c9397f5-859b-4fba-bcc5-4ecbd454a189', 'optional': 'false', 'deviceId': '15f606c1-757f-44a8-adc7-85e75ff02944', 'address': {'bus': '0x00', 'slot': '0x06', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'type': 'disk', 'bootOrder': '1'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:68:a9:cc', 'linkActive': 'true', 'network': 'ovirtmgmt', 'specParams': {}, 'deviceId': '', 'address': {'bus': '0x00', 'slot': '0x03', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'type': 'console'}, {'device': 'vga', 'alias': 'video0', 'type': 'video'}, {'device': 'vnc', 'type': 'graphics'}, {'device': 'virtio', 'specParams': {'source': 'urandom'}, 'model': 'virtio', 'type': 'rng'}]}} failed: (code=100, message=General Exception: ("'xml'",)) VM failed to launch [root@novirt2 530490b0-935e-46a8-84f6-6cd6618c6f0d]# In vdsm.log 2020-04-20 17:16:51,401+0200 ERROR (jsonrpc/3) [api] FINISH create error='xml' (api:134) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 124, in method ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/API.py", line 190, in create xml = vmParams.get('_srcDomXML') or vmParams['xml'] KeyError: 'xml' 2020-04-20 17:16:51,402+0200 INFO (jsonrpc/3) [api.virt] FINISH create return={'status': {'code': 100, 'message': 'General Exception: ("\'xml\'",)'}} from=::1,44384, vmId=22082519-70bb-476f-a373-45c2d402a870 (api:54) 2020-04-20 17:16:51,402+0200 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.create failed (error 100) in 0.00 seconds (__init__:312) 2020-04-20 17:16:51,598+0200 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) and the same if I leave vm.conf unmodified.... Gianluca
On April 20, 2020 6:23:29 PM GMT+03:00, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, Apr 20, 2020 at 4:57 PM Strahil Nikolov <hunter86_bg@yahoo.com> wrote:
Try with 'hosted-engine --vm-conf' options.
Is it just a guess or did you try and worked with this parameter? Several devs has recommended it before - but it never worked for me :)
[root@novirt2 ovirt-hosted-engine]# hosted-engine --vm-start --help Usage: /sbin/hosted-engine --vm-start Start the engine VM on this host. Available only after deployment has completed.
--vm-conf=<file> Load an alternative vm.conf file as a recovery action. [root@novirt2 ovirt-hosted-engine]#
But hwo to give my vm.conf? Eg if I untar the vm.conf on shared storage
# tar -C /tmp/q -xvf 2e0f414c-2589-4545-a0a4-3bfed24ef3fc
and then modify vm.conf removing the +tsx-ctrl
# hosted-engine --vm-start --vm-conf=/tmp/q/vm.conf VM exists and is Down, cleaning up and restarting Command VM.create with args {'vmID': '22082519-70bb-476f-a373-45c2d402a870', 'vmParams': {'vmId': '22082519-70bb-476f-a373-45c2d402a870', 'memSize': '16384', 'display': 'vnc', 'vmName': 'HostedEngine', 'spiceSecureChannels': 'smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir', 'smp': '2', 'maxVCpus': '8', 'cpuType': 'Cascadelake-Server,+md-clear,+mds-no,-hle,-rtm,+arch-capabilities', 'emulatedMachine': '', 'devices': [{'index': '2', 'iface': 'ide', 'address': {'controller': '0', 'target': '0', 'unit': '0', 'bus': '1', 'type': 'drive'}, 'specParams': {}, 'readonly': 'true', 'deviceId': '', 'path': '', 'device': 'cdrom', 'shared': 'false', 'type': 'disk'}, {'index': '0', 'iface': 'virtio', 'format': 'raw', 'poolID': '00000000-0000-0000-0000-000000000000', 'volumeID': '15f606c1-757f-44a8-adc7-85e75ff02944', 'imageID': '8c2b8355-ba2d-4ffa-8055-0034d56f0dd7', 'specParams': {}, 'readonly': 'false', 'domainID': '7c9397f5-859b-4fba-bcc5-4ecbd454a189', 'optional': 'false', 'deviceId': '15f606c1-757f-44a8-adc7-85e75ff02944', 'address': {'bus': '0x00', 'slot': '0x06', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'exclusive', 'propagateErrors': 'off', 'type': 'disk', 'bootOrder': '1'}, {'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}, {'nicModel': 'pv', 'macAddr': '00:16:3e:68:a9:cc', 'linkActive': 'true', 'network': 'ovirtmgmt', 'specParams': {}, 'deviceId': '', 'address': {'bus': '0x00', 'slot': '0x03', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'device': 'console', 'type': 'console'}, {'device': 'vga', 'alias': 'video0', 'type': 'video'}, {'device': 'vnc', 'type': 'graphics'}, {'device': 'virtio', 'specParams': {'source': 'urandom'}, 'model': 'virtio', 'type': 'rng'}]}} failed: (code=100, message=General Exception: ("'xml'",)) VM failed to launch [root@novirt2 530490b0-935e-46a8-84f6-6cd6618c6f0d]#
I would start the engine the regular way , then use virsh dumpxml HostedEngine > HE.xml -> edit to your needs.
In vdsm.log
2020-04-20 17:16:51,401+0200 ERROR (jsonrpc/3) [api] FINISH create error='xml' (api:134) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 124, in method ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/API.py", line 190, in create xml = vmParams.get('_srcDomXML') or vmParams['xml'] KeyError: 'xml' 2020-04-20 17:16:51,402+0200 INFO (jsonrpc/3) [api.virt] FINISH create return={'status': {'code': 100, 'message': 'General Exception: ("\'xml\'",)'}} from=::1,44384, vmId=22082519-70bb-476f-a373-45c2d402a870 (api:54) 2020-04-20 17:16:51,402+0200 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call VM.create failed (error 100) in 0.00 seconds (__init__:312) 2020-04-20 17:16:51,598+0200 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312)
and the same if I leave vm.conf unmodified....
Gianluca
On Mon, Apr 20, 2020 at 6:01 PM Strahil Nikolov <hunter86_bg@yahoo.com> wrote: [snip]
I would start the engine the regular way , then use virsh dumpxml HostedEngine > HE.xml -> edit to your needs.
As you can see if you read carefully the mail, the engine doesn't start at all, due to the unknown feature set: this is the reason I'm going to try to modify it, otherwise I would have no need to interact. As the engine is not able to start, obviously I cannot dump any xml..... Gianluca
On April 20, 2020 10:06:30 PM GMT+03:00, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Mon, Apr 20, 2020 at 6:01 PM Strahil Nikolov <hunter86_bg@yahoo.com> wrote: [snip]
I would start the engine the regular way , then use virsh dumpxml HostedEngine > HE.xml -> edit to your needs.
As you can see if you read carefully the mail, the engine doesn't start at all, due to the unknown feature set: this is the reason I'm going to try to modify it, otherwise I would have no need to interact. As the engine is not able to start, obviously I cannot dump any xml.....
Gianluca
You can still start it in pause mode, or to get the config from the vdsm.log on the machine it is being started. Best Rregards, Strahil Nikolov
participants (2)
-
Gianluca Cecchi -
Strahil Nikolov