Restoring hosted engine from backup fails on new FC storage domain creation
by Juhani Rautiainen
Hi!
I started the upgrade from 4.3->4.4. Now I'm stuck as restoring the
backup fails to create the correct storage domain for Hosted Engine.
How can I create one? Error from ansible task is:
[ ERROR ] ovirtsdk4.Error: Fault reason is "Operation Failed". Fault
detail is "[Domain format is different from master storage domain
format]". HTTP response code is 400.
[ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg":
"Fault reason is \"Operation Failed\". Fault detail is \"[Domain
format is different from master storage domain format]\". HTTP
response code is 400."}
And from UI I can see that what has been created is a data domain. Not
data (master) domain as old one was. Now I'm stuck here. This is a
fibre channel system where I'm trying to do this if it is relevant.
What I could find from the logs is this:
2021-04-27 09:36:06,925+0300 DEBUG
otopi.ovirt_hosted_engine_setup.ansible_utils
ansible_utils._process_output:105 storage_domain_details: {'changed':
False, 'ovirt_storage_domains': [{'href':
'/ovirt-engine/api/storagedomains/dd52022b-7616-47f6-9534-6f1a4084fdf4',
'comment': '', 'description': '', 'id':
'dd52022b-7616-47f6-9534-6f1a4084fdf4', 'name': 'hosted_storage',
'available': 531502202880, 'backup': False, 'block_size': 512,
'committed': 0, 'critical_space_action_blocker': 5,
'discard_after_delete': True, 'disk_profiles': [], 'disk_snapshots':
[], 'disks': [], 'external_status': 'ok', 'master': False,
'permissions': [], 'status': 'unattached', 'storage': {'type': 'fcp',
'volume_group': {'id': 'HRLDCn-p7X2-5X2O-vm4h-1Wb9-wAMu-WkIwit',
'logical_units': [{'discard_max_size': 268435456,
'discard_zeroes_data': False, 'id':
'36000d31005b4f6000000000000000029', 'lun_mapping': 3, 'paths': 0,
'product_id': 'Compellent Vol', 'serial':
'SCOMPELNTCompellent_Vol_0005b4f6-00000029', 'size': 536870912000,
'storage_domain_id': 'dd52022b-7616-47f6-9534-6f1a4084fdf4',
'vendor_id': 'COMPELNT', 'volume_group_id':
'HRLDCn-p7X2-5X2O-vm4h-1Wb9-wAMu-WkIwit'}]}}, 'storage_connections':
[], 'storage_format': 'v5', 'supports_discard': True,
'supports_discard_zeroes_data': False, 'templates': [], 'type':
'data', 'used': 4294967296, 'vms': [], 'warning_low_space_indicator':
10, 'wipe_after_delete': False}], 'failed': False}
'master': False? I'm not sure if this creation or check. I tried this
opration twice. I removed the new hosted_storage and remove also old
hosted_engine domain on second try to make sure that it doesn't
prevent creation of another master. No luck with that.
Thanks,
Juhani
3 years, 6 months
Import VM with Xen Based in Ovirt
by hemaa＠emiratesnbd.com
Hi All,
I am trying to migrate a VM from a Xen Based virtualization to Ovirt with the import option in GUI
But I am not able to list any stopped VMs on the Xen host. So what I did is I started the VM and got it listed. Then selected the VM and moved to right hand section. Clicked on next button. But then it gave an error
"All chosen VMs couldn't be retrieved by the external system and therefore have been filtered. Please see log for details."
When I check the libvirt logs in xen hypervisor I see error as below. There is no other errors in engine logs or vdsm logs
2021-04-22 08:52:36.796+0000: 30219: error : do_connect:116 : internal error: failed to connect to xend
2021-04-22 08:52:36.830+0000: 30219: error : virNodeSuspendSupportsTarget:332 : internal error: Cannot probe for supported suspend types
2021-04-22 08:52:36.830+0000: 30219: warning : xenHypervisorMakeCapabilities:2550 : Failed to get host power management capabilities
2021-04-22 08:52:36.832+0000: 30220: error : virConnectListAllDomains:6803 : this function is not supported by the connection driver: virConnectListAllDomains
This is the same error I got when I tried importing a running VM as well. Any idea on what could be a possible reason behind this?
3 years, 6 months
Faild ovirt-websocket-proxy.
by jesado74＠gmail.com
Hello again.
After getting the hosted-engine vm back, now I have the following problem. I can't use the graphic console. The websocket-proy fails me. Surely when I update the certificates I have missed some intermediate step, but I do not know how to follow.
Anyone know of a link to follow the certificate change procedure in the websocket-proxy service.
Thank you.
PS: Here I leave the output of the service:
● ovirt-websocket-proxy.service - oVirt Engine websockets proxy
Loaded: loaded (/usr/lib/systemd/system/ovirt-websocket-proxy.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2021-04-28 02:03:54 UTC; 7h ago
Process: 1694 ExecStart = / usr / share / ovirt-engine / services / ovirt-websocket-proxy / ovirt-websocket-proxy.py --systemd = notify $ EXTRA_ARGS start (code = exited, status = 1 / FAILURE)
Main PID: 1694 (code = exited, status = 1 / FAILURE)
Apr 28 02:03:53 ovirt.myhostedengine.com systemd [1]: Starting oVirt Engine websockets proxy ...
Apr 28 02:03:54 ovirt.myhostedengine.com ovirt-websocket [1694]: 2021-04-28 02: 03: 54,614 + 0000 ovirt-websocket-proxy: ERROR run: 554 Error: SSL_ONLY is set but SSL_CERTIFICATE or SSL_KEY file not found.
Apr 28 02:03:54ovirt.hostedengine.com ovirt-websocket-proxy.py [1694]: ovirt-websocket-proxy [1694] ERROR run: 554 Error: SSL_ONLY is set but SSL_CERTIFICATE or SSL_KEY file not found.
Apr 28 02:03:54 ovirt.myhostedengine.com systemd [1]: ovirt-websocket-proxy.service: main process exited, code = exited, status = 1 / FAILURE
Apr 28 02:03:54 ovirt.myhostedengine.com systemd [1]: Failed to start oVirt Engine websockets proxy.
Apr 28 02:03:54 ovirt.myhostedengine.com systemd [1]: Unit ovirt-websocket-proxy.service entered failed state.
Apr 28 02:03:54 ovirt.myhostedengine.com systemd [1]: ovirt-websocket-proxy.service failed.
Sorry if I say any inconsistencies, but I'm new to this whole ovirt topic.
Thanks again.
3 years, 6 months
Ovirt-engine Setup Error-Failed to execute stage 'Misc configuration
by rajkumar madhu
Hi Team,
Unable to install Ovirt in my centos server.while executing Engine-setup
command ,I am getting below error.Please help me on this .
Please confirm installation settings (OK, Cancel) [OK]: [ INFO ] Stage:
Transaction setup
[ INFO ] Stopping engine service
[ INFO ] Stopping ovirt-fence-kdump-listener service
[ INFO ] Stopping dwh service
[ INFO ] Stopping vmconsole-proxy service
[ INFO ] Stopping websocket-proxy service
[ INFO ] Stage: Misc configuration (early)
[ INFO ] Stage: Package installation
[ INFO ] Stage: Misc configuration
[ ERROR ] Failed to execute stage 'Misc configuration': [Errno 2] No such
file or directory: '/etc/ovirt-engine/extensions.d'
[ INFO ] DNF Performing DNF transaction rollback
[ INFO ] Stage: Clean up
Log file is located at
/var/log/ovirt-engine/setup/ovirt-engine-setup-20210427090149-r0i80v.log
[ INFO ] Generating answer file
'/var/lib/ovirt-engine/setup/answers/20210427090311-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Execution of setup failed
Regards,
Rajkumar M.
9176772077
3 years, 6 months
Ovirt 4.4.6.5-1 engine-setup failed
by Don Dupuis
I am installing Ovirt 4.4.6.5-1 engine on CentOS Stream 8 on a dedicated
vm. Towards the end of the engine-setup session I have a failure:
[ INFO ] Stage: Transaction setup
[ INFO ] Stopping engine service
[ INFO ] Stopping ovirt-fence-kdump-listener service
[ INFO ] Stopping dwh service
[ INFO ] Stopping vmconsole-proxy service
[ INFO ] Stopping websocket-proxy service
[ INFO ] Stage: Misc configuration (early)
[ INFO ] Stage: Package installation
[ INFO ] Stage: Misc configuration
[ INFO ] Upgrading CA
[ INFO ] Initializing PostgreSQL
[ INFO ] Creating PostgreSQL 'engine' database
[ INFO ] Configuring PostgreSQL
[ INFO ] Creating PostgreSQL 'ovirt_engine_history' database
[ INFO ] Configuring PostgreSQL
[ INFO ] Creating CA: /etc/pki/ovirt-engine/ca.pem
[ INFO ] Creating CA: /etc/pki/ovirt-engine/qemu-ca.pem
[ INFO ] Updating OVN SSL configuration
[ INFO ] Updating OVN timeout configuration
[ INFO ] Creating/refreshing DWH database schema
[ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts
[ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts
[ INFO ] Configuring WebSocket Proxy
[ INFO ] Creating/refreshing Engine database schema
[ INFO ] Creating a user for Grafana
[ INFO ] Creating/refreshing Engine 'internal' domain database schema
[ INFO ] Creating default mac pool range
[ INFO ] Adding default OVN provider to database
[ INFO ] Adding OVN provider secret to database
[ INFO ] Setting a password for internal user admin
[ INFO ] Install selinux module
/usr/share/ovirt-engine/selinux/ansible-runner-service.cil
[ INFO ] Generating post install configuration file
'/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'
[ INFO ] Stage: Transaction commit
[ INFO ] Stage: Closing up
[ INFO ] Starting engine service
[ INFO ] Starting dwh service
[ INFO ] Starting Grafana service
[ ERROR ] Failed to execute stage 'Closing up': Failed to start service
'ovirt-imageio'
[ INFO ] Stage: Clean up
Log file is located at
/var/log/ovirt-engine/setup/ovirt-engine-setup-20210426111225-g31wa5.log
[ INFO ] Generating answer file
'/var/lib/ovirt-engine/setup/answers/20210426111501-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Execution of setup failed
Below is output of /var/log/ovirt-imageio/daemon.log:
2021-04-26 11:15:01,481 ERROR (MainThread) [server] Server failed
Traceback (most recent call last):
File
"/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py",
line 46, in main
server = Server(cfg)
File
"/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/server.py",
line 110, in __init__
self.remote_service = services.RemoteService(self.config, self.auth)
File
"/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/services.py",
line 73, in __init__
self._server = http.Server((config.remote.host, port), http.Connection)
File
"/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line
91, in __init__
self.create_socket(prefer_ipv4)
File
"/usr/lib64/python3.6/site-packages/ovirt_imageio/_internal/http.py", line
123, in create_socket
self.socket = socket.socket(self.address_family, self.socket_type)
File "/usr/lib64/python3.6/socket.py", line 144, in __init__
_socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 97] Address family not supported by protocol
I am not seeing errors in /var/log/ovirt-engine/setup/setup.log. What other
information do you need that is revelant to this issue?
Thanks
Don
3 years, 6 months
Brick didn't come up after planned Poweroutage
by Marcel d'Heureuse
Hi Guys,
We hat a planned power outage and we have stopped all VMS and try to keep
the servers alive as long as possible.
Some Servers made a Reboot by itself but the IPMI interface was not
reachable for the engine. The some hosts have made a restart.
Now all systems running and all 180 VMs too. But one Brick of the hosted
engine didn't come up.
Did you have an idea. Please find attached some screenshots.
Our Hosted Engine has 6 bricks on 6 different Servers.
A restart of the Host didn't work. The Server Restart of glusterd didn't
work.
I hope you have some more Ideas?
Thanks
Br
Marcel
3 years, 6 months
Re: Attempting to detach a storage domain
by Vojtech Juranek
On Tuesday, 27 April 2021 02:18:18 CEST Matthew.Stier(a)fujitsu.com wrote:
> There is no /var/log/vdsm directory on the hosted-engine.
>
> This is from the current host, hosting the engine: aaa.bbb.ccc.100 is the
> hosted-engine
> 2021-04-26 19:12:42,760-0500 INFO (jsonrpc/2) [api.host] START
> getCapabilities() from=::ffff:aaa.bbb.ccc.100,37812 (api:48)
2021-04-26
> 19:12:42,763-0500 INFO (jsonrpc/2) [api.host] FINISH getCapabilities
> error=internal error: Duplicate key from=::ffff:168.127.151.100,37812
> (api:52) 2021-04-26 19:12:42,763-0500 ERROR (jsonrpc/2)
> [jsonrpc.JsonRpcServer] Internal server error (__init__:350) Traceback
> (most recent call last):
> File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345,
> in _handle_request
res = method(**params)
> File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in
> _dynamicMethod
result = fn(*methodArgs)
> File "<string>", line 2, in getCapabilities
> File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in
> method
ret = func(*args, **kwargs)
> File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1371, in
> getCapabilities
c = caps.get()
> File "/usr/lib/python2.7/site-packages/vdsm/host/caps.py", line 95, in
> get
machinetype.compatible_cpu_models())
> File "/usr/lib/python2.7/site-packages/vdsm/common/cache.py", line 43, in
> __call__
value = self.func(*args)
> File "/usr/lib/python2.7/site-packages/vdsm/machinetype.py", line 142, in
> compatible_cpu_models
all_models = domain_cpu_models(c, arch, cpu_mode)
> File "/usr/lib/python2.7/site-packages/vdsm/machinetype.py", line 97, in
> domain_cpu_models
domcaps = conn.getDomainCapabilities(None, arch, None,
> virt_type, 0) File
> "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line
> 131, in wrapper ret = f(*args, **kwargs)
> File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94,
> in wrapper
return func(inst, *args, **kwargs)
> File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3844, in
> getDomainCapabilities
if ret is None: raise libvirtError
> ('virConnectGetDomainCapabilities() failed', conn=self) libvirtError:
> internal error: Duplicate key
looks like https://bugzilla.redhat.com/show_bug.cgi?id=1791790
The fix should be contained in recent oVirt relases.
> 2021-04-26 19:12:42,764-0500 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC
> call Host.getCapabilities failed (error -32603) in 0.00 seconds
> (__init__:312)
>
> -----Original Message-----
> From: Nir Soffer <nsoffer(a)redhat.com>
> Sent: Monday, April 26, 2021 5:12 PM
> To: Stier, Matthew <Matthew.Stier(a)fujitsu.com>
> Cc: Benny Zlotnik <bzlotnik(a)redhat.com>; users(a)ovirt.org
> Subject: Re: [ovirt-users] Re: Attempting to detach a storage domain
>
> On Tue, Apr 27, 2021 at 12:48 AM Matthew.Stier(a)fujitsu.com
> <Matthew.Stier(a)fujitsu.com> wrote:
> >
> >
> > I'm getting tons of these in the
> > hosted-engine:/var/log/ovirt-engine/engine.log
> >
> >
> >
> > 2021-04-26 16:40:40,260-05 ERROR
> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> > (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] EVENT_ID:
> > VDS_BROKER_COMMAND_FAILURE(10,802), VDSM xxxx.yyyy.zzz command Get
> > Host Capabilities failed: Internal JSON-RPC error: {'reason':
> > 'internal error: Duplicate key'}
>
>
> This looks like bad response from vdsm. Can you share the vdsm log from the
> host xxxx.yyyy.zzz?
> /var/log/vdsm/vdsm.log
>
> Nir
>
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/ List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/Z2GOGEZ2DZX34
> ADZ5L2BATIINYGVWYJD/
3 years, 6 months
Cannot delete snapshot
by Giulio Casella
Hi,
I'm having (another) issue working with snapshots.
Since a few days my backup system (storware vprotect) cannot delete a
snapshot. Message reported by SPM is:
HSMGetAllTasksStatusesVDS failed: value=Volume does not exist:
('6180c2e8-141a-4126-bcb1-5af5caa98175',) abortedcode=201
Other snapshots from other VMs are working fine (created and deleted as
needed).
I verified it, I can reach on SPM filesystem other snapshots from the
same VM, but that one doesn't exist.
I suspect that during normal backup management that snapshot has been
correctly removed from file system, but still live in ovirt database.
I'm pretty sure vProtect use ovirt API to operate, it doesn't directly
interact with ovirt database. That's why I'm talking to you and not to
storware guys.
Is there a (safe) way to get rid of that snapshot?
TIA
Regards,
Giulio Casella
3 years, 6 months
