Non-responsive host, VM's are still running - how to resolve?
by Artem Tambovskiy
Apparently, i lost the host which was running hosted-engine and another 4
VM's exactly during migration of second host from bare-metal to second host
in the cluster. For some reason first host entered the "Non reponsive"
state. The interesting thing is that hosted-engine and all other VM's up
and running, so its like a communication problem between hosted-engine and
host.
The engine.log at hosted-engine is full of following messages:
2017-11-14 17:06:43,158Z INFO
[org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor)
[] Connecting to ovirt2/80.239.162.106
2017-11-14 17:06:43,159Z ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand]
(DefaultQuartzScheduler9) [50938c3] Command
'GetAllVmStatsVDSCommand(HostName = ovirt2.telia.ru,
VdsIdVDSCommandParametersBase:{runAsync='true',
hostId='3970247c-69eb-4bd8-b263-9100703a8243'})' execution failed:
java.net.NoRouteToHostException: No route to host
2017-11-14 17:06:43,159Z INFO
[org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher]
(DefaultQuartzScheduler9) [50938c3] Failed to fetch vms info for host '
ovirt2.telia.ru' - skipping VMs monitoring.
2017-11-14 17:06:45,929Z INFO
[org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor)
[] Connecting to ovirt2/80.239.162.106
2017-11-14 17:06:45,930Z ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
(DefaultQuartzScheduler2) [6080f1cc] Command
'GetCapabilitiesVDSCommand(HostName = ovirt2.telia.ru,
VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
hostId='3970247c-69eb-4bd8-b263-9100703a8243',
vds='Host[ovirt2.telia.ru,3970247c-69eb-4bd8-b263-9100703a8243]'})'
execution failed: java.net.NoRouteToHostException: No route to host
2017-11-14 17:06:45,930Z ERROR
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
(DefaultQuartzScheduler2) [6080f1cc] Failure to refresh host '
ovirt2.telia.ru' runtime info: java.net.NoRouteToHostException: No route to
host
2017-11-14 17:06:48,933Z INFO
[org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor)
[] Connecting to ovirt2/80.239.162.106
2017-11-14 17:06:48,934Z ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
(DefaultQuartzScheduler6) [1a64dfea] Command
'GetCapabilitiesVDSCommand(HostName = ovirt2.telia.ru,
VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
hostId='3970247c-69eb-4bd8-b263-9100703a8243',
vds='Host[ovirt2.telia.ru,3970247c-69eb-4bd8-b263-9100703a8243]'})'
execution failed: java.net.NoRouteToHostException: No route to host
2017-11-14 17:06:48,934Z ERROR
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
(DefaultQuartzScheduler6) [1a64dfea] Failure to refresh host '
ovirt2.telia.ru' runtime info: java.net.NoRouteToHostException: No route to
host
2017-11-14 17:06:50,931Z INFO
[org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor)
[] Connecting to ovirt2/80.239.162.106
2017-11-14 17:06:50,932Z ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand]
(DefaultQuartzScheduler4) [6b19d168] Command 'SpmStatusVDSCommand(HostName
= ovirt2.telia.ru, SpmStatusVDSCommandParameters:{runAsync='true',
hostId='3970247c-69eb-4bd8-b263-9100703a8243',
storagePoolId='5a044257-02ec-0382-0243-0000000001f2'})' execution failed:
java.net.NoRouteToHostException: No route to host
2017-11-14 17:06:50,939Z INFO
[org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor)
[] Connecting to ovirt2/80.239.162.106
2017-11-14 17:06:50,940Z ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
(DefaultQuartzScheduler4) [6b19d168]
IrsBroker::Failed::GetStoragePoolInfoVDS
2017-11-14 17:06:50,940Z ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand]
(DefaultQuartzScheduler4) [6b19d168] Command 'GetStoragePoolInfoVDSCommand(
GetStoragePoolInfoVDSCommandParameters:{runAsync='true',
storagePoolId='5a044257-02ec-0382-0243-0000000001f2',
ignoreFailoverLimit='true'})' execution failed: IRSProtocolException:
2017-11-14 17:06:51,937Z INFO
[org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor)
[] Connecting to ovirt2/80.239.162.106
2017-11-14 17:06:51,938Z ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
(DefaultQuartzScheduler7) [7f23a3bd] Command
'GetCapabilitiesVDSCommand(HostName = ovirt2.telia.ru,
VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
hostId='3970247c-69eb-4bd8-b263-9100703a8243',
vds='Host[ovirt2.telia.ru,3970247c-69eb-4bd8-b263-9100703a8243]'})'
execution failed: java.net.NoRouteToHostException: No route to host
2017-11-14 17:06:51,938Z ERROR
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
(DefaultQuartzScheduler7) [7f23a3bd] Failure to refresh host '
ovirt2.telia.ru' runtime info: java.net.NoRouteToHostException: No route to
host
2017-11-14 17:06:54,941Z INFO
[org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor)
[] Connecting to ovirt2/80.239.162.106
2017-11-14 17:06:54,942Z ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
(DefaultQuartzScheduler2) [7a769f6c] Command
'GetCapabilitiesVDSCommand(HostName = ovirt2.telia.ru,
VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
hostId='3970247c-69eb-4bd8-b263-9100703a8243',
vds='Host[ovirt2.telia.ru,3970247c-69eb-4bd8-b263-9100703a8243]'})'
execution failed: java.net.NoRouteToHostException: No route to host
2017-11-14 17:06:54,942Z ERROR
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
(DefaultQuartzScheduler2) [7a769f6c] Failure to refresh host '
ovirt2.telia.ru' runtime info: java.net.NoRouteToHostException: No route to
host
Its a bit weird, since I can ping and login via ssh to the host from
hosted-engine with no problem. I have added second host to the cluster, but
it not running hosted-engine. Any suggestion for the further steps? Just
reboot the host and hope for the best?
Regards,
Artem
6 years, 6 months
hosted exchange failed to install
by Rudi Ahlers
Hi,
Can someone please help?
I installed hosted exchange but specified the wrong interface, and thus
couldn't access it. So I removed it (yum install) and reinstalled it, and
re-ran the deploy but got the following error:
Please confirm installation settings (Yes, No)[Yes]:
[ INFO ] Stage: Transaction setup
[ INFO ] Stage: Misc configuration
[ INFO ] Stage: Package installation
[ INFO ] Stage: Misc configuration
[ INFO ] Configuring libvirt
[ INFO ] Configuring VDSM
[WARNING] VDSM configuration file not found: creating a new configuration
file
[ INFO ] Starting vdsmd
[ INFO ] Creating Storage Domain
[ ERROR ] Failed to execute stage 'Misc configuration': Storage domain is
not empty - requires cleaning: (u'srv1:/engine',)
[ INFO ] Yum Performing yum transaction rollback
[ INFO ] Stage: Clean up
[ INFO ] Generating answer file
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20171114162130.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Hosted Engine deployment failed: this system is not reliable,
please check the issue,fix and redeploy
Log file is located at
/var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20171114161520-km3qok.log
I am honestly not sure why it would think "this system is not reliable".
How do I check what is actually wrong?
The log file shows the same error:
tail -f
/var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20171114161520-km3qok.log
2017-11-14 16:21:30 DEBUG otopi.context context._executeMethod:134
condition False
2017-11-14 16:21:30 INFO otopi.context context.runSequence:687 Stage:
Termination
2017-11-14 16:21:30 DEBUG otopi.context context.runSequence:691 STAGE
terminate
2017-11-14 16:21:30 DEBUG otopi.context context._executeMethod:128 Stage
terminate METHOD otopi.plugins.gr_he_common.core.misc.Plugin._terminate
2017-11-14 16:21:30 ERROR otopi.plugins.gr_he_common.core.misc
misc._terminate:178 Hosted Engine deployment failed: this system is not
reliable, please check the issue,fix and redeploy
2017-11-14 16:21:30 DEBUG otopi.plugins.otopi.dialog.human
dialog.__logString:204 DIALOG:SEND Log file is located at
/var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20171114161520-km3qok.log
2017-11-14 16:21:30 DEBUG otopi.context context._executeMethod:128 Stage
terminate METHOD otopi.plugins.otopi.dialog.human.Plugin._terminate
2017-11-14 16:21:30 DEBUG otopi.context context._executeMethod:128 Stage
terminate METHOD otopi.plugins.otopi.dialog.machine.Plugin._terminate
2017-11-14 16:21:30 DEBUG otopi.context context._executeMethod:134
condition False
2017-11-14 16:21:30 DEBUG otopi.context context._executeMethod:128 Stage
terminate METHOD otopi.plugins.otopi.core.log.Plugin._terminate
--
Kind Regards
Rudi Ahlers
Website: http://www.rudiahlers.co.za
6 years, 6 months
Issue migrating hard drive to new vm store
by Bryan Sockel
Having an issue moving a hard disk from one vm data store new a newly
created gluster data store. I can shut down the machine and copy the hard
drive, detach the old hard drive and attach the new hard drive, but i would
prefer to keep the vm on line when moving the disk.
I have attached a portion of the vdsm.log file.
Thanks
Bryan
6 years, 6 months
4.1 engine-iso-uploader / root password glitch
by andreil1＠starlett.lv
Hi,
I'm trying to upload iso with this coomand.
engine-iso-uploader --ssh-user=root --iso-domain=iso upload suse.iso
Please provide the REST API password for the admin@internal oVirt Engine
user (CTRL+D to abort):
This go OK.
However, then it asks root password, I enter it, then it asks again and
again. Root password is correct for sure, becuase I can coonect vis ssh
from terminal.
How to fix this problem?
May be its possible just to copy files manually?
Thanks in advance.
Andrei
6 years, 6 months
Re: [ovirt-users] CIFS Share
by Arthur Melo
Great answare. Thanks!
Atenciosamente,
Arthur Melo
Linux User #302250
2017-11-12 6:23 GMT-02:00 Yaniv Kaul <ykaul(a)redhat.com>:
>
>
> On Thu, Nov 9, 2017 at 8:28 PM, Arthur Melo <arthur(a)afabrica.net> wrote:
>
>> Is it possible to mount a export share using CIFS?
>>
>
> We generally support any POSIX compliant file system which also support
> Direct IO for data domain - I'm not sure if CIFS in general and the
> specific implementation you use support both.
> If it does, it should work for the data domain - which you can detach and
> attach between environments.
> Export domain functionality is really done with NFS.
>
> You could also upload and download disks via your browser and place the
> disks on a CIFS share.
> Y.
>
>
>>
>> Atenciosamente,
>> Arthur Melo
>> Linux User #302250
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>
6 years, 6 months
how to clean stuck task
by Gianluca Cecchi
Hello,
I have a task that seems stuck in webadmin gui, in the sens tha I have
"Tasks(1)" listed
The task is
Restoring VM Snapshot Active VM before the preview of VM snaptest
and the VM is powered down.
Screenshot of expanded steps of task, that actually seem all completed, is
here:
https://drive.google.com/file/d/1bfl_gEfVotIrxGC9TDzPHPCeRub41mUa/view?us...
Any hint on what to do to clean things? I'm on oVirt 4.1.6.2-1.el7.centos
and I would like to clean before upgrading to 4.1.7.
Thanks
Gianluca
6 years, 6 months
Error during SSO authentication Cannot authenticate user 'admin@internal'
by Sverker Abrahamsson
Since upgrading my test lab to ovirt 4.2 I can't get ovirt-provider-ovn
to work. From ovirt-provider-ovn.log:
2017-11-14 00:40:15,795 Request: POST : /v2.0///tokens
2017-11-14 00:40:15,795 Request body:
{
"auth" : {
"passwordCredentials" : {
"username" : "admin@internal",
"password" : "xxxxxxxxx"
}
}
}
2017-11-14 00:40:15,819 Starting new HTTPS connection (1): h2-int
2017-11-14 00:40:20,829 "POST /ovirt-engine/sso/oauth/token HTTP/1.1"
400 118
2017-11-14 00:40:20,830 Error during SSO authentication Cannot
authenticate user 'admin@internal': The username or password is
incorrect.. : access_deniedNone
Traceback (most recent call last):
File "/usr/share/ovirt-provider-ovn/handlers/base_handler.py", line
119, in _handle_request
method, path_parts, content)
File "/usr/share/ovirt-provider-ovn/handlers/selecting_handler.py",
line 177, in handle_request
handler, content, parameters
File "/usr/share/ovirt-provider-ovn/handlers/keystone.py", line 28,
in call_response_handler
return response_handler(content, parameters)
File "/usr/share/ovirt-provider-ovn/handlers/keystone_responses.py",
line 58, in post_tokens
user_password=user_password)
File "/usr/share/ovirt-provider-ovn/auth/plugin_facade.py", line 26,
in create_token
return auth.core.plugin.create_token(user_at_domain, user_password)
File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/plugin.py",
line 48, in create_token
timeout=self._timeout())
File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line
62, in create_token
username, password, engine_url, ca_file, timeout)
File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line
54, in wrapper
_check_for_error(response)
File "/usr/share/ovirt-provider-ovn/auth/plugins/ovirt/sso.py", line
168, in _check_for_error
result['error'], details))
Unauthorized: Error during SSO authentication Cannot authenticate user
'admin@internal': The username or password is incorrect.. :
access_deniedNone
And in engine.log:
2017-11-14 00:40:20,828+01 ERROR
[org.ovirt.engine.core.sso.utils.SsoUtils] (default task-16) []
OAuthException access_denied: Cannot authenticate user 'admin@internal':
The username or password is incorrect..
The password in the request is the same as used to log in to the admin
portal and works fine there.
/Sverker
6 years, 6 months
Host Power Management Configuration questions
by Artem Tambovskiy
Trying to configure power management for a certain host and fence agent
always fail when I'm pressing Test button.
At the same time from command line on the same host all looks good:
[root@ovirt ~]# fence_ipmilan -a 172.16.22.1 -l user -p pwd -o status -v -P
Executing: /usr/bin/ipmitool -I lanplus -H 172.16.22.1 -p 623 -U user -P
pwd -L ADMINISTRATOR chassis power status
0 Chassis Power is on
Status: ON
[root@ovirt ~]#
What could be the reason?
Regards,
Artem
6 years, 6 months
Transfer from one Storage to Other is very slow
by Jon bae
Hello everybody,
I have a node where I installed nfs storage and I have a second nfs network
storage. My node have a bond (4) with two nics and the other storage have a
bond with 4 nics.
My oVirt engine runs as a VM on my network storage.
The bond on the node side is relative new, before I had this setup, the
speed was good. But at the same time I also move my ovirt engine from a
third server to the network storage.
My problem is now, when I move a VM disk from the network storage to the
storage on the node, I have very pure speed. A VM with 140GB takes more
then an hour to transfer.
When I make speed tests with iperf3 I got this speed:
- from oVirt to network storage: 20Gbits/s
- from network storage to node storage: 9.5Gbits/s
- from ovrit to node storage 9.5 Gbits/s
When I transfer a VM disk, iftop shows almost no traffic.
Have you an idea what is happen here?
Regards
Jonathan
6 years, 6 months
