[ovirt-users] hosted-engine vm-status stale data and cluster seems "broken"
Sahina Bose
sabose at redhat.com
Wed Jun 15 05:09:45 UTC 2016
Are the ovirt-ha-agent and ovirt-ha-broker services running on all the
nodes? If they are, check the agent.log and broker.log for errors.
On 06/14/2016 05:29 PM, Charles Kozler wrote:
> Anyone have any other possible information on this? I've noticed this
> issue before and usually it just takes a bit of time for the cluster
> to 'settle' after some node reboots but its been a few days and its
> still marked as stale.
>
>
>
> --== Host 1 status ==--
>
> Status up-to-date : False
> Hostname : njsevcnp01
> Host ID : 1
> Engine status : unknown stale-data
> Score : 0
> stopped : True
> Local maintenance : False
> crc32 : 260dbf06
> Host timestamp : 327
>
>
> --== Host 2 status ==--
>
> Status up-to-date : False
> Hostname : njsevcnp02
> Host ID : 2
> Engine status : unknown stale-data
> Score : 0
> stopped : True
> Local maintenance : False
> crc32 : 25da07df
> Host timestamp : 3030
>
>
> --== Host 3 status ==--
>
> Status up-to-date : False
> Hostname : njsevcnp03
> Host ID : 3
> Engine status : unknown stale-data
> Score : 0
> stopped : True
> Local maintenance : False
> crc32 : c67818cb
> Host timestamp : 10877406
>
>
> && vdsClient on node2 showing hosted engine is up on node 2
>
> 48207078-8cb0-413c-8984-40aa772f4d94
> Status = Up
> nicModel = rtl8139,pv
> statusTime = 4540044460
> emulatedMachine = pc
> pid = 30571
> vmName = HostedEngine
> devices = [{'device': 'memballoon', 'specParams': {'model': 'none'},
> 'type': 'balloon', 'alias': 'balloon0'}, {'alias': 'scsi0',
> 'deviceId': '17f10db1-2e9e-4422-9ea5-61a628072e29', 'address':
> {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci',
> 'function': '0x0'}, 'device': 'scsi', 'model': 'virtio-scsi', 'type':
> 'controller'}, {'device': 'usb', 'alias': 'usb', 'type': 'controller',
> 'deviceId': '9be34ac0-7d00-4a95-bdfe-5b328fc1355b', 'address':
> {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci',
> 'function': '0x2'}}, {'device': 'ide', 'alias': 'ide', 'type':
> 'controller', 'deviceId': '222629a8-0dd6-4e8e-9b42-43aac314c0c2',
> 'address': {'slot': '0x01', 'bus': '0x00', 'domain': '0x0000', 'type':
> 'pci', 'function': '0x1'}}, {'device': 'virtio-serial', 'alias':
> 'virtio-serial0', 'type': 'controller', 'deviceId':
> '7cbccd04-853a-408f-94c2-5b10b641b7af', 'address': {'slot': '0x05',
> 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}},
> {'device': 'vnc', 'specParams': {'spiceSecureChannels':
> 'smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir',
> 'displayIp': '0'}, 'type': 'graphics', 'port': '5900'}, {'nicModel':
> 'pv', 'macAddr': '00:16:3e:16:83:91', 'linkActive': True, 'network':
> 'ovirtmgmt', 'alias': 'net0', 'deviceId':
> '3f679659-142c-41f3-a69d-4264d7234fbc', 'address': {'slot': '0x03',
> 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'},
> 'device': 'bridge', 'type': 'interface', 'name': 'vnet0'}, {'address':
> {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci',
> 'function': '0x0'}, 'volumeInfo': {'domainID':
> 'c6323975-2966-409d-b9e0-48370a513a98', 'volType': 'path',
> 'leaseOffset': 0, 'volumeID': 'aa66d378-5a5f-490c-b0ab-993b79838d95',
> 'leasePath':
> '/rhev/data-center/mnt/nas01:_volume1_vm__os_ovirt36__engine/c6323975-2966-409d-b9e0-48370a513a98/images/8518ef4a-7b17-4291-856c-81875ba4e264/aa66d378-5a5f-490c-b0ab-993b79838d95.lease',
> 'imageID': '8518ef4a-7b17-4291-856c-81875ba4e264', 'path':
> '/rhev/data-center/mnt/nas01:_volume1_vm__os_ovirt36__engine/c6323975-2966-409d-b9e0-48370a513a98/images/8518ef4a-7b17-4291-856c-81875ba4e264/aa66d378-5a5f-490c-b0ab-993b79838d95'},
> 'index': '0', 'iface': 'virtio', 'apparentsize': '10737418240',
> 'imageID': '8518ef4a-7b17-4291-856c-81875ba4e264', 'readonly':
> 'False', 'shared': 'exclusive', 'truesize': '6899802112', 'type':
> 'disk', 'domainID': 'c6323975-2966-409d-b9e0-48370a513a98', 'reqsize':
> '0', 'format': 'raw', 'deviceId':
> '8518ef4a-7b17-4291-856c-81875ba4e264', 'poolID':
> '00000000-0000-0000-0000-000000000000', 'device': 'disk', 'path':
> '/var/run/vdsm/storage/c6323975-2966-409d-b9e0-48370a513a98/8518ef4a-7b17-4291-856c-81875ba4e264/aa66d378-5a5f-490c-b0ab-993b79838d95',
> 'propagateErrors': 'off', 'name': 'vda', 'bootOrder': '1', 'volumeID':
> 'aa66d378-5a5f-490c-b0ab-993b79838d95', 'alias': 'virtio-disk0',
> 'volumeChain': [{'domainID': 'c6323975-2966-409d-b9e0-48370a513a98',
> 'volType': 'path', 'leaseOffset': 0, 'volumeID':
> 'aa66d378-5a5f-490c-b0ab-993b79838d95', 'leasePath':
> '/rhev/data-center/mnt/nas01:_volume1_vm__os_ovirt36__engine/c6323975-2966-409d-b9e0-48370a513a98/images/8518ef4a-7b17-4291-856c-81875ba4e264/aa66d378-5a5f-490c-b0ab-993b79838d95.lease',
> 'imageID': '8518ef4a-7b17-4291-856c-81875ba4e264', 'path':
> '/rhev/data-center/mnt/nas01:_volume1_vm__os_ovirt36__engine/c6323975-2966-409d-b9e0-48370a513a98/images/8518ef4a-7b17-4291-856c-81875ba4e264/aa66d378-5a5f-490c-b0ab-993b79838d95'}]},
> {'index': '2', 'iface': 'ide', 'name': 'hdc', 'alias': 'ide0-1-0',
> 'readonly': 'True', 'deviceId':
> '8c3179ac-b322-4f5c-9449-c52e3665e0ae', 'address': {'bus': '1',
> 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'},
> 'device': 'cdrom', 'shared': 'false', 'path': '', 'type': 'disk'},
> {'device': 'unix', 'alias': 'channel0', 'type': 'channel', 'address':
> {'bus': '0', 'controller': '0', 'type': 'virtio-serial', 'port':
> '1'}}, {'device': 'unix', 'alias': 'channel1', 'type': 'channel',
> 'address': {'bus': '0', 'controller': '0', 'type': 'virtio-serial',
> 'port': '2'}}, {'device': 'unix', 'alias': 'channel2', 'type':
> 'channel', 'address': {'bus': '0', 'controller': '0', 'type':
> 'virtio-serial', 'port': '3'}}, {'device': '', 'alias': 'video0',
> 'type': 'video', 'address': {'slot': '0x02', 'bus': '0x00', 'domain':
> '0x0000', 'type': 'pci', 'function': '0x0'}}]
> guestDiskMapping = {'8518ef4a-7b17-4291-8': {'name': '/dev/vda'},
> 'QEMU_DVD-ROM_QM00003': {'name': '/dev/sr0'}}
> vmType = kvm
> displaySecurePort = -1
> memSize = 4096
> displayPort = 5900
> clientIp =
> spiceSecureChannels =
> smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir
> smp = 4
> displayIp = 0
> display = vnc
> pauseCode = NOERR
>
>
>
>
> On Mon, Jun 13, 2016 at 8:25 AM, Charles Kozler <charles at fixflyer.com
> <mailto:charles at fixflyer.com>> wrote:
>
> It is up. I can do "ps -Aef | grep -i qemu-kvm | grep -i hosted"
> and see it running. I also forcefully shut it down with
> hosted-engine --vm-stop when it was on node1 and then did
> --vm-start on node 2 and it came up. Also the Web UI is reachable
> so thats how I also know the hosted engine VM is running
>
> On Mon, Jun 13, 2016 at 8:24 AM, Alexis HAUSER
> <alexis.hauser at telecom-bretagne.eu
> <mailto:alexis.hauser at telecom-bretagne.eu>> wrote:
>
>
> > http://imgur.com/a/6xkaS
>
> I had similar errors with one single host and a hosted-engine VM.
> My case should be totally different, but one thing you could
> try first is to check VM is really up.
> In my issues, VM was shown by hosted-engine command as up, but
> was down. with vdsClient command, you can check if it's status
> with more details.
>
> What is the result for you of the following command ?
>
> vdsClient -s 0 list
>
>
>
>
> --
>
> *Charles Kozler*
> /Vice President, IT Operations/
> FIX Flyer, LLC
> 225 Broadway | Suite 1600 | New York, NY 10007
> 1-888-349-3593 <tel:1-888-349-3593>
> http://www.fixflyer.com <http://fixflyer.com>
> NOTICE TO RECIPIENT: THIS E-MAIL IS MEANT ONLY FOR THE INTENDED
> RECIPIENT(S) OF THE TRANSMISSION, AND CONTAINS CONFIDENTIAL
> INFORMATION WHICH IS PROPRIETARY TO FIX FLYER LLC. ANY
> UNAUTHORIZED USE, COPYING, DISTRIBUTION, OR DISSEMINATION IS
> STRICTLY PROHIBITED. ALL RIGHTS TO THIS INFORMATION IS RESERVED
> BY FIX FLYER LLC. IF YOU ARE NOT THE INTENDED RECIPIENT, PLEASE
> CONTACT THE SENDER BY REPLY E-MAIL AND PLEASE DELETE THIS E-MAIL
> FROM YOUR SYSTEM AND DESTROY ANY COPIES.
>
>
>
>
> --
>
> *Charles Kozler*
> /Vice President, IT Operations/
> FIX Flyer, LLC
> 225 Broadway | Suite 1600 | New York, NY 10007
> 1-888-349-3593
> http://www.fixflyer.com <http://fixflyer.com>
> NOTICE TO RECIPIENT: THIS E-MAIL IS MEANT ONLY FOR THE INTENDED
> RECIPIENT(S) OF THE TRANSMISSION, AND CONTAINS CONFIDENTIAL
> INFORMATION WHICH IS PROPRIETARY TO FIX FLYER LLC. ANY UNAUTHORIZED
> USE, COPYING, DISTRIBUTION, OR DISSEMINATION IS STRICTLY PROHIBITED.
> ALL RIGHTS TO THIS INFORMATION IS RESERVED BY FIX FLYER LLC. IF YOU
> ARE NOT THE INTENDED RECIPIENT, PLEASE CONTACT THE SENDER BY REPLY
> E-MAIL AND PLEASE DELETE THIS E-MAIL FROM YOUR SYSTEM AND DESTROY ANY
> COPIES.
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160615/6c01195c/attachment-0001.html>
More information about the Users
mailing list