[ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 05/02/2017 ] [ assign_hosts_network_label ]

Sun Mar 5 19:42:45 UTC 2017

Eyal,

This test looks like a victim rather than an offender.

At the moment it tries to assign a label, host1 is
VDS_HOST_NOT_RESPONDING because of "FullListVDS failed: Unrecognized
message received".

I think that in [1] a response to Host.getVMFullList from VDSM is
incomplete and as a result the host is declared as "not responding".

[1] http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experimental_master/5673/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-005_network_by_label.py/lago-basic-suite-master-engine/_var_log/ovirt-engine/engine.log

2017-03-05 06:12:12,030-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand]
(DefaultQuartzScheduler3) [] START, FullListVDSCommand(HostName =
lago-basic-suite-master-host1,
FullListVDSCommandParameters:{runAsync='true',
hostId='dae8590a-e672-44b0-8be2-f5e019641314',
vmIds='[b0b35145-95eb-40cb-a46c-17937a2d67ef]'}), log id: 2327aeb9
2017-03-05 06:12:12,030-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message]
(DefaultQuartzScheduler3) [] SEND
destination:jms.topic.vdsm_requests
reply-to:jms.topic.vdsm_responses
content-length:152

{"jsonrpc":"2.0","method":"Host.getVMFullList","params":{"vmList":["b0b35145-95eb-40cb-a46c-17937a2d67ef"]},"id":"2189a48e-fc88-43fe-b5d1-ec3d82d0587a"}�
2017-03-05 06:12:12,030-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient]
(DefaultQuartzScheduler3) [] Message sent: SEND
destination:jms.topic.vdsm_requests
content-length:152
reply-to:jms.topic.vdsm_responses

<JsonRpcRequest id: "2189a48e-fc88-43fe-b5d1-ec3d82d0587a", method:
Host.getVMFullList, params:
{vmList=[b0b35145-95eb-40cb-a46c-17937a2d67ef]}>
2017-03-05 06:12:12,062-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (SSL Stomp
Reactor) [48435f7c] �MESSAGE
content-length:12133
destination:jms.topic.vdsm_responses
content-type:application/json
subscription:88ae7775-7c8f-4489-ad0e-6fd40d9990ea

{"jsonrpc": "2.0", "id": "2189a48e-fc88-43fe-b5d1-ec3d82d0587a",
"result": [{"acpiEnable": "true", "emulatedMachine":
"pc-i440fx-rhel7.3.0", "afterMigrationStatus": "", "pid": "19518",
"memGuaranteedSize": 256, "transparentHugePages": "false",
"timeOffset": "0", "cpuType": "Westmere", "smp": "1",
"guestNumaNodes": [{"nodeIndex": 0, "cpus": "0", "memory": "512"}],
"smartcardEnable": "true", "custom": {}, "vmType": "kvm", "memSize":
512, "smpCoresPerSocket": "1", "vmName": "vm0", "nice": "0", "status":
"Up", "maxMemSize": 4096, "bootMenuEnable": "false", "vmId":
"b0b35145-95eb-40cb-a46c-17937a2d67ef", "smpThreadsPerCore": "1",
"agentChannelName": "ovirt-guest-agent.0", "guestDiskMapping": {},
"kvmEnable": "true", "pitReinjection": "false", "displayNetwork":
"ovirtmgmt", "devices": [{"alias": "sound0", "specParams": {},
"deviceId": "e71ac881-b31e-4986-a68a-574f22fbfc
2017-03-05 06:12:12,062-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (SSL Stomp
Reactor) [48435f7c] Message received: �MESSAGE
content-length:12133
destination:jms.topic.vdsm_responses
content-type:application/json
subscription:88ae7775-7c8f-4489-ad0e-6fd40d9990ea

{"jsonrpc": "2.0", "id": "2189a48e-fc88-43fe-b5d1-ec3d82d0587a",
"result": [{"acpiEnable": "true", "emulatedMachine":
"pc-i440fx-rhel7.3.0", "afterMigrationStatus": "", "pid": "19518",
"memGuaranteedSize": 256, "transparentHugePages": "false",
"timeOffset": "0", "cpuType": "Westmere", "smp": "1",
"guestNumaNodes": [{"nodeIndex": 0, "cpus": "0", "memory": "512"}],
"smartcardEnable": "true", "custom": {}, "vmType": "kvm", "memSize":
512, "smpCoresPerSocket": "1", "vmName": "vm0", "nice": "0", "status":
"Up", "maxMemSize": 4096, "bootMenuEnable": "false", "vmId":
"b0b35145-95eb-40cb-a46c-17937a2d67ef", "smpThreadsPerCore": "1",
"agentChannelName": "ovirt-guest-agent.0", "guestDiskMapping": {},
"kvmEnable": "true", "pitReinjection": "false", "displayNetwork":
"ovirtmgmt", "devices": [{"alias": "sound0", "specParams": {},
"deviceId": "e71ac881-b31e-4986-a68a-574f22fbfc
2017-03-05 06:12:12,063-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor)
[48435f7c] Unable to process messages Unrecognized message received :
org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Unrecognized
message received
	at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message.parse(Message.java:219)
[vdsm-jsonrpc-java-client.jar:]
	at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.getMessage(StompCommonClient.java:171)
[vdsm-jsonrpc-java-client.jar:]

On Sun, Mar 5, 2017 at 4:01 PM, Eyal Edri <eedri at redhat.com> wrote:

> Dan,
> This test seems to be flaky and fail once a week,
> Can we disable / remove it until its back to stable?
>
> e.
>
> On Sun, Mar 5, 2017 at 3:55 PM, Daniel Belenky <dbelenky at redhat.com>
> wrote:
>
>> *Test failed:* assign_hosts_network_label
>>
>> *Link to failed Job:*
>>   -  http://jenkins.ovirt.org/view/experimental%20jobs/job/test-r
>> epo_ovirt_experimental_master/5673/
>>
>>
>> *Link to all logs:  -  *http://jenkins.ovirt.org/view/
>> experimental%20jobs/job/test-repo_ovirt_experimental_master/
>> 5673/artifact/exported-artifacts/basic-suit-master-el7/test_
>> logs/basic-suite-master/
>>
>> *Eror snippet from log:*
>>
>>  RequestError:
>>  status: 409
>>  reason: Conflict
>>  detail: Cannot add Label. Operation can be performed only when Host status is  Maintenance, Up, NonOperational.
>>
>> --
>>
>> *Daniel Belenky*
>>
>> *RHV DevOps*
>>
>> *Red Hat Israel*
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
>
> --
> Eyal Edri
> Associate Manager
> RHV DevOps
> EMEA ENG Virtualization R&D
> Red Hat Israel
>
> phone: +972-9-7692018 <+972%209-769-2018>
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
>
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20170305/66dee8ba/attachment-0001.html>