Project:
https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/
Build:
https://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/...
Build Number: 350
Build Status: Still Failing
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #340
[Galit Rosenthal] Update NetworkManager Repo
[Barak Korren] Refactor: Move python tools to their own directory
[Barak Korren] Refactor: Move STDCI Python libs to their own dir
Changes for Build #341
[Galit Rosenthal] Fix wrong error exception
[Sandro Bonazzola] pipelines: add ovirt-engine-extension-aaa-jdbc
[Sandro Bonazzola] ovirt-engine-api-explorer: drop stdci V1 jobs
Changes for Build #342
[Galit Rosenthal] Fix wrong error exception
Changes for Build #343
[Galit Rosenthal] Fix wrong error exception
Changes for Build #344
[Galit Rosenthal] Fix wrong error exception
[Sandro Bonazzola] pipelines: drop vcredist
[Galit Rosenthal] Fix old mirros url which doesnt exists
Changes for Build #345
[Anton Marchukov] Added "sar" system resources collection on VMs.
[Martin Perina] Add ovirt-engine-extensions-api into standard pipeline
Changes for Build #346
[Anton Marchukov] Added "sar" system resources collection on VMs.
[Martin Perina] Remove STDCIv1 config for aaa-jdbc extension
[Gal Ben Haim] dockerfile-utils: Add registry if needed
[Shlomi] Fixing fedora-base-fc29 URL
Changes for Build #347
[Anton Marchukov] Added "sar" system resources collection on VMs.
[Barak Korren] Enable STDCI tools container to run playbooks & oc
[Galit Rosenthal] remove rmmod in containers
Changes for Build #348
[Anton Marchukov] Added "sar" system resources collection on VMs.
[Barak Korren] usrc.py: Be better at importing `pusher.py`
Changes for Build #349
[Anton Marchukov] Added "sar" system resources collection on VMs.
Changes for Build #350
[Anton Marchukov] Added "sar" system resources collection on VMs.
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 001_initialize_engine.initialize_engine_43
Error Message:
engine-setup failed. Exit code is 1
-------------------- >> begin captured logging << --------------------
lago.plugins.vm: INFO: * Copy
/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/engine-answer-file.conf
to lago-compat-4-3-suite-master-engine:/tmp/answer-file: ?[0m?[0m
lago.ssh: DEBUG: start task:d3787e8f-68de-430c-bd36-57c4b24df194:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.ssh: DEBUG: end task:d3787e8f-68de-430c-bd36-57c4b24df194:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.plugins.vm: INFO: * Copy
/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/engine-answer-file.conf
to lago-compat-4-3-suite-master-engine:/tmp/answer-file: ?[32mSuccess?[0m (in 0:00:01)
lago.plugins.vm: INFO: * Copy /tmp/tmpnX4CjZ to
lago-compat-4-3-suite-master-engine:/etc/ovirt-engine/engine.conf.d/99-custom-fqdn.conf:
?[0m?[0m
lago.ssh: DEBUG: start task:45d0f385-0795-4dbd-8efd-48e7faa2ea5e:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.ssh: DEBUG: end task:45d0f385-0795-4dbd-8efd-48e7faa2ea5e:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.plugins.vm: INFO: * Copy /tmp/tmpnX4CjZ to
lago-compat-4-3-suite-master-engine:/etc/ovirt-engine/engine.conf.d/99-custom-fqdn.conf:
?[32mSuccess?[0m (in 0:00:00)
lago.ssh: DEBUG: start task:931bc9ab-f438-4a48-b8ad-b04b4b911702:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.ssh: DEBUG: end task:931bc9ab-f438-4a48-b8ad-b04b4b911702:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.ssh: DEBUG: Running c2c0b754 on lago-compat-4-3-suite-master-engine: chmod 644
/etc/ovirt-engine/engine.conf.d/99-custom-fqdn.conf
lago.ssh: DEBUG: Command c2c0b754 on lago-compat-4-3-suite-master-engine returned with 0
lago.ssh: DEBUG: start task:1c6768a2-1ea5-483f-ab26-8ce656ed312b:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.ssh: DEBUG: end task:1c6768a2-1ea5-483f-ab26-8ce656ed312b:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.ssh: DEBUG: Running c3435a60 on lago-compat-4-3-suite-master-engine: engine-setup
--config-append=/tmp/answer-file --accept-defaults
lago.ssh: DEBUG: Command c3435a60 on lago-compat-4-3-suite-master-engine returned with 1
lago.ssh: DEBUG: Command c3435a60 on lago-compat-4-3-suite-master-engine output:
[ INFO ] Stage: Initializing
[ INFO ] Stage: Environment setup
Configuration files: /etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf,
/etc/ovirt-engine-setup.conf.d/10-packaging.conf, /tmp/answer-file
Log file:
/var/log/ovirt-engine/setup/ovirt-engine-setup-20200215200046-4xjbqs.log
Version: otopi-1.9.0 (otopi-1.9.0-1.el7)
[ INFO ] Stage: Environment packages setup
[ INFO ] Stage: Programs detection
[ INFO ] Stage: Environment setup (late)
[ INFO ] Stage: Environment customization
--== PRODUCT OPTIONS ==--
Set up Cinderlib integration
(Currently in tech preview)
(Yes, No) [No]:
Configure ovirt-provider-ovn (Yes, No) [Yes]:
--== PACKAGES ==--
[ INFO ] Checking for an update for Setup...
[ ERROR ] Yum Cannot queue package ovirt-host-deploy: Package ovirt-host-deploy cannot be
found
[ INFO ] Yum Performing yum transaction rollback
[ ERROR ] Failed to execute stage 'Environment customization': Package
ovirt-host-deploy cannot be found
[ INFO ] Stage: Clean up
Log file is located at
/var/log/ovirt-engine/setup/ovirt-engine-setup-20200215200046-4xjbqs.log
[ INFO ] Generating answer file
'/var/lib/ovirt-engine/setup/answers/20200215200124-setup.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Execution of setup failed
lago.ssh: DEBUG: start task:af8edffe-37ec-4f86-81a9-c4fffc1b491f:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.ssh: DEBUG: end task:af8edffe-37ec-4f86-81a9-c4fffc1b491f:Get ssh client for
lago-compat-4-3-suite-master-engine:
lago.ssh: DEBUG: Running db78c980 on lago-compat-4-3-suite-master-engine: ss -anp
lago.ssh: DEBUG: Command db78c980 on lago-compat-4-3-suite-master-engine returned with 0
lago.ssh: DEBUG: Command db78c980 on lago-compat-4-3-suite-master-engine output:
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
nl UNCONN 0 0 0:1224738933 *
nl UNCONN 0 0 0:732 *
nl UNCONN 0 0 0:3942 *
nl UNCONN 0 0 0:0 *
nl UNCONN 0 0 0:-1769994123 *
nl UNCONN 0 0 0:2230 *
nl UNCONN 0 0 0:3942 *
nl UNCONN 0 0 0:2230 *
nl UNCONN 0 0 0:732 *
nl UNCONN 4352 0 4:4196 *
nl UNCONN 768 0 4:0 *
nl UNCONN 0 0 6:0 *
nl UNCONN 0 0 7:0 *
nl UNCONN 0 0 7:1 *
nl UNCONN 0 0 7:678 *
nl UNCONN 0 0 7:678 *
nl UNCONN 0 0 7:1 *
nl UNCONN 0 0 9:646 *
nl UNCONN 0 0 9:0 *
nl UNCONN 0 0 9:1 *
nl UNCONN 0 0 9:678 *
nl UNCONN 0 0 10:0 *
nl UNCONN 0 0 11:0 *
nl UNCONN 0 0 12:0 *
nl UNCONN 0 0 15:-4123 *
nl UNCONN 0 0 15:-4122 *
nl UNCONN 0 0 15:0 *
nl UNCONN 0 0 15:1 *
nl UNCONN 768 0 15:-4109 *
nl UNCONN 0 0 15:-4121 *
nl UNCONN 0 0 15:2165 *
nl UNCONN 0 0 15:732 *
nl UNCONN 0 0 15:1135 *
nl UNCONN 0 0 15:-4124 *
nl UNCONN 0 0 15:694 *
nl UNCONN 0 0 15:1807747189 *
nl UNCONN 0 0 15:2165 *
nl UNCONN 0 0 15:1807747189 *
nl UNCONN 0 0 15:1135 *
nl UNCONN 0 0 15:-4124 *
nl UNCONN 0 0 15:732 *
nl UNCONN 0 0 15:-4123 *
nl UNCONN 0 0 15:-4122 *
nl UNCONN 0 0 15:-4121 *
nl UNCONN 0 0 15:694 *
nl UNCONN 768 0 15:-4109 *
nl UNCONN 0 0 15:1 *
nl UNCONN 0 0 16:732 *
nl UNCONN 0 0 16:0 *
nl UNCONN 0 0 18:0 *
p_dgr UNCONN 0 0 arp:virbr0 *
users:(("NetworkManager",pid=732,fd=22))
p_raw UNCONN 0 0 *:eth0 *
users:(("dhclient",pid=887,fd=5))
p_raw UNCONN 0 0 *:eth1 *
users:(("dhclient",pid=889,fd=5))
u_str LISTEN 0 128 /var/run/libvirt/virtlockd-sock 25358 * 0
users:(("systemd",pid=1,fd=66))
u_str LISTEN 0 10 /var/run/NetworkManager/private-dhcp 17437
* 0 users:(("NetworkManager",pid=732,fd=19))
u_str LISTEN 0 128 /run/dbus/system_bus_socket 15399 * 0
users:(("dbus-daemon",pid=678,fd=3),("systemd",pid=1,fd=49))
u_str LISTEN 0 128 /run/systemd/private 12348 * 0
users:(("systemd",pid=1,fd=12))
u_str LISTEN 0 128 @ISCSID_UIP_ABSTRACT_NAMESPACE 21211 * 0
users:(("systemd",pid=1,fd=95))
u_str LISTEN 0 128 /var/run/libvirt/virtlogd-sock 24201 * 0
users:(("systemd",pid=1,fd=27))
u_seq LISTEN 0 128 /run/udev/control 10138 * 0
users:(("systemd-udevd",pid=460,fd=3),("systemd",pid=1,fd=32))
u_dgr UNCONN 0 0 /run/systemd/shutdownd 10140 * 0
users:(("systemd",pid=1,fd=62))
u_str LISTEN 0 128 /run/lvm/lvmetad.socket 22699 * 0
users:(("lvmetad",pid=1645,fd=3),("systemd",pid=1,fd=67))
u_dgr UNCONN 0 0 /run/systemd/notify 8365 * 0
users:(("systemd",pid=1,fd=23))
u_dgr UNCONN 0 0 /run/systemd/cgroups-agent 8367 * 0
users:(("systemd",pid=1,fd=24))
u_str LISTEN 0 128 /run/systemd/journal/stdout 8382 * 0
users:(("systemd-journal",pid=433,fd=3),("systemd",pid=1,fd=92))
u_str LISTEN 0 128 @ISCSIADM_ABSTRACT_NAMESPACE 21359 * 0
users:(("systemd",pid=1,fd=50))
u_dgr UNCONN 255 0 /run/systemd/journal/socket 8385 * 0
users:(("systemd-journal",pid=433,fd=4),("systemd",pid=1,fd=93))
u_dgr UNCONN 0 0 /dev/log 8387 * 0
users:(("systemd-journal",pid=433,fd=5),("systemd",pid=1,fd=94))
u_str LISTEN 0 128 /var/run/libvirt/libvirt-sock 26080 * 0
users:(("libvirtd",pid=2165,fd=10))
u_str LISTEN 0 128 /var/run/libvirt/libvirt-sock-ro 26082 * 0
users:(("libvirtd",pid=2165,fd=11))
u_str LISTEN 0 20 /var/run/libvirt/libvirt-admin-sock 26084
* 0 users:(("libvirtd",pid=2165,fd=12))
u_str LISTEN 0 128 /var/run/rpcbind.sock 21482 * 0
users:(("systemd",pid=1,fd=76))
u_str LISTEN 0 128 /run/lvm/lvmpolld.socket 22003 * 0
users:(("systemd",pid=1,fd=96))
u_dgr UNCONN 0 0 * 16397 * 8387
users:(("dbus-daemon",pid=678,fd=15))
u_dgr UNCONN 0 0 * 26546 * 8387
users:(("dnsmasq",pid=2234,fd=7))
u_dgr UNCONN 0 0 * 37631 * 8387
users:(("sshd",pid=4192,fd=4))
u_str ESTAB 0 0 * 21873 * 22727
users:(("lvmetad",pid=1645,fd=2),("lvmetad",pid=1645,fd=1))
u_str ESTAB 0 0 /run/systemd/journal/stdout 22727 * 21873
users:(("systemd-journal",pid=433,fd=24),("systemd",pid=1,fd=48))
u_str ESTAB 0 0 /run/dbus/system_bus_socket 17971 * 17970
users:(("dbus-daemon",pid=678,fd=16))
u_dgr UNCONN 0 0 * 33429 * 8385
users:(("systemd",pid=1,fd=14))
u_str ESTAB 0 0 /run/systemd/journal/stdout 15313 * 16095
users:(("systemd-journal",pid=433,fd=18),("systemd",pid=1,fd=45))
u_str ESTAB 0 0 * 15584 * 15009
users:(("dbus-daemon",pid=678,fd=2),("dbus-daemon",pid=678,fd=1))
u_str ESTAB 0 0 * 16264 * 16988
users:(("NetworkManager",pid=732,fd=16))
u_str ESTAB 0 0 * 26089 * 26090
users:(("libvirtd",pid=2165,fd=14))
u_str ESTAB 0 0 /run/systemd/journal/stdout 15009 * 15584
users:(("systemd-journal",pid=433,fd=19),("systemd",pid=1,fd=41))
u_str ESTAB 0 0 * 16095 * 15313
users:(("NetworkManager",pid=732,fd=2),("NetworkManager",pid=732,fd=1))
u_str ESTAB 0 0 /run/systemd/journal/stdout 17792 * 18759
users:(("systemd-journal",pid=433,fd=20),("systemd",pid=1,fd=46))
u_str ESTAB 0 0 /run/systemd/journal/stdout 17848 * 17847
users:(("systemd-journal",pid=433,fd=25),("systemd",pid=1,fd=47))
u_str ESTAB 0 0 /run/dbus/system_bus_socket 15683 * 15545
users:(("dbus-daemon",pid=678,fd=10))
u_str ESTAB 0 0 * 18759 * 17792
users:(("sshd",pid=1134,fd=2),("sshd",pid=1134,fd=1))
u_str ESTAB 0 0 * 17970 * 17971
users:(("tuned",pid=1135,fd=6))
u_str ESTAB 0 0 /run/dbus/system_bus_socket 16988 * 16264
users:(("dbus-daemon",pid=678,fd=19))
u_str ESTAB 0 0 /run/systemd/journal/stdout 15120 * 15743
users:(("systemd-journal",pid=433,fd=17),("systemd",pid=1,fd=43))
u_str ESTAB 0 0 /run/dbus/system_bus_socket 26090 * 26089
users:(("dbus-daemon",pid=678,fd=18))
u_dgr UNCONN 0 0 * 15181 * 8385
users:(("systemd-logind",pid=694,fd=3))
u_str ESTAB 0 0 * 15743 * 15120
users:(("systemd-logind",pid=694,fd=2),("systemd-logind",pid=694,fd=1))
u_str ESTAB 0 0 * 15545 * 15683
users:(("systemd",pid=1,fd=35))
u_dgr UNCONN 0 0 * 18434 * 8387
users:(("dhclient",pid=887,fd=3))
u_str ESTAB 0 0 * 15681 * 15682
users:(("dbus-daemon",pid=678,fd=8))
u_str ESTAB 0 0 /run/systemd/journal/stdout 15189 * 15888
users:(("systemd-journal",pid=433,fd=23),("systemd",pid=1,fd=44))
u_str ESTAB 0 0 * 14846 * 14845
users:(("auditd",pid=646,fd=10))
u_str ESTAB 0 0 * 15682 * 15681
users:(("dbus-daemon",pid=678,fd=9))
u_dgr UNCONN 0 0 * 22955 * 8387
users:(("rsyslogd",pid=1737,fd=4))
u_str ESTAB 0 0 * 17847 * 17848
users:(("tuned",pid=1135,fd=2),("tuned",pid=1135,fd=1))
u_dgr UNCONN 0 0 * 18435 * 8387
users:(("dhclient",pid=889,fd=3))
u_str ESTAB 0 0 * 15888 * 15189
users:(("crond",pid=700,fd=2),("crond",pid=700,fd=1))
u_dgr UNCONN 0 0 * 14836 * 8387
users:(("auditd",pid=646,fd=8))
u_dgr UNCONN 0 0 * 38346 * 8387
users:(("anacron",pid=4187,fd=4))
u_str ESTAB 0 0 * 15663 * 15018
users:(("irqbalance",pid=674,fd=2),("irqbalance",pid=674,fd=1))
u_dgr UNCONN 0 0 * 15328 * 8387
users:(("NetworkManager",pid=732,fd=5))
u_str ESTAB 0 0 /run/systemd/journal/stdout 15018 * 15663
users:(("systemd-journal",pid=433,fd=22),("systemd",pid=1,fd=42))
u_str ESTAB 0 0 * 14845 * 14846
users:(("auditd",pid=646,fd=9))
u_str ESTAB 0 0 /run/dbus/system_bus_socket 15356 * 15355
users:(("dbus-daemon",pid=678,fd=14))
u_str ESTAB 0 0 /run/dbus/system_bus_socket 15296 * 15295
users:(("dbus-daemon",pid=678,fd=13))
u_str ESTAB 0 0 * 15113 * 15688
users:(("polkitd",pid=675,fd=6))
u_str ESTAB 0 0 * 15295 * 15296
users:(("firewalld",pid=709,fd=4))
u_dgr UNCONN 0 0 * 15953 * 8387
users:(("crond",pid=700,fd=4))
u_dgr UNCONN 0 0 * 12721 * 12722
users:(("systemd-udevd",pid=460,fd=9))
u_str ESTAB 0 0 /run/dbus/system_bus_socket 15688 * 15113
users:(("dbus-daemon",pid=678,fd=11))
u_str ESTAB 0 0 /run/systemd/journal/stdout 13434 * 12664
users:(("systemd-journal",pid=433,fd=14),("systemd",pid=1,fd=37))
u_dgr UNCONN 0 0 * 29449 * 8387
users:(("firewalld",pid=709,fd=10))
u_str ESTAB 0 0 * 12664 * 13434
users:(("systemd-udevd",pid=460,fd=2),("systemd-udevd",pid=460,fd=1))
u_str ESTAB 0 0 * 26067 * 24541
users:(("libvirtd",pid=2165,fd=2),("libvirtd",pid=2165,fd=1))
u_str ESTAB 0 0 * 15355 * 15356
users:(("NetworkManager",pid=732,fd=11))
u_str ESTAB 0 0 * 15195 * 15895
users:(("systemd-logind",pid=694,fd=12))
u_str ESTAB 0 0 /run/dbus/system_bus_socket 15895 * 15195
users:(("dbus-daemon",pid=678,fd=12))
u_str ESTAB 0 0 /run/systemd/journal/stdout 14961 * 14960
users:(("systemd-journal",pid=433,fd=15),("systemd",pid=1,fd=39))
u_dgr UNCONN 0 3840 * 12709 * 8385
users:(("systemd-udevd",pid=460,fd=5))
u_str ESTAB 0 0 /run/systemd/journal/stdout 14964 * 14963
users:(("systemd-journal",pid=433,fd=16),("systemd",pid=1,fd=40))
u_str ESTAB 0 0 * 14963 * 14964
users:(("qemu-ga",pid=671,fd=2))
u_dgr UNCONN 0 0 * 15948 * 8387
users:(("polkitd",pid=675,fd=11))
u_dgr UNCONN 0 0 * 12722 * 12721
users:(("systemd-udevd",pid=460,fd=10))
u_str ESTAB 0 0 /run/systemd/journal/stdout 24541 * 26067
users:(("systemd-journal",pid=433,fd=26),("systemd",pid=1,fd=36))
u_dgr UNCONN 0 0 * 10063 * 8365
users:(("systemd-journal",pid=433,fd=11))
u_str ESTAB 0 0 * 14960 * 14961
users:(("qemu-ga",pid=671,fd=1))
u_str ESTAB 0 0 /run/systemd/journal/stdout 36084 * 36083
users:(("systemd-journal",pid=433,fd=21),("systemd",pid=1,fd=34))
u_dgr UNCONN 0 0 * 26554 * 8387
users:(("dnsmasq",pid=2234,fd=11))
u_dgr UNCONN 0 0 * 33166 * 8387
users:(("ntpd",pid=3942,fd=3))
u_str ESTAB 0 0 * 36083 * 36084
users:(("snmptrapd",pid=4012,fd=2),("snmptrapd",pid=4012,fd=1))
u_dgr UNCONN 0 0 * 33365 * 8387
users:(("snmptrapd",pid=4012,fd=9))
udp UNCONN 0 0 192.168.122.1:53 *:*
users:(("dnsmasq",pid=2234,fd=5))
udp UNCONN 0 0 *%virbr0:67 *:*
users:(("dnsmasq",pid=2234,fd=3))
udp UNCONN 0 0 *:68 *:*
users:(("dhclient",pid=889,fd=6))
udp UNCONN 0 0 *:68 *:*
users:(("dhclient",pid=887,fd=6))
udp UNCONN 0 0 *:111 *:*
users:(("systemd",pid=1,fd=89))
udp UNCONN 0 0 192.168.122.1:123 *:*
users:(("ntpd",pid=3942,fd=21))
udp UNCONN 0 0 192.168.202.4:123 *:*
users:(("ntpd",pid=3942,fd=20))
udp UNCONN 0 0 192.168.201.4:123 *:*
users:(("ntpd",pid=3942,fd=19))
udp UNCONN 0 0 127.0.0.1:123 *:*
users:(("ntpd",pid=3942,fd=18))
udp UNCONN 0 0 *:123 *:*
users:(("ntpd",pid=3942,fd=16))
udp UNCONN 0 0 *:162 *:*
users:(("snmptrapd",pid=4012,fd=7))
udp UNCONN 0 0 [::]:111 [::]:*
users:(("systemd",pid=1,fd=91))
udp UNCONN 0 0 [fe80::5652:c0ff:fea8:c904]%eth0:123 [::]:*
users:(("ntpd",pid=3942,fd=24))
udp UNCONN 0 0 [fe80::5652:c0ff:fea8:ca04]%eth1:123 [::]:*
users:(("ntpd",pid=3942,fd=23))
udp UNCONN 0 0 [::1]:123 [::]:*
users:(("ntpd",pid=3942,fd=22))
udp UNCONN 0 0 [::]:123 [::]:*
users:(("ntpd",pid=3942,fd=17))
tcp LISTEN 0 128 *:111 *:*
users:(("systemd",pid=1,fd=77))
tcp LISTEN 0 5 192.168.122.1:53 *:*
users:(("dnsmasq",pid=2234,fd=6))
tcp LISTEN 0 128 *:22 *:*
users:(("sshd",pid=1134,fd=3))
tcp TIME-WAIT 0 0 192.168.201.4:41018 66.187.230.28:80
tcp ESTAB 0 0 192.168.201.4:22 192.168.201.1:56924
users:(("sshd",pid=4192,fd=3))
tcp TIME-WAIT 0 0 192.168.201.4:41016 66.187.230.28:80
tcp LISTEN 0 128 [::]:111 [::]:*
users:(("systemd",pid=1,fd=90))
tcp LISTEN 0 128 [::]:22 [::]:*
users:(("sshd",pid=1134,fd=4))
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in
wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in
wrapper
return func(get_test_prefix(), *args, **kwargs)
File
"/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/001_initialize_engine.py",
line 76, in initialize_engine
result.code, 0, 'engine-setup failed. Exit code is %s' % result.code
File "/usr/lib/python2.7/site-packages/nose/tools/trivial.py", line 29, in
eq_
raise AssertionError(msg or "%r != %r" % (a, b))
'engine-setup failed. Exit code is 1\n-------------------- >> begin captured
logging << --------------------\nlago.plugins.vm: INFO: * Copy
/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/engine-answer-file.conf
to lago-compat-4-3-suite-master-engine:/tmp/answer-file: \x1b[0m\x1b[0m\nlago.ssh: DEBUG:
start task:d3787e8f-68de-430c-bd36-57c4b24df194:Get ssh client for
lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end
task:d3787e8f-68de-430c-bd36-57c4b24df194:Get ssh client for
lago-compat-4-3-suite-master-engine:\nlago.plugins.vm: INFO: * Copy
/home/jenkins/agent/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/engine-answer-file.conf
to lago-compat-4-3-suite-master-engine:/tmp/answer-file: \x1b[32mSuccess\x1b[0m (in
0:00:01)\nlago.plugins.vm: INFO: * Copy /tmp/tmpnX4CjZ to
lago-compat-4-3-suite-master-engine:/etc/ovirt-engine/engine.conf.d/99-custom-fqdn.conf:
\x1b[0m\x1b[0m\nlago.ssh: DEBUG: start task:45d0f385-0795-4dbd-8efd-48e7faa2ea5e:Get ssh
client for lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end
task:45d0f385-0795-4dbd-8efd-48e7faa2ea5e:Get ssh client for
lago-compat-4-3-suite-master-engine:\nlago.plugins.vm: INFO: * Copy /tmp/tmpnX4CjZ to
lago-compat-4-3-suite-master-engine:/etc/ovirt-engine/engine.conf.d/99-custom-fqdn.conf:
\x1b[32mSuccess\x1b[0m (in 0:00:00)\nlago.ssh: DEBUG: start
task:931bc9ab-f438-4a48-b8ad-b04b4b911702:Get ssh client for
lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end
task:931bc9ab-f438-4a48-b8ad-b04b4b911702:Get ssh client for
lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: Running c2c0b754 on
lago-compat-4-3-suite-master-engine: chmod 644
/etc/ovirt-engine/engine.conf.d/99-custom-fqdn.conf\nlago.ssh: DEBUG: Command c2c0b754 on
lago-compat-4-3-suite-master-engine returned with 0\nlago.ssh: DEBUG: start
task:1c6768a2-1ea5-483f-ab26-8ce656ed312b:Get ssh client for
lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end
task:1c6768a2-1ea5-483f-ab26-8ce656ed312b:Get ssh client for
lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: Running c3435a60 on
lago-compat-4-3-suite-master-engine: engine-setup --config-append=/tmp/answer-file
--accept-defaults\nlago.ssh: DEBUG: Command c3435a60 on
lago-compat-4-3-suite-master-engine returned with 1\nlago.ssh: DEBUG: Command c3435a60 on
lago-compat-4-3-suite-master-engine output:\n [ INFO ] Stage: Initializing\n[ INFO ]
Stage: Environment setup\n Configuration files:
/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf,
/etc/ovirt-engine-setup.conf.d/10-packaging.conf, /tmp/answer-file\n Log file:
/var/log/ovirt-engine/setup/ovirt-engine-setup-20200215200046-4xjbqs.log\n
Version: otopi-1.9.0 (otopi-1.9.0-1.el7)\n[ INFO ] Stage: Environment packages setup\n[
INFO ] Stage: Programs detection\n[ INFO ] Stage: Environment setup (late)\n[ INFO ]
Stage: Environment customization\n \n --== PRODUCT OPTIONS ==--\n
\n Set up Cinderlib integration\n (Currently in tech preview)\n
(Yes, No) [No]: \n Configure ovirt-provider-ovn (Yes, No) [Yes]: \n \n
--== PACKAGES ==--\n \n[ INFO ] Checking for an update for Setup...\n[
ERROR ] Yum Cannot queue package ovirt-host-deploy: Package ovirt-host-deploy cannot be
found\n[ INFO ] Yum Performing yum transaction rollback\n[ ERROR ] Failed to execute
stage \'Environment customization\': Package ovirt-host-deploy cannot be found\n[
INFO ] Stage: Clean up\n Log file is located at
/var/log/ovirt-engine/setup/ovirt-engine-setup-20200215200046-4xjbqs.log\n[ INFO ]
Generating answer file
\'/var/lib/ovirt-engine/setup/answers/20200215200124-setup.conf\'\n[ INFO ]
Stage: Pre-termination\n[ INFO ] Stage: Termination\n[ ERROR ] Execution of setup
failed\n\nlago.ssh: DEBUG: start task:af8edffe-37ec-4f86-81a9-c4fffc1b491f:Get ssh client
for lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end
task:af8edffe-37ec-4f86-81a9-c4fffc1b491f:Get ssh client for
lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: Running db78c980 on
lago-compat-4-3-suite-master-engine: ss -anp\nlago.ssh: DEBUG: Command db78c980 on
lago-compat-4-3-suite-master-engine returned with 0\nlago.ssh: DEBUG: Command db78c980 on
lago-compat-4-3-suite-master-engine output:\n Netid State Recv-Q Send-Q Local
Address:Port Peer Address:Port \nnl UNCONN 0 0
0:1224738933 * \nnl UNCONN 0 0
0:732 * \nnl UNCONN 0 0 0:3942
* \nnl UNCONN 0 0 0:0
* \nnl UNCONN 0 0 0:-1769994123 *
\nnl UNCONN 0 0 0:2230 *
\nnl UNCONN 0 0 0:3942 *
\nnl UNCONN 0 0 0:2230 * \nnl
UNCONN 0 0 0:732 * \nnl UNCONN
4352 0 4:4196 * \nnl UNCONN 768
0 4:0 * \nnl UNCONN 0 0
6:0 * \nnl UNCONN 0 0 7:0
* \nnl UNCONN 0 0 7:1
* \nnl UNCONN 0 0 7:678
* \nnl UNCONN 0 0 7:678 *
\nnl UNCONN 0 0 7:1 *
\nnl UNCONN 0 0 9:646 * \nnl
UNCONN 0 0 9:0 * \nnl UNCONN
0 0 9:1 * \nnl UNCONN 0
0 9:678 * \nnl UNCONN 0 0
10:0 * \nnl UNCONN 0 0 11:0
* \nnl UNCONN 0 0 12:0
* \nnl UNCONN 0 0 15:-4123
* \nnl UNCONN 0 0 15:-4122 *
\nnl UNCONN 0 0 15:0 *
\nnl UNCONN 0 0 15:1 * \nnl
UNCONN 768 0 15:-4109 * \nnl
UNCONN 0 0 15:-4121 * \nnl UNCONN
0 0 15:2165 * \nnl UNCONN 0
0 15:732 * \nnl UNCONN 0 0
15:1135 * \nnl UNCONN 0 0 15:-4124
* \nnl UNCONN 0 0 15:694
* \nnl UNCONN 0 0 15:1807747189
* \nnl UNCONN 0 0 15:2165 *
\nnl UNCONN 0 0 15:1807747189 *
\nnl UNCONN 0 0 15:1135 * \nnl
UNCONN 0 0 15:-4124 * \nnl UNCONN
0 0 15:732 * \nnl UNCONN 0
0 15:-4123 * \nnl UNCONN 0 0
15:-4122 * \nnl UNCONN 0 0
15:-4121 * \nnl UNCONN 0 0 15:694
* \nnl UNCONN 768 0 15:-4109
* \nnl UNCONN 0 0 15:1
* \nnl UNCONN 0 0 16:732 *
\nnl UNCONN 0 0 16:0 *
\nnl UNCONN 0 0 18:0 *
\np_dgr UNCONN 0 0 arp:virbr0 *
users:(("NetworkManager",pid=732,fd=22))\np_raw UNCONN 0 0
*:eth0 *
users:(("dhclient",pid=887,fd=5))\np_raw UNCONN 0 0 *:eth1
* users:(("dhclient",pid=889,fd=5))\nu_str
LISTEN 0 128 /var/run/libvirt/virtlockd-sock 25358 * 0
users:(("systemd",pid=1,fd=66))\nu_str LISTEN 0 10
/var/run/NetworkManager/private-dhcp 17437 * 0
users:(("NetworkManager",pid=732,fd=19))\nu_str LISTEN 0 128
/run/dbus/system_bus_socket 15399 * 0
users:(("dbus-daemon",pid=678,fd=3),("systemd",pid=1,fd=49))\nu_str
LISTEN 0 128 /run/systemd/private 12348 * 0
users:(("systemd",pid=1,fd=12))\nu_str LISTEN 0 128
@ISCSID_UIP_ABSTRACT_NAMESPACE 21211 * 0
users:(("systemd",pid=1,fd=95))\nu_str LISTEN 0 128
/var/run/libvirt/virtlogd-sock 24201 * 0
users:(("systemd",pid=1,fd=27))\nu_seq LISTEN 0 128
/run/udev/control 10138 * 0
users:(("systemd-udevd",pid=460,fd=3),("systemd",pid=1,fd=32))\nu_dgr
UNCONN 0 0 /run/systemd/shutdownd 10140 * 0
users:(("systemd",pid=1,fd=62))\nu_str LISTEN 0 128
/run/lvm/lvmetad.socket 22699 * 0
users:(("lvmetad",pid=1645,fd=3),("systemd",pid=1,fd=67))\nu_dgr
UNCONN 0 0 /run/systemd/notify 8365 * 0
users:(("systemd",pid=1,fd=23))\nu_dgr UNCONN 0 0
/run/systemd/cgroups-agent 8367 * 0
users:(("systemd",pid=1,fd=24))\nu_str LISTEN 0 128
/run/systemd/journal/stdout 8382 * 0
users:(("systemd-journal",pid=433,fd=3),("systemd",pid=1,fd=92))\nu_str
LISTEN 0 128 @ISCSIADM_ABSTRACT_NAMESPACE 21359 * 0
users:(("systemd",pid=1,fd=50))\nu_dgr UNCONN 255 0
/run/systemd/journal/socket 8385 * 0
users:(("systemd-journal",pid=433,fd=4),("systemd",pid=1,fd=93))\nu_dgr
UNCONN 0 0 /dev/log 8387 * 0
users:(("systemd-journal",pid=433,fd=5),("systemd",pid=1,fd=94))\nu_str
LISTEN 0 128 /var/run/libvirt/libvirt-sock 26080 * 0
users:(("libvirtd",pid=2165,fd=10))\nu_str LISTEN 0 128
/var/run/libvirt/libvirt-sock-ro 26082 * 0
users:(("libvirtd",pid=2165,fd=11))\nu_str LISTEN 0 20
/var/run/libvirt/libvirt-admin-sock 26084 * 0
users:(("libvirtd",pid=2165,fd=12))\nu_str LISTEN 0 128
/var/run/rpcbind.sock 21482 * 0
users:(("systemd",pid=1,fd=76))\nu_str LISTEN 0 128
/run/lvm/lvmpolld.socket 22003 * 0
users:(("systemd",pid=1,fd=96))\nu_dgr UNCONN 0 0 * 16397
* 8387 users:(("dbus-daemon",pid=678,fd=15))\nu_dgr
UNCONN 0 0 * 26546 * 8387
users:(("dnsmasq",pid=2234,fd=7))\nu_dgr UNCONN 0 0 * 37631
* 8387 users:(("sshd",pid=4192,fd=4))\nu_str ESTAB
0 0 * 21873 * 22727
users:(("lvmetad",pid=1645,fd=2),("lvmetad",pid=1645,fd=1))\nu_str
ESTAB 0 0 /run/systemd/journal/stdout 22727 * 21873
users:(("systemd-journal",pid=433,fd=24),("systemd",pid=1,fd=48))\nu_str
ESTAB 0 0 /run/dbus/system_bus_socket 17971 * 17970
users:(("dbus-daemon",pid=678,fd=16))\nu_dgr UNCONN 0 0
* 33429 * 8385
users:(("systemd",pid=1,fd=14))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 15313 * 16095
users:(("systemd-journal",pid=433,fd=18),("systemd",pid=1,fd=45))\nu_str
ESTAB 0 0 * 15584 * 15009
users:(("dbus-daemon",pid=678,fd=2),("dbus-daemon",pid=678,fd=1))\nu_str
ESTAB 0 0 * 16264 * 16988
users:(("NetworkManager",pid=732,fd=16))\nu_str ESTAB 0 0 *
26089 * 26090
users:(("libvirtd",pid=2165,fd=14))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 15009 * 15584
users:(("systemd-journal",pid=433,fd=19),("systemd",pid=1,fd=41))\nu_str
ESTAB 0 0 * 16095 * 15313
users:(("NetworkManager",pid=732,fd=2),("NetworkManager",pid=732,fd=1))\nu_str
ESTAB 0 0 /run/systemd/journal/stdout 17792 * 18759
users:(("systemd-journal",pid=433,fd=20),("systemd",pid=1,fd=46))\nu_str
ESTAB 0 0 /run/systemd/journal/stdout 17848 * 17847
users:(("systemd-journal",pid=433,fd=25),("systemd",pid=1,fd=47))\nu_str
ESTAB 0 0 /run/dbus/system_bus_socket 15683 * 15545
users:(("dbus-daemon",pid=678,fd=10))\nu_str ESTAB 0 0
* 18759 * 17792
users:(("sshd",pid=1134,fd=2),("sshd",pid=1134,fd=1))\nu_str ESTAB
0 0 * 17970 * 17971
users:(("tuned",pid=1135,fd=6))\nu_str ESTAB 0 0
/run/dbus/system_bus_socket 16988 * 16264
users:(("dbus-daemon",pid=678,fd=19))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 15120 * 15743
users:(("systemd-journal",pid=433,fd=17),("systemd",pid=1,fd=43))\nu_str
ESTAB 0 0 /run/dbus/system_bus_socket 26090 * 26089
users:(("dbus-daemon",pid=678,fd=18))\nu_dgr UNCONN 0 0
* 15181 * 8385
users:(("systemd-logind",pid=694,fd=3))\nu_str ESTAB 0 0 *
15743 * 15120
users:(("systemd-logind",pid=694,fd=2),("systemd-logind",pid=694,fd=1))\nu_str
ESTAB 0 0 * 15545 * 15683
users:(("systemd",pid=1,fd=35))\nu_dgr UNCONN 0 0 * 18434
* 8387 users:(("dhclient",pid=887,fd=3))\nu_str ESTAB
0 0 * 15681 * 15682
users:(("dbus-daemon",pid=678,fd=8))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 15189 * 15888
users:(("systemd-journal",pid=433,fd=23),("systemd",pid=1,fd=44))\nu_str
ESTAB 0 0 * 14846 * 14845
users:(("auditd",pid=646,fd=10))\nu_str ESTAB 0 0 * 15682
* 15681 users:(("dbus-daemon",pid=678,fd=9))\nu_dgr
UNCONN 0 0 * 22955 * 8387
users:(("rsyslogd",pid=1737,fd=4))\nu_str ESTAB 0 0 * 17847
* 17848
users:(("tuned",pid=1135,fd=2),("tuned",pid=1135,fd=1))\nu_dgr UNCONN
0 0 * 18435 * 8387
users:(("dhclient",pid=889,fd=3))\nu_str ESTAB 0 0 * 15888
* 15189
users:(("crond",pid=700,fd=2),("crond",pid=700,fd=1))\nu_dgr UNCONN
0 0 * 14836 * 8387
users:(("auditd",pid=646,fd=8))\nu_dgr UNCONN 0 0 * 38346
* 8387 users:(("anacron",pid=4187,fd=4))\nu_str ESTAB
0 0 * 15663 * 15018
users:(("irqbalance",pid=674,fd=2),("irqbalance",pid=674,fd=1))\nu_dgr
UNCONN 0 0 * 15328 * 8387
users:(("NetworkManager",pid=732,fd=5))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 15018 * 15663
users:(("systemd-journal",pid=433,fd=22),("systemd",pid=1,fd=42))\nu_str
ESTAB 0 0 * 14845 * 14846
users:(("auditd",pid=646,fd=9))\nu_str ESTAB 0 0
/run/dbus/system_bus_socket 15356 * 15355
users:(("dbus-daemon",pid=678,fd=14))\nu_str ESTAB 0 0
/run/dbus/system_bus_socket 15296 * 15295
users:(("dbus-daemon",pid=678,fd=13))\nu_str ESTAB 0 0 *
15113 * 15688
users:(("polkitd",pid=675,fd=6))\nu_str ESTAB 0 0 * 15295
* 15296 users:(("firewalld",pid=709,fd=4))\nu_dgr
UNCONN 0 0 * 15953 * 8387
users:(("crond",pid=700,fd=4))\nu_dgr UNCONN 0 0 * 12721
* 12722 users:(("systemd-udevd",pid=460,fd=9))\nu_str
ESTAB 0 0 /run/dbus/system_bus_socket 15688 * 15113
users:(("dbus-daemon",pid=678,fd=11))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 13434 * 12664
users:(("systemd-journal",pid=433,fd=14),("systemd",pid=1,fd=37))\nu_dgr
UNCONN 0 0 * 29449 * 8387
users:(("firewalld",pid=709,fd=10))\nu_str ESTAB 0 0 * 12664
* 13434
users:(("systemd-udevd",pid=460,fd=2),("systemd-udevd",pid=460,fd=1))\nu_str
ESTAB 0 0 * 26067 * 24541
users:(("libvirtd",pid=2165,fd=2),("libvirtd",pid=2165,fd=1))\nu_str
ESTAB 0 0 * 15355 * 15356
users:(("NetworkManager",pid=732,fd=11))\nu_str ESTAB 0 0 *
15195 * 15895
users:(("systemd-logind",pid=694,fd=12))\nu_str ESTAB 0 0
/run/dbus/system_bus_socket 15895 * 15195
users:(("dbus-daemon",pid=678,fd=12))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 14961 * 14960
users:(("systemd-journal",pid=433,fd=15),("systemd",pid=1,fd=39))\nu_dgr
UNCONN 0 3840 * 12709 * 8385
users:(("systemd-udevd",pid=460,fd=5))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 14964 * 14963
users:(("systemd-journal",pid=433,fd=16),("systemd",pid=1,fd=40))\nu_str
ESTAB 0 0 * 14963 * 14964
users:(("qemu-ga",pid=671,fd=2))\nu_dgr UNCONN 0 0 * 15948
* 8387 users:(("polkitd",pid=675,fd=11))\nu_dgr
UNCONN 0 0 * 12722 * 12721
users:(("systemd-udevd",pid=460,fd=10))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 24541 * 26067
users:(("systemd-journal",pid=433,fd=26),("systemd",pid=1,fd=36))\nu_dgr
UNCONN 0 0 * 10063 * 8365
users:(("systemd-journal",pid=433,fd=11))\nu_str ESTAB 0 0 *
14960 * 14961
users:(("qemu-ga",pid=671,fd=1))\nu_str ESTAB 0 0
/run/systemd/journal/stdout 36084 * 36083
users:(("systemd-journal",pid=433,fd=21),("systemd",pid=1,fd=34))\nu_dgr
UNCONN 0 0 * 26554 * 8387
users:(("dnsmasq",pid=2234,fd=11))\nu_dgr UNCONN 0 0 * 33166
* 8387 users:(("ntpd",pid=3942,fd=3))\nu_str ESTAB
0 0 * 36083 * 36084
users:(("snmptrapd",pid=4012,fd=2),("snmptrapd",pid=4012,fd=1))\nu_dgr
UNCONN 0 0 * 33365 * 8387
users:(("snmptrapd",pid=4012,fd=9))\nudp UNCONN 0 0
192.168.122.1:53 *:*
users:(("dnsmasq",pid=2234,fd=5))\nudp UNCONN 0 0 *%virbr0:67
*:* users:(("dnsmasq",pid=2234,fd=3))\nudp
UNCONN 0 0 *:68 *:*
users:(("dhclient",pid=889,fd=6))\nudp UNCONN 0 0 *:68
*:* users:(("dhclient",pid=887,fd=6))\nudp
UNCONN 0 0 *:111 *:*
users:(("systemd",pid=1,fd=89))\nudp UNCONN 0 0
192.168.122.1:123 *:*
users:(("ntpd",pid=3942,fd=21))\nudp UNCONN 0 0
192.168.202.4:123 *:*
users:(("ntpd",pid=3942,fd=20))\nudp UNCONN 0 0
192.168.201.4:123 *:*
users:(("ntpd",pid=3942,fd=19))\nudp UNCONN 0 0 127.0.0.1:123
*:* users:(("ntpd",pid=3942,fd=18))\nudp
UNCONN 0 0 *:123 *:*
users:(("ntpd",pid=3942,fd=16))\nudp UNCONN 0 0 *:162
*:* users:(("snmptrapd",pid=4012,fd=7))\nudp
UNCONN 0 0 [::]:111 [::]:*
users:(("systemd",pid=1,fd=91))\nudp UNCONN 0 0
[fe80::5652:c0ff:fea8:c904]%eth0:123 [::]:*
users:(("ntpd",pid=3942,fd=24))\nudp UNCONN 0 0
[fe80::5652:c0ff:fea8:ca04]%eth1:123 [::]:*
users:(("ntpd",pid=3942,fd=23))\nudp UNCONN 0 0 [::1]:123
[::]:* users:(("ntpd",pid=3942,fd=22))\nudp
UNCONN 0 0 [::]:123 [::]:*
users:(("ntpd",pid=3942,fd=17))\ntcp LISTEN 0 128 *:111
*:* users:(("systemd",pid=1,fd=77))\ntcp LISTEN
0 5 192.168.122.1:53 *:*
users:(("dnsmasq",pid=2234,fd=6))\ntcp LISTEN 0 128 *:22
*:* users:(("sshd",pid=1134,fd=3))\ntcp
TIME-WAIT 0 0 192.168.201.4:41018 66.187.230.28:80
\ntcp ESTAB 0 0 192.168.201.4:22 192.168.201.1:56924
users:(("sshd",pid=4192,fd=3))\ntcp TIME-WAIT 0 0
192.168.201.4:41016 66.187.230.28:80 \ntcp LISTEN 0
128 [::]:111 [::]:*
users:(("systemd",pid=1,fd=90))\ntcp LISTEN 0 128 [::]:22
[::]:*
users:(("sshd",pid=1134,fd=4))\n\n--------------------- >> end captured
logging << ---------------------'