Well, now that I’ve gone and read through that bug again in detail, I’m not sure I’ve worked around it after all. I do seem to recall additional discussion on the original bug for HA engine ligfapi a mention that RR-DNS would work to resolve the issue, but can’t remember the bug ID at the moment. I will test thoroughly the next time I update my glusterfs servers. But I firmly believe that I’ve never encountered that issue in over 3 years of running gluster with libgfapi enabled..
I use round robin DNS, and in theory, QEMU retries until it gets a working server. I also have said DNS setup in host files on all my hosts and gluster servers, having discovered the hard way that when your DNS server runs on an ovirt managed VM, you have a bootstrap problem when thing break badly :) Somewhere around gluster 3.12, I added backup servers to the mount options for my gluster storage volumes as well, and have’t had any issues with that.
And to be frank, the significant performance bonus from libgfapi is still absolutely worth it to me even if it means automatic HA won’t work if one particular server is down. I can always intervene in the DNS on my hosts if I have to, and it just hasn’t come up yet.
-Darrell
On February 13, 2020 11:51:41 PM GMT+02:00, Stephen Panicho <s.panicho@gmail.com> wrote:Darrell, would you care to elaborate on your HA workaround?
As far as I understand, only the primary Gluster host is visible to
libvirt
when using gfapi, so if that host goes down, all VMs break. I imagine
you're using a round-robin DNS entry for the primary Gluster host, but
I'd
like to be sure.
On Wed, Feb 12, 2020 at 11:01 AM Darrell Budic <budic@onholyground.com>
wrote:
Yes. I’m using libgfapi access on gluster 6.7 with overt 4.3.8 just
fine,
but I don’t use snapshots. You can work around the HA issue with DNS
and
backup server entries on the storage domain as well. Worth it to me
for the
performance, YMMV.
On Feb 12, 2020, at 8:04 AM, Jayme <jaymef@gmail.com> wrote:
From my understanding it's not a default option but many users are
still
using libgfapi successfully. I'm not sure about its status in the
latest
4.3.8 release but I know it is/was working for people in previous
versions.
The libgfapi bugs affect HA and snapshots (on 3 way replica HCI) but
it
should still be working otherwise, unless like I said something
changed in
more recent releases of oVirt.
On Wed, Feb 12, 2020 at 9:43 AM Guillaume Pavese <
guillaume.pavese@interactiv-group.com> wrote:
Libgfapi is not supported because of an old bug in qemu. That qemu
bug is
slowly getting fixed, but the bugs about Libgfapi support in ovirt
have
since been closed as WONTFIX and DEFERRED
See :
https://bugzilla.redhat.com/show_bug.cgi?id=1465810
https://bugzilla.redhat.com/show_bug.cgi?id=1484660
<https://bugzilla.redhat.com/show_bug.cgi?id=1484227> : "No plans to
enable libgfapi in RHHI-V for now. Closing this bug"
https://bugzilla.redhat.com/show_bug.cgi?id=1484227 : "No plans to
enable libgfapi in RHHI-V for now. Closing this bug"
https://bugzilla.redhat.com/show_bug.cgi?id=1633642 : "Closing this
as
no action taken from long back.Please reopen if required."
Would be nice if someone could reopen the closed bugs so this
feature
doesn't get forgotten
Guillaume Pavese
Ingénieur Système et Réseau
Interactiv-Group
On Tue, Feb 11, 2020 at 9:58 AM Stephen Panicho
<s.panicho@gmail.com>
wrote:
I used the cockpit-based hc setup and "option
rpc-auth-allow-insecure"
is absent from /etc/glusterfs/glusterd.vol.
I'm going to redo the cluster this week and report back. Thanks for
the
tip!
On Mon, Feb 10, 2020 at 6:01 PM Darrell Budic
<budic@onholyground.com>
wrote:
The hosts will still mount the volume via FUSE, but you might
double
check you set the storage up as Gluster and not NFS.
Then gluster used to need some config in glusterd.vol to set
option rpc-auth-allow-insecure on
I’m not sure if that got added to a hyper converged setup or not,
but
I’d check it.
On Feb 10, 2020, at 4:41 PM, Stephen Panicho <s.panicho@gmail.com>
wrote:
No, this was a relatively new cluster-- only a couple days old.
Just a
handful of VMs including the engine.
On Mon, Feb 10, 2020 at 5:26 PM Jayme <jaymef@gmail.com> wrote:
Curious do the vms have active snapshots?
On Mon, Feb 10, 2020 at 5:59 PM <s.panicho@gmail.com> wrote:
Hello, all. I have a 3-node Hyperconverged oVirt 4.3.8 cluster
running on CentOS 7.7 hosts. I was investigating poor Gluster
performance
and heard about libgfapi, so I thought I'd give it a shot.
Looking through
the documentation, followed by lots of threads and BZ reports,
I've done
the following to enable it:
First, I shut down all VMs except the engine. Then...
On the hosts:
1. setsebool -P virt_use_glusterfs on
2. dynamic_ownership=0 in /etc/libvirt/qemu.conf
On the engine VM:
1. engine-config -s LibgfApiSupported=true --cver=4.3
2. systemctl restart ovirt-engine
VMs now fail to launch. Am I doing this correctly? I should also
note
that the hosts still have the Gluster domain mounted via FUSE.
Here's a relevant bit from engine.log:
2020-02-06T16:38:32.573511Z qemu-kvm: -drive file=gluster://
node1.fs.trashnet.xyz:24007/vmstore/781717e5-1cff-43a1-b586-9941503544e8/images/a1d56b14-6d72-4f46-a0aa-eb0870c36bc4/a2314816-7970-49ce-a80c-ab0d1cf17c78,file.debug=4,format=qcow2,if=none,id=drive-ua-a1d56b14-6d72-4f46-a0aa-eb0870c36bc4,serial=a1d56b14-6d72-4f46-a0aa-eb0870c36bc4,werror=stop,rerror=stop,cache=none,discard=unmap,aio=native:
Could not read qcow2 header: Invalid argument.
The full engine.log from one of the attempts:
2020-02-06 16:38:24,909Z INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-12) [] add VM
'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'(yumcache) to rerun
treatment
2020-02-06 16:38:25,010Z ERROR
[org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring]
(ForkJoinPool-1-worker-12) [] Rerun VM
'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'. Called from VDS '
node2.ovirt.trashnet.xyz'
2020-02-06 16:38:25,091Z WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-216) [] EVENT_ID:
USER_INITIATED_RUN_VM_FAILED(151), Failed to run VM yumcache on
Host
node2.ovirt.trashnet.xyz.
2020-02-06 16:38:25,166Z INFO
[org.ovirt.engine.core.bll.RunVmCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] Lock Acquired to
object
'EngineLock:{exclusiveLocks='[df9dbac4-35c0-40ee-acd4-a1cfc959aa8b=VM]',
sharedLocks=''}'
2020-02-06 16:38:25,179Z INFO
[org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] START,
IsVmDuringInitiatingVDSCommand(
IsVmDuringInitiatingVDSCommandParameters:{vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'}),
log id: 2107f52a
2020-02-06 16:38:25,181Z INFO
[org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] FINISH,
IsVmDuringInitiatingVDSCommand, return: false, log id: 2107f52a
2020-02-06 16:38:25,298Z INFO
[org.ovirt.engine.core.bll.RunVmCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] Running command:
RunVmCommand internal: false. Entities affected : ID:
df9dbac4-35c0-40ee-acd4-a1cfc959aa8b Type: VMAction group RUN_VM
with role
type USER
2020-02-06 16:38:25,313Z INFO
[org.ovirt.engine.core.bll.utils.EmulatedMachineUtils]
(EE-ManagedThreadFactory-engine-Thread-216) [] Emulated machine
'pc-q35-rhel7.6.0' which is different than that of the cluster
is set for
'yumcache'(df9dbac4-35c0-40ee-acd4-a1cfc959aa8b)
2020-02-06 16:38:25,382Z INFO
[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] START,
UpdateVmDynamicDataVDSCommand(
UpdateVmDynamicDataVDSCommandParameters:{hostId='null',
vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b',
vmDynamic='org.ovirt.engine.core.common.businessentities.VmDynamic@9774a64'}),
log id: 4a83911f
2020-02-06 16:38:25,417Z INFO
[org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] FINISH,
UpdateVmDynamicDataVDSCommand, return: , log id: 4a83911f
2020-02-06 16:38:25,418Z INFO
[org.ovirt.engine.core.vdsbroker.CreateVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] START,
CreateVDSCommand(
CreateVDSCommandParameters:{hostId='c3465ca2-395e-4c0c-b72e-b5b7153df452',
vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b', vm='VM
[yumcache]'}), log id:
5e07ba66
2020-02-06 16:38:25,420Z INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] START,
CreateBrokerVDSCommand(HostName = node1.ovirt.trashnet.xyz,
CreateVDSCommandParameters:{hostId='c3465ca2-395e-4c0c-b72e-b5b7153df452',
vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b', vm='VM
[yumcache]'}), log id:
1bfa03c4
2020-02-06 16:38:25,424Z INFO
[org.ovirt.engine.core.vdsbroker.builder.vminfo.VmInfoBuildUtils]
(EE-ManagedThreadFactory-engine-Thread-216) [] Kernel FIPS -
Guid:
c3465ca2-395e-4c0c-b72e-b5b7153df452 fips: false
2020-02-06 16:38:25,435Z INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] VM <?xml
version="1.0"
encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="
http://ovirt.org/vm/tune/1.0"
xmlns:ovirt-vm="http://ovirt.org/vm/1.0
">
<name>yumcache</name>
<uuid>df9dbac4-35c0-40ee-acd4-a1cfc959aa8b</uuid>
<memory>1048576</memory>
<currentMemory>1048576</currentMemory>
<iothreads>1</iothreads>
<maxMemory slots="16">4194304</maxMemory>
<vcpu current="1">16</vcpu>
<sysinfo type="smbios">
<system>
<entry name="manufacturer">oVirt</entry>
<entry name="product">OS-NAME:</entry>
<entry name="version">OS-VERSION:</entry>
<entry name="serial">HOST-SERIAL:</entry>
<entry
name="uuid">df9dbac4-35c0-40ee-acd4-a1cfc959aa8b</entry>
</system>
</sysinfo>
<clock offset="variable" adjustment="0">
<timer name="rtc" tickpolicy="catchup"/>
<timer name="pit" tickpolicy="delay"/>
<timer name="hpet" present="no"/>
</clock>
<features>
<acpi/>
</features>
<cpu match="exact">
<model>EPYC</model>
<feature name="ibpb" policy="require"/>
<feature name="virt-ssbd" policy="require"/>
<topology cores="1" threads="1" sockets="16"/>
<numa>
<cell id="0" cpus="0" memory="1048576"/>
</numa>
</cpu>
<cputune/>
<devices>
<input type="tablet" bus="usb"/>
<channel type="unix">
<target type="virtio" name="ovirt-guest-agent.0"/>
<source mode="bind"
path="/var/lib/libvirt/qemu/channels/df9dbac4-35c0-40ee-acd4-a1cfc959aa8b.ovirt-guest-agent.0"/>
</channel>
<channel type="unix">
<target type="virtio" name="org.qemu.guest_agent.0"/>
<source mode="bind"
path="/var/lib/libvirt/qemu/channels/df9dbac4-35c0-40ee-acd4-a1cfc959aa8b.org.qemu.guest_agent.0"/>
</channel>
<controller type="pci" model="pcie-root-port" index="1">
<address bus="0x00" domain="0x0000" function="0x0"
slot="0x02"
type="pci" multifunction="on"/>
</controller>
<memballoon model="virtio">
<stats period="5"/>
<alias name="ua-27c77007-3a3c-4431-958d-90fd1c7257dd"/>
<address bus="0x05" domain="0x0000" function="0x0"
slot="0x00"
type="pci"/>
</memballoon>
<controller type="pci" model="pcie-root-port" index="2">
<address bus="0x00" domain="0x0000" function="0x1"
slot="0x02"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="9">
<address bus="0x00" domain="0x0000" function="0x0"
slot="0x03"
type="pci" multifunction="on"/>
</controller>
<controller type="sata" index="0">
<address bus="0x00" domain="0x0000" function="0x2"
slot="0x1f"
type="pci"/>
</controller>
<rng model="virtio">
<backend model="random">/dev/urandom</backend>
<alias name="ua-51960005-6b95-47e9-82a7-67d5e0d6cf8a"/>
</rng>
<controller type="pci" model="pcie-root-port" index="6">
<address bus="0x00" domain="0x0000" function="0x5"
slot="0x02"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="15">
<address bus="0x00" domain="0x0000" function="0x6"
slot="0x03"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="13">
<address bus="0x00" domain="0x0000" function="0x4"
slot="0x03"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="7">
<address bus="0x00" domain="0x0000" function="0x6"
slot="0x02"
type="pci"/>
</controller>
<graphics type="vnc" port="-1" autoport="yes" passwd="*****"
passwdValidTo="1970-01-01T00:00:01" keymap="en-us">
<listen type="network" network="vdsm-ovirtmgmt"/>
</graphics>
<controller type="pci" model="pcie-root-port" index="16">
<address bus="0x00" domain="0x0000" function="0x7"
slot="0x03"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="12">
<address bus="0x00" domain="0x0000" function="0x3"
slot="0x03"
type="pci"/>
</controller>
<video>
<model type="qxl" vram="32768" heads="1" ram="65536"
vgamem="16384"/>
<alias name="ua-8a295e96-40c3-44de-a3b0-1c4a685a5473"/>
<address bus="0x00" domain="0x0000" function="0x0"
slot="0x01"
type="pci"/>
</video>
<graphics type="spice" port="-1" autoport="yes"
passwd="*****"
passwdValidTo="1970-01-01T00:00:01" tlsPort="-1">
<channel name="main" mode="secure"/>
<channel name="inputs" mode="secure"/>
<channel name="cursor" mode="secure"/>
<channel name="playback" mode="secure"/>
<channel name="record" mode="secure"/>
<channel name="display" mode="secure"/>
<channel name="smartcard" mode="secure"/>
<channel name="usbredir" mode="secure"/>
<listen type="network" network="vdsm-ovirtmgmt"/>
</graphics>
<controller type="pci" model="pcie-root-port" index="5">
<address bus="0x00" domain="0x0000" function="0x4"
slot="0x02"
type="pci"/>
</controller>
<controller type="usb" model="qemu-xhci" index="0"
ports="8">
<address bus="0x02" domain="0x0000" function="0x0"
slot="0x00"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="4">
<address bus="0x00" domain="0x0000" function="0x3"
slot="0x02"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="3">
<address bus="0x00" domain="0x0000" function="0x2"
slot="0x02"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="11">
<address bus="0x00" domain="0x0000" function="0x2"
slot="0x03"
type="pci"/>
</controller>
<controller type="scsi" model="virtio-scsi" index="0">
<driver iothread="1"/>
<alias name="ua-d0bf6fcd-7aa2-4658-b7cc-3dac259b7ad2"/>
<address bus="0x03" domain="0x0000" function="0x0"
slot="0x00"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="8">
<address bus="0x00" domain="0x0000" function="0x7"
slot="0x02"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="14">
<address bus="0x00" domain="0x0000" function="0x5"
slot="0x03"
type="pci"/>
</controller>
<controller type="pci" model="pcie-root-port" index="10">
<address bus="0x00" domain="0x0000" function="0x1"
slot="0x03"
type="pci"/>
</controller>
<controller type="virtio-serial" index="0" ports="16">
<address bus="0x04" domain="0x0000" function="0x0"
slot="0x00"
type="pci"/>
</controller>
<channel type="spicevmc">
<target type="virtio" name="com.redhat.spice.0"/>
</channel>
<controller type="pci" model="pcie-root"/>
<interface type="bridge">
<model type="virtio"/>
<link state="up"/>
<source bridge="vmnet"/>
<alias name="ua-ceda0ef6-9139-4e5c-8840-86fe344ecbd3"/>
<address bus="0x01" domain="0x0000" function="0x0"
slot="0x00"
type="pci"/>
<mac address="56:6f:91:b9:00:05"/>
<mtu size="1500"/>
<filterref filter="vdsm-no-mac-spoofing"/>
<bandwidth/>
</interface>
<disk type="file" device="cdrom" snapshot="no">
<driver name="qemu" type="raw" error_policy="report"/>
<source file="" startupPolicy="optional">
<seclabel model="dac" type="none" relabel="no"/>
</source>
<target dev="sdc" bus="sata"/>
<readonly/>
<alias name="ua-bdf99844-2d02-411b-90bb-671ee26764cb"/>
<address bus="0" controller="0" unit="2" type="drive"
target="0"/>
</disk>
<disk snapshot="no" type="network" device="disk">
<target dev="sda" bus="scsi"/>
<source protocol="gluster"
name="vmstore/781717e5-1cff-43a1-b586-9941503544e8/images/a1d56b14-6d72-4f46-a0aa-eb0870c36bc4/a2314816-7970-49ce-a80c-ab0d1cf17c78">
<host name="node1.fs.trashnet.xyz" port="0"/>
<seclabel model="dac" type="none" relabel="no"/>
</source>
<driver name="qemu" discard="unmap" io="native"
type="qcow2"
error_policy="stop" cache="none"/>
<alias name="ua-a1d56b14-6d72-4f46-a0aa-eb0870c36bc4"/>
<address bus="0" controller="0" unit="0" type="drive"
target="0"/>
<boot order="1"/>
<serial>a1d56b14-6d72-4f46-a0aa-eb0870c36bc4</serial>
</disk>
<lease>
<key>df9dbac4-35c0-40ee-acd4-a1cfc959aa8b</key>
<lockspace>781717e5-1cff-43a1-b586-9941503544e8</lockspace>
<target offset="6291456"
path="/rhev/data-center/mnt/glusterSD/
node1.fs.trashnet.xyz
:_vmstore/781717e5-1cff-43a1-b586-9941503544e8/dom_md/xleases"/>
</lease>
</devices>
<pm>
<suspend-to-disk enabled="no"/>
<suspend-to-mem enabled="no"/>
</pm>
<os>
<type arch="x86_64" machine="pc-q35-rhel7.6.0">hvm</type>
<smbios mode="sysinfo"/>
</os>
<metadata>
<ovirt-tune:qos/>
<ovirt-vm:vm>
<ovirt-vm:minGuaranteedMemoryMb
type="int">512</ovirt-vm:minGuaranteedMemoryMb>
<ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion>
<ovirt-vm:custom/>
<ovirt-vm:device mac_address="56:6f:91:b9:00:05">
<ovirt-vm:custom/>
</ovirt-vm:device>
<ovirt-vm:device devtype="disk" name="sda">
<ovirt-vm:poolID>2ffaec76-462c-11ea-b155-00163e512202</ovirt-vm:poolID>
<ovirt-vm:volumeID>a2314816-7970-49ce-a80c-ab0d1cf17c78</ovirt-vm:volumeID>
<ovirt-vm:imageID>a1d56b14-6d72-4f46-a0aa-eb0870c36bc4</ovirt-vm:imageID>
<ovirt-vm:domainID>781717e5-1cff-43a1-b586-9941503544e8</ovirt-vm:domainID>
</ovirt-vm:device>
<ovirt-vm:launchPaused>false</ovirt-vm:launchPaused>
<ovirt-vm:resumeBehavior>kill</ovirt-vm:resumeBehavior>
</ovirt-vm:vm>
</metadata>
</domain>
2020-02-06 16:38:25,455Z INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] FINISH,
CreateBrokerVDSCommand, return: , log id: 1bfa03c4
2020-02-06 16:38:25,494Z INFO
[org.ovirt.engine.core.vdsbroker.CreateVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] FINISH,
CreateVDSCommand,
return: WaitForLaunch, log id: 5e07ba66
2020-02-06 16:38:25,495Z INFO
[org.ovirt.engine.core.bll.RunVmCommand]
(EE-ManagedThreadFactory-engine-Thread-216) [] Lock freed to
object
'EngineLock:{exclusiveLocks='[df9dbac4-35c0-40ee-acd4-a1cfc959aa8b=VM]',
sharedLocks=''}'
2020-02-06 16:38:25,533Z INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-216) [] EVENT_ID:
USER_STARTED_VM(153), VM yumcache was started by
admin@internal-authz
(Host: node1.ovirt.trashnet.xyz).
2020-02-06 16:38:33,300Z INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-5) [] VM
'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b' was
reported as Down on VDS 'c3465ca2-395e-4c0c-b72e-b5b7153df452'(
node1.ovirt.trashnet.xyz)
2020-02-06 16:38:33,301Z INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand]
(ForkJoinPool-1-worker-5) [] START, DestroyVDSCommand(HostName =
node1.ovirt.trashnet.xyz,
DestroyVmVDSCommandParameters:{hostId='c3465ca2-395e-4c0c-b72e-b5b7153df452',
vmId='df9dbac4-35c0-40ee-acd4-a1cfc959aa8b', secondsToWait='0',
gracefully='false', reason='', ignoreNoVm='true'}), log id:
1f951ea9
2020-02-06 16:38:33,478Z INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher]
(EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Fetched 2
VMs from
VDS 'c3465ca2-395e-4c0c-b72e-b5b7153df452'
2020-02-06 16:38:33,545Z INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand]
(ForkJoinPool-1-worker-5) [] FINISH, DestroyVDSCommand, return:
, log id:
1f951ea9
2020-02-06 16:38:33,546Z INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-5) [] VM
'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'(yumcache) moved from
'WaitForLaunch'
--> 'Down'
2020-02-06 16:38:33,623Z ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJoinPool-1-worker-5) [] EVENT_ID: VM_DOWN_ERROR(119), VM
yumcache is
down with error. Exit message: internal error: qemu unexpectedly
closed the
monitor: [2020-02-06 16:38:31.723977] E [MSGID: 108006]
[afr-common.c:5323:__afr_handle_child_down_event]
0-vmstore-replicate-0:
All subvolumes are down. Going offline until at least one of
them comes
back up.
[2020-02-06 16:38:31.724765] I [io-stats.c:4027:fini] 0-vmstore:
io-stats translator unloaded
2020-02-06T16:38:32.573511Z qemu-kvm: -drive file=gluster://
node1.fs.trashnet.xyz:24007/vmstore/781717e5-1cff-43a1-b586-9941503544e8/images/a1d56b14-6d72-4f46-a0aa-eb0870c36bc4/a2314816-7970-49ce-a80c-ab0d1cf17c78,file.debug=4,format=qcow2,if=none,id=drive-ua-a1d56b14-6d72-4f46-a0aa-eb0870c36bc4,serial=a1d56b14-6d72-4f46-a0aa-eb0870c36bc4,werror=stop,rerror=stop,cache=none,discard=unmap,aio=native:
Could not read qcow2 header: Invalid argument.
2020-02-06 16:38:33,624Z INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-5) [] add VM
'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'(yumcache) to rerun
treatment
2020-02-06 16:38:33,796Z ERROR
[org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring]
(ForkJoinPool-1-worker-5) [] Rerun VM
'df9dbac4-35c0-40ee-acd4-a1cfc959aa8b'. Called from VDS '
node1.ovirt.trashnet.xyz'
2020-02-06 16:38:33,899Z WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-223) [] EVENT_ID:
USER_INITIATED_RUN_VM_FAILED(151), Failed to run VM yumcache on
Host
node1.ovirt.trashnet.xyz.
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/6GTBANZ4R44HJE2BU55GAEBLTETUXTKT/
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/5EZYM4OADD65RVNDIQVS25EGZOK65PWX/
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KKUVN3ZR5QOVKKCOR5XIUYYC6EL7FYXS/
Ce message et toutes les pièces jointes (ci-après le “message”) sont
établis à l’intention exclusive de ses destinataires et sont
confidentiels.
Si vous recevez ce message par erreur, merci de le détruire et d’en
avertir
immédiatement l’expéditeur. Toute utilisation de ce message non
conforme a
sa destination, toute diffusion ou toute publication, totale ou
partielle,
est interdite, sauf autorisation expresse. L’internet ne permettant
pas
d’assurer l’intégrité de ce message . Interactiv-group (et ses
filiales)
décline(nt) toute responsabilité au titre de ce message, dans
l’hypothèse
ou il aurait été modifié. IT, ES, UK.
<https://interactiv-group.com/disclaimer.html>
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/S2RYKJLRE533BZB5A3ZDDMIZ573W7HTC/
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/23SAGPLPE5MHCGGQJXUK6D7MMO6CFWWJ/
My guess is different - a real Load Balancer , like HAproxy, can take care when a backend server is down.Still, it will be nice to learn Darrell's solution.Best Regards,Strahil Nikolov