Yes, so the bug has been fixed upstream and the backports to
release-7 and release-8 of gluster
pending merge. The fix should be available in the next .x release of gluster-7 and 8.
Until then
like Nir suggested, please turn off performance.stat-prefetch on your volumes.
-Krutika
On Wed, Jun 17, 2020 at 5:59 AM Nir Soffer <nsoffer(a)redhat.com> wrote:
> On Mon, Jun 8, 2020 at 3:10 PM Joop <jvdwege(a)xs4all.nl> wrote:
>>
>> On 3-6-2020 14:58, Joop wrote:
>>> Hi All,
>>>
>>> Just had a rather new experience in that starting a VM worked but the
>>> kernel entered grub2 rescue console due to the fact that something was
>>> wrong with its virtio-scsi disk.
>>> The message is Booting from Hard Disk ....
>>> error: ../../grub-core/kern/dl.c:266:invalid arch-independent ELF maginc.
>>> entering rescue mode...
>>>
>>> Doing a CTRL-ALT-Del through the spice console let the VM boot
>>> correctly. Shutting it down and repeating the procedure I get a disk
>>> problem everytime. Weird thing is if I activate the BootMenu and then
>>> straight away start the VM all is OK.
>>> I don't see any ERROR messages in either vdsm.log, engine.log
>>>
>>> If I would have to guess it looks like the disk image isn't connected
>>> yet when the VM boots but thats weird isn't it?
>>>
>>>
>> As an update to this:
>> Just had the same problem with a Windows VM but more importantly also
>> with HostedEngine itself.
>> On the host did:
>> hosted-engine --set-maintenance --mode=global
>> hosted-engine --vm-shutdown
>>
>> Stopped all oVirt related services, cleared all oVirt related logs from
>> /var/log/..., restarted the host, ran hosted-engine --set-maintenance
>> --mode=none
>> Watched /var/spool/mail/root to see the engine coming up. It went to
>> starting but never came into the Up status.
>> Set a password and used vncviewer to see the console, see attached
>> screenschot.
>
> The screenshot "engine.png" show gluster bug we discovered a few weeks
ago:
>
https://bugzilla.redhat.com/1823423
>
> Until you get a fixed version, this may fix the issues:
>
> # gluster volume set engine performance.stat-prefetch off
>
> See
https://bugzilla.redhat.com/show_bug.cgi?id=1823423#c55.
>
> Krutica, can this bug affect upstream gluster?
>
> Joop, please share the gluster version in your setup.
gluster-ansible-cluster-1.0.0-1.el8.noarch
gluster-ansible-features-1.0.5-6.el8.noarch
gluster-ansible-infra-1.0.4-10.el8.noarch
gluster-ansible-maintenance-1.0.1-3.el8.noarch
gluster-ansible-repositories-1.0.1-2.el8.noarch
gluster-ansible-roles-1.0.5-12.el8.noarch
glusterfs-7.5-1.el8.x86_64
glusterfs-api-7.5-1.el8.x86_64
glusterfs-cli-7.5-1.el8.x86_64
glusterfs-client-xlators-7.5-1.el8.x86_64
glusterfs-events-7.5-1.el8.x86_64
glusterfs-fuse-7.5-1.el8.x86_64
glusterfs-geo-replication-7.5-1.el8.x86_64
glusterfs-libs-7.5-1.el8.x86_64
glusterfs-rdma-7.5-1.el8.x86_64
glusterfs-server-7.5-1.el8.x86_64
libvirt-daemon-driver-storage-gluster-6.0.0-17.el8.x86_64
python3-gluster-7.5-1.el8.x86_64
qemu-kvm-block-gluster-4.2.0-19.el8.x86_64
vdsm-gluster-4.40.16-1.el8.x86_64
I tried friday to import my VMs but had not much success with the stat-prefetch off
setting. Some VMs imported correctly some didn't and no correlation between size or
whatever.
Sunday I decided to use features.shard off and I was able to import 1Tb worth of VM images
without a hitch.
I'm on HCI so I'm assuming that turning sharding off won't be a performance
problem?
If its fixed I can still move all VM disk to a other storage domain and back after turning
sharding back on.
Regards,
Joop