[ovirt-users] Problems with some vms
Endre Karlson
endre.karlson at gmail.com
Wed Jan 17 11:07:36 UTC 2018
One brick was at a point down for replacement.
It has been replaced and all vols are up
Status of volume: data
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick ovirt0:/gluster/brick3/data 49152 0 Y
22467
Brick ovirt2:/gluster/brick3/data 49152 0 Y
20736
Brick ovirt3:/gluster/brick3/data 49152 0 Y
23148
Brick ovirt0:/gluster/brick4/data 49153 0 Y
22497
Brick ovirt2:/gluster/brick4/data 49153 0 Y
20742
Brick ovirt3:/gluster/brick4/data 49153 0 Y
23158
Brick ovirt0:/gluster/brick5/data 49154 0 Y
22473
Brick ovirt2:/gluster/brick5/data 49154 0 Y
20748
Brick ovirt3:/gluster/brick5/data 49154 0 Y
23156
Brick ovirt0:/gluster/brick6/data 49155 0 Y
22479
Brick ovirt2:/gluster/brick6_1/data 49161 0 Y
21203
Brick ovirt3:/gluster/brick6/data 49155 0 Y
23157
Brick ovirt0:/gluster/brick7/data 49156 0 Y
22485
Brick ovirt2:/gluster/brick7/data 49156 0 Y
20763
Brick ovirt3:/gluster/brick7/data 49156 0 Y
23155
Brick ovirt0:/gluster/brick8/data 49157 0 Y
22491
Brick ovirt2:/gluster/brick8/data 49157 0 Y
20771
Brick ovirt3:/gluster/brick8/data 49157 0 Y
23154
Self-heal Daemon on localhost N/A N/A Y
23238
Bitrot Daemon on localhost N/A N/A Y
24870
Scrubber Daemon on localhost N/A N/A Y
24889
Self-heal Daemon on ovirt2 N/A N/A Y
24271
Bitrot Daemon on ovirt2 N/A N/A Y
24856
Scrubber Daemon on ovirt2 N/A N/A Y
24866
Self-heal Daemon on ovirt0 N/A N/A Y
29409
Bitrot Daemon on ovirt0 N/A N/A Y
5457
Scrubber Daemon on ovirt0 N/A N/A Y
5468
Task Status of Volume data
------------------------------------------------------------------------------
There are no active volume tasks
Status of volume: engine
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick ovirt0:/gluster/brick1/engine 49158 0 Y
22511
Brick ovirt2:/gluster/brick1/engine 49158 0 Y
20780
Brick ovirt3:/gluster/brick1/engine 49158 0 Y
23199
Self-heal Daemon on localhost N/A N/A Y
23238
Self-heal Daemon on ovirt0 N/A N/A Y
29409
Self-heal Daemon on ovirt2 N/A N/A Y
24271
Task Status of Volume engine
------------------------------------------------------------------------------
There are no active volume tasks
Status of volume: iso
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick ovirt0:/gluster/brick2/iso 49159 0 Y
22520
Brick ovirt2:/gluster/brick2/iso 49159 0 Y
20789
Brick ovirt3:/gluster/brick2/iso 49159 0 Y
23208
NFS Server on localhost N/A N/A N
N/A
Self-heal Daemon on localhost N/A N/A Y
23238
NFS Server on ovirt2 N/A N/A N
N/A
Self-heal Daemon on ovirt2 N/A N/A Y
24271
NFS Server on ovirt0 N/A N/A N
N/A
Self-heal Daemon on ovirt0 N/A N/A Y
29409
Task Status of Volume iso
------------------------------------------------------------------------------
There are no active volume tasks
2018-01-17 8:13 GMT+01:00 Gobinda Das <godas at redhat.com>:
> Hi,
> I can see some error in log:
> [2018-01-14 11:19:49.886571] E [socket.c:2309:socket_connect_finish]
> 0-engine-client-0: connection to 10.2.0.120:24007 failed (Connection
> timed out)
> [2018-01-14 11:20:05.630669] E [socket.c:2309:socket_connect_finish]
> 0-engine-client-0: connection to 10.2.0.120:24007 failed (Connection
> timed out)
> [2018-01-14 12:01:09.089925] E [MSGID: 114058]
> [client-handshake.c:1527:client_query_portmap_cbk] 0-engine-client-0:
> failed to get the port number for remote subvolume. Please run 'gluster
> volume status' on server to see if brick process is running.
> [2018-01-14 12:01:09.090048] I [MSGID: 114018]
> [client.c:2280:client_rpc_notify] 0-engine-client-0: disconnected from
> engine-client-0. Client process will keep trying to connect to glusterd
> until brick's port is available
>
> Can you please check gluster volume status and see if all bricks are up?
>
> On Wed, Jan 17, 2018 at 12:24 PM, Endre Karlson <endre.karlson at gmail.com>
> wrote:
>
>> It's there now for each of the hosts. ovirt1 is not in service yet.
>>
>> 2018-01-17 5:52 GMT+01:00 Gobinda Das <godas at redhat.com>:
>>
>>> In the above url only data and iso mnt log present,But there is no
>>> engine and vmstore mount log.
>>>
>>> On Wed, Jan 17, 2018 at 1:26 AM, Endre Karlson <endre.karlson at gmail.com>
>>> wrote:
>>>
>>>> Hi, all logs are located here: https://www.dropbox.com/
>>>> sh/3qzmwe76rkt09fk/AABzM9rJKbH5SBPWc31Npxhma?dl=0 for the mounts
>>>>
>>>> additionally we replaced a broken disk that is now resynced.
>>>>
>>>> 2018-01-15 11:17 GMT+01:00 Gobinda Das <godas at redhat.com>:
>>>>
>>>>> Hi Endre,
>>>>> Mount logs will be in below format inside /var/log/glusterfs :
>>>>>
>>>>> /var/log/glusterfs/rhev-data-center-mnt-glusterSD-*\:_engine.log
>>>>> /var/log/glusterfs/rhev-data-center-mnt-glusterSD-*\:_data.log
>>>>> /var/log/glusterfs/rhev-data-center-mnt-glusterSD-*\:_vmstore.log
>>>>>
>>>>> On Mon, Jan 15, 2018 at 11:57 AM, Endre Karlson <
>>>>> endre.karlson at gmail.com> wrote:
>>>>>
>>>>>> Hi.
>>>>>>
>>>>>> What are the gluster mount logs ?
>>>>>>
>>>>>> I have these gluster logs.
>>>>>> cli.log etc-glusterfs-glusterd.vol.log
>>>>>> glfsheal-engine.log glusterd.log nfs.log
>>>>>> rhev-data-center-mnt-glusterSD-ovirt0:_engine.log
>>>>>> rhev-data-center-mnt-glusterSD-ovirt3:_iso.log
>>>>>> cmd_history.log glfsheal-data.log glfsheal-iso.log
>>>>>> glustershd.log rhev-data-center-mnt-glusterSD-ovirt0:_data.log
>>>>>> rhev-data-center-mnt-glusterSD-ovirt0:_iso.log statedump.log
>>>>>>
>>>>>>
>>>>>> I am running version
>>>>>> glusterfs-server-3.12.4-1.el7.x86_64
>>>>>> glusterfs-geo-replication-3.12.4-1.el7.x86_64
>>>>>> libvirt-daemon-driver-storage-gluster-3.2.0-14.el7_4.7.x86_64
>>>>>> glusterfs-libs-3.12.4-1.el7.x86_64
>>>>>> glusterfs-api-3.12.4-1.el7.x86_64
>>>>>> python2-gluster-3.12.4-1.el7.x86_64
>>>>>> glusterfs-client-xlators-3.12.4-1.el7.x86_64
>>>>>> glusterfs-cli-3.12.4-1.el7.x86_64
>>>>>> glusterfs-events-3.12.4-1.el7.x86_64
>>>>>> glusterfs-rdma-3.12.4-1.el7.x86_64
>>>>>> vdsm-gluster-4.20.9.3-1.el7.centos.noarch
>>>>>> glusterfs-3.12.4-1.el7.x86_64
>>>>>> glusterfs-fuse-3.12.4-1.el7.x86_64
>>>>>>
>>>>>> // Endre
>>>>>>
>>>>>> 2018-01-15 6:11 GMT+01:00 Gobinda Das <godas at redhat.com>:
>>>>>>
>>>>>>> Hi Endre,
>>>>>>> Can you please provide glusterfs mount logs?
>>>>>>>
>>>>>>> On Mon, Jan 15, 2018 at 6:16 AM, Darrell Budic <
>>>>>>> budic at onholyground.com> wrote:
>>>>>>>
>>>>>>>> What version of gluster are you running? I’ve seen a few of these
>>>>>>>> since moving my storage cluster to 12.3, but still haven’t been able to
>>>>>>>> determine what’s causing it. Seems to be happening most often on VMs that
>>>>>>>> haven’t been switches over to libgfapi mounts yet, but even one of those
>>>>>>>> has paused once so far. They generally restart fine from the GUI, and
>>>>>>>> nothing seems to need healing.
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>> *From:* Endre Karlson <endre.karlson at gmail.com>
>>>>>>>> *Subject:* [ovirt-users] Problems with some vms
>>>>>>>> *Date:* January 14, 2018 at 12:55:45 PM CST
>>>>>>>> *To:* users
>>>>>>>>
>>>>>>>> Hi, we are getting some errors with some of our vms in a 3 node
>>>>>>>> server setup.
>>>>>>>>
>>>>>>>> 2018-01-14 15:01:44,015+0100 INFO (libvirt/events) [virt.vm]
>>>>>>>> (vmId='2c34f52d-140b-4dbe-a4bd-d2cb467b0b7c') abnormal vm stop
>>>>>>>> device virtio-disk0 error eother (vm:4880)
>>>>>>>>
>>>>>>>> We are running glusterfs for shared storage.
>>>>>>>>
>>>>>>>> I have tried setting global maintenance on the first server and
>>>>>>>> then issuing a 'hosted-engine --vm-start' but that leads to nowhere.
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list
>>>>>>>> Users at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list
>>>>>>>> Users at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> Gobinda
>>>>>>> +91-9019047912 <+91%2090190%2047912>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Gobinda
>>>>> +91-9019047912 <+91%2090190%2047912>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Gobinda
>>> +91-9019047912 <+91%2090190%2047912>
>>>
>>
>>
>
>
> --
> Thanks,
> Gobinda
> +91-9019047912 <+91%2090190%2047912>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20180117/0235a34d/attachment.html>
More information about the Users
mailing list