Hi Kaushal,
I am really thankful to you and the guy form ovirt-china "huntxu" to help
me to resolve this issue... once again thanks to all...
Punit
On Wed, Mar 25, 2015 at 6:52 PM, Kaushal M <kshlmster(a)gmail.com> wrote:
Awesome Punit! I'm happy to have been a part of the debugging
process.
~kaushal
On Wed, Mar 25, 2015 at 3:09 PM, Punit Dambiwal <hypunit(a)gmail.com> wrote:
> Hi All,
>
> With the help of gluster community and ovirt-china community...my issue
> got resolved...
>
> The main root cause was the following :-
>
> 1. the glob operation takes quite a long time, longer than the ioprocess
> default 60s..
> 2. python-ioprocess updated which makes a single change of configuration
> file doesn't work properly, only because this we should hack the code
> manually...
>
> Solution (Need to do on all the hosts) :-
>
> 1. Add the the ioprocess timeout value in the /etc/vdsm/vdsm.conf file
> as :-
>
> ------------
> [irs]
> process_pool_timeout = 180
> -------------
>
> 2. Check /usr/share/vdsm/storage/outOfProcess.py, line 71 and see whether
> there is still "IOProcess(DEFAULT_TIMEOUT)" in it,if yes...then changing
> the configuration file takes no effect because now timeout is the third
> parameter not the second of IOProcess.__init__().
>
> 3. Change IOProcess(DEFAULT_TIMEOUT) to
> IOProcess(timeout=DEFAULT_TIMEOUT) and remove the
> /usr/share/vdsm/storage/outOfProcess.pyc file and restart vdsm and
> supervdsm service on all hosts....
>
> Thanks,
> Punit Dambiwal
>
>
> On Mon, Mar 23, 2015 at 9:18 AM, Punit Dambiwal <hypunit(a)gmail.com>
> wrote:
>
>> Hi All,
>>
>> Still i am facing the same issue...please help me to overcome this
>> issue...
>>
>> Thanks,
>> punit
>>
>> On Fri, Mar 20, 2015 at 12:22 AM, Thomas Holkenbrink <
>> thomas.holkenbrink(a)fibercloud.com> wrote:
>>
>>> I’ve seen this before. The system thinks the storage system us up and
>>> running and then attempts to utilize it.
>>>
>>> The way I got around it was to put a delay in the startup of the
>>> gluster Node on the interface that the clients use to communicate.
>>>
>>>
>>>
>>> I use a bonded link, I then add a LINKDELAY to the interface to get the
>>> underlying system up and running before the network comes up. This then
>>> causes Network dependent features to wait for the network to finish.
>>>
>>> It adds about 10seconds to the startup time, in our environment it
>>> works well, you may not need as long of a delay.
>>>
>>>
>>>
>>> CentOS
>>>
>>> root@gls1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
>>>
>>>
>>>
>>> DEVICE=bond0
>>>
>>> ONBOOT=yes
>>>
>>> BOOTPROTO=static
>>>
>>> USERCTL=no
>>>
>>> NETMASK=255.255.248.0
>>>
>>> IPADDR=10.10.1.17
>>>
>>> MTU=9000
>>>
>>> IPV6INIT=no
>>>
>>> IPV6_AUTOCONF=no
>>>
>>> NETWORKING_IPV6=no
>>>
>>> NM_CONTROLLED=no
>>>
>>> LINKDELAY=10
>>>
>>> NAME="System Storage Bond0"
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Hi Michal,
>>>
>>>
>>>
>>> The Storage domain is up and running and mounted on all the host
>>> nodes...as i updated before that it was working perfectly before but just
>>> after reboot can not make the VM poweron...
>>>
>>>
>>>
>>> [image: Inline image 1]
>>>
>>>
>>>
>>> [image: Inline image 2]
>>>
>>>
>>>
>>> [root@cpu01 log]# gluster volume info
>>>
>>>
>>>
>>> Volume Name: ds01
>>>
>>> Type: Distributed-Replicate
>>>
>>> Volume ID: 369d3fdc-c8eb-46b7-a33e-0a49f2451ff6
>>>
>>> Status: Started
>>>
>>> Number of Bricks: 48 x 2 = 96
>>>
>>> Transport-type: tcp
>>>
>>> Bricks:
>>>
>>> Brick1: cpu01:/bricks/1/vol1
>>>
>>> Brick2: cpu02:/bricks/1/vol1
>>>
>>> Brick3: cpu03:/bricks/1/vol1
>>>
>>> Brick4: cpu04:/bricks/1/vol1
>>>
>>> Brick5: cpu01:/bricks/2/vol1
>>>
>>> Brick6: cpu02:/bricks/2/vol1
>>>
>>> Brick7: cpu03:/bricks/2/vol1
>>>
>>> Brick8: cpu04:/bricks/2/vol1
>>>
>>> Brick9: cpu01:/bricks/3/vol1
>>>
>>> Brick10: cpu02:/bricks/3/vol1
>>>
>>> Brick11: cpu03:/bricks/3/vol1
>>>
>>> Brick12: cpu04:/bricks/3/vol1
>>>
>>> Brick13: cpu01:/bricks/4/vol1
>>>
>>> Brick14: cpu02:/bricks/4/vol1
>>>
>>> Brick15: cpu03:/bricks/4/vol1
>>>
>>> Brick16: cpu04:/bricks/4/vol1
>>>
>>> Brick17: cpu01:/bricks/5/vol1
>>>
>>> Brick18: cpu02:/bricks/5/vol1
>>>
>>> Brick19: cpu03:/bricks/5/vol1
>>>
>>> Brick20: cpu04:/bricks/5/vol1
>>>
>>> Brick21: cpu01:/bricks/6/vol1
>>>
>>> Brick22: cpu02:/bricks/6/vol1
>>>
>>> Brick23: cpu03:/bricks/6/vol1
>>>
>>> Brick24: cpu04:/bricks/6/vol1
>>>
>>> Brick25: cpu01:/bricks/7/vol1
>>>
>>> Brick26: cpu02:/bricks/7/vol1
>>>
>>> Brick27: cpu03:/bricks/7/vol1
>>>
>>> Brick28: cpu04:/bricks/7/vol1
>>>
>>> Brick29: cpu01:/bricks/8/vol1
>>>
>>> Brick30: cpu02:/bricks/8/vol1
>>>
>>> Brick31: cpu03:/bricks/8/vol1
>>>
>>> Brick32: cpu04:/bricks/8/vol1
>>>
>>> Brick33: cpu01:/bricks/9/vol1
>>>
>>> Brick34: cpu02:/bricks/9/vol1
>>>
>>> Brick35: cpu03:/bricks/9/vol1
>>>
>>> Brick36: cpu04:/bricks/9/vol1
>>>
>>> Brick37: cpu01:/bricks/10/vol1
>>>
>>> Brick38: cpu02:/bricks/10/vol1
>>>
>>> Brick39: cpu03:/bricks/10/vol1
>>>
>>> Brick40: cpu04:/bricks/10/vol1
>>>
>>> Brick41: cpu01:/bricks/11/vol1
>>>
>>> Brick42: cpu02:/bricks/11/vol1
>>>
>>> Brick43: cpu03:/bricks/11/vol1
>>>
>>> Brick44: cpu04:/bricks/11/vol1
>>>
>>> Brick45: cpu01:/bricks/12/vol1
>>>
>>> Brick46: cpu02:/bricks/12/vol1
>>>
>>> Brick47: cpu03:/bricks/12/vol1
>>>
>>> Brick48: cpu04:/bricks/12/vol1
>>>
>>> Brick49: cpu01:/bricks/13/vol1
>>>
>>> Brick50: cpu02:/bricks/13/vol1
>>>
>>> Brick51: cpu03:/bricks/13/vol1
>>>
>>> Brick52: cpu04:/bricks/13/vol1
>>>
>>> Brick53: cpu01:/bricks/14/vol1
>>>
>>> Brick54: cpu02:/bricks/14/vol1
>>>
>>> Brick55: cpu03:/bricks/14/vol1
>>>
>>> Brick56: cpu04:/bricks/14/vol1
>>>
>>> Brick57: cpu01:/bricks/15/vol1
>>>
>>> Brick58: cpu02:/bricks/15/vol1
>>>
>>> Brick59: cpu03:/bricks/15/vol1
>>>
>>> Brick60: cpu04:/bricks/15/vol1
>>>
>>> Brick61: cpu01:/bricks/16/vol1
>>>
>>> Brick62: cpu02:/bricks/16/vol1
>>>
>>> Brick63: cpu03:/bricks/16/vol1
>>>
>>> Brick64: cpu04:/bricks/16/vol1
>>>
>>> Brick65: cpu01:/bricks/17/vol1
>>>
>>> Brick66: cpu02:/bricks/17/vol1
>>>
>>> Brick67: cpu03:/bricks/17/vol1
>>>
>>> Brick68: cpu04:/bricks/17/vol1
>>>
>>> Brick69: cpu01:/bricks/18/vol1
>>>
>>> Brick70: cpu02:/bricks/18/vol1
>>>
>>> Brick71: cpu03:/bricks/18/vol1
>>>
>>> Brick72: cpu04:/bricks/18/vol1
>>>
>>> Brick73: cpu01:/bricks/19/vol1
>>>
>>> Brick74: cpu02:/bricks/19/vol1
>>>
>>> Brick75: cpu03:/bricks/19/vol1
>>>
>>> Brick76: cpu04:/bricks/19/vol1
>>>
>>> Brick77: cpu01:/bricks/20/vol1
>>>
>>> Brick78: cpu02:/bricks/20/vol1
>>>
>>> Brick79: cpu03:/bricks/20/vol1
>>>
>>> Brick80: cpu04:/bricks/20/vol1
>>>
>>> Brick81: cpu01:/bricks/21/vol1
>>>
>>> Brick82: cpu02:/bricks/21/vol1
>>>
>>> Brick83: cpu03:/bricks/21/vol1
>>>
>>> Brick84: cpu04:/bricks/21/vol1
>>>
>>> Brick85: cpu01:/bricks/22/vol1
>>>
>>> Brick86: cpu02:/bricks/22/vol1
>>>
>>> Brick87: cpu03:/bricks/22/vol1
>>>
>>> Brick88: cpu04:/bricks/22/vol1
>>>
>>> Brick89: cpu01:/bricks/23/vol1
>>>
>>> Brick90: cpu02:/bricks/23/vol1
>>>
>>> Brick91: cpu03:/bricks/23/vol1
>>>
>>> Brick92: cpu04:/bricks/23/vol1
>>>
>>> Brick93: cpu01:/bricks/24/vol1
>>>
>>> Brick94: cpu02:/bricks/24/vol1
>>>
>>> Brick95: cpu03:/bricks/24/vol1
>>>
>>> Brick96: cpu04:/bricks/24/vol1
>>>
>>> Options Reconfigured:
>>>
>>> diagnostics.count-fop-hits: on
>>>
>>> diagnostics.latency-measurement: on
>>>
>>> nfs.disable: on
>>>
>>> user.cifs: enable
>>>
>>> auth.allow: 10.10.0.*
>>>
>>> performance.quick-read: off
>>>
>>> performance.read-ahead: off
>>>
>>> performance.io-cache: off
>>>
>>> performance.stat-prefetch: off
>>>
>>> cluster.eager-lock: enable
>>>
>>> network.remote-dio: enable
>>>
>>> cluster.quorum-type: auto
>>>
>>> cluster.server-quorum-type: server
>>>
>>> storage.owner-uid: 36
>>>
>>> storage.owner-gid: 36
>>>
>>> server.allow-insecure: on
>>>
>>> network.ping-timeout: 100
>>>
>>> [root@cpu01 log]#
>>>
>>>
>>>
>>> -----------------------------------------
>>>
>>>
>>>
>>> [root@cpu01 log]# gluster volume status
>>>
>>> Status of volume: ds01
>>>
>>> Gluster process Port Online
>>> Pid
>>>
>>>
>>>
------------------------------------------------------------------------------
>>>
>>> Brick cpu01:/bricks/1/vol1 49152 Y
>>> 33474
>>>
>>> Brick cpu02:/bricks/1/vol1 49152 Y
>>> 40717
>>>
>>> Brick cpu03:/bricks/1/vol1 49152 Y
>>> 18080
>>>
>>> Brick cpu04:/bricks/1/vol1 49152 Y
>>> 40447
>>>
>>> Brick cpu01:/bricks/2/vol1 49153 Y
>>> 33481
>>>
>>> Brick cpu02:/bricks/2/vol1 49153 Y
>>> 40724
>>>
>>> Brick cpu03:/bricks/2/vol1 49153 Y
>>> 18086
>>>
>>> Brick cpu04:/bricks/2/vol1 49153 Y
>>> 40453
>>>
>>> Brick cpu01:/bricks/3/vol1 49154 Y
>>> 33489
>>>
>>> Brick cpu02:/bricks/3/vol1 49154 Y
>>> 40731
>>>
>>> Brick cpu03:/bricks/3/vol1 49154 Y
>>> 18097
>>>
>>> Brick cpu04:/bricks/3/vol1 49154 Y
>>> 40460
>>>
>>> Brick cpu01:/bricks/4/vol1 49155 Y
>>> 33495
>>>
>>> Brick cpu02:/bricks/4/vol1 49155 Y
>>> 40738
>>>
>>> Brick cpu03:/bricks/4/vol1 49155 Y
>>> 18103
>>>
>>> Brick cpu04:/bricks/4/vol1 49155 Y
>>> 40468
>>>
>>> Brick cpu01:/bricks/5/vol1 49156 Y
>>> 33502
>>>
>>> Brick cpu02:/bricks/5/vol1 49156 Y
>>> 40745
>>>
>>> Brick cpu03:/bricks/5/vol1 49156 Y
>>> 18110
>>>
>>> Brick cpu04:/bricks/5/vol1 49156 Y
>>> 40474
>>>
>>> Brick cpu01:/bricks/6/vol1 49157 Y
>>> 33509
>>>
>>> Brick cpu02:/bricks/6/vol1 49157 Y
>>> 40752
>>>
>>> Brick cpu03:/bricks/6/vol1 49157 Y
>>> 18116
>>>
>>> Brick cpu04:/bricks/6/vol1 49157 Y
>>> 40481
>>>
>>> Brick cpu01:/bricks/7/vol1 49158 Y
>>> 33516
>>>
>>> Brick cpu02:/bricks/7/vol1 49158 Y
>>> 40759
>>>
>>> Brick cpu03:/bricks/7/vol1 49158 Y
>>> 18122
>>>
>>> Brick cpu04:/bricks/7/vol1 49158 Y
>>> 40488
>>>
>>> Brick cpu01:/bricks/8/vol1 49159 Y
>>> 33525
>>>
>>> Brick cpu02:/bricks/8/vol1 49159 Y
>>> 40766
>>>
>>> Brick cpu03:/bricks/8/vol1 49159 Y
>>> 18130
>>>
>>> Brick cpu04:/bricks/8/vol1 49159 Y
>>> 40495
>>>
>>> Brick cpu01:/bricks/9/vol1 49160 Y
>>> 33530
>>>
>>> Brick cpu02:/bricks/9/vol1 49160 Y
>>> 40773
>>>
>>> Brick cpu03:/bricks/9/vol1 49160 Y
>>> 18137
>>>
>>> Brick cpu04:/bricks/9/vol1 49160 Y
>>> 40502
>>>
>>> Brick cpu01:/bricks/10/vol1 49161 Y
>>> 33538
>>>
>>> Brick cpu02:/bricks/10/vol1 49161 Y
>>> 40780
>>>
>>> Brick cpu03:/bricks/10/vol1 49161 Y
>>> 18143
>>>
>>> Brick cpu04:/bricks/10/vol1 49161 Y
>>> 40509
>>>
>>> Brick cpu01:/bricks/11/vol1 49162 Y
>>> 33544
>>>
>>> Brick cpu02:/bricks/11/vol1 49162 Y
>>> 40787
>>>
>>> Brick cpu03:/bricks/11/vol1 49162 Y
>>> 18150
>>>
>>> Brick cpu04:/bricks/11/vol1 49162 Y
>>> 40516
>>>
>>> Brick cpu01:/bricks/12/vol1 49163 Y
>>> 33551
>>>
>>> Brick cpu02:/bricks/12/vol1 49163 Y
>>> 40794
>>>
>>> Brick cpu03:/bricks/12/vol1 49163 Y
>>> 18157
>>>
>>> Brick cpu04:/bricks/12/vol1 49163 Y
>>> 40692
>>>
>>> Brick cpu01:/bricks/13/vol1 49164 Y
>>> 33558
>>>
>>> Brick cpu02:/bricks/13/vol1 49164 Y
>>> 40801
>>>
>>> Brick cpu03:/bricks/13/vol1 49164 Y
>>> 18165
>>>
>>> Brick cpu04:/bricks/13/vol1 49164 Y
>>> 40700
>>>
>>> Brick cpu01:/bricks/14/vol1 49165 Y
>>> 33566
>>>
>>> Brick cpu02:/bricks/14/vol1 49165 Y
>>> 40809
>>>
>>> Brick cpu03:/bricks/14/vol1 49165 Y
>>> 18172
>>>
>>> Brick cpu04:/bricks/14/vol1 49165 Y
>>> 40706
>>>
>>> Brick cpu01:/bricks/15/vol1 49166 Y
>>> 33572
>>>
>>> Brick cpu02:/bricks/15/vol1 49166 Y
>>> 40815
>>>
>>> Brick cpu03:/bricks/15/vol1 49166 Y
>>> 18179
>>>
>>> Brick cpu04:/bricks/15/vol1 49166 Y
>>> 40714
>>>
>>> Brick cpu01:/bricks/16/vol1 49167 Y
>>> 33579
>>>
>>> Brick cpu02:/bricks/16/vol1 49167 Y
>>> 40822
>>>
>>> Brick cpu03:/bricks/16/vol1 49167 Y
>>> 18185
>>>
>>> Brick cpu04:/bricks/16/vol1 49167 Y
>>> 40722
>>>
>>> Brick cpu01:/bricks/17/vol1 49168 Y
>>> 33586
>>>
>>> Brick cpu02:/bricks/17/vol1 49168 Y
>>> 40829
>>>
>>> Brick cpu03:/bricks/17/vol1 49168 Y
>>> 18192
>>>
>>> Brick cpu04:/bricks/17/vol1 49168 Y
>>> 40727
>>>
>>> Brick cpu01:/bricks/18/vol1 49169 Y
>>> 33593
>>>
>>> Brick cpu02:/bricks/18/vol1 49169 Y
>>> 40836
>>>
>>> Brick cpu03:/bricks/18/vol1 49169 Y
>>> 18201
>>>
>>> Brick cpu04:/bricks/18/vol1 49169 Y
>>> 40735
>>>
>>> Brick cpu01:/bricks/19/vol1 49170 Y
>>> 33600
>>>
>>> Brick cpu02:/bricks/19/vol1 49170 Y
>>> 40843
>>>
>>> Brick cpu03:/bricks/19/vol1 49170 Y
>>> 18207
>>>
>>> Brick cpu04:/bricks/19/vol1 49170 Y
>>> 40741
>>>
>>> Brick cpu01:/bricks/20/vol1 49171 Y
>>> 33608
>>>
>>> Brick cpu02:/bricks/20/vol1 49171 Y
>>> 40850
>>>
>>> Brick cpu03:/bricks/20/vol1 49171 Y
>>> 18214
>>>
>>> Brick cpu04:/bricks/20/vol1 49171 Y
>>> 40748
>>>
>>> Brick cpu01:/bricks/21/vol1 49172 Y
>>> 33614
>>>
>>> Brick cpu02:/bricks/21/vol1 49172 Y
>>> 40858
>>>
>>> Brick cpu03:/bricks/21/vol1 49172 Y
>>> 18222
>>>
>>> Brick cpu04:/bricks/21/vol1 49172 Y
>>> 40756
>>>
>>> Brick cpu01:/bricks/22/vol1 49173 Y
>>> 33621
>>>
>>> Brick cpu02:/bricks/22/vol1 49173 Y
>>> 40864
>>>
>>> Brick cpu03:/bricks/22/vol1 49173 Y
>>> 18227
>>>
>>> Brick cpu04:/bricks/22/vol1 49173 Y
>>> 40762
>>>
>>> Brick cpu01:/bricks/23/vol1 49174 Y
>>> 33626
>>>
>>> Brick cpu02:/bricks/23/vol1 49174 Y
>>> 40869
>>>
>>> Brick cpu03:/bricks/23/vol1 49174 Y
>>> 18234
>>>
>>> Brick cpu04:/bricks/23/vol1 49174 Y
>>> 40769
>>>
>>> Brick cpu01:/bricks/24/vol1 49175 Y
>>> 33631
>>>
>>> Brick cpu02:/bricks/24/vol1 49175 Y
>>> 40874
>>>
>>> Brick cpu03:/bricks/24/vol1 49175 Y
>>> 18239
>>>
>>> Brick cpu04:/bricks/24/vol1 49175 Y
>>> 40774
>>>
>>> Self-heal Daemon on localhost N/A Y
>>> 33361
>>>
>>> Self-heal Daemon on cpu05 N/A Y
>>> 2353
>>>
>>> Self-heal Daemon on cpu04 N/A Y
>>> 40786
>>>
>>> Self-heal Daemon on cpu02 N/A Y
>>> 32442
>>>
>>> Self-heal Daemon on cpu03 N/A Y
>>> 18664
>>>
>>>
>>>
>>> Task Status of Volume ds01
>>>
>>>
>>>
------------------------------------------------------------------------------
>>>
>>> Task : Rebalance
>>>
>>> ID : 5db24b30-4b9f-4b65-8910-a7a0a6d327a4
>>>
>>> Status : completed
>>>
>>>
>>>
>>> [root@cpu01 log]#
>>>
>>>
>>>
>>> [root@cpu01 log]# gluster pool list
>>>
>>> UUID Hostname State
>>>
>>> 626c9360-8c09-480f-9707-116e67cc38e6 cpu02 Connected
>>>
>>> dc475d62-b035-4ee6-9006-6f03bf68bf24 cpu05 Connected
>>>
>>> 41b5b2ff-3671-47b4-b477-227a107e718d cpu03 Connected
>>>
>>> c0afe114-dfa7-407d-bad7-5a3f97a6f3fc cpu04 Connected
>>>
>>> 9b61b0a5-be78-4ac2-b6c0-2db588da5c35 localhost Connected
>>>
>>> [root@cpu01 log]#
>>>
>>>
>>>
>>> [image: Inline image 3]
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Punit
>>>
>>>
>>>
>>> On Thu, Mar 19, 2015 at 2:53 PM, Michal Skrivanek <
>>> michal.skrivanek(a)redhat.com> wrote:
>>>
>>>
>>> On Mar 19, 2015, at 03:18 , Punit Dambiwal <hypunit(a)gmail.com> wrote:
>>>
>>> > Hi All,
>>> >
>>> > Is there any one have any idea about this problem...it seems it's
bug
>>> either in Ovirt or Glusterfs...that's why no one has the idea about
>>> it....please correct me if i am wrong….
>>>
>>> Hi,
>>> as I said, storage access times out; so it seems to me as a gluster
>>> setup problem, the storage domain you have your VMs on is not working…
>>>
>>> Thanks,
>>> michal
>>>
>>>
>>> >
>>> > Thanks,
>>> > Punit
>>> >
>>> > On Wed, Mar 18, 2015 at 5:05 PM, Punit Dambiwal
<hypunit(a)gmail.com>
>>> wrote:
>>> > Hi Michal,
>>> >
>>> > Would you mind to let me know the possible messedup things...i will
>>> check and try to resolve it....still i am communicating gluster community
>>> to resolve this issue...
>>> >
>>> > But in the ovirt....gluster setup is quite straight....so how come it
>>> will be messedup with reboot ?? if it can be messedup with reboot then it
>>> seems not good and stable technology for the production storage....
>>> >
>>> > Thanks,
>>> > Punit
>>> >
>>> > On Wed, Mar 18, 2015 at 3:51 PM, Michal Skrivanek <
>>> michal.skrivanek(a)redhat.com> wrote:
>>> >
>>> > On Mar 18, 2015, at 03:33 , Punit Dambiwal <hypunit(a)gmail.com>
wrote:
>>> >
>>> > > Hi,
>>> > >
>>> > > Is there any one from community can help me to solve this
issue...??
>>> > >
>>> > > Thanks,
>>> > > Punit
>>> > >
>>> > > On Tue, Mar 17, 2015 at 12:52 PM, Punit Dambiwal
<hypunit(a)gmail.com>
>>> wrote:
>>> > > Hi,
>>> > >
>>> > > I am facing one strange issue with ovirt/glusterfs....still
didn't
>>> find this issue is related with glusterfs or Ovirt....
>>> > >
>>> > > Ovirt :- 3.5.1
>>> > > Glusterfs :- 3.6.1
>>> > > Host :- 4 Hosts (Compute+ Storage)...each server has 24 bricks
>>> > > Guest VM :- more then 100
>>> > >
>>> > > Issue :- When i deploy this cluster first time..it work well for
>>> me(all the guest VM created and running successfully)....but suddenly one
>>> day my one of the host node rebooted and none of the VM can boot up
>>> now...and failed with the following error "Bad Volume
Specification"
>>> > >
>>> > > VMId :- d877313c18d9783ca09b62acf5588048
>>> > >
>>> > > VDSM Logs :-
http://ur1.ca/jxabi
>>> >
>>> > you've got timeouts while accessing storage…so I guess something
got
>>> messed up on reboot, it may also be just a gluster misconfiguration…
>>> >
>>> > > Engine Logs :-
http://ur1.ca/jxabv
>>> > >
>>> > > ------------------------
>>> > > [root@cpu01 ~]# vdsClient -s 0 getVolumeInfo
>>> e732a82f-bae9-4368-8b98-dedc1c3814de 00000002-0002-0002-0002-000000000145
>>> 6d123509-6867-45cf-83a2-6d679b77d3c5 9030bb43-6bc9-462f-a1b9-f6d5a02fb180
>>> > > status = OK
>>> > > domain = e732a82f-bae9-4368-8b98-dedc1c3814de
>>> > > capacity = 21474836480
>>> > > voltype = LEAF
>>> > > description =
>>> > > parent = 00000000-0000-0000-0000-000000000000
>>> > > format = RAW
>>> > > image = 6d123509-6867-45cf-83a2-6d679b77d3c5
>>> > > uuid = 9030bb43-6bc9-462f-a1b9-f6d5a02fb180
>>> > > disktype = 2
>>> > > legality = LEGAL
>>> > > mtime = 0
>>> > > apparentsize = 21474836480
>>> > > truesize = 4562972672
>>> > > type = SPARSE
>>> > > children = []
>>> > > pool =
>>> > > ctime = 1422676305
>>> > > ---------------------
>>> > >
>>> > > I opened same thread earlier but didn't get any perfect answers
to
>>> solve this issue..so i reopen it...
>>> > >
>>> > >
https://www.mail-archive.com/users@ovirt.org/msg25011.html
>>> > >
>>> > > Thanks,
>>> > > Punit
>>> > >
>>> > >
>>> > >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>
>>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users(a)gluster.org
>
http://www.gluster.org/mailman/listinfo/gluster-users
>