<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <br>
    Hi Brett,<br>
    Can you truncate the gluster brick and mount logs on all three
    nodes, try creating the storage domain again and then share these
    logs along with the VDSM logs? <br>
    <br>
    i.e. on all 3 nodes, <br>
    1. echo &gt; /var/log/glusterfs/<span>rhev-data-center-mnt-glusterSD-sjcstorage01:_vmstore.log</span><br>
    2. echo &gt; export-vmstore-brick01.log<br>
    3. Create the storage domain (at which point VDSM supposedly fails
    with the truncate error)<br>
    4. Share the logs.<br>
    <br>
    Also, what timezone are you in? That would be needed to co-relate
    the timestamps in the vdsm log (local time) and gluster log (UTC)<br>
    <br>
    Thanks!<br>
    Ravi<br>
    <blockquote cite="mid:560A221C.5030204@redhat.com" type="cite">
      <div class="moz-forward-container"> <br>
        -------- Forwarded Message --------
        <table class="moz-email-headers-table" border="0"
          cellpadding="0" cellspacing="0">
          <tbody>
            <tr>
              <th nowrap="nowrap" valign="BASELINE" align="RIGHT">Subject:

              </th>
              <td>Re: [ovirt-users] adding gluster domains</td>
            </tr>
            <tr>
              <th nowrap="nowrap" valign="BASELINE" align="RIGHT">Date:
              </th>
              <td>Tue, 29 Sep 2015 08:38:49 +1000</td>
            </tr>
            <tr>
              <th nowrap="nowrap" valign="BASELINE" align="RIGHT">From:
              </th>
              <td>Brett Stevens <a moz-do-not-send="true"
                  class="moz-txt-link-rfc2396E"
                  href="mailto:gorttman@i3sec.com.au">&lt;gorttman@i3sec.com.au&gt;</a></td>
            </tr>
            <tr>
              <th nowrap="nowrap" valign="BASELINE" align="RIGHT">Reply-To:

              </th>
              <td><a moz-do-not-send="true"
                  class="moz-txt-link-abbreviated"
                  href="mailto:brett@i3sec.com.au">brett@i3sec.com.au</a></td>
            </tr>
            <tr>
              <th nowrap="nowrap" valign="BASELINE" align="RIGHT">To: </th>
              <td>Sahina Bose <a moz-do-not-send="true"
                  class="moz-txt-link-rfc2396E"
                  href="mailto:sabose@redhat.com">&lt;sabose@redhat.com&gt;</a></td>
            </tr>
          </tbody>
        </table>
        <br>
        <br>
        <div dir="ltr">Sorry about the delay, I've run the truncate. I'm
          not sure what results you were expecting, but it executed
          fine, no delays no errors no problems. 
          <div><br>
          </div>
          <div>thanks</div>
          <div>Brett Stevens</div>
        </div>
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Thu, Sep 24, 2015 at 7:29 PM,
            Brett Stevens <span dir="ltr">&lt;<a moz-do-not-send="true"
                href="mailto:gorttman@i3sec.com.au" target="_blank">gorttman@i3sec.com.au</a>&gt;</span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0
              .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div dir="ltr">Thanks I'll do that tomorrow morning. 
                <div><br>
                </div>
                <div>Just out of interest, I keep getting warn errors in
                  the engine.log allong the lines of node not present
                  (sjcvhost02 which is the arbiter) and no gluster
                  network present even after I have added the gluster
                  network option in the network management gui.</div>
                <div><br>
                </div>
                <div>thanks</div>
                <div><br>
                </div>
                <div>Brett Stevens</div>
                <div><br>
                </div>
              </div>
              <div class="gmail_extra"><br>
                <div class="gmail_quote">On Thu, Sep 24, 2015 at 7:26
                  PM, Sahina Bose <span dir="ltr">&lt;<a
                      moz-do-not-send="true"
                      class="moz-txt-link-abbreviated"
                      href="mailto:sabose@redhat.com"><a class="moz-txt-link-abbreviated" href="mailto:sabose@redhat.com">sabose@redhat.com</a></a>&gt;</span>
                  wrote:<br>
                  <blockquote class="gmail_quote" style="margin:0 0 0
                    .8ex;border-left:1px #ccc solid;padding-left:1ex">
                    <div text="#000000" bgcolor="#FFFFFF"> Sorry, I
                      intended to forward it to a gluster devel.<br>
                      <br>
                      Btw, there were no errors in the mount log - so
                      unable to root cause why truncate of file failed
                      with IO error. Was the log from vhost03 -
                      /var/log/glusterfs/<span>rhev-data-center-mnt-glusterSD-sjcstorage01:_vmstore.log


                        ?<br>
                        <br>
                        We will look into the logs you attached to see
                        if there are any errors reported at the bricks.
                        (But there should have been some error in mount
                        log!)<br>
                        <br>
                        Could you also try "truncate -s 10M test" from
                        the mount point ( manually mount gluster using -
                        #mount -t glusterfs </span><span><span>sjcstorage01:/vmstore


                          &lt;mountpoint&gt;) and report results.</span><br>
                      </span><br>
                      <div>On 09/24/2015 02:32 PM, Brett Stevens wrote:<br>
                      </div>
                      <blockquote type="cite">
                        <div dir="ltr">Hi Sahina.
                          <div><br>
                          </div>
                          <div>Something has gone wrong with your last
                            email. I have received a message from you,
                            but did not get any text to go with it.
                            could you resend please?</div>
                          <div><br>
                          </div>
                          <div>thanks</div>
                          <div><br>
                          </div>
                        </div>
                        <div class="gmail_extra"><br>
                          <div class="gmail_quote">On Thu, Sep 24, 2015
                            at 6:48 PM, Sahina Bose <span dir="ltr">&lt;<a
                                moz-do-not-send="true"
                                class="moz-txt-link-abbreviated"
                                href="mailto:sabose@redhat.com"><a class="moz-txt-link-abbreviated" href="mailto:sabose@redhat.com">sabose@redhat.com</a></a>&gt;</span>
                            wrote:<br>
                            <blockquote class="gmail_quote"
                              style="margin:0 0 0 .8ex;border-left:1px
                              #ccc solid;padding-left:1ex">
                              <div text="#000000" bgcolor="#FFFFFF"> <br>
                                <br>
                                <div>On 09/24/2015 04:21 AM, Brett
                                  Stevens wrote:<br>
                                </div>
                                <blockquote type="cite">
                                  <div dir="ltr">Hi Sahina. 
                                    <div><br>
                                    </div>
                                    <div>vhost02 is the engine node
                                      vhost03 is the hypervisor
                                      storage01 and 02 the gluster
                                      nodes. I've put arbiter on vhost02</div>
                                    <div><br>
                                    </div>
                                    <div>all tasks are separated (except
                                      engine and arbiter) </div>
                                    <div><br>
                                    </div>
                                    <div>thanks</div>
                                    <div><br>
                                    </div>
                                  </div>
                                  <div class="gmail_extra"><br>
                                    <div class="gmail_quote">On Wed, Sep
                                      23, 2015 at 9:48 PM, Sahina Bose <span
                                        dir="ltr">&lt;<a
                                          moz-do-not-send="true"
                                          class="moz-txt-link-abbreviated"
href="mailto:sabose@redhat.com"><a class="moz-txt-link-abbreviated" href="mailto:sabose@redhat.com">sabose@redhat.com</a></a>&gt;</span> wrote:<br>
                                      <blockquote class="gmail_quote"
                                        style="margin:0 0 0
                                        .8ex;border-left:1px #ccc
                                        solid;padding-left:1ex">
                                        <div text="#000000"
                                          bgcolor="#FFFFFF"> +
                                          ovirt-users<br>
                                          <br>
                                          Some clarity on your setup - <br>
                                          <span>sjcvhost03 - is this
                                            your arbiter node and ovirt
                                            management node? And are you
                                            running a compute + storage
                                            on the same nodes - i.e, </span><span>sjcstorage01,


                                          </span><span>sjcstorage02, </span><span>sjcvhost03


                                            (arbiter).<br>
                                            <br>
                                          </span><br>
                                          <span>
                                            CreateStorageDomainVDSCommand(HostName
                                            = sjcvhost03,
                                            CreateStorageDomainVDSCommandParameters:{runAsync='true',
                                            hostId='80245ac2-32a3-4d5d-b0fe-08019e2d1c9c',


                                            storageDomain='StorageDomainStatic:{name='sjcvmstore',



                                            id='597d5b5b-7c09-4de9-8840-6993bd9b61a6'}',


                                            args='sjcstorage01:/vmstore'}),

                                            log id: b9fe587<br>
                                            <br>
                                            - fails with </span><span>Error

                                            creating a storage domain's
                                            metadata: ("create meta file
                                            'outbox' failed: [Errno 5]
                                            Input/output error",<br>
                                            <br>
                                            Are the vdsm logs you
                                            provided from </span><span>sjcvhost03?

                                            There are no errors to be
                                            seen in the gluster log you
                                            provided. Could you provide
                                            mount log from </span><span><span>sjcvhost03</span>
                                            (at
                                            /rhev/data-center/mnt/glusterSD/sjcstorage01:_vmstore.log
                                            most likely)<br>
                                            If possible,
                                            /var/log/glusterfs/* from
                                            the 3 storage nodes.<br>
                                            <br>
                                            thanks<br>
                                            sahina<br>
                                            <br>
                                          </span>
                                          <div>On 09/23/2015 05:02 AM,
                                            Brett Stevens wrote:<br>
                                          </div>
                                          <blockquote type="cite">
                                            <div dir="ltr">Hi Sahina, 
                                              <div><br>
                                              </div>
                                              <div>as requested here is
                                                some logs taken during a
                                                domain create.</div>
                                              <div><br>
                                              </div>
                                              <div>
                                                <p><span>2015-09-22
                                                    18:46:44,320 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
                                                    (DefaultQuartzScheduler_Worker-88)

                                                    [] START,
                                                    GlusterVolumesListVDSCommand(HostName
                                                    = sjcstorage01,
                                                    GlusterVolumesListVDSParameters:{runAsync='true',
                                                    hostId='c75682ba-1e4c-42a3-85c7-16e4bb2ce5da'}),



                                                    log id: 2205ff1</span></p>
                                                <p><span>2015-09-22
                                                    18:46:44,413 WARN 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]




                                                    (DefaultQuartzScheduler_Worker-88)

                                                    [] Could not
                                                    associate brick
                                                    'sjcstorage01:/export/vmstore/brick01'
                                                    of volume
                                                    '030f270a-0999-4df4-9b14-ae56eb0a2fb9'
                                                    with correct network
                                                    as no gluster
                                                    network found in
                                                    cluster
                                                    'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:44,417 WARN 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]




                                                    (DefaultQuartzScheduler_Worker-88)

                                                    [] Could not
                                                    associate brick
                                                    'sjcstorage02:/export/vmstore/brick01'
                                                    of volume
                                                    '030f270a-0999-4df4-9b14-ae56eb0a2fb9'
                                                    with correct network
                                                    as no gluster
                                                    network found in
                                                    cluster
                                                    'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:44,417 WARN 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]




                                                    (DefaultQuartzScheduler_Worker-88)

                                                    [] Could not add
                                                    brick
                                                    'sjcvhost02:/export/vmstore/brick01'
                                                    to volume
                                                    '030f270a-0999-4df4-9b14-ae56eb0a2fb9'
                                                    - server uuid
                                                    '29b58278-9aa3-47c5-bfb4-1948ef7fdbba'
                                                    not found in cluster
'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:44,418 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
                                                    (DefaultQuartzScheduler_Worker-88)

                                                    [] FINISH,
                                                    GlusterVolumesListVDSCommand,
                                                    return:
                                                    {030f270a-0999-4df4-9b14-ae56eb0a2fb9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a0628f36},




                                                    log id: 2205ff1</span></p>
                                                <p><span>2015-09-22
                                                    18:46:45,215 INFO 
                                                    [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand]
                                                    (default task-24)
                                                    [5099cda3] Lock
                                                    Acquired to object
                                                    'EngineLock:{exclusiveLocks='[sjcstorage01:/vmstore=&lt;STORAGE_CONNECTION,




                                                    ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;]',



                                                    sharedLocks='null'}'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:45,230 INFO 
                                                    [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand]
                                                    (default task-24)
                                                    [5099cda3] Running
                                                    command:
                                                    AddStorageServerConnectionCommand
                                                    internal: false.
                                                    Entities affected : 
                                                    ID:
                                                    aaa00000-0000-0000-0000-123456789aaa
                                                    Type: SystemAction
                                                    group
                                                    CREATE_STORAGE_DOMAIN
                                                    with role type ADMIN</span></p>
                                                <p><span>2015-09-22
                                                    18:46:45,233 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]




                                                    (default task-24)
                                                    [5099cda3] START,
                                                    ConnectStorageServerVDSCommand(HostName
                                                    = sjcvhost03,
                                                    StorageServerConnectionManagementVDSParameters:{runAsync='true',
                                                    hostId='80245ac2-32a3-4d5d-b0fe-08019e2d1c9c',



                                                    storagePoolId='00000000-0000-0000-0000-000000000000',



                                                    storageType='GLUSTERFS',


                                                    connectionList='[StorageServerConnections:{id='null',


                                                    connection='sjcstorage01:/vmstore',


                                                    iqn='null',
                                                    vfsType='glusterfs',
                                                    mountOptions='null',
                                                    nfsVersion='null',
                                                    nfsRetrans='null',
                                                    nfsTimeo='null',
                                                    iface='null',
                                                    netIfaceName='null'}]'}),
                                                    log id: 6a112292</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,065 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]




                                                    (default task-24)
                                                    [5099cda3] FINISH,
                                                    ConnectStorageServerVDSCommand,
                                                    return:
                                                    {00000000-0000-0000-0000-000000000000=0},
                                                    log id: 6a112292</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,073 INFO 
                                                    [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand]
                                                    (default task-24)
                                                    [5099cda3] Lock
                                                    freed to object
                                                    'EngineLock:{exclusiveLocks='[sjcstorage01:/vmstore=&lt;STORAGE_CONNECTION,




                                                    ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;]',



                                                    sharedLocks='null'}'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,188 INFO 
                                                    [org.ovirt.engine.core.bll.storage.AddGlusterFsStorageDomainCommand]
                                                    (default task-23)
                                                    [6410419] Running
                                                    command:
                                                    AddGlusterFsStorageDomainCommand
                                                    internal: false.
                                                    Entities affected : 
                                                    ID:
                                                    aaa00000-0000-0000-0000-123456789aaa
                                                    Type: SystemAction
                                                    group
                                                    CREATE_STORAGE_DOMAIN
                                                    with role type ADMIN</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,206 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]




                                                    (default task-23)
                                                    [6410419] START,
                                                    ConnectStorageServerVDSCommand(HostName
                                                    = sjcvhost03,
                                                    StorageServerConnectionManagementVDSParameters:{runAsync='true',
                                                    hostId='80245ac2-32a3-4d5d-b0fe-08019e2d1c9c',



                                                    storagePoolId='00000000-0000-0000-0000-000000000000',



                                                    storageType='GLUSTERFS',


                                                    connectionList='[StorageServerConnections:{id='ec5ab31e-b5b9-4a8e-a2b2-0876df71a21e',




                                                    connection='sjcstorage01:/vmstore',


                                                    iqn='null',
                                                    vfsType='glusterfs',
                                                    mountOptions='null',
                                                    nfsVersion='null',
                                                    nfsRetrans='null',
                                                    nfsTimeo='null',
                                                    iface='null',
                                                    netIfaceName='null'}]'}),
                                                    log id: 38a2b0d</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,219 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]




                                                    (default task-23)
                                                    [6410419] FINISH,
                                                    ConnectStorageServerVDSCommand,
                                                    return:
                                                    {ec5ab31e-b5b9-4a8e-a2b2-0876df71a21e=0},
                                                    log id: 38a2b0d</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,221 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand]




                                                    (default task-23)
                                                    [6410419] START,
                                                    CreateStorageDomainVDSCommand(HostName
                                                    = sjcvhost03,
                                                    CreateStorageDomainVDSCommandParameters:{runAsync='true',
                                                    hostId='80245ac2-32a3-4d5d-b0fe-08019e2d1c9c',



                                                    storageDomain='StorageDomainStatic:{name='sjcvmstore',



                                                    id='597d5b5b-7c09-4de9-8840-6993bd9b61a6'}',



                                                    args='sjcstorage01:/vmstore'}),

                                                    log id: b9fe587</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,744 ERROR
                                                    [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
                                                    (default task-23)
                                                    [6410419]
                                                    Correlation ID:
                                                    null, Call Stack:
                                                    null, Custom Event
                                                    ID: -1, Message:
                                                    VDSM sjcvhost03
                                                    command failed:
                                                    Error creating a
                                                    storage domain's
                                                    metadata: ("create
                                                    meta file 'outbox'
                                                    failed: [Errno 5]
                                                    Input/output
                                                    error",)</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,744 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand]




                                                    (default task-23)
                                                    [6410419] Command
                                                    'org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand'




                                                    return value
                                                    'StatusOnlyReturnForXmlRpc
                                                    [status=StatusForXmlRpc

                                                    [code=362,
                                                    message=Error
                                                    creating a storage
                                                    domain's metadata:
                                                    ("create meta file
                                                    'outbox' failed:
                                                    [Errno 5]
                                                    Input/output
                                                    error",)]]'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,744 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand]




                                                    (default task-23)
                                                    [6410419] HostName =
                                                    sjcvhost03</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,745 ERROR
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand]




                                                    (default task-23)
                                                    [6410419] Command
                                                    'CreateStorageDomainVDSCommand(HostName
                                                    = sjcvhost03,
                                                    CreateStorageDomainVDSCommandParameters:{runAsync='true',
                                                    hostId='80245ac2-32a3-4d5d-b0fe-08019e2d1c9c',



                                                    storageDomain='StorageDomainStatic:{name='sjcvmstore',



                                                    id='597d5b5b-7c09-4de9-8840-6993bd9b61a6'}',



                                                    args='sjcstorage01:/vmstore'})'


                                                    execution failed:
                                                    VDSGenericException:
                                                    VDSErrorException:
                                                    Failed in vdscommand
                                                    to
                                                    CreateStorageDomainVDS,
                                                    error = Error
                                                    creating a storage
                                                    domain's metadata:
                                                    ("create meta file
                                                    'outbox' failed:
                                                    [Errno 5]
                                                    Input/output
                                                    error",)</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,745 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand]




                                                    (default task-23)
                                                    [6410419] FINISH,
                                                    CreateStorageDomainVDSCommand,
                                                    log id: b9fe587</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,745 ERROR
                                                    [org.ovirt.engine.core.bll.storage.AddGlusterFsStorageDomainCommand]
                                                    (default task-23)
                                                    [6410419] Command
                                                    'org.ovirt.engine.core.bll.storage.AddGlusterFsStorageDomainCommand'
                                                    failed:
                                                    EngineException:
                                                    org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException:
                                                    VDSGenericException:
                                                    VDSErrorException:
                                                    Failed in vdscommand
                                                    to
                                                    CreateStorageDomainVDS,
                                                    error = Error
                                                    creating a storage
                                                    domain's metadata:
                                                    ("create meta file
                                                    'outbox' failed:
                                                    [Errno 5]
                                                    Input/output
                                                    error",) (Failed
                                                    with error
                                                    StorageDomainMetadataCreationError
                                                    and code 362)</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,755 INFO 
                                                    [org.ovirt.engine.core.bll.storage.AddGlusterFsStorageDomainCommand]
                                                    (default task-23)
                                                    [6410419] Command
                                                    [id=5ae15f53-69a1-47c5-b3a5-82f32c20e48f]:
                                                    Compensating
                                                    NEW_ENTITY_ID of
                                                    org.ovirt.engine.core.common.businessentities.StorageDomainDynamic;
                                                    snapshot:
                                                    597d5b5b-7c09-4de9-8840-6993bd9b61a6.</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,758 INFO 
                                                    [org.ovirt.engine.core.bll.storage.AddGlusterFsStorageDomainCommand]
                                                    (default task-23)
                                                    [6410419] Command
                                                    [id=5ae15f53-69a1-47c5-b3a5-82f32c20e48f]:
                                                    Compensating
                                                    NEW_ENTITY_ID of
                                                    org.ovirt.engine.core.common.businessentities.StorageDomainStatic;
                                                    snapshot:
                                                    597d5b5b-7c09-4de9-8840-6993bd9b61a6.</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,769 ERROR
                                                    [org.ovirt.engine.core.bll.storage.AddGlusterFsStorageDomainCommand]
                                                    (default task-23)
                                                    [6410419]
                                                    Transaction
                                                    rolled-back for
                                                    command
                                                    'org.ovirt.engine.core.bll.storage.AddGlusterFsStorageDomainCommand'.</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,784 ERROR
                                                    [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
                                                    (default task-23)
                                                    [6410419]
                                                    Correlation ID:
                                                    6410419, Job ID:
                                                    78692780-a06f-49a5-b6b1-e6c24a820d62,
                                                    Call Stack: null,
                                                    Custom Event ID: -1,
                                                    Message: Failed to
                                                    add Storage Domain
                                                    sjcvmstore. (User:
                                                    admin@internal)</span></p>
                                                <p><span>2015-09-22
                                                    18:46:48,996 INFO 
                                                    [org.ovirt.engine.core.bll.storage.RemoveStorageServerConnectionCommand]
                                                    (default task-32)
                                                    [1635a244] Lock
                                                    Acquired to object
                                                    'EngineLock:{exclusiveLocks='[ec5ab31e-b5b9-4a8e-a2b2-0876df71a21e=&lt;STORAGE_CONNECTION,




                                                    ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;,


                                                    sjcstorage01:/vmstore=&lt;STORAGE_CONNECTION,



                                                    ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;]',



                                                    sharedLocks='null'}'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,018 INFO 
                                                    [org.ovirt.engine.core.bll.storage.RemoveStorageServerConnectionCommand]
                                                    (default task-32)
                                                    [1635a244] Running
                                                    command:
                                                    RemoveStorageServerConnectionCommand
                                                    internal: false.
                                                    Entities affected : 
                                                    ID:
                                                    aaa00000-0000-0000-0000-123456789aaa
                                                    Type: SystemAction
                                                    group
                                                    CREATE_STORAGE_DOMAIN
                                                    with role type ADMIN</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,024 INFO 
                                                    [org.ovirt.engine.core.bll.storage.RemoveStorageServerConnectionCommand]
                                                    (default task-32)
                                                    [1635a244] Removing
                                                    connection
                                                    'ec5ab31e-b5b9-4a8e-a2b2-0876df71a21e'
                                                    from database </span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,026 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand]




                                                    (default task-32)
                                                    [1635a244] START,
                                                    DisconnectStorageServerVDSCommand(HostName
                                                    = sjcvhost03,
                                                    StorageServerConnectionManagementVDSParameters:{runAsync='true',
                                                    hostId='80245ac2-32a3-4d5d-b0fe-08019e2d1c9c',



                                                    storagePoolId='00000000-0000-0000-0000-000000000000',



                                                    storageType='GLUSTERFS',


                                                    connectionList='[StorageServerConnections:{id='ec5ab31e-b5b9-4a8e-a2b2-0876df71a21e',




                                                    connection='sjcstorage01:/vmstore',


                                                    iqn='null',
                                                    vfsType='glusterfs',
                                                    mountOptions='null',
                                                    nfsVersion='null',
                                                    nfsRetrans='null',
                                                    nfsTimeo='null',
                                                    iface='null',
                                                    netIfaceName='null'}]'}),
                                                    log id: 39d3b568</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,248 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand]




                                                    (default task-32)
                                                    [1635a244] FINISH,
                                                    DisconnectStorageServerVDSCommand,
                                                    return:
                                                    {ec5ab31e-b5b9-4a8e-a2b2-0876df71a21e=0},
                                                    log id: 39d3b568</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,252 INFO 
                                                    [org.ovirt.engine.core.bll.storage.RemoveStorageServerConnectionCommand]
                                                    (default task-32)
                                                    [1635a244] Lock
                                                    freed to object
                                                    'EngineLock:{exclusiveLocks='[ec5ab31e-b5b9-4a8e-a2b2-0876df71a21e=&lt;STORAGE_CONNECTION,




                                                    ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;,


                                                    sjcstorage01:/vmstore=&lt;STORAGE_CONNECTION,



                                                    ACTION_TYPE_FAILED_OBJECT_LOCKED&gt;]',



                                                    sharedLocks='null'}'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,431 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
                                                    (DefaultQuartzScheduler_Worker-3)

                                                    [] START,
                                                    GlusterVolumesListVDSCommand(HostName
                                                    = sjcstorage01,
                                                    GlusterVolumesListVDSParameters:{runAsync='true',
                                                    hostId='c75682ba-1e4c-42a3-85c7-16e4bb2ce5da'}),



                                                    log id: 17014ae8</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,511 WARN 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]




                                                    (DefaultQuartzScheduler_Worker-3)

                                                    [] Could not
                                                    associate brick
                                                    'sjcstorage01:/export/vmstore/brick01'
                                                    of volume
                                                    '030f270a-0999-4df4-9b14-ae56eb0a2fb9'
                                                    with correct network
                                                    as no gluster
                                                    network found in
                                                    cluster
                                                    'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,515 WARN 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]




                                                    (DefaultQuartzScheduler_Worker-3)

                                                    [] Could not
                                                    associate brick
                                                    'sjcstorage02:/export/vmstore/brick01'
                                                    of volume
                                                    '030f270a-0999-4df4-9b14-ae56eb0a2fb9'
                                                    with correct network
                                                    as no gluster
                                                    network found in
                                                    cluster
                                                    'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,516 WARN 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]




                                                    (DefaultQuartzScheduler_Worker-3)

                                                    [] Could not add
                                                    brick
                                                    'sjcvhost02:/export/vmstore/brick01'
                                                    to volume
                                                    '030f270a-0999-4df4-9b14-ae56eb0a2fb9'
                                                    - server uuid
                                                    '29b58278-9aa3-47c5-bfb4-1948ef7fdbba'
                                                    not found in cluster
'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                <p><span>2015-09-22
                                                    18:46:49,516 INFO 
                                                    [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
                                                    (DefaultQuartzScheduler_Worker-3)

                                                    [] FINISH,
                                                    GlusterVolumesListVDSCommand,
                                                    return:
                                                    {030f270a-0999-4df4-9b14-ae56eb0a2fb9=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@92ed0f75},




                                                    log id: 17014ae8</span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p><span>ovirt engine
                                                    thinks that
                                                    sjcstorage01 is
                                                    sjcstorage01, its
                                                    all testbed at the
                                                    moment and is all
                                                    short names, defined
                                                    in /etc/hosts (all
                                                    copied to each
                                                    server for
                                                    consistancy)</span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p><span>volume info for
                                                    vmstore is</span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p><span>Status of
                                                    volume: vmstore</span></p>
                                                <p><span>Gluster process
                                                                       
                                                            TCP Port 
                                                    RDMA Port  Online 
                                                    Pid</span></p>
                                                <p><span>------------------------------------------------------------------------------</span></p>
                                                <p><span>Brick
                                                    sjcstorage01:/export/vmstore/brick01 
                                                    49157     0         
                                                    Y       7444 </span></p>
                                                <p><span>Brick
                                                    sjcstorage02:/export/vmstore/brick01 
                                                    49157     0         
                                                    Y       4063 </span></p>
                                                <p><span>Brick
                                                    sjcvhost02:/export/vmstore/brick01 
                                                      49156     0       
                                                      Y       3243 </span></p>
                                                <p><span>NFS Server on
                                                    localhost          
                                                              2049     
                                                    0          Y      
                                                    3268 </span></p>
                                                <p><span>Self-heal
                                                    Daemon on localhost
                                                                  N/A  
                                                        N/A        Y    
                                                      3284 </span></p>
                                                <p><span>NFS Server on
                                                    sjcstorage01       
                                                              2049     
                                                    0          Y      
                                                    7463 </span></p>
                                                <p><span>Self-heal
                                                    Daemon on
                                                    sjcstorage01       
                                                        N/A       N/A   
                                                        Y       7472 </span></p>
                                                <p><span>NFS Server on
                                                    sjcstorage02       
                                                              2049     
                                                    0          Y      
                                                    4082 </span></p>
                                                <p><span>Self-heal
                                                    Daemon on
                                                    sjcstorage02       
                                                        N/A       N/A   
                                                        Y       4090 </span></p>
                                                <p><span> </span></p>
                                                <p><span>Task Status of
                                                    Volume vmstore</span></p>
                                                <p><span>------------------------------------------------------------------------------</span></p>
                                                <p> </p>
                                                <p><span>There are no
                                                    active volume tasks</span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p><span>vdsm logs from
                                                    time the domain is
                                                    added</span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p>hread-789::DEBUG::2015-09-22


                                                  19:12:05,865::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-790::DEBUG::2015-09-22


                                                  19:12:07,797::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`93731f26-a48f-45c9-9959-42c96b09cf85`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-790::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:07,797::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats(options=None)</p>
                                                <p>Thread-790::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:07,797::logUtils::51::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats, Return
                                                  response: {}</p>
                                                <p>Thread-790::DEBUG::2015-09-22


                                                  19:12:07,797::task::1191::Storage.TaskManager.Task::(prepare)


                                                  Task=`93731f26-a48f-45c9-9959-42c96b09cf85`::finished:



                                                  {}</p>
                                                <p>Thread-790::DEBUG::2015-09-22


                                                  19:12:07,797::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`93731f26-a48f-45c9-9959-42c96b09cf85`::moving



                                                  from state preparing
                                                  -&gt; state finished</p>
                                                <p>Thread-790::DEBUG::2015-09-22


                                                  19:12:07,797::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-790::DEBUG::2015-09-22


                                                  19:12:07,797::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-790::DEBUG::2015-09-22


                                                  19:12:07,797::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`93731f26-a48f-45c9-9959-42c96b09cf85`::ref


                                                  0 aborting False</p>
                                                <p>Thread-790::DEBUG::2015-09-22


                                                  19:12:07,802::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:14,816::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept)




                                                  Accepting connection
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52510" target="_blank">127.0.0.1:52510</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:12:14,822::protocoldetector::82::ProtocolDetector.Detector::(__init__)




                                                  Using required_size=11</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:14,823::protocoldetector::118::ProtocolDetector.Detector::(handle_read)




                                                  Detected protocol xml
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52510" target="_blank">127.0.0.1:52510</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:12:14,823::bindingxmlrpc::1297::XmlDetector::(handle_socket)


                                                  xml over http detected
                                                  from ('127.0.0.1',
                                                  52510)</p>
                                                <p>BindingXMLRPC::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:14,823::xmlrpc::73::vds.XMLRPCServer::(handle_request)




                                                  Starting request
                                                  handler for <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52510" target="_blank">127.0.0.1:52510</a></p>
                                                <p>Thread-791::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:14,823::xmlrpc::84::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52510" target="_blank">127.0.0.1:52510</a>
                                                  started</p>
                                                <p>Thread-791::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:14,825::xmlrpc::92::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52510" target="_blank">127.0.0.1:52510</a>
                                                  stopped</p>
                                                <p>Thread-792::DEBUG::2015-09-22


                                                  19:12:20,872::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-793::DEBUG::2015-09-22


                                                  19:12:22,832::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`a1f48f6f-a9ba-4dac-b024-ae6289f4a7dd`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-793::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:22,832::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats(options=None)</p>
                                                <p>Thread-793::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:22,832::logUtils::51::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats, Return
                                                  response: {}</p>
                                                <p>Thread-793::DEBUG::2015-09-22


                                                  19:12:22,832::task::1191::Storage.TaskManager.Task::(prepare)


                                                  Task=`a1f48f6f-a9ba-4dac-b024-ae6289f4a7dd`::finished:



                                                  {}</p>
                                                <p>Thread-793::DEBUG::2015-09-22


                                                  19:12:22,832::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`a1f48f6f-a9ba-4dac-b024-ae6289f4a7dd`::moving



                                                  from state preparing
                                                  -&gt; state finished</p>
                                                <p>Thread-793::DEBUG::2015-09-22


                                                  19:12:22,833::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-793::DEBUG::2015-09-22


                                                  19:12:22,833::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-793::DEBUG::2015-09-22


                                                  19:12:22,833::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`a1f48f6f-a9ba-4dac-b024-ae6289f4a7dd`::ref


                                                  0 aborting False</p>
                                                <p>Thread-793::DEBUG::2015-09-22


                                                  19:12:22,837::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:29,841::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept)




                                                  Accepting connection
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52511" target="_blank">127.0.0.1:52511</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:12:29,848::protocoldetector::82::ProtocolDetector.Detector::(__init__)




                                                  Using required_size=11</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:29,849::protocoldetector::118::ProtocolDetector.Detector::(handle_read)




                                                  Detected protocol xml
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52511" target="_blank">127.0.0.1:52511</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:12:29,849::bindingxmlrpc::1297::XmlDetector::(handle_socket)


                                                  xml over http detected
                                                  from ('127.0.0.1',
                                                  52511)</p>
                                                <p>BindingXMLRPC::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:29,849::xmlrpc::73::vds.XMLRPCServer::(handle_request)




                                                  Starting request
                                                  handler for <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52511" target="_blank">127.0.0.1:52511</a></p>
                                                <p>Thread-794::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:29,849::xmlrpc::84::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52511" target="_blank">127.0.0.1:52511</a>
                                                  started</p>
                                                <p>Thread-794::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:29,851::xmlrpc::92::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52511" target="_blank">127.0.0.1:52511</a>
                                                  stopped</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,520::__init__::503::jsonrpc.JsonRpcServer::(_serveRequest)


                                                  Calling
                                                  'StoragePool.connectStorageServer'
                                                  in bridge with
                                                  {u'connectionParams':
                                                  [{u'id':
                                                  u'00000000-0000-0000-0000-000000000000',
                                                  u'connection':
                                                  u'sjcstorage01:/vmstore',
                                                  u'iqn': u'', u'user':
                                                  u'', u'tpgt': u'1',
                                                  u'vfs_type':
                                                  u'glusterfs',
                                                  u'password':
                                                  '********', u'port':
                                                  u''}],
                                                  u'storagepoolID':
                                                  u'00000000-0000-0000-0000-000000000000',
                                                  u'domainType': 7}</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,520::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`6e8aec06-556f-4659-9ee8-efc60b637ff6`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-795::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:35,521::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  connectStorageServer(domType=7,
                                                  spUUID=u'00000000-0000-0000-0000-000000000000',


                                                  conList=[{u'id':
                                                  u'00000000-0000-0000-0000-000000000000',
                                                  u'connection':
                                                  u'sjcstorage01:/vmstore',
                                                  u'iqn': u'', u'user':
                                                  u'', u'tpgt': u'1',
                                                  u'vfs_type':
                                                  u'glusterfs',
                                                  u'password':
                                                  '********', u'port':
                                                  u''}], options=None)</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,539::fileUtils::143::Storage.fileUtils::(createdir)


                                                  Creating directory:
                                                  /rhev/data-center/mnt/glusterSD/sjcstorage01:_vmstore
                                                  mode: None</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,540::mount::229::Storage.Misc.excCmd::(_runcmd)


                                                  /usr/bin/sudo -n
                                                  /usr/bin/systemd-run
                                                  --scope
                                                  --slice=vdsm-glusterfs
                                                  /usr/bin/mount -t
                                                  glusterfs -o
                                                  backup-volfile-servers=sjcstorage02:sjcvhost02
                                                  sjcstorage01:/vmstore
                                                  /rhev/data-center/mnt/glusterSD/sjcstorage01:_vmstore

                                                  (cwd None)</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,706::hsm::2417::Storage.HSM::(__prefetchDomains)


                                                  glusterDomPath:
                                                  glusterSD/*</p>
                                                <p>Thread-796::DEBUG::2015-09-22


                                                  19:12:35,707::__init__::298::IOProcessClient::(_run)


                                                  Starting IOProcess...</p>
                                                <p>Thread-797::DEBUG::2015-09-22


                                                  19:12:35,712::__init__::298::IOProcessClient::(_run)


                                                  Starting IOProcess...</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,721::hsm::2429::Storage.HSM::(__prefetchDomains)


                                                  Found SD uuids: ()</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,721::hsm::2489::Storage.HSM::(connectStorageServer)


                                                  knownSDs:
                                                  {41b75ca9-9783-42a7-9a23-10a2ae3cbb96:
                                                  storage.glusterSD.findDomain,


                                                  597d5b5b-7c09-4de9-8840-6993bd9b61a6:


                                                  storage.glusterSD.findDomain,


ef17fec4-fecf-4d7e-b815-d1db4ef65225:
storage.glusterSD.findDomain}</p>
                                                <p>Thread-795::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:35,721::logUtils::51::dispatcher::(wrapper)



                                                  Run and protect:
                                                  connectStorageServer,
                                                  Return response:
                                                  {'statuslist':
                                                  [{'status': 0, 'id':
                                                  u'00000000-0000-0000-0000-000000000000'}]}</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,722::task::1191::Storage.TaskManager.Task::(prepare)


                                                  Task=`6e8aec06-556f-4659-9ee8-efc60b637ff6`::finished:




                                                  {'statuslist':
                                                  [{'status': 0, 'id':
                                                  u'00000000-0000-0000-0000-000000000000'}]}</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,722::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`6e8aec06-556f-4659-9ee8-efc60b637ff6`::moving



                                                  from state preparing
                                                  -&gt; state finished</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,722::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,722::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,722::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`6e8aec06-556f-4659-9ee8-efc60b637ff6`::ref


                                                  0 aborting False</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,722::__init__::533::jsonrpc.JsonRpcServer::(_serveRequest)


                                                  Return
                                                  'StoragePool.connectStorageServer'
                                                  in bridge with
                                                  [{'status': 0, 'id':
                                                  u'00000000-0000-0000-0000-000000000000'}]</p>
                                                <p>Thread-795::DEBUG::2015-09-22


                                                  19:12:35,722::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,775::__init__::503::jsonrpc.JsonRpcServer::(_serveRequest)


                                                  Calling
                                                  'StoragePool.connectStorageServer'
                                                  in bridge with
                                                  {u'connectionParams':
                                                  [{u'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054',
                                                  u'connection':
                                                  u'sjcstorage01:/vmstore',
                                                  u'iqn': u'', u'user':
                                                  u'', u'tpgt': u'1',
                                                  u'vfs_type':
                                                  u'glusterfs',
                                                  u'password':
                                                  '********', u'port':
                                                  u''}],
                                                  u'storagepoolID':
                                                  u'00000000-0000-0000-0000-000000000000',
                                                  u'domainType': 7}</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,775::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`b2c91515-bdda-45e5-a031-61a1e2c53c4d`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-798::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:35,776::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  connectStorageServer(domType=7,
                                                  spUUID=u'00000000-0000-0000-0000-000000000000',


                                                  conList=[{u'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054',
                                                  u'connection':
                                                  u'sjcstorage01:/vmstore',
                                                  u'iqn': u'', u'user':
                                                  u'', u'tpgt': u'1',
                                                  u'vfs_type':
                                                  u'glusterfs',
                                                  u'password':
                                                  '********', u'port':
                                                  u''}], options=None)</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,777::hsm::2417::Storage.HSM::(__prefetchDomains)


                                                  glusterDomPath:
                                                  glusterSD/*</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,782::hsm::2429::Storage.HSM::(__prefetchDomains)


                                                  Found SD uuids: ()</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,782::hsm::2489::Storage.HSM::(connectStorageServer)


                                                  knownSDs:
                                                  {41b75ca9-9783-42a7-9a23-10a2ae3cbb96:
                                                  storage.glusterSD.findDomain,


                                                  597d5b5b-7c09-4de9-8840-6993bd9b61a6:


                                                  storage.glusterSD.findDomain,


ef17fec4-fecf-4d7e-b815-d1db4ef65225:
storage.glusterSD.findDomain}</p>
                                                <p>Thread-798::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:35,782::logUtils::51::dispatcher::(wrapper)



                                                  Run and protect:
                                                  connectStorageServer,
                                                  Return response:
                                                  {'statuslist':
                                                  [{'status': 0, 'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054'}]}</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,783::task::1191::Storage.TaskManager.Task::(prepare)


                                                  Task=`b2c91515-bdda-45e5-a031-61a1e2c53c4d`::finished:




                                                  {'statuslist':
                                                  [{'status': 0, 'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054'}]}</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,783::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`b2c91515-bdda-45e5-a031-61a1e2c53c4d`::moving



                                                  from state preparing
                                                  -&gt; state finished</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,783::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,783::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,783::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`b2c91515-bdda-45e5-a031-61a1e2c53c4d`::ref


                                                  0 aborting False</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,783::__init__::533::jsonrpc.JsonRpcServer::(_serveRequest)


                                                  Return
                                                  'StoragePool.connectStorageServer'
                                                  in bridge with
                                                  [{'status': 0, 'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054'}]</p>
                                                <p>Thread-798::DEBUG::2015-09-22


                                                  19:12:35,783::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,787::__init__::503::jsonrpc.JsonRpcServer::(_serveRequest)


                                                  Calling
                                                  'StorageDomain.create'
                                                  in bridge with
                                                  {u'name':
                                                  u'sjcvmstore01',
                                                  u'domainType': 7,
                                                  u'domainClass': 1,
                                                  u'typeArgs':
                                                  u'sjcstorage01:/vmstore',
                                                  u'version': u'3',
                                                  u'storagedomainID':
                                                  u'c02fda97-62e3-40d3-9a6e-ac5d100f8ad3'}</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,788::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-801::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:35,788::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  createStorageDomain(storageType=7,
                                                  sdUUID=u'c02fda97-62e3-40d3-9a6e-ac5d100f8ad3',


                                                  domainName=u'sjcvmstore01',


                                                  typeSpecificArg=u'sjcstorage01:/vmstore',


                                                  domClass=1,
                                                  domVersion=u'3',
                                                  options=None)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,788::misc::733::Storage.SamplingMethod::(__call__)


                                                  Trying to enter
                                                  sampling method
                                                  (storage.sdc.refreshStorage)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,788::misc::736::Storage.SamplingMethod::(__call__)


                                                  Got in to sampling
                                                  method</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,788::misc::733::Storage.SamplingMethod::(__call__)


                                                  Trying to enter
                                                  sampling method
                                                  (storage.iscsi.rescan)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,788::misc::736::Storage.SamplingMethod::(__call__)


                                                  Got in to sampling
                                                  method</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,788::iscsi::431::Storage.ISCSI::(rescan)


                                                  Performing SCSI scan,
                                                  this will take up to
                                                  30 seconds</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,788::iscsiadm::97::Storage.Misc.excCmd::(_runCmd)


                                                  /usr/bin/sudo -n
                                                  /sbin/iscsiadm -m
                                                  session -R (cwd None)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,821::misc::743::Storage.SamplingMethod::(__call__)


                                                  Returning last result</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,821::misc::733::Storage.SamplingMethod::(__call__)


                                                  Trying to enter
                                                  sampling method
                                                  (storage.hba.rescan)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,821::misc::736::Storage.SamplingMethod::(__call__)


                                                  Got in to sampling
                                                  method</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,821::hba::56::Storage.HBA::(rescan)


                                                  Starting scan</p>
                                                <p>Thread-802::DEBUG::2015-09-22


                                                  19:12:35,882::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,912::hba::62::Storage.HBA::(rescan)


                                                  Scan finished</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,912::misc::743::Storage.SamplingMethod::(__call__)


                                                  Returning last result</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,912::multipath::77::Storage.Misc.excCmd::(rescan)


                                                  /usr/bin/sudo -n
                                                  /usr/sbin/multipath
                                                  (cwd None)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,936::multipath::77::Storage.Misc.excCmd::(rescan)


                                                  SUCCESS: &lt;err&gt; =
                                                  ''; &lt;rc&gt; = 0</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,936::utils::661::root::(execCmd)


                                                  /sbin/udevadm settle
                                                  --timeout=5 (cwd None)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,946::utils::679::root::(execCmd)


                                                  SUCCESS: &lt;err&gt; =
                                                  ''; &lt;rc&gt; = 0</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,947::lvm::498::Storage.OperationMutex::(_invalidateAllPvs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  got the operation
                                                  mutex</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,947::lvm::500::Storage.OperationMutex::(_invalidateAllPvs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  released the operation
                                                  mutex</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,947::lvm::509::Storage.OperationMutex::(_invalidateAllVgs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  got the operation
                                                  mutex</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,948::lvm::511::Storage.OperationMutex::(_invalidateAllVgs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  released the operation
                                                  mutex</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,948::lvm::529::Storage.OperationMutex::(_invalidateAllLvs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  got the operation
                                                  mutex</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,948::lvm::531::Storage.OperationMutex::(_invalidateAllLvs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  released the operation
                                                  mutex</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,948::misc::743::Storage.SamplingMethod::(__call__)


                                                  Returning last result</p>
                                                <p>Thread-801::ERROR::2015-09-22


                                                  19:12:35,949::sdc::138::Storage.StorageDomainCache::(_findDomain)


                                                  looking for unfetched
                                                  domain
                                                  c02fda97-62e3-40d3-9a6e-ac5d100f8ad3</p>
                                                <p>Thread-801::ERROR::2015-09-22


                                                  19:12:35,949::sdc::155::Storage.StorageDomainCache::(_findUnfetchedDomain)




                                                  looking for domain
                                                  c02fda97-62e3-40d3-9a6e-ac5d100f8ad3</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,949::lvm::371::Storage.OperationMutex::(_reloadvgs)


                                                  Operation 'lvm reload
                                                  operation' got the
                                                  operation mutex</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,950::lvm::291::Storage.Misc.excCmd::(cmd)


                                                  /usr/bin/sudo -n
                                                  /usr/sbin/lvm vgs
                                                  --config ' devices {
                                                  preferred_names =
                                                  ["^/dev/mapper/"]
                                                  ignore_suspended_devices=1
                                                  write_cache_state=0
                                                  disable_after_error_count=3
                                                  obtain_device_list_from_udev=0

                                                  filter = [
                                                  '\''r|.*|'\'' ] }
                                                   global {
                                                   locking_type=1
                                                   prioritise_write_locks=1
                                                   wait_for_locks=1
                                                   use_lvmetad=0 }
                                                   backup {  retain_min
                                                  = 50  retain_days = 0
                                                  } ' --noheadings
                                                  --units b --nosuffix
                                                  --separator '|'
                                                  --ignoreskippedcluster
                                                  -o
                                                  uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name




                                                  c02fda97-62e3-40d3-9a6e-ac5d100f8ad3


                                                  (cwd None)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,985::lvm::291::Storage.Misc.excCmd::(cmd)


                                                  FAILED: &lt;err&gt; =
                                                  '  WARNING: lvmetad is
                                                  running but disabled.
                                                  Restart lvmetad before
                                                  enabling it!\n  Volume
                                                  group
                                                  "c02fda97-62e3-40d3-9a6e-ac5d100f8ad3"
                                                  not found\n  Cannot
                                                  process volume group
                                                  c02fda97-62e3-40d3-9a6e-ac5d100f8ad3\n';
                                                  &lt;rc&gt; = 5</p>
                                                <p>Thread-801::WARNING::2015-09-22


                                                  19:12:35,986::lvm::376::Storage.LVM::(_reloadvgs)


                                                  lvm vgs failed: 5 []
                                                  ['  WARNING: lvmetad
                                                  is running but
                                                  disabled. Restart
                                                  lvmetad before
                                                  enabling it!', '
                                                   Volume group
                                                  "c02fda97-62e3-40d3-9a6e-ac5d100f8ad3"
                                                  not found', '  Cannot
                                                  process volume group
                                                  c02fda97-62e3-40d3-9a6e-ac5d100f8ad3']</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:35,987::lvm::416::Storage.OperationMutex::(_reloadvgs)


                                                  Operation 'lvm reload
                                                  operation' released
                                                  the operation mutex</p>
                                                <p>Thread-801::ERROR::2015-09-22


                                                  19:12:35,997::sdc::144::Storage.StorageDomainCache::(_findDomain)


                                                  domain
                                                  c02fda97-62e3-40d3-9a6e-ac5d100f8ad3
                                                  not found</p>
                                                <p>Traceback (most
                                                  recent call last):</p>
                                                <p>  File
                                                  "/usr/share/vdsm/storage/sdc.py",
                                                  line 142, in
                                                  _findDomain</p>
                                                <p>    dom =
                                                  findMethod(sdUUID)</p>
                                                <p>  File
                                                  "/usr/share/vdsm/storage/sdc.py",
                                                  line 172, in
                                                  _findUnfetchedDomain</p>
                                                <p>    raise
                                                  se.StorageDomainDoesNotExist(sdUUID)</p>
                                                <p>StorageDomainDoesNotExist:

                                                  Storage domain does
                                                  not exist:
                                                  (u'c02fda97-62e3-40d3-9a6e-ac5d100f8ad3',)</p>
                                                <p>Thread-801::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:35,998::nfsSD::69::Storage.StorageDomain::(create)




                                                  sdUUID=c02fda97-62e3-40d3-9a6e-ac5d100f8ad3



                                                  domainName=sjcvmstore01

                                                  remotePath=sjcstorage01:/vmstore


                                                  domClass=1</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,015::__init__::298::IOProcessClient::(_run)


                                                  Starting IOProcess...</p>
                                                <p>Thread-801::ERROR::2015-09-22


                                                  19:12:36,067::task::866::Storage.TaskManager.Task::(_setError)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::Unexpected




                                                  error</p>
                                                <p>Traceback (most
                                                  recent call last):</p>
                                                <p>  File
                                                  "/usr/share/vdsm/storage/task.py",
                                                  line 873, in _run</p>
                                                <p>    return fn(*args,
                                                  **kargs)</p>
                                                <p>  File
                                                  "/usr/share/vdsm/logUtils.py",
                                                  line 49, in wrapper</p>
                                                <p>    res = f(*args,
                                                  **kwargs)</p>
                                                <p>  File
                                                  "/usr/share/vdsm/storage/hsm.py",
                                                  line 2697, in
                                                  createStorageDomain</p>
                                                <p>    domVersion)</p>
                                                <p>  File
                                                  "/usr/share/vdsm/storage/nfsSD.py",
                                                  line 84, in create</p>
                                                <p>    remotePath,
                                                  storageType, version)</p>
                                                <p>  File
                                                  "/usr/share/vdsm/storage/fileSD.py",
                                                  line 264, in
                                                  _prepareMetadata</p>
                                                <p>    "create meta file
                                                  '%s' failed: %s" %
                                                  (metaFile, str(e)))</p>
                                                <p>StorageDomainMetadataCreationError:


                                                  Error creating a
                                                  storage domain's
                                                  metadata: ("create
                                                  meta file 'outbox'
                                                  failed: [Errno 5]
                                                  Input/output error",)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,067::task::885::Storage.TaskManager.Task::(_run)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::Task._run:




                                                  d2d29352-8677-45cb-a4ab-06aa32cf1acb

                                                  (7,
                                                  u'c02fda97-62e3-40d3-9a6e-ac5d100f8ad3',
                                                  u'sjcvmstore01',
                                                  u'sjcstorage01:/vmstore',
                                                  1, u'3') {} failed -
                                                  stopping task</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,067::task::1246::Storage.TaskManager.Task::(stop)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::stopping



                                                  in state preparing
                                                  (force False)</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,067::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::ref


                                                  1 aborting True</p>
                                                <p>Thread-801::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:36,067::task::1171::Storage.TaskManager.Task::(prepare)




                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::aborting:



                                                  Task is aborted:
                                                  "Error creating a
                                                  storage domain's
                                                  metadata" - code 362</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::task::1176::Storage.TaskManager.Task::(prepare)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::Prepare:




                                                  aborted: Error
                                                  creating a storage
                                                  domain's metadata</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::ref


                                                  0 aborting True</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::task::928::Storage.TaskManager.Task::(_doAbort)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::Task._doAbort:




                                                  force False</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::moving



                                                  from state preparing
                                                  -&gt; state aborting</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::task::550::Storage.TaskManager.Task::(__state_aborting)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::_aborting:




                                                  recover policy none</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`d2d29352-8677-45cb-a4ab-06aa32cf1acb`::moving



                                                  from state aborting
                                                  -&gt; state failed</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,068::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-801::ERROR::2015-09-22


                                                  19:12:36,068::dispatcher::76::Storage.Dispatcher::(wrapper)


                                                  {'status': {'message':
                                                  'Error creating a
                                                  storage domain\'s
                                                  metadata: ("create
                                                  meta file \'outbox\'
                                                  failed: [Errno 5]
                                                  Input/output
                                                  error",)', 'code':
                                                  362}}</p>
                                                <p>Thread-801::DEBUG::2015-09-22


                                                  19:12:36,069::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,180::__init__::503::jsonrpc.JsonRpcServer::(_serveRequest)


                                                  Calling
                                                  'StoragePool.disconnectStorageServer'
                                                  in bridge with
                                                  {u'connectionParams':
                                                  [{u'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054',
                                                  u'connection':
                                                  u'sjcstorage01:/vmstore',
                                                  u'iqn': u'', u'user':
                                                  u'', u'tpgt': u'1',
                                                  u'vfs_type':
                                                  u'glusterfs',
                                                  u'password':
                                                  '********', u'port':
                                                  u''}],
                                                  u'storagepoolID':
                                                  u'00000000-0000-0000-0000-000000000000',
                                                  u'domainType': 7}</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,181::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`01af6594-9c7b-4ec7-b08f-02627db8f421`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-807::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:36,182::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  disconnectStorageServer(domType=7,
                                                  spUUID=u'00000000-0000-0000-0000-000000000000',


                                                  conList=[{u'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054',
                                                  u'connection':
                                                  u'sjcstorage01:/vmstore',
                                                  u'iqn': u'', u'user':
                                                  u'', u'tpgt': u'1',
                                                  u'vfs_type':
                                                  u'glusterfs',
                                                  u'password':
                                                  '********', u'port':
                                                  u''}], options=None)</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,182::mount::229::Storage.Misc.excCmd::(_runcmd)


                                                  /usr/bin/sudo -n
                                                  /usr/bin/umount -f -l
                                                  /rhev/data-center/mnt/glusterSD/sjcstorage01:_vmstore

                                                  (cwd None)</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,222::misc::733::Storage.SamplingMethod::(__call__)


                                                  Trying to enter
                                                  sampling method
                                                  (storage.sdc.refreshStorage)</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,222::misc::736::Storage.SamplingMethod::(__call__)


                                                  Got in to sampling
                                                  method</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,222::misc::733::Storage.SamplingMethod::(__call__)


                                                  Trying to enter
                                                  sampling method
                                                  (storage.iscsi.rescan)</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,222::misc::736::Storage.SamplingMethod::(__call__)


                                                  Got in to sampling
                                                  method</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,223::iscsi::431::Storage.ISCSI::(rescan)


                                                  Performing SCSI scan,
                                                  this will take up to
                                                  30 seconds</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,223::iscsiadm::97::Storage.Misc.excCmd::(_runCmd)


                                                  /usr/bin/sudo -n
                                                  /sbin/iscsiadm -m
                                                  session -R (cwd None)</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,258::misc::743::Storage.SamplingMethod::(__call__)


                                                  Returning last result</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,258::misc::733::Storage.SamplingMethod::(__call__)


                                                  Trying to enter
                                                  sampling method
                                                  (storage.hba.rescan)</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,258::misc::736::Storage.SamplingMethod::(__call__)


                                                  Got in to sampling
                                                  method</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,258::hba::56::Storage.HBA::(rescan)


                                                  Starting scan</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,350::hba::62::Storage.HBA::(rescan)


                                                  Scan finished</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,350::misc::743::Storage.SamplingMethod::(__call__)


                                                  Returning last result</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,350::multipath::77::Storage.Misc.excCmd::(rescan)


                                                  /usr/bin/sudo -n
                                                  /usr/sbin/multipath
                                                  (cwd None)</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,374::multipath::77::Storage.Misc.excCmd::(rescan)


                                                  SUCCESS: &lt;err&gt; =
                                                  ''; &lt;rc&gt; = 0</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,374::utils::661::root::(execCmd)


                                                  /sbin/udevadm settle
                                                  --timeout=5 (cwd None)</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,383::utils::679::root::(execCmd)


                                                  SUCCESS: &lt;err&gt; =
                                                  ''; &lt;rc&gt; = 0</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,384::lvm::498::Storage.OperationMutex::(_invalidateAllPvs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  got the operation
                                                  mutex</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,385::lvm::500::Storage.OperationMutex::(_invalidateAllPvs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  released the operation
                                                  mutex</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,385::lvm::509::Storage.OperationMutex::(_invalidateAllVgs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  got the operation
                                                  mutex</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,385::lvm::511::Storage.OperationMutex::(_invalidateAllVgs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  released the operation
                                                  mutex</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,386::lvm::529::Storage.OperationMutex::(_invalidateAllLvs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  got the operation
                                                  mutex</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,386::lvm::531::Storage.OperationMutex::(_invalidateAllLvs)


                                                  Operation 'lvm
                                                  invalidate operation'
                                                  released the operation
                                                  mutex</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,386::misc::743::Storage.SamplingMethod::(__call__)


                                                  Returning last result</p>
                                                <p>Thread-807::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:36,386::logUtils::51::dispatcher::(wrapper)



                                                  Run and protect:
                                                  disconnectStorageServer,
                                                  Return response:
                                                  {'statuslist':
                                                  [{'status': 0, 'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054'}]}</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,387::task::1191::Storage.TaskManager.Task::(prepare)


                                                  Task=`01af6594-9c7b-4ec7-b08f-02627db8f421`::finished:




                                                  {'statuslist':
                                                  [{'status': 0, 'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054'}]}</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,387::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`01af6594-9c7b-4ec7-b08f-02627db8f421`::moving



                                                  from state preparing
                                                  -&gt; state finished</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,387::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,387::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,387::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`01af6594-9c7b-4ec7-b08f-02627db8f421`::ref


                                                  0 aborting False</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,388::__init__::533::jsonrpc.JsonRpcServer::(_serveRequest)


                                                  Return
                                                  'StoragePool.disconnectStorageServer'
                                                  in bridge with
                                                  [{'status': 0, 'id':
                                                  u'cd55e6a1-022a-4b32-8a94-cab506a9b054'}]</p>
                                                <p>Thread-807::DEBUG::2015-09-22


                                                  19:12:36,388::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-808::DEBUG::2015-09-22


                                                  19:12:37,868::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`adddaa68-dd1d-4d2e-9853-b7894ee4809c`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-808::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:37,868::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats(options=None)</p>
                                                <p>Thread-808::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:37,868::logUtils::51::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats, Return
                                                  response: {}</p>
                                                <p>Thread-808::DEBUG::2015-09-22


                                                  19:12:37,868::task::1191::Storage.TaskManager.Task::(prepare)


                                                  Task=`adddaa68-dd1d-4d2e-9853-b7894ee4809c`::finished:



                                                  {}</p>
                                                <p>Thread-808::DEBUG::2015-09-22


                                                  19:12:37,868::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`adddaa68-dd1d-4d2e-9853-b7894ee4809c`::moving



                                                  from state preparing
                                                  -&gt; state finished</p>
                                                <p>Thread-808::DEBUG::2015-09-22


                                                  19:12:37,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-808::DEBUG::2015-09-22


                                                  19:12:37,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-808::DEBUG::2015-09-22


                                                  19:12:37,868::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`adddaa68-dd1d-4d2e-9853-b7894ee4809c`::ref


                                                  0 aborting False</p>
                                                <p>Thread-808::DEBUG::2015-09-22


                                                  19:12:37,873::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:44,867::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept)




                                                  Accepting connection
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52512" target="_blank">127.0.0.1:52512</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:12:44,874::protocoldetector::82::ProtocolDetector.Detector::(__init__)




                                                  Using required_size=11</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:44,875::protocoldetector::118::ProtocolDetector.Detector::(handle_read)




                                                  Detected protocol xml
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52512" target="_blank">127.0.0.1:52512</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:12:44,875::bindingxmlrpc::1297::XmlDetector::(handle_socket)


                                                  xml over http detected
                                                  from ('127.0.0.1',
                                                  52512)</p>
                                                <p>BindingXMLRPC::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:44,875::xmlrpc::73::vds.XMLRPCServer::(handle_request)




                                                  Starting request
                                                  handler for <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52512" target="_blank">127.0.0.1:52512</a></p>
                                                <p>Thread-809::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:44,876::xmlrpc::84::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52512" target="_blank">127.0.0.1:52512</a>
                                                  started</p>
                                                <p>Thread-809::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:44,877::xmlrpc::92::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52512" target="_blank">127.0.0.1:52512</a>
                                                  stopped</p>
                                                <p>Thread-810::DEBUG::2015-09-22


                                                  19:12:50,889::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-811::DEBUG::2015-09-22


                                                  19:12:52,902::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`d9fb30bc-dff3-4df3-a25e-2ad09a940fde`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-811::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:52,902::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats(options=None)</p>
                                                <p>Thread-811::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:52,902::logUtils::51::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats, Return
                                                  response: {}</p>
                                                <p>Thread-811::DEBUG::2015-09-22


                                                  19:12:52,902::task::1191::Storage.TaskManager.Task::(prepare)


                                                  Task=`d9fb30bc-dff3-4df3-a25e-2ad09a940fde`::finished:



                                                  {}</p>
                                                <p>Thread-811::DEBUG::2015-09-22


                                                  19:12:52,903::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`d9fb30bc-dff3-4df3-a25e-2ad09a940fde`::moving



                                                  from state preparing
                                                  -&gt; state finished</p>
                                                <p>Thread-811::DEBUG::2015-09-22


                                                  19:12:52,903::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-811::DEBUG::2015-09-22


                                                  19:12:52,903::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-811::DEBUG::2015-09-22


                                                  19:12:52,903::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`d9fb30bc-dff3-4df3-a25e-2ad09a940fde`::ref


                                                  0 aborting False</p>
                                                <p>Thread-811::DEBUG::2015-09-22


                                                  19:12:52,908::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:59,895::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept)




                                                  Accepting connection
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52513" target="_blank">127.0.0.1:52513</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:12:59,902::protocoldetector::82::ProtocolDetector.Detector::(__init__)




                                                  Using required_size=11</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:59,902::protocoldetector::118::ProtocolDetector.Detector::(handle_read)




                                                  Detected protocol xml
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52513" target="_blank">127.0.0.1:52513</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:12:59,902::bindingxmlrpc::1297::XmlDetector::(handle_socket)


                                                  xml over http detected
                                                  from ('127.0.0.1',
                                                  52513)</p>
                                                <p>BindingXMLRPC::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:59,903::xmlrpc::73::vds.XMLRPCServer::(handle_request)




                                                  Starting request
                                                  handler for <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52513" target="_blank">127.0.0.1:52513</a></p>
                                                <p>Thread-812::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:59,903::xmlrpc::84::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52513" target="_blank">127.0.0.1:52513</a>
                                                  started</p>
                                                <p>Thread-812::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:12:59,904::xmlrpc::92::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52513" target="_blank">127.0.0.1:52513</a>
                                                  stopped</p>
                                                <p>Thread-813::DEBUG::2015-09-22


                                                  19:13:05,898::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Thread-814::DEBUG::2015-09-22


                                                  19:13:07,934::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`c526c56c-6254-4497-9c3e-ffe09ed54af2`::moving



                                                  from state init -&gt;
                                                  state preparing</p>
                                                <p>Thread-814::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:13:07,935::logUtils::48::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats(options=None)</p>
                                                <p>Thread-814::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:13:07,935::logUtils::51::dispatcher::(wrapper)



                                                  Run and protect:
                                                  repoStats, Return
                                                  response: {}</p>
                                                <p>Thread-814::DEBUG::2015-09-22


                                                  19:13:07,935::task::1191::Storage.TaskManager.Task::(prepare)


                                                  Task=`c526c56c-6254-4497-9c3e-ffe09ed54af2`::finished:



                                                  {}</p>
                                                <p>Thread-814::DEBUG::2015-09-22


                                                  19:13:07,935::task::595::Storage.TaskManager.Task::(_updateState)


                                                  Task=`c526c56c-6254-4497-9c3e-ffe09ed54af2`::moving



                                                  from state preparing
                                                  -&gt; state finished</p>
                                                <p>Thread-814::DEBUG::2015-09-22


                                                  19:13:07,935::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll)




                                                  Owner.releaseAll
                                                  requests {} resources
                                                  {}</p>
                                                <p>Thread-814::DEBUG::2015-09-22


                                                  19:13:07,935::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll)




                                                  Owner.cancelAll
                                                  requests {}</p>
                                                <p>Thread-814::DEBUG::2015-09-22


                                                  19:13:07,935::task::993::Storage.TaskManager.Task::(_decref)


                                                  Task=`c526c56c-6254-4497-9c3e-ffe09ed54af2`::ref


                                                  0 aborting False</p>
                                                <p>Thread-814::DEBUG::2015-09-22


                                                  19:13:07,939::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:13:14,921::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept)




                                                  Accepting connection
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52515" target="_blank">127.0.0.1:52515</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:13:14,927::protocoldetector::82::ProtocolDetector.Detector::(__init__)




                                                  Using required_size=11</p>
                                                <p>Reactor thread::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:13:14,928::protocoldetector::118::ProtocolDetector.Detector::(handle_read)




                                                  Detected protocol xml
                                                  from <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52515" target="_blank">127.0.0.1:52515</a></p>
                                                <p>Reactor
                                                  thread::DEBUG::2015-09-22
                                                  19:13:14,928::bindingxmlrpc::1297::XmlDetector::(handle_socket)


                                                  xml over http detected
                                                  from ('127.0.0.1',
                                                  52515)</p>
                                                <p>BindingXMLRPC::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:13:14,928::xmlrpc::73::vds.XMLRPCServer::(handle_request)




                                                  Starting request
                                                  handler for <a
                                                    moz-do-not-send="true"
href="http://127.0.0.1:52515" target="_blank">127.0.0.1:52515</a></p>
                                                <p>Thread-815::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:13:14,928::xmlrpc::84::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52515" target="_blank">127.0.0.1:52515</a>
                                                  started</p>
                                                <p>Thread-815::<a
                                                    moz-do-not-send="true"
class="moz-txt-link-freetext" href="INFO::2015-09-22"><a class="moz-txt-link-freetext" href="INFO::2015-09-22">INFO::2015-09-22</a></a>
                                                  19:13:14,930::xmlrpc::92::vds.XMLRPCServer::(_process_requests)




                                                  Request handler for <a
moz-do-not-send="true" href="http://127.0.0.1:52515" target="_blank">127.0.0.1:52515</a>
                                                  stopped</p>
                                                <p><span></span></p>
                                                <p>Thread-816::DEBUG::2015-09-22


                                                  19:13:20,906::stompreactor::304::yajsonrpc.StompServer::(send)


                                                  Sending response</p>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div><br>
                                              </div>
                                              <div>gluster logs</div>
                                              <div><br>
                                              </div>
                                              <div>
                                                <p><span>+------------------------------------------------------------------------------+</span></p>
                                                <p><span>  1: volume
                                                    vmstore-client-0</span></p>
                                                <p><span>  2:     type
                                                    protocol/client</span></p>
                                                <p><span>  3:     option
                                                    ping-timeout 42</span></p>
                                                <p><span>  4:     option
                                                    remote-host
                                                    sjcstorage01</span></p>
                                                <p><span>  5:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span>  6:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span>  7:     option
                                                    send-gids true</span></p>
                                                <p><span>  8: end-volume</span></p>
                                                <p><span>  9:  </span></p>
                                                <p><span> 10: volume
                                                    vmstore-client-1</span></p>
                                                <p><span> 11:     type
                                                    protocol/client</span></p>
                                                <p><span> 12:     option
                                                    ping-timeout 42</span></p>
                                                <p><span> 13:     option
                                                    remote-host
                                                    sjcstorage02</span></p>
                                                <p><span> 14:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span> 15:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span> 16:     option
                                                    send-gids true</span></p>
                                                <p><span> 17: end-volume</span></p>
                                                <p><span> 18:  </span></p>
                                                <p><span> 19: volume
                                                    vmstore-client-2</span></p>
                                                <p><span> 20:     type
                                                    protocol/client</span></p>
                                                <p><span> 21:     option
                                                    ping-timeout 42</span></p>
                                                <p><span> 22:     option
                                                    remote-host
                                                    sjcvhost02</span></p>
                                                <p><span> 23:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span> 24:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span> 25:     option
                                                    send-gids true</span></p>
                                                <p><span> 26: end-volume</span></p>
                                                <p><span> 27:  </span></p>
                                                <p><span> 28: volume
                                                    vmstore-replicate-0</span></p>
                                                <p><span> 29:     type
                                                    cluster/replicate</span></p>
                                                <p><span> 30:     option
                                                    arbiter-count 1</span></p>
                                                <p><span> 31:    
                                                    subvolumes
                                                    vmstore-client-0
                                                    vmstore-client-1
                                                    vmstore-client-2</span></p>
                                                <p><span> 32: end-volume</span></p>
                                                <p><span> 33:  </span></p>
                                                <p><span> 34: volume
                                                    vmstore-dht</span></p>
                                                <p><span> 35:     type
                                                    cluster/distribute</span></p>
                                                <p><span> 36:    
                                                    subvolumes
                                                    vmstore-replicate-0</span></p>
                                                <p><span> 37: end-volume</span></p>
                                                <p><span> 38:  </span></p>
                                                <p><span> 39: volume
                                                    vmstore-write-behind</span></p>
                                                <p><span> 40:     type
                                                    performance/write-behind</span></p>
                                                <p><span> 41:    
                                                    subvolumes
                                                    vmstore-dht</span></p>
                                                <p><span> 42: end-volume</span></p>
                                                <p><span> 43:  </span></p>
                                                <p><span> 44: volume
                                                    vmstore-read-ahead</span></p>
                                                <p><span> 45:     type
                                                    performance/read-ahead</span></p>
                                                <p><span> 46:    
                                                    subvolumes
                                                    vmstore-write-behind</span></p>
                                                <p><span> 47: end-volume</span></p>
                                                <p><span> 48:  </span></p>
                                                <p><span> 49: volume
                                                    vmstore-readdir-ahead</span></p>
                                                <p><span> 50:     type
                                                    performance/readdir-ahead</span></p>
                                                <p><span>52: end-volume</span></p>
                                                <p><span> 53:  </span></p>
                                                <p><span> 54: volume
                                                    vmstore-io-cache</span></p>
                                                <p><span> 55:     type
                                                    performance/io-cache</span></p>
                                                <p><span> 56:    
                                                    subvolumes
                                                    vmstore-readdir-ahead</span></p>
                                                <p><span> 57: end-volume</span></p>
                                                <p><span> 58:  </span></p>
                                                <p><span> 59: volume
                                                    vmstore-quick-read</span></p>
                                                <p><span> 60:     type
                                                    performance/quick-read</span></p>
                                                <p><span> 61:    
                                                    subvolumes
                                                    vmstore-io-cache</span></p>
                                                <p><span> 62: end-volume</span></p>
                                                <p><span> 63:  </span></p>
                                                <p><span> 64: volume
                                                    vmstore-open-behind</span></p>
                                                <p><span> 65:     type
                                                    performance/open-behind</span></p>
                                                <p><span> 66:    
                                                    subvolumes
                                                    vmstore-quick-read</span></p>
                                                <p><span> 67: end-volume</span></p>
                                                <p><span> 68:  </span></p>
                                                <p><span> 69: volume
                                                    vmstore-md-cache</span></p>
                                                <p><span> 70:     type
                                                    performance/md-cache</span></p>
                                                <p><span> 71:    
                                                    subvolumes
                                                    vmstore-open-behind</span></p>
                                                <p><span> 72: end-volume</span></p>
                                                <p><span> 73:  </span></p>
                                                <p><span> 74: volume
                                                    vmstore</span></p>
                                                <p><span> 75:     type
                                                    debug/io-stats</span></p>
                                                <p><span> 76:     option
                                                    latency-measurement
                                                    off</span></p>
                                                <p><span> 77:     option
                                                    count-fop-hits off</span></p>
                                                <p><span> 78:    
                                                    subvolumes
                                                    vmstore-md-cache</span></p>
                                                <p><span> 79: end-volume</span></p>
                                                <p><span> 80:  </span></p>
                                                <p><span> 81: volume
                                                    meta-autoload</span></p>
                                                <p><span> 82:     type
                                                    meta</span></p>
                                                <p><span> 83:    
                                                    subvolumes vmstore</span></p>
                                                <p><span> 84: end-volume</span></p>
                                                <p><span> 85:  </span></p>
                                                <p><span>+------------------------------------------------------------------------------+</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.586205] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-0:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.586325] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-1:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.586480] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-2:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.595052] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-0:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.595397] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-1:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.595576] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-2:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.595721] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-0:
                                                    Connected to
                                                    vmstore-client-0,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.595738] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-0:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.596044] I
                                                    [MSGID: 108005]
                                                    [afr-common.c:3998:afr_notify]
                                                    0-vmstore-replicate-0:

                                                    Subvolume
                                                    'vmstore-client-0'
                                                    came back up; going
                                                    online.</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.596170] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-1:
                                                    Connected to
                                                    vmstore-client-1,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.596189] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-1:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span> </span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.596495] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-2:
                                                    Connected to
                                                    vmstore-client-2,
                                                    attached to remote
                                                    volume :</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.596189] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-1:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.596495] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-2:
                                                    Connected to
                                                    vmstore-client-2,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.596506] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-2:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.608758] I
                                                    [fuse-bridge.c:5053:fuse_graph_setup]
                                                    0-fuse: switched to
                                                    graph 0</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.608910] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-0:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.608936] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-1:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.608950] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-2:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.609695] I
                                                    [MSGID: 101190]
                                                    [event-epoll.c:632:event_dispatch_epoll_worker]
                                                    0-epoll: Started
                                                    thread with index 2</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.609868] I
                                                    [fuse-bridge.c:3979:fuse_init]
                                                    0-glusterfs-fuse:
                                                    FUSE inited with
                                                    protocol versions:
                                                    glusterfs 7.22
                                                    kernel 7.22</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.616577] I
                                                    [MSGID: 109063]
                                                    [dht-layout.c:702:dht_layout_normalize]
                                                    0-vmstore-dht: Found
                                                    anomalies in / (gfid
                                                    =
                                                    00000000-0000-0000-0000-000000000001).
                                                    Holes=1 overlaps=0</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:07.620230] I
                                                    [MSGID: 109036]
                                                    [dht-common.c:7754:dht_log_new_layout_for_dir_selfheal]
                                                    0-vmstore-dht:
                                                    Setting layout of /
                                                    with [Subvol_name:
                                                    vmstore-replicate-0,
                                                    Err: -1 , Start: 0 ,
                                                    Stop: 4294967295 ,
                                                    Hash: 1 ], </span></p>
                                                <p><span>[2015-09-22
                                                    05:29:08.122415] W
                                                    [fuse-bridge.c:1230:fuse_err_cbk]
                                                    0-glusterfs-fuse:
                                                    26: REMOVEXATTR()
                                                    /__DIRECT_IO_TEST__
                                                    =&gt; -1 (No data
                                                    available)</span></p>
                                                <p><span>[2015-09-22
                                                    05:29:08.<a
                                                      moz-do-not-send="true"
                                                      href="tel:137359"
                                                      value="+61137359"
                                                      target="_blank">137359</a>]
                                                    I [MSGID: 109036]
                                                    [dht-common.c:7754:dht_log_new_layout_for_dir_selfheal]
                                                    0-vmstore-dht:
                                                    Setting layout of
                                                    /061b73d5-ae59-462e-b674-ea9c60d436c2
                                                    with [Subvol_name:
                                                    vmstore-replicate-0,
                                                    Err: -1 , Start: 0 ,
                                                    Stop: 4294967295 ,
                                                    Hash: 1 ], </span></p>
                                                <p><span>[2015-09-22
                                                    05:29:08.145835] I
                                                    [MSGID: 109036]
                                                    [dht-common.c:7754:dht_log_new_layout_for_dir_selfheal]
                                                    0-vmstore-dht:
                                                    Setting layout of
                                                    /061b73d5-ae59-462e-b674-ea9c60d436c2/dom_md
                                                    with [Subvol_name:
                                                    vmstore-replicate-0,
                                                    Err: -1 , Start: 0 ,
                                                    Stop: 4294967295 ,
                                                    Hash: 1 ], </span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.897819] I
                                                    [MSGID: 100030]
                                                    [glusterfsd.c:2301:main]
                                                    0-/usr/sbin/glusterfs:

                                                    Started running
                                                    /usr/sbin/glusterfs
                                                    version 3.7.4 (args:
                                                    /usr/sbin/glusterfs
                                                    --volfile-server=sjcvhost02

                                                    --volfile-server=sjcstorage01


                                                    --volfile-server=sjcstorage02


--volfile-id=/vmstore
/rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore)</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.909889] I
                                                    [MSGID: 101190]
                                                    [event-epoll.c:632:event_dispatch_epoll_worker]
                                                    0-epoll: Started
                                                    thread with index 1</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.923087] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-0:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.925701] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-1:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.927984] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-2:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p><span>Final graph:</span></p>
                                                <p><span>+------------------------------------------------------------------------------+</span></p>
                                                <p><span>  1: volume
                                                    vmstore-client-0</span></p>
                                                <p><span>  2:     type
                                                    protocol/client</span></p>
                                                <p><span>  3:     option
                                                    ping-timeout 42</span></p>
                                                <p><span>  4:     option
                                                    remote-host
                                                    sjcstorage01</span></p>
                                                <p><span>  5:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span>  6:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span>  7:     option
                                                    send-gids true</span></p>
                                                <p><span>  8: end-volume</span></p>
                                                <p><span>  9:  </span></p>
                                                <p><span> 10: volume
                                                    vmstore-client-1</span></p>
                                                <p><span> 11:     type
                                                    protocol/client</span></p>
                                                <p><span> 12:     option
                                                    ping-timeout 42</span></p>
                                                <p><span> 13:     option
                                                    remote-host
                                                    sjcstorage02</span></p>
                                                <p><span> </span></p>
                                                <p><span> 14:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span> 15:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span> 16:     option
                                                    send-gids true</span></p>
                                                <p><span> 17: end-volume</span></p>
                                                <p><span> 18:  </span></p>
                                                <p><span> 19: volume
                                                    vmstore-client-2</span></p>
                                                <p><span> 20:     type
                                                    protocol/client</span></p>
                                                <p><span> 21:     option
                                                    ping-timeout 42</span></p>
                                                <p><span> 22:     option
                                                    remote-host
                                                    sjcvhost02</span></p>
                                                <p><span> 23:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span> 24:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span> 25:     option
                                                    send-gids true</span></p>
                                                <p><span> 26: end-volume</span></p>
                                                <p><span> 27:  </span></p>
                                                <p><span> 28: volume
                                                    vmstore-replicate-0</span></p>
                                                <p><span> 29:     type
                                                    cluster/replicate</span></p>
                                                <p><span> 30:     option
                                                    arbiter-count 1</span></p>
                                                <p><span> 31:    
                                                    subvolumes
                                                    vmstore-client-0
                                                    vmstore-client-1
                                                    vmstore-client-2</span></p>
                                                <p><span> 32: end-volume</span></p>
                                                <p><span> 33:  </span></p>
                                                <p><span> 34: volume
                                                    vmstore-dht</span></p>
                                                <p><span> 35:     type
                                                    cluster/distribute</span></p>
                                                <p><span> 36:    
                                                    subvolumes
                                                    vmstore-replicate-0</span></p>
                                                <p><span> 37: end-volume</span></p>
                                                <p><span> 38:  </span></p>
                                                <p><span> 39: volume
                                                    vmstore-write-behind</span></p>
                                                <p><span> 40:     type
                                                    performance/write-behind</span></p>
                                                <p><span> 41:    
                                                    subvolumes
                                                    vmstore-dht</span></p>
                                                <p><span> 42: end-volume</span></p>
                                                <p><span> 43:  </span></p>
                                                <p><span> 44: volume
                                                    vmstore-read-ahead</span></p>
                                                <p><span> 45:     type
                                                    performance/read-ahead</span></p>
                                                <p><span> 46:    
                                                    subvolumes
                                                    vmstore-write-behind</span></p>
                                                <p><span> 47: end-volume</span></p>
                                                <p><span> 48:  </span></p>
                                                <p><span> 49: volume
                                                    vmstore-readdir-ahead</span></p>
                                                <p><span> 50:     type
                                                    performance/readdir-ahead</span></p>
                                                <p><span> 51:    
                                                    subvolumes
                                                    vmstore-read-ahead</span></p>
                                                <p><span> 52: end-volume</span></p>
                                                <p><span> 53:  </span></p>
                                                <p><span> 54: volume
                                                    vmstore-io-cache</span></p>
                                                <p><span> 55:     type
                                                    performance/io-cache</span></p>
                                                <p><span> 56:    
                                                    subvolumes
                                                    vmstore-readdir-ahead</span></p>
                                                <p><span> 57: end-volume</span></p>
                                                <p><span> 58:  </span></p>
                                                <p><span> 59: volume
                                                    vmstore-quick-read</span></p>
                                                <p><span> 60:     type
                                                    performance/quick-read</span></p>
                                                <p><span> 61:    
                                                    subvolumes
                                                    vmstore-io-cache</span></p>
                                                <p><span> 62: end-volume</span></p>
                                                <p><span> 63:  </span></p>
                                                <p><span> 64: volume
                                                    vmstore-open-behind</span></p>
                                                <p><span> 65:     type
                                                    performance/open-behind</span></p>
                                                <p><span> 66:    
                                                    subvolumes
                                                    vmstore-quick-read</span></p>
                                                <p><span> 67: end-volume</span></p>
                                                <p><span> 68:  </span></p>
                                                <p><span> 69: volume
                                                    vmstore-md-cache</span></p>
                                                <p><span> </span></p>
                                                <p><span> 70:     type
                                                    performance/md-cache</span></p>
                                                <p><span> 71:    
                                                    subvolumes
                                                    vmstore-open-behind</span></p>
                                                <p><span> 72: end-volume</span></p>
                                                <p><span> 73:  </span></p>
                                                <p><span> 74: volume
                                                    vmstore</span></p>
                                                <p><span> 75:     type
                                                    debug/io-stats</span></p>
                                                <p><span> 76:     option
                                                    latency-measurement
                                                    off</span></p>
                                                <p><span> 77:     option
                                                    count-fop-hits off</span></p>
                                                <p><span> 78:    
                                                    subvolumes
                                                    vmstore-md-cache</span></p>
                                                <p><span> 79: end-volume</span></p>
                                                <p><span> 80:  </span></p>
                                                <p><span> 81: volume
                                                    meta-autoload</span></p>
                                                <p><span> 82:     type
                                                    meta</span></p>
                                                <p><span> 83:    
                                                    subvolumes vmstore</span></p>
                                                <p><span> 84: end-volume</span></p>
                                                <p><span> 85:  </span></p>
                                                <p><span>+------------------------------------------------------------------------------+</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.934021] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-0:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.934145] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-1:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.934491] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-2:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.942198] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-0:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.942545] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-1:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.942659] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-2:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.942797] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-0:
                                                    Connected to
                                                    vmstore-client-0,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.942808] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-0:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.943036] I
                                                    [MSGID: 108005]
                                                    [afr-common.c:3998:afr_notify]
                                                    0-vmstore-replicate-0:

                                                    Subvolume
                                                    'vmstore-client-0'
                                                    came back up; going
                                                    online.</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.943078] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-1:
                                                    Connected to
                                                    vmstore-client-1,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.943086] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-1:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.943292] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-2:
                                                    Connected to
                                                    vmstore-client-2,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.943302] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-2:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.953887] I
                                                    [fuse-bridge.c:5053:fuse_graph_setup]
                                                    0-fuse: switched to
                                                    graph 0</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.954071] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-0:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.954105] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-1:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.954124] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-2:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.955282] I
                                                    [fuse-bridge.c:3979:fuse_init]
                                                    0-glusterfs-fuse:
                                                    FUSE inited with
                                                    protocol versions:
                                                    glusterfs 7.22
                                                    kernel 7.22</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.955738] I
                                                    [MSGID: 101190]
                                                    [event-epoll.c:632:event_dispatch_epoll_worker]
                                                    0-epoll: Started
                                                    thread with index 2</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.970232] I
                                                    [fuse-bridge.c:4900:fuse_thread_proc]
                                                    0-fuse: unmounting
                                                    /rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.970834] W
                                                    [glusterfsd.c:1219:cleanup_and_exit]
                                                    (--&gt;/lib64/libpthread.so.0(+0x7df5)


                                                    [0x7f187139fdf5]
                                                    --&gt;/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5)
                                                    [0x7f1872a09785]
                                                    --&gt;/usr/sbin/glusterfs(cleanup_and_exit+0x69)
                                                    [0x7f1872a09609] )
                                                    0-: received signum
                                                    (15), shutting down</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:57.970848] I
                                                    [fuse-bridge.c:5595:fini]
                                                    0-fuse: Unmounting
                                                    '/rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:58.420973] I
                                                    [fuse-bridge.c:4900:fuse_thread_proc]
                                                    0-fuse: unmounting
                                                    /rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:58.421355] W
                                                    [glusterfsd.c:1219:cleanup_and_exit]
                                                    (--&gt;/lib64/libpthread.so.0(+0x7df5)


                                                    [0x7f8267cd4df5]
                                                    --&gt;/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5)
                                                    [0x7f826933e785]
                                                    --&gt;/usr/sbin/glusterfs(cleanup_and_exit+0x69)
                                                    [0x7f826933e609] )
                                                    0-: received signum
                                                    (15), shutting down</span></p>
                                                <p><span>[2015-09-22
                                                    05:30:58.421369] I
                                                    [fuse-bridge.c:5595:fini]
                                                    0-fuse: Unmounting
                                                    '/rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.534410] I
                                                    [MSGID: 100030]
                                                    [glusterfsd.c:2301:main]
                                                    0-/usr/sbin/glusterfs:

                                                    Started running
                                                    /usr/sbin/glusterfs
                                                    version 3.7.4 (args:
                                                    /usr/sbin/glusterfs
                                                    --volfile-server=sjcvhost02

                                                    --volfile-server=sjcstorage01


                                                    --volfile-server=sjcstorage02


--volfile-id=/vmstore
/rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore)</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.545686] I
                                                    [MSGID: 101190]
                                                    [event-epoll.c:632:event_dispatch_epoll_worker]
                                                    0-epoll: Started
                                                    thread with index 1</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.553019] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-0:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p> </p>
                                                <p><span>[2015-09-22
                                                    05:31:09.555552] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-1:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.557989] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-2:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p><span>Final graph:</span></p>
                                                <p><span>+------------------------------------------------------------------------------+</span></p>
                                                <p><span>  1: volume
                                                    vmstore-client-0</span></p>
                                                <p><span>  2:     type
                                                    protocol/client</span></p>
                                                <p><span>  3:     option
                                                    ping-timeout 42</span></p>
                                                <p><span>  4:     option
                                                    remote-host
                                                    sjcstorage01</span></p>
                                                <p><span>  5:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span>  6:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span>  7:     option
                                                    send-gids true</span></p>
                                                <p><span>  8: end-volume</span></p>
                                                <p><span>  9:  </span></p>
                                                <p><span> 10: volume
                                                    vmstore-client-1</span></p>
                                                <p><span> 11:     type
                                                    protocol/client</span></p>
                                                <p><span> 12:     option
                                                    ping-timeout 42</span></p>
                                                <p><span> 13:     option
                                                    remote-host
                                                    sjcstorage02</span></p>
                                                <p><span> 14:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span> 15:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span> 16:     option
                                                    send-gids true</span></p>
                                                <p><span> 17: end-volume</span></p>
                                                <p><span> 18:  </span></p>
                                                <p><span> 19: volume
                                                    vmstore-client-2</span></p>
                                                <p><span> 20:     type
                                                    protocol/client</span></p>
                                                <p><span> 21:     option
                                                    ping-timeout 42</span></p>
                                                <p><span> 22:     option
                                                    remote-host
                                                    sjcvhost02</span></p>
                                                <p><span> 23:     option
                                                    remote-subvolume
                                                    /export/vmstore/brick01</span></p>
                                                <p><span> 24:     option
                                                    transport-type
                                                    socket</span></p>
                                                <p><span> 25:     option
                                                    send-gids true</span></p>
                                                <p><span> 26: end-volume</span></p>
                                                <p><span> 27:  </span></p>
                                                <p><span> 28: volume
                                                    vmstore-replicate-0</span></p>
                                                <p><span> 29:     type
                                                    cluster/replicate</span></p>
                                                <p><span> 30:     option
                                                    arbiter-count 1</span></p>
                                                <p><span> 31:    
                                                    subvolumes
                                                    vmstore-client-0
                                                    vmstore-client-1
                                                    vmstore-client-2</span></p>
                                                <p><span> 32: end-volume</span></p>
                                                <p><span> 33:  </span></p>
                                                <p><span> 34: volume
                                                    vmstore-dht</span></p>
                                                <p><span> 35:     type
                                                    cluster/distribute</span></p>
                                                <p><span> 36:    
                                                    subvolumes
                                                    vmstore-replicate-0</span></p>
                                                <p><span> 37: end-volume</span></p>
                                                <p><span> 38:  </span></p>
                                                <p><span> 39: volume
                                                    vmstore-write-behind</span></p>
                                                <p><span> 40:     type
                                                    performance/write-behind</span></p>
                                                <p><span> 41:    
                                                    subvolumes
                                                    vmstore-dht</span></p>
                                                <p><span> 42: end-volume</span></p>
                                                <p><span> 43:  </span></p>
                                                <p><span> 44: volume
                                                    vmstore-read-ahead</span></p>
                                                <p><span> 45:     type
                                                    performance/read-ahead</span></p>
                                                <p><span> 46:    
                                                    subvolumes
                                                    vmstore-write-behind</span></p>
                                                <p><span> 47: end-volume</span></p>
                                                <p><span> 48:  </span></p>
                                                <p><span> 49: volume
                                                    vmstore-readdir-ahead</span></p>
                                                <p><span> 50:     type
                                                    performance/readdir-ahead</span></p>
                                                <p><span> 51:    
                                                    subvolumes
                                                    vmstore-read-ahead</span></p>
                                                <p> </p>
                                                <p><span> 52: end-volume</span></p>
                                                <p><span> 53:  </span></p>
                                                <p><span> 54: volume
                                                    vmstore-io-cache</span></p>
                                                <p><span> 55:     type
                                                    performance/io-cache</span></p>
                                                <p><span> 56:    
                                                    subvolumes
                                                    vmstore-readdir-ahead</span></p>
                                                <p><span> 57: end-volume</span></p>
                                                <p><span> 58:  </span></p>
                                                <p><span> 59: volume
                                                    vmstore-quick-read</span></p>
                                                <p><span> 60:     type
                                                    performance/quick-read</span></p>
                                                <p><span> 61:    
                                                    subvolumes
                                                    vmstore-io-cache</span></p>
                                                <p><span> 62: end-volume</span></p>
                                                <p><span> 63:  </span></p>
                                                <p><span> 64: volume
                                                    vmstore-open-behind</span></p>
                                                <p><span> 65:     type
                                                    performance/open-behind</span></p>
                                                <p><span> 66:    
                                                    subvolumes
                                                    vmstore-quick-read</span></p>
                                                <p><span> 67: end-volume</span></p>
                                                <p><span> 68:  </span></p>
                                                <p><span> 69: volume
                                                    vmstore-md-cache</span></p>
                                                <p><span> 70:     type
                                                    performance/md-cache</span></p>
                                                <p><span> 71:    
                                                    subvolumes
                                                    vmstore-open-behind</span></p>
                                                <p><span> 72: end-volume</span></p>
                                                <p><span> 73:  </span></p>
                                                <p><span> 74: volume
                                                    vmstore</span></p>
                                                <p><span> 75:     type
                                                    debug/io-stats</span></p>
                                                <p><span> 76:     option
                                                    latency-measurement
                                                    off</span></p>
                                                <p><span> 77:     option
                                                    count-fop-hits off</span></p>
                                                <p><span> 78:    
                                                    subvolumes
                                                    vmstore-md-cache</span></p>
                                                <p><span> 79: end-volume</span></p>
                                                <p><span> 80:  </span></p>
                                                <p><span> 81: volume
                                                    meta-autoload</span></p>
                                                <p><span> 82:     type
                                                    meta</span></p>
                                                <p><span> 83:    
                                                    subvolumes vmstore</span></p>
                                                <p><span> 84: end-volume</span></p>
                                                <p><span> 85:  </span></p>
                                                <p><span>+------------------------------------------------------------------------------+</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.563262] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-0:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.563431] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-1:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.563877] I
                                                    [rpc-clnt.c:1851:rpc_clnt_reconfig]
                                                    0-vmstore-client-2:
                                                    changing port to
                                                    49153 (from 0)</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.572443] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-1:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.572599] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-0:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.572742] I
                                                    [MSGID: 114057]
                                                    [client-handshake.c:1437:select_server_supported_programs]
                                                    0-vmstore-client-2:
                                                    Using Program
                                                    GlusterFS 3.3, Num
                                                    (1298437), Version
                                                    (330)</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.573165] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-1:
                                                    Connected to
                                                    vmstore-client-1,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.573186] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-1:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.573395] I
                                                    [MSGID: 108005]
                                                    [afr-common.c:3998:afr_notify]
                                                    0-vmstore-replicate-0:

                                                    Subvolume
                                                    'vmstore-client-1'
                                                    came back up; going
                                                    online.</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.573427] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-0:
                                                    Connected to
                                                    vmstore-client-0,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.573435] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-0:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopening the fds</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.573754] I
                                                    [MSGID: 114046]
                                                    [client-handshake.c:1213:client_setvolume_cbk]
                                                    0-vmstore-client-2:
                                                    Connected to
                                                    vmstore-client-2,
                                                    attached to remote
                                                    volume
                                                    '/export/vmstore/brick01'.</span></p>
                                                <p> </p>
                                                <p><span>[2015-09-22
                                                    05:31:09.573783] I
                                                    [MSGID: 114047]
                                                    [client-handshake.c:1224:client_setvolume_cbk]
                                                    0-vmstore-client-2:
                                                    Server and Client
                                                    lk-version numbers
                                                    are not same,
                                                    reopen:</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.577192] I
                                                    [fuse-bridge.c:5053:fuse_graph_setup]
                                                    0-fuse: switched to
                                                    graph 0</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.577302] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-1:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.577325] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-0:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.577339] I
                                                    [MSGID: 114035]
                                                    [client-handshake.c:193:client_set_lk_version_cbk]
                                                    0-vmstore-client-2:
                                                    Server lk version =
                                                    1</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.578125] I
                                                    [fuse-bridge.c:3979:fuse_init]
                                                    0-glusterfs-fuse:
                                                    FUSE inited with
                                                    protocol versions:
                                                    glusterfs 7.22
                                                    kernel 7.22</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:09.578636] I
                                                    [MSGID: 101190]
                                                    [event-epoll.c:632:event_dispatch_epoll_worker]
                                                    0-epoll: Started
                                                    thread with index 2</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:10.073698] I
                                                    [fuse-bridge.c:4900:fuse_thread_proc]
                                                    0-fuse: unmounting
                                                    /rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:10.073977] W
                                                    [glusterfsd.c:1219:cleanup_and_exit]
                                                    (--&gt;/lib64/libpthread.so.0(+0x7df5)


                                                    [0x7f6b9ba88df5]
                                                    --&gt;/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5)
                                                    [0x7f6b9d0f2785]
                                                    --&gt;/usr/sbin/glusterfs(cleanup_and_exit+0x69)
                                                    [0x7f6b9d0f2609] )
                                                    0-: received signum
                                                    (15), shutting down</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:10.073993] I
                                                    [fuse-bridge.c:5595:fini]
                                                    0-fuse: Unmounting
                                                    '/rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore'.</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:20.184700] I
                                                    [MSGID: 100030]
                                                    [glusterfsd.c:2301:main]
                                                    0-/usr/sbin/glusterfs:

                                                    Started running
                                                    /usr/sbin/glusterfs
                                                    version 3.7.4 (args:
                                                    /usr/sbin/glusterfs
                                                    --volfile-server=sjcvhost02

                                                    --volfile-server=sjcstorage01


                                                    --volfile-server=sjcstorage02


--volfile-id=/vmstore
/rhev/data-center/mnt/glusterSD/sjcvhost02:_vmstore)</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:20.194928] I
                                                    [MSGID: 101190]
                                                    [event-epoll.c:632:event_dispatch_epoll_worker]
                                                    0-epoll: Started
                                                    thread with index 1</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:20.200701] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-0:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:20.203110] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-1:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p><span>[2015-09-22
                                                    05:31:20.205708] I
                                                    [MSGID: 114020]
                                                    [client.c:2118:notify]
                                                    0-vmstore-client-2:
                                                    parent translators
                                                    are ready,
                                                    attempting connect
                                                    on transport</span></p>
                                                <p><span> </span></p>
                                                <p><span>Final graph:</span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p><span>Hope this
                                                    helps. </span></p>
                                                <p><span><br>
                                                  </span></p>
                                                <p>thanks again</p>
                                                <p><br>
                                                </p>
                                                <p>Brett Stevens</p>
                                                <p><br>
                                                </p>
                                              </div>
                                            </div>
                                            <div class="gmail_extra"><br>
                                              <div class="gmail_quote">On

                                                Tue, Sep 22, 2015 at
                                                10:14 PM, Sahina Bose <span
                                                  dir="ltr">&lt;<a
                                                    moz-do-not-send="true"
class="moz-txt-link-abbreviated" href="mailto:sabose@redhat.com"><a class="moz-txt-link-abbreviated" href="mailto:sabose@redhat.com">sabose@redhat.com</a></a>&gt;</span>
                                                wrote:<br>
                                                <blockquote
                                                  class="gmail_quote"
                                                  style="margin:0 0 0
                                                  .8ex;border-left:1px
                                                  #ccc
                                                  solid;padding-left:1ex">
                                                  <div text="#000000"
                                                    bgcolor="#FFFFFF"><span>
                                                      <br>
                                                      <br>
                                                      <div>On 09/22/2015
                                                        02:17 PM, Brett
                                                        Stevens wrote:<br>
                                                      </div>
                                                      <blockquote
                                                        type="cite">
                                                        <div dir="ltr">Hi.

                                                          First time on
                                                          the lists.
                                                          I've searched
                                                          for this but
                                                          no luck so
                                                          sorry if this
                                                          has been
                                                          covered
                                                          before.
                                                          <div><br>
                                                          </div>
                                                          <div>Im
                                                          working with
                                                          the latest 3.6
                                                          beta with the
                                                          following
                                                          infrastructure. </div>
                                                          <div><br>
                                                          </div>
                                                          <div>1
                                                          management
                                                          host (to be
                                                          used for a
                                                          number of
                                                          tasks so chose
                                                          not to use
                                                          self hosted,
                                                          we are a
                                                          school and
                                                          will need to
                                                          keep an eye on
                                                          hardware
                                                          costs)</div>
                                                          <div>2 compute
                                                          nodes</div>
                                                          <div>2 gluster
                                                          nodes</div>
                                                          <div><br>
                                                          </div>
                                                          <div>so far
                                                          built one
                                                          gluster volume
                                                          using the
                                                          gluster cli to
                                                          give me 2
                                                          nodes and one
                                                          arbiter node
                                                          (management
                                                          host)</div>
                                                          <div><br>
                                                          </div>
                                                          <div>so far,
                                                          every time I
                                                          create a
                                                          volume, it
                                                          shows up
                                                          strait away on
                                                          the ovirt gui.
                                                          however no
                                                          matter what I
                                                          try, I cannot
                                                          create or
                                                          import it as a
                                                          data domain. </div>
                                                          <div><br>
                                                          </div>
                                                          <div>the
                                                          current error
                                                          in the ovirt
                                                          gui is "Error
                                                          while
                                                          executing
                                                          action
                                                          AddGlusterFsStorageDomain:
                                                          Error creating
                                                          a storage
                                                          domain's
                                                          metadata"</div>
                                                        </div>
                                                      </blockquote>
                                                      <br>
                                                    </span> Please
                                                    provide vdsm and
                                                    gluster logs<span><br>
                                                      <br>
                                                      <blockquote
                                                        type="cite">
                                                        <div dir="ltr">
                                                          <div><br>
                                                          </div>
                                                          <div>logs,
                                                          continuously
                                                          rolling the
                                                          following
                                                          errors around</div>
                                                          <div>
                                                          <p><span>Scheduler_Worker-53)


                                                          [] START,
                                                          GlusterVolumesListVDSCommand(HostName
                                                          =
                                                          sjcstorage02,
                                                          GlusterVolumesListVDSParameters:{runAsync='true',


                                                          hostId='c75682ba-1e4c-42a3-85c7-16e4bb2ce5da'}),




                                                          log id:
                                                          24198fbf</span></p>
                                                          <p><span>2015-09-22


                                                          03:57:29,903
                                                          WARN 
                                                          [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]





                                                          (DefaultQuartzScheduler_Worker-53)



                                                          [] Could not
                                                          associate
                                                          brick
                                                          'sjcstorage01:/export/vmstore/brick01'
                                                          of volume
                                                          '878a316d-2394-4aae-bdf8-e10eea38225e'
                                                          with correct
                                                          network as no
                                                          gluster
                                                          network found
                                                          in cluster
                                                          'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                          </div>
                                                        </div>
                                                      </blockquote>
                                                      <br>
                                                    </span> What is the
                                                    hostname provided in
                                                    ovirt engine for <span>sjcstorage01


                                                      ? Does this host
                                                      have multiple
                                                      nics?<br>
                                                      <br>
                                                      Could you provide
                                                      output of gluster
                                                      volume info?<br>
                                                      Please note, that
                                                      these errors are
                                                      not related to
                                                      error in creating
                                                      storage domain.
                                                      However, these
                                                      errors could
                                                      prevent you from
                                                      monitoring the
                                                      state of gluster
                                                      volume from oVirt<br>
                                                      <br>
                                                    </span>
                                                    <blockquote
                                                      type="cite"><span>
                                                        <div dir="ltr">
                                                          <div>
                                                          <p><span>2015-09-22


                                                          03:57:29,905
                                                          WARN 
                                                          [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]





                                                          (DefaultQuartzScheduler_Worker-53)



                                                          [] Could not
                                                          associate
                                                          brick
                                                          'sjcstorage02:/export/vmstore/brick01'
                                                          of volume
                                                          '878a316d-2394-4aae-bdf8-e10eea38225e'
                                                          with correct
                                                          network as no
                                                          gluster
                                                          network found
                                                          in cluster
                                                          'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                          <p><span>2015-09-22


                                                          03:57:29,905
                                                          WARN 
                                                          [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]





                                                          (DefaultQuartzScheduler_Worker-53)



                                                          [] Could not
                                                          add brick
                                                          'sjcvhost02:/export/vmstore/brick01'
                                                          to volume
                                                          '878a316d-2394-4aae-bdf8-e10eea38225e'
                                                          - server uuid
                                                          '29b58278-9aa3-47c5-bfb4-1948ef7fdbba'


                                                          not found in
                                                          cluster
                                                          'b00d3c6d-fdfb-49e8-9f1a-f749c3d42486'</span></p>
                                                          <p><span>2015-09-22


                                                          03:57:29,905
                                                          INFO 
                                                          [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
                                                          (DefaultQuartzScheduler_Worker-53)



                                                          [] FINISH,
                                                          GlusterVolumesListVDSCommand,
                                                          return:
                                                          {878a316d-2394-4aae-bdf8-e10eea38225e=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@41e93fb1},





                                                          log id:
                                                          24198fbf</span></p>
                                                          <p><span><br>
                                                          </span></p>
                                                          <p><span>I'm
                                                          new to ovirt
                                                          and gluster,
                                                          so any help
                                                          would be great</span></p>
                                                          <p><span><br>
                                                          </span></p>
                                                          <p><span>thanks</span></p>
                                                          <p><span><br>
                                                          </span></p>
                                                          <p><span>Brett
                                                          Stevens</span></p>
                                                          </div>
                                                        </div>
                                                        <br>
                                                        <fieldset></fieldset>
                                                        <br>
                                                      </span><span>
                                                        <pre>_______________________________________________
Users mailing list
<a moz-do-not-send="true" href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a>
<a moz-do-not-send="true" href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
                                                      </span></blockquote>
                                                    <br>
                                                  </div>
                                                </blockquote>
                                              </div>
                                              <br>
                                            </div>
                                          </blockquote>
                                          <br>
                                        </div>
                                      </blockquote>
                                    </div>
                                    <br>
                                  </div>
                                </blockquote>
                                <br>
                              </div>
                            </blockquote>
                          </div>
                          <br>
                        </div>
                      </blockquote>
                      <br>
                    </div>
                  </blockquote>
                </div>
                <br>
              </div>
            </blockquote>
          </div>
          <br>
        </div>
        <br>
      </div>
      <br>
    </blockquote>
    <br>
  </body>
</html>