Re: [ovirt-users] [Gluster-users] Gluster command [<UNKNOWN>] failed on server...

Hi Albereto, I am using the bonding eth2+eth3=bond0 for gluster communication....and i am not using the NetworkManager.service....as all the interface configured by Ovirt itself.....please let me know how i can achieve this...?? [root@cpu04 ~]# systemctl list-unit-files | grep Network NetworkManager-dispatcher.service disabled NetworkManager-wait-online.service disabled NetworkManager.service disabled [root@cpu04 ~]# cat /usr/lib/systemd/system/glusterd.service [Unit] Description=GlusterFS, a clustered file-system server After=network.target rpcbind.service Wants=network-online.target [Service] Type=forking PIDFile=/var/run/glusterd.pid LimitNOFILE=65536 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid KillMode=process [Install] WantedBy=multi-user.target [root@cpu04 ~]# Thanks, Punit On Wed, Dec 10, 2014 at 6:21 AM, Alberto Larraz < alarraz(a)escoladeltreball.org&gt; wrote: > I post a bug in centos about a problem with > NetworkManager-wait-online.service, this service is executed before > network-online.target > > http://bugs.centos.org/view.php?id=7825 > > I had a problem with mount drbd in a centos7 at boot time. I think it > could be similar to the problem to mount gluster in boot time in centos7. > > I modified the service adding one line that wait to NetworkManager to set > up the ip address in the interface that is used to connect de drbd, the the > ip address in eth0 is 10.88.0.1 and the line I added is: > > ExecStart=/bin/sh -c 'until ip a s eth0 | grep -q 10.88; do usleep > 100000; done' > > > > > # cat /usr/lib/systemd/system/NetworkManager-wait-online.service > > [Unit] > Description=Network Manager Wait Online > Requisite=NetworkManager.service > After=NetworkManager.service > Wants=network.target > Before=network.target network-online.target > > [Service] > Type=oneshot > ExecStart=/usr/bin/nm-online -q --timeout=30 > ExecStart=/bin/sh -c 'until ip a s eth0 | grep -q 10.88; do usleep > 100000; done' > > > > Alberto > > 2014-12-05 14:50 GMT+01:00 Kaushal M <kshlmster(a)gmail.com&gt;: > >> Can you replace 'Before=network-online.target' with >> 'Wants=network-online.target' and try the boot again? This should >> force the network to be online before starting GlusterD. >> >> If even that fails, you could try adding an entry into /etc/hosts with >> the hostname of the system. This should prevent any more failures. >> >> I still don't believe it's a problem with Gluster. Gluster uses apis >> provided by the system to perform name resolution. These definitely >> work correctly because you can start GlusterD later. Since the >> resolution failure only happens during boot, it points to system or >> network setup issues during boot. To me it seems like the network >> isn't completely setup at that point of time. >> >> ~kaushal >> >> On Fri, Dec 5, 2014 at 12:47 PM, Punit Dambiwal <hypunit(a)gmail.com&gt; >> wrote: >> > Hi Kaushal, >> > >> > It seems it's bug in glusterfs 3.6....even i manage my systemd to >> start the >> > network service before glusterd...but it's still fail... >> > >> > --------------- >> > [Unit] >> > Description=GlusterFS, a clustered file-system server >> > After=network.target rpcbind.service >> > Before=network-online.target >> > >> > [Service] >> > Type=forking >> > PIDFile=/var/run/glusterd.pid >> > LimitNOFILE=65536 >> > ExecStartPre=/etc/rc.d/init.d/network start >> > ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid >> > KillMode=process >> > >> > [Install] >> > WantedBy=multi-user.target >> > ---------------- >> > >> > Thanks, >> > Punit >> > >> > On Wed, Dec 3, 2014 at 8:56 PM, Kaushal M <kshlmster(a)gmail.com&gt; wrote: >> >> >> >> I just remembered this. >> >> >> >> There was another user having a similar issue of GlusterD failing to >> >> start on the mailing list a while back. The cause of his problem was >> >> the way his network was brought up. >> >> IIRC, he was using a static network configuration. The problem >> >> vanished when he began using dhcp. Or it might have been he was using >> >> dhcp.service and it got solved after switching to NetworkManager. >> >> >> >> This could be one more thing you could look at. >> >> >> >> I'll try to find the mail thread to see if it was the same problem as >> you. >> >> >> >> ~kaushal >> >> >> >> On Wed, Dec 3, 2014 at 6:22 PM, Kaushal M <kshlmster(a)gmail.com&gt; >> wrote: >> >> > I don't know much about how the network target is brought up in >> >> > CentOS7, but I'll try as much as I can. >> >> > >> >> > It seems to me that, after the network has been brought up and by >> the >> >> > time GlusterD is started, >> >> > a. The machine hasn't yet recieved it's hostname, or >> >> > b. It hasn't yet registered with the name server. >> >> > >> >> > This is causing name resolution failures. >> >> > >> >> > I don't know if the network target could come up without the machine >> >> > getting its hostname, so I'm pretty sure it's not a. >> >> > >> >> > So it seems to be b. But these kind of signing in happens only in >> DDNS >> >> > systems, which doesn't seem to be the case for you. >> >> > >> >> > Both of these reasons might be wrong (most likely wrong). You'd do >> >> > good if you could ask for help from someone with more experience in >> >> > systemd + networking. >> >> > >> >> > ~kaushal >> >> > >> >> > On Wed, Dec 3, 2014 at 10:54 AM, Punit Dambiwal <hypunit(a)gmail.com&gt; >> >> > wrote: >> >> >> Hi Kaushal, >> >> >> >> >> >> This is the host...which i rebooted...would you mind to let me >> know how >> >> >> i >> >> >> can make the glusterd sevice come up after network...i am using >> >> >> centos7...if >> >> >> network is the issue... >> >> >> >> >> >> On Wed, Dec 3, 2014 at 11:54 AM, Kaushal M <kshlmster(a)gmail.com&gt; >> wrote: >> >> >>> >> >> >>> This peer cannot be identified. >> >> >>> >> >> >>> " [2014-12-03 02:29:25.998153] D >> >> >>> [glusterd-peer-utils.c:121:glusterd_peerinfo_find_by_hostname] >> >> >>> 0-management: >> >> >>> Unable to find friend: cpu05.zne01.hkg1.ovt.36stack.com" >> >> >>> >> >> >>> I don't know why this address is not being resolved during boot >> time. >> >> >>> If >> >> >>> this is a valid peer, the the only reason I can think of this >> that the >> >> >>> network is not up. >> >> >>> >> >> >>> If you had previously detached the peer forcefully, the that could >> >> >>> have >> >> >>> left stale entries in some volumes. In this case as well, GlusterD >> >> >>> will fail >> >> >>> to identify the peer. >> >> >>> >> >> >>> Do either of these reasons seem a possibility to you? >> >> >>> >> >> >>> On Dec 3, 2014 8:07 AM, "Punit Dambiwal" <hypunit(a)gmail.com&gt; >> wrote: >> >> >>>> >> >> >>>> Hi Kaushal, >> >> >>>> >> >> >>>> Please find the logs here :- http://ur1.ca/iyoe5 and >> >> >>>> http://ur1.ca/iyoed >> >> >>>> >> >> >>>> On Tue, Dec 2, 2014 at 10:43 PM, Kaushal M <kshlmster(a)gmail.com&gt; >> >> >>>> wrote: >> >> >>>>> >> >> >>>>> Hey Punit, >> >> >>>>> In the logs you've provided, GlusterD appears to be running >> >> >>>>> correctly. >> >> >>>>> Could you provide the logs for the time period when GlusterD >> >> >>>>> attempts to >> >> >>>>> start but fails. >> >> >>>>> >> >> >>>>> ~kaushal >> >> >>>>> >> >> >>>>> On Dec 2, 2014 8:03 PM, "Punit Dambiwal" <hypunit(a)gmail.com&gt; >> wrote: >> >> >>>>>> >> >> >>>>>> Hi Kaushal, >> >> >>>>>> >> >> >>>>>> Please find the logs here :- http://ur1.ca/iyhs5 and >> >> >>>>>> http://ur1.ca/iyhue >> >> >>>>>> >> >> >>>>>> Thanks, >> >> >>>>>> punit >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> On Tue, Dec 2, 2014 at 12:00 PM, Kaushal M < >> kshlmster(a)gmail.com&gt; >> >> >>>>>> wrote: >> >> >>>>>>> >> >> >>>>>>> Hey Punit, >> >> >>>>>>> Could you start Glusterd in debug mode and provide the logs >> here? >> >> >>>>>>> To start it in debug mode, append '-LDEBUG' to the ExecStart >> line >> >> >>>>>>> in >> >> >>>>>>> the service file. >> >> >>>>>>> >> >> >>>>>>> ~kaushal >> >> >>>>>>> >> >> >>>>>>> On Mon, Dec 1, 2014 at 9:05 AM, Punit Dambiwal < >> hypunit(a)gmail.com&gt; >> >> >>>>>>> wrote: >> >> >>>>>>> > Hi, >> >> >>>>>>> > >> >> >>>>>>> > Can Any body help me on this ?? >> >> >>>>>>> > >> >> >>>>>>> > On Thu, Nov 27, 2014 at 9:29 AM, Punit Dambiwal >> >> >>>>>>> > <hypunit(a)gmail.com&gt; >> >> >>>>>>> > wrote: >> >> >>>>>>> >> >> >> >>>>>>> >> Hi Kaushal, >> >> >>>>>>> >> >> >> >>>>>>> >> Thanks for the detailed reply....let me explain my setup >> first >> >> >>>>>>> >> :- >> >> >>>>>>> >> >> >> >>>>>>> >> 1. Ovirt Engine >> >> >>>>>>> >> 2. 4* host as well as storage machine (Host and gluster >> >> >>>>>>> >> combined) >> >> >>>>>>> >> 3. Every host has 24 bricks... >> >> >>>>>>> >> >> >> >>>>>>> >> Now whenever the host machine reboot...it can come up but >> can >> >> >>>>>>> >> not >> >> >>>>>>> >> join the >> >> >>>>>>> >> cluster again and through the following error "Gluster >> command >> >> >>>>>>> >> [<UNKNOWN>] >> >> >>>>>>> >> failed on server.." >> >> >>>>>>> >> >> >> >>>>>>> >> Please check my comment in line :- >> >> >>>>>>> >> >> >> >>>>>>> >> 1. Use the same string for doing the peer probe and for the >> >> >>>>>>> >> brick >> >> >>>>>>> >> address >> >> >>>>>>> >> during volume create/add-brick. Ideally, we suggest you use >> >> >>>>>>> >> properly >> >> >>>>>>> >> resolvable FQDNs everywhere. If that is not possible, then >> use >> >> >>>>>>> >> only >> >> >>>>>>> >> IP >> >> >>>>>>> >> addresses. Try to avoid short names. >> >> >>>>>>> >> --------------- >> >> >>>>>>> >> [root@cpu05 ~]# gluster peer status >> >> >>>>>>> >> Number of Peers: 3 >> >> >>>>>>> >> >> >> >>>>>>> >> Hostname: cpu03.stack.com >> >> >>>>>>> >> Uuid: 5729b8c4-e80d-4353-b456-6f467bddbdfb >> >> >>>>>>> >> State: Peer in Cluster (Connected) >> >> >>>>>>> >> >> >> >>>>>>> >> Hostname: cpu04.stack.com >> >> >>>>>>> >> Uuid: d272b790-c4b2-4bed-ba68-793656e6d7b0 >> >> >>>>>>> >> State: Peer in Cluster (Connected) >> >> >>>>>>> >> Other names: >> >> >>>>>>> >> 10.10.0.8 >> >> >>>>>>> >> >> >> >>>>>>> >> Hostname: cpu02.stack.com >> >> >>>>>>> >> Uuid: 8d8a7041-950e-40d0-85f9-58d14340ca25 >> >> >>>>>>> >> State: Peer in Cluster (Connected) >> >> >>>>>>> >> [root@cpu05 ~]# >> >> >>>>>>> >> ---------------- >> >> >>>>>>> >> 2. During boot up, make sure to launch glusterd only after >> the >> >> >>>>>>> >> network is >> >> >>>>>>> >> up. This will allow the new peer identification mechanism >> to do >> >> >>>>>>> >> its >> >> >>>>>>> >> job correctly. >> >> >>>>>>> >> >> I think the service itself doing the same job.... >> >> >>>>>>> >> >> >> >>>>>>> >> [root@cpu05 ~]# cat >> /usr/lib/systemd/system/glusterd.service >> >> >>>>>>> >> [Unit] >> >> >>>>>>> >> Description=GlusterFS, a clustered file-system server >> >> >>>>>>> >> After=network.target rpcbind.service >> >> >>>>>>> >> Before=network-online.target >> >> >>>>>>> >> >> >> >>>>>>> >> [Service] >> >> >>>>>>> >> Type=forking >> >> >>>>>>> >> PIDFile=/var/run/glusterd.pid >> >> >>>>>>> >> LimitNOFILE=65536 >> >> >>>>>>> >> ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid >> >> >>>>>>> >> KillMode=process >> >> >>>>>>> >> >> >> >>>>>>> >> [Install] >> >> >>>>>>> >> WantedBy=multi-user.target >> >> >>>>>>> >> [root@cpu05 ~]# >> >> >>>>>>> >> -------------------- >> >> >>>>>>> >> >> >> >>>>>>> >> gluster logs :- >> >> >>>>>>> >> >> >> >>>>>>> >> [2014-11-24 09:22:22.147471] I [MSGID: 100030] >> >> >>>>>>> >> [glusterfsd.c:2018:main] >> >> >>>>>>> >> 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd >> >> >>>>>>> >> version >> >> >>>>>>> >> 3.6.1 >> >> >>>>>>> >> (args: /usr/sbin/glusterd -p /var/run/glusterd.pid) >> >> >>>>>>> >> [2014-11-24 09:22:22.151565] I [glusterd.c:1214:init] >> >> >>>>>>> >> 0-management: >> >> >>>>>>> >> Maximum allowed open file descriptors set to 65536 >> >> >>>>>>> >> [2014-11-24 09:22:22.151599] I [glusterd.c:1259:init] >> >> >>>>>>> >> 0-management: >> >> >>>>>>> >> Using >> >> >>>>>>> >> /var/lib/glusterd as working directory >> >> >>>>>>> >> [2014-11-24 09:22:22.155216] W >> >> >>>>>>> >> [rdma.c:4195:__gf_rdma_ctx_create] >> >> >>>>>>> >> 0-rpc-transport/rdma: rdma_cm event channel creation >> failed (No >> >> >>>>>>> >> such device) >> >> >>>>>>> >> [2014-11-24 09:22:22.155264] E [rdma.c:4483:init] >> >> >>>>>>> >> 0-rdma.management: >> >> >>>>>>> >> Failed to initialize IB Device >> >> >>>>>>> >> [2014-11-24 09:22:22.155285] E >> >> >>>>>>> >> [rpc-transport.c:333:rpc_transport_load] >> >> >>>>>>> >> 0-rpc-transport: 'rdma' initialization failed >> >> >>>>>>> >> [2014-11-24 09:22:22.155354] W >> >> >>>>>>> >> [rpcsvc.c:1524:rpcsvc_transport_create] >> >> >>>>>>> >> 0-rpc-service: cannot create listener, initing the >> transport >> >> >>>>>>> >> failed >> >> >>>>>>> >> [2014-11-24 09:22:22.156290] I >> >> >>>>>>> >> [glusterd.c:413:glusterd_check_gsync_present] 0-glusterd: >> >> >>>>>>> >> geo-replication >> >> >>>>>>> >> module not installed in the system >> >> >>>>>>> >> [2014-11-24 09:22:22.161318] I >> >> >>>>>>> >> [glusterd-store.c:2043:glusterd_restore_op_version] >> 0-glusterd: >> >> >>>>>>> >> retrieved >> >> >>>>>>> >> op-version: 30600 >> >> >>>>>>> >> [2014-11-24 09:22:22.821800] I >> >> >>>>>>> >> [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] >> >> >>>>>>> >> 0-management: >> >> >>>>>>> >> connect returned 0 >> >> >>>>>>> >> [2014-11-24 09:22:22.825810] I >> >> >>>>>>> >> [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] >> >> >>>>>>> >> 0-management: >> >> >>>>>>> >> connect returned 0 >> >> >>>>>>> >> [2014-11-24 09:22:22.828705] I >> >> >>>>>>> >> [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] >> >> >>>>>>> >> 0-management: >> >> >>>>>>> >> connect returned 0 >> >> >>>>>>> >> [2014-11-24 09:22:22.828771] I >> >> >>>>>>> >> [rpc-clnt.c:969:rpc_clnt_connection_init] >> >> >>>>>>> >> 0-management: setting frame-timeout to 600 >> >> >>>>>>> >> [2014-11-24 09:22:22.832670] I >> >> >>>>>>> >> [rpc-clnt.c:969:rpc_clnt_connection_init] >> >> >>>>>>> >> 0-management: setting frame-timeout to 600 >> >> >>>>>>> >> [2014-11-24 09:22:22.835919] I >> >> >>>>>>> >> [rpc-clnt.c:969:rpc_clnt_connection_init] >> >> >>>>>>> >> 0-management: setting frame-timeout to 600 >> >> >>>>>>> >> [2014-11-24 09:22:22.840209] E >> >> >>>>>>> >> [glusterd-store.c:4248:glusterd_resolve_all_bricks] >> 0-glusterd: >> >> >>>>>>> >> resolve >> >> >>>>>>> >> brick failed in restore >> >> >>>>>>> >> [2014-11-24 09:22:22.840233] E [xlator.c:425:xlator_init] >> >> >>>>>>> >> 0-management: >> >> >>>>>>> >> Initialization of volume 'management' failed, review your >> >> >>>>>>> >> volfile >> >> >>>>>>> >> again >> >> >>>>>>> >> [2014-11-24 09:22:22.840245] E >> >> >>>>>>> >> [graph.c:322:glusterfs_graph_init] >> >> >>>>>>> >> 0-management: initializing translator failed >> >> >>>>>>> >> [2014-11-24 09:22:22.840264] E >> >> >>>>>>> >> [graph.c:525:glusterfs_graph_activate] >> >> >>>>>>> >> 0-graph: init failed >> >> >>>>>>> >> [2014-11-24 09:22:22.840754] W >> >> >>>>>>> >> [glusterfsd.c:1194:cleanup_and_exit] >> >> >>>>>>> >> (--> >> >> >>>>>>> >> 0-: received signum (0), shutting down >> >> >>>>>>> >> >> >> >>>>>>> >> Thanks, >> >> >>>>>>> >> Punit >> >> >>>>>>> >> >> >> >>>>>>> >> >> >> >>>>>>> >> >> >> >>>>>>> >> >> >> >>>>>>> >> On Wed, Nov 26, 2014 at 7:14 PM, Kaushal M >> >> >>>>>>> >> <kshlmster(a)gmail.com&gt; >> >> >>>>>>> >> wrote: >> >> >>>>>>> >>> >> >> >>>>>>> >>> Based on the logs I can guess that glusterd is being >> started >> >> >>>>>>> >>> before >> >> >>>>>>> >>> the network has come up and that the addresses given to >> bricks >> >> >>>>>>> >>> do >> >> >>>>>>> >>> not >> >> >>>>>>> >>> directly match the addresses used in during peer probe. >> >> >>>>>>> >>> >> >> >>>>>>> >>> The gluster_after_reboot log has the line "[2014-11-25 >> >> >>>>>>> >>> 06:46:09.972113] E >> >> >>>>>>> >>> [glusterd-store.c:2632:glusterd_resolve_all_bricks] >> >> >>>>>>> >>> 0-glusterd: resolve brick failed in restore". >> >> >>>>>>> >>> >> >> >>>>>>> >>> Brick resolution fails when glusterd cannot match the >> address >> >> >>>>>>> >>> for >> >> >>>>>>> >>> the >> >> >>>>>>> >>> brick, with one of the peers. Brick resolution happens in >> two >> >> >>>>>>> >>> phases, >> >> >>>>>>> >>> 1. We first try to identify the peer by performing string >> >> >>>>>>> >>> comparisions >> >> >>>>>>> >>> with the brick address and the peer addresses (The peer >> names >> >> >>>>>>> >>> will >> >> >>>>>>> >>> be >> >> >>>>>>> >>> the names/addresses that were given when the peer was >> probed). >> >> >>>>>>> >>> 2. If we don't find a match from step 1, we will then >> resolve >> >> >>>>>>> >>> all >> >> >>>>>>> >>> the >> >> >>>>>>> >>> brick address and the peer addresses into addrinfo >> structs, >> >> >>>>>>> >>> and >> >> >>>>>>> >>> then >> >> >>>>>>> >>> compare these structs to find a match. This process should >> >> >>>>>>> >>> generally >> >> >>>>>>> >>> find a match if available. This will fail only if the >> network >> >> >>>>>>> >>> is >> >> >>>>>>> >>> not >> >> >>>>>>> >>> up yet as we cannot resolve addresses. >> >> >>>>>>> >>> >> >> >>>>>>> >>> The above steps are applicable only to glusterfs versions >> >> >>>>>>> >>> >=3.6. >> >> >>>>>>> >>> They >> >> >>>>>>> >>> were introduced to reduce problems with peer >> identification, >> >> >>>>>>> >>> like >> >> >>>>>>> >>> the >> >> >>>>>>> >>> one you encountered >> >> >>>>>>> >>> >> >> >>>>>>> >>> Since both of the steps failed to find a match in one >> run, but >> >> >>>>>>> >>> succeeded later, we can come to the conclusion that, >> >> >>>>>>> >>> a) the bricks don't have the exact same string used in >> peer >> >> >>>>>>> >>> probe >> >> >>>>>>> >>> for >> >> >>>>>>> >>> their addresses as step 1 failed, and >> >> >>>>>>> >>> b) the network was not up in the initial run, as step 2 >> failed >> >> >>>>>>> >>> during >> >> >>>>>>> >>> the initial run, but passed in the second run. >> >> >>>>>>> >>> >> >> >>>>>>> >>> Please let me know if my conclusion is correct. >> >> >>>>>>> >>> >> >> >>>>>>> >>> If it is, you can solve your problem in two ways. >> >> >>>>>>> >>> 1. Use the same string for doing the peer probe and for >> the >> >> >>>>>>> >>> brick >> >> >>>>>>> >>> address during volume create/add-brick. Ideally, we >> suggest >> >> >>>>>>> >>> you >> >> >>>>>>> >>> use >> >> >>>>>>> >>> properly resolvable FQDNs everywhere. If that is not >> possible, >> >> >>>>>>> >>> then >> >> >>>>>>> >>> use only IP addresses. Try to avoid short names. >> >> >>>>>>> >>> 2. During boot up, make sure to launch glusterd only >> after the >> >> >>>>>>> >>> network >> >> >>>>>>> >>> is up. This will allow the new peer identification >> mechanism >> >> >>>>>>> >>> to do >> >> >>>>>>> >>> its >> >> >>>>>>> >>> job correctly. >> >> >>>>>>> >>> >> >> >>>>>>> >>> >> >> >>>>>>> >>> If you have already followed these steps and yet still >> hit the >> >> >>>>>>> >>> problem, then please provide more information (setup, >> logs, >> >> >>>>>>> >>> etc.). >> >> >>>>>>> >>> It >> >> >>>>>>> >>> could be much different problem that you are facing. >> >> >>>>>>> >>> >> >> >>>>>>> >>> ~kaushal >> >> >>>>>>> >>> >> >> >>>>>>> >>> On Wed, Nov 26, 2014 at 4:01 PM, Punit Dambiwal >> >> >>>>>>> >>> <hypunit(a)gmail.com&gt; >> >> >>>>>>> >>> wrote: >> >> >>>>>>> >>> > Is there any one can help on this ?? >> >> >>>>>>> >>> > >> >> >>>>>>> >>> > Thanks, >> >> >>>>>>> >>> > punit >> >> >>>>>>> >>> > >> >> >>>>>>> >>> > On Wed, Nov 26, 2014 at 9:42 AM, Punit Dambiwal >> >> >>>>>>> >>> > <hypunit(a)gmail.com&gt; >> >> >>>>>>> >>> > wrote: >> >> >>>>>>> >>> >> >> >> >>>>>>> >>> >> Hi, >> >> >>>>>>> >>> >> >> >> >>>>>>> >>> >> My Glusterfs version is :- glusterfs-3.6.1-1.el7 >> >> >>>>>>> >>> >> >> >> >>>>>>> >>> >> On Wed, Nov 26, 2014 at 1:59 AM, Kanagaraj Mayilsamy >> >> >>>>>>> >>> >> <kmayilsa(a)redhat.com&gt; >> >> >>>>>>> >>> >> wrote: >> >> >>>>>>> >>> >>> >> >> >>>>>>> >>> >>> [+Gluster-users(a)gluster.org] >> >> >>>>>>> >>> >>> >> >> >>>>>>> >>> >>> "Initialization of volume 'management' failed, review >> your >> >> >>>>>>> >>> >>> volfile >> >> >>>>>>> >>> >>> again", glusterd throws this error when the service is >> >> >>>>>>> >>> >>> started >> >> >>>>>>> >>> >>> automatically >> >> >>>>>>> >>> >>> after the reboot. But the service is successfully >> started >> >> >>>>>>> >>> >>> later >> >> >>>>>>> >>> >>> manually by >> >> >>>>>>> >>> >>> the user. >> >> >>>>>>> >>> >>> >> >> >>>>>>> >>> >>> can somebody from gluster-users please help on this? >> >> >>>>>>> >>> >>> >> >> >>>>>>> >>> >>> glusterfs version: 3.5.1 >> >> >>>>>>> >>> >>> >> >> >>>>>>> >>> >>> Thanks, >> >> >>>>>>> >>> >>> Kanagaraj >> >> >>>>>>> >>> >>> >> >> >>>>>>> >>> >>> ----- Original Message ----- >> >> >>>>>>> >>> >>> > From: "Punit Dambiwal" <hypunit(a)gmail.com&gt; >> >> >>>>>>> >>> >>> > To: "Kanagaraj" <kmayilsa(a)redhat.com&gt; >> >> >>>>>>> >>> >>> > Cc: users(a)ovirt.org >> >> >>>>>>> >>> >>> > Sent: Tuesday, November 25, 2014 7:24:45 PM >> >> >>>>>>> >>> >>> > Subject: Re: [ovirt-users] Gluster command >> [<UNKNOWN>] >> >> >>>>>>> >>> >>> > failed on >> >> >>>>>>> >>> >>> > server... >> >> >>>>>>> >>> >>> > >> >> >>>>>>> >>> >>> > Hi Kanagraj, >> >> >>>>>>> >>> >>> > >> >> >>>>>>> >>> >>> > Please check the attached log files....i didn't >> find any >> >> >>>>>>> >>> >>> > thing >> >> >>>>>>> >>> >>> > special.... >> >> >>>>>>> >>> >>> > >> >> >>>>>>> >>> >>> > On Tue, Nov 25, 2014 at 12:12 PM, Kanagaraj >> >> >>>>>>> >>> >>> > <kmayilsa(a)redhat.com&gt; >> >> >>>>>>> >>> >>> > wrote: >> >> >>>>>>> >>> >>> > >> >> >>>>>>> >>> >>> > > Do you see any errors in >> >> >>>>>>> >>> >>> > > /var/log/glusterfs/etc-glusterfs-glusterd.vol.log >> or >> >> >>>>>>> >>> >>> > > vdsm.log >> >> >>>>>>> >>> >>> > > when >> >> >>>>>>> >>> >>> > > the >> >> >>>>>>> >>> >>> > > service is trying to start automatically after the >> >> >>>>>>> >>> >>> > > reboot? >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > Thanks, >> >> >>>>>>> >>> >>> > > Kanagaraj >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > On 11/24/2014 08:13 PM, Punit Dambiwal wrote: >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > Hi Kanagaraj, >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > Yes...once i will start the gluster service and >> then >> >> >>>>>>> >>> >>> > > vdsmd >> >> >>>>>>> >>> >>> > > ...the >> >> >>>>>>> >>> >>> > > host >> >> >>>>>>> >>> >>> > > can connect to cluster...but the question is why >> it's >> >> >>>>>>> >>> >>> > > not >> >> >>>>>>> >>> >>> > > started >> >> >>>>>>> >>> >>> > > even it >> >> >>>>>>> >>> >>> > > has chkconfig enabled... >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > I have tested it in two host cluster >> >> >>>>>>> >>> >>> > > environment...(Centos 6.6 >> >> >>>>>>> >>> >>> > > and >> >> >>>>>>> >>> >>> > > centos 7.0) on both hypervisior cluster..it's >> failed >> >> >>>>>>> >>> >>> > > to >> >> >>>>>>> >>> >>> > > reconnect >> >> >>>>>>> >>> >>> > > in >> >> >>>>>>> >>> >>> > > to >> >> >>>>>>> >>> >>> > > cluster after reboot.... >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > In both the environment glusterd enabled for next >> >> >>>>>>> >>> >>> > > boot....but >> >> >>>>>>> >>> >>> > > it's >> >> >>>>>>> >>> >>> > > failed with the same error....seems it's bug in >> either >> >> >>>>>>> >>> >>> > > gluster or >> >> >>>>>>> >>> >>> > > Ovirt ?? >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > Please help me to find the workaround here if >> can not >> >> >>>>>>> >>> >>> > > resolve >> >> >>>>>>> >>> >>> > > it...as >> >> >>>>>>> >>> >>> > > without this the Host machine can not connect >> after >> >> >>>>>>> >>> >>> > > reboot....that >> >> >>>>>>> >>> >>> > > means >> >> >>>>>>> >>> >>> > > engine will consider it as down and every time >> need to >> >> >>>>>>> >>> >>> > > manually >> >> >>>>>>> >>> >>> > > start >> >> >>>>>>> >>> >>> > > the >> >> >>>>>>> >>> >>> > > gluster service and vdsmd... ?? >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > Thanks, >> >> >>>>>>> >>> >>> > > Punit >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > > On Mon, Nov 24, 2014 at 10:20 PM, Kanagaraj >> >> >>>>>>> >>> >>> > > <kmayilsa(a)redhat.com&gt; >> >> >>>>>>> >>> >>> > > wrote: >> >> >>>>>>> >>> >>> > > >> >> >>>>>>> >>> >>> > >> From vdsm.log "error: Connection failed. Please >> >> >>>>>>> >>> >>> > >> check if >> >> >>>>>>> >>> >>> > >> gluster >> >> >>>>>>> >>> >>> > >> daemon >> >> >>>>>>> >>> >>> > >> is operational." >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> Starting glusterd service should fix this issue. >> >> >>>>>>> >>> >>> > >> 'service >> >> >>>>>>> >>> >>> > >> glusterd >> >> >>>>>>> >>> >>> > >> start' >> >> >>>>>>> >>> >>> > >> But i am wondering why the glusterd was not >> started >> >> >>>>>>> >>> >>> > >> automatically >> >> >>>>>>> >>> >>> > >> after >> >> >>>>>>> >>> >>> > >> the reboot. >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> Thanks, >> >> >>>>>>> >>> >>> > >> Kanagaraj >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> On 11/24/2014 07:18 PM, Punit Dambiwal wrote: >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> Hi Kanagaraj, >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> Please find the attached VDSM logs :- >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> ---------------- >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> 21:41:17,182::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) >> >> >>>>>>> >>> >>> > >> Owner.cancelAll requests {} >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> 21:41:17,182::task::993::Storage.TaskManager.Task::(_decref) >> >> >>>>>>> >>> >>> > >> Task=`1691d409-9b27-4585-8281-5ec26154367a`::ref >> 0 >> >> >>>>>>> >>> >>> > >> aborting >> >> >>>>>>> >>> >>> > >> False >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> 21:41:32,393::task::595::Storage.TaskManager.Task::(_updateState) >> >> >>>>>>> >>> >>> > >> >> Task=`994c7bc3-a236-4d03-a732-e068c7ed9ed4`::moving >> >> >>>>>>> >>> >>> > >> from >> >> >>>>>>> >>> >>> > >> state >> >> >>>>>>> >>> >>> > >> init >> >> >>>>>>> >>> >>> > >> -> >> >> >>>>>>> >>> >>> > >> state preparing >> >> >>>>>>> >>> >>> > >> Thread-13::INFO::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> 21:41:32,393::logUtils::44::dispatcher::(wrapper) Run >> >> >>>>>>> >>> >>> > >> and >> >> >>>>>>> >>> >>> > >> protect: >> >> >>>>>>> >>> >>> > >> repoStats(options=None) >> >> >>>>>>> >>> >>> > >> Thread-13::INFO::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> 21:41:32,393::logUtils::47::dispatcher::(wrapper) Run >> >> >>>>>>> >>> >>> > >> and >> >> >>>>>>> >>> >>> > >> protect: >> >> >>>>>>> >>> >>> > >> repoStats, Return response: {} >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> 21:41:32,393::task::1191::Storage.TaskManager.Task::(prepare) >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> Task=`994c7bc3-a236-4d03-a732-e068c7ed9ed4`::finished: {} >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> 21:41:32,394::task::595::Storage.TaskManager.Task::(_updateState) >> >> >>>>>>> >>> >>> > >> >> Task=`994c7bc3-a236-4d03-a732-e068c7ed9ed4`::moving >> >> >>>>>>> >>> >>> > >> from >> >> >>>>>>> >>> >>> > >> state >> >> >>>>>>> >>> >>> > >> preparing >> >> >>>>>>> >>> >>> > >> -> >> >> >>>>>>> >>> >>> > >> state finished >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> 21:41:32,394::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) >> >> >>>>>>> >>> >>> > >> Owner.releaseAll requests {} resources {} >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> 21:41:32,394::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) >> >> >>>>>>> >>> >>> > >> Owner.cancelAll requests {} >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> 21:41:32,394::task::993::Storage.TaskManager.Task::(_decref) >> >> >>>>>>> >>> >>> > >> Task=`994c7bc3-a236-4d03-a732-e068c7ed9ed4`::ref >> 0 >> >> >>>>>>> >>> >>> > >> aborting >> >> >>>>>>> >>> >>> > >> False >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,550::BindingXMLRPC::1132::vds::(wrapper) >> >> >>>>>>> >>> >>> > >> client >> >> >>>>>>> >>> >>> > >> [10.10.10.2]::call >> >> >>>>>>> >>> >>> > >> getCapabilities with () {} >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,553::utils::738::root::(execCmd) >> >> >>>>>>> >>> >>> > >> /sbin/ip route show to 0.0.0.0/0 table all (cwd >> None) >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,560::utils::758::root::(execCmd) >> >> >>>>>>> >>> >>> > >> SUCCESS: <err> = ''; <rc> = 0 >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,588::caps::728::root::(_getKeyPackages) >> rpm >> >> >>>>>>> >>> >>> > >> package >> >> >>>>>>> >>> >>> > >> ('gluster-swift',) not found >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,592::caps::728::root::(_getKeyPackages) >> rpm >> >> >>>>>>> >>> >>> > >> package >> >> >>>>>>> >>> >>> > >> ('gluster-swift-object',) not found >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,593::caps::728::root::(_getKeyPackages) >> rpm >> >> >>>>>>> >>> >>> > >> package >> >> >>>>>>> >>> >>> > >> ('gluster-swift-plugin',) not found >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,598::caps::728::root::(_getKeyPackages) >> rpm >> >> >>>>>>> >>> >>> > >> package >> >> >>>>>>> >>> >>> > >> ('gluster-swift-account',) not found >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,598::caps::728::root::(_getKeyPackages) >> rpm >> >> >>>>>>> >>> >>> > >> package >> >> >>>>>>> >>> >>> > >> ('gluster-swift-proxy',) not found >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,598::caps::728::root::(_getKeyPackages) >> rpm >> >> >>>>>>> >>> >>> > >> package >> >> >>>>>>> >>> >>> > >> ('gluster-swift-doc',) not found >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,599::caps::728::root::(_getKeyPackages) >> rpm >> >> >>>>>>> >>> >>> > >> package >> >> >>>>>>> >>> >>> > >> ('gluster-swift-container',) not found >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,599::caps::728::root::(_getKeyPackages) >> rpm >> >> >>>>>>> >>> >>> > >> package >> >> >>>>>>> >>> >>> > >> ('glusterfs-geo-replication',) not found >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,600::caps::646::root::(get) >> >> >>>>>>> >>> >>> > >> VirtioRNG DISABLED: libvirt version >> 0.10.2-29.el6_5.9 >> >> >>>>>>> >>> >>> > >> required >> >> >>>>>>> >>> >>> > >> >= >> >> >>>>>>> >>> >>> > >> 0.10.2-31 >> >> >>>>>>> >>> >>> > >> Thread-13::DEBUG::2014-11-24 >> >> >>>>>>> >>> >>> > >> 21:41:41,603::BindingXMLRPC::1139::vds::(wrapper) >> >> >>>>>>> >>> >>> > >> return >> >> >>>>>>> >>> >>> > >> getCapabilities >> >> >>>>>>> >>> >>> > >> with {'status': {'message': 'Done', 'code': 0}, >> >> >>>>>>> >>> >>> > >> 'info': >> >> >>>>>>> >>> >>> > >> {'HBAInventory': >> >> >>>>>>> >>> >>> > >> {'iSCSI': [{'InitiatorName': >> >> >>>>>>> >>> >>> > >> 'iqn.1994-05.com.redhat:32151ce183c8'}], >> >> >>>>>>> >>> >>> > >> 'FC': >> >> >>>>>>> >>> >>> > >> []}, 'packages2': {'kernel': {'release': >> >> >>>>>>> >>> >>> > >> '431.el6.x86_64', >> >> >>>>>>> >>> >>> > >> 'buildtime': >> >> >>>>>>> >>> >>> > >> 1385061309.0, 'version': '2.6.32'}, >> 'glusterfs-rdma': >> >> >>>>>>> >>> >>> > >> {'release': >> >> >>>>>>> >>> >>> > >> '1.el6', >> >> >>>>>>> >>> >>> > >> 'buildtime': 1403622628L, 'version': '3.5.1'}, >> >> >>>>>>> >>> >>> > >> 'glusterfs-fuse': >> >> >>>>>>> >>> >>> > >> {'release': '1.el6', 'buildtime': 1403622628L, >> >> >>>>>>> >>> >>> > >> 'version': >> >> >>>>>>> >>> >>> > >> '3.5.1'}, >> >> >>>>>>> >>> >>> > >> 'spice-server': {'release': '6.el6_5.2', >> 'buildtime': >> >> >>>>>>> >>> >>> > >> 1402324637L, >> >> >>>>>>> >>> >>> > >> 'version': '0.12.4'}, 'vdsm': {'release': >> >> >>>>>>> >>> >>> > >> '1.gitdb83943.el6', >> >> >>>>>>> >>> >>> > >> 'buildtime': >> >> >>>>>>> >>> >>> > >> 1412784567L, 'version': '4.16.7'}, 'qemu-kvm': >> >> >>>>>>> >>> >>> > >> {'release': >> >> >>>>>>> >>> >>> > >> '2.415.el6_5.10', 'buildtime': 1402435700L, >> >> >>>>>>> >>> >>> > >> 'version': >> >> >>>>>>> >>> >>> > >> '0.12.1.2'}, >> >> >>>>>>> >>> >>> > >> 'qemu-img': {'release': '2.415.el6_5.10', >> >> >>>>>>> >>> >>> > >> 'buildtime': >> >> >>>>>>> >>> >>> > >> 1402435700L, >> >> >>>>>>> >>> >>> > >> 'version': '0.12.1.2'}, 'libvirt': {'release': >> >> >>>>>>> >>> >>> > >> '29.el6_5.9', >> >> >>>>>>> >>> >>> > >> 'buildtime': >> >> >>>>>>> >>> >>> > >> 1402404612L, 'version': '0.10.2'}, 'glusterfs': >> >> >>>>>>> >>> >>> > >> {'release': >> >> >>>>>>> >>> >>> > >> '1.el6', >> >> >>>>>>> >>> >>> > >> 'buildtime': 1403622628L, 'version': '3.5.1'}, >> 'mom': >> >> >>>>>>> >>> >>> > >> {'release': >> >> >>>>>>> >>> >>> > >> '2.el6', >> >> >>>>>>> >>> >>> > >> 'buildtime': 1403794344L, 'version': '0.4.1'}, >> >> >>>>>>> >>> >>> > >> 'glusterfs-server': >> >> >>>>>>> >>> >>> > >> {'release': '1.el6', 'buildtime': 1403622628L, >> >> >>>>>>> >>> >>> > >> 'version': >> >> >>>>>>> >>> >>> > >> '3.5.1'}}, >> >> >>>>>>> >>> >>> > >> 'numaNodeDistance': {'1': [20, 10], '0': [10, >> 20]}, >> >> >>>>>>> >>> >>> > >> 'cpuModel': >> >> >>>>>>> >>> >>> > >> 'Intel(R) >> >> >>>>>>> >>> >>> > >> Xeon(R) CPU X5650 @ 2.67GHz', >> 'liveMerge': >> >> >>>>>>> >>> >>> > >> 'false', >> >> >>>>>>> >>> >>> > >> 'hooks': >> >> >>>>>>> >>> >>> > >> {}, >> >> >>>>>>> >>> >>> > >> 'cpuSockets': '2', 'vmTypes': ['kvm'], 'selinux': >> >> >>>>>>> >>> >>> > >> {'mode': '1'}, >> >> >>>>>>> >>> >>> > >> 'kdumpStatus': 0, 'supportedProtocols': ['2.2', >> >> >>>>>>> >>> >>> > >> '2.3'], >> >> >>>>>>> >>> >>> > >> 'networks': >> >> >>>>>>> >>> >>> > >> {'ovirtmgmt': {'iface': u'bond0.10', 'addr': >> >> >>>>>>> >>> >>> > >> '43.252.176.16', >> >> >>>>>>> >>> >>> > >> 'bridged': >> >> >>>>>>> >>> >>> > >> False, 'ipv6addrs': >> ['fe80::62eb:69ff:fe20:b46c/64'], >> >> >>>>>>> >>> >>> > >> 'mtu': >> >> >>>>>>> >>> >>> > >> '1500', >> >> >>>>>>> >>> >>> > >> 'bootproto4': 'none', 'netmask': '255.255.255.0', >> >> >>>>>>> >>> >>> > >> 'ipv4addrs': >> >> >>>>>>> >>> >>> > >> [' >> >> >>>>>>> >>> >>> > >> 43.252.176.16/24' <http://43.252.176.16/24%27>], >> >> >>>>>>> >>> >>> > >> 'interface': >> >> >>>>>>> >>> >>> > >> u'bond0.10', 'ipv6gateway': '::', 'gateway': >> >> >>>>>>> >>> >>> > >> '43.25.17.1'}, >> >> >>>>>>> >>> >>> > >> 'Internal': >> >> >>>>>>> >>> >>> > >> {'iface': 'Internal', 'addr': '', 'cfg': >> {'DEFROUTE': >> >> >>>>>>> >>> >>> > >> 'no', >> >> >>>>>>> >>> >>> > >> 'HOTPLUG': >> >> >>>>>>> >>> >>> > >> 'no', 'MTU': '9000', 'DELAY': '0', >> 'NM_CONTROLLED': >> >> >>>>>>> >>> >>> > >> 'no', >> >> >>>>>>> >>> >>> > >> 'BOOTPROTO': >> >> >>>>>>> >>> >>> > >> 'none', 'STP': 'off', 'DEVICE': 'Internal', >> 'TYPE': >> >> >>>>>>> >>> >>> > >> 'Bridge', >> >> >>>>>>> >>> >>> > >> 'ONBOOT': >> >> >>>>>>> >>> >>> > >> 'no'}, 'bridged': True, 'ipv6addrs': >> >> >>>>>>> >>> >>> > >> ['fe80::210:18ff:fecd:daac/64'], >> >> >>>>>>> >>> >>> > >> 'gateway': '', 'bootproto4': 'none', 'netmask': >> '', >> >> >>>>>>> >>> >>> > >> 'stp': >> >> >>>>>>> >>> >>> > >> 'off', >> >> >>>>>>> >>> >>> > >> 'ipv4addrs': [], 'mtu': '9000', 'ipv6gateway': >> '::', >> >> >>>>>>> >>> >>> > >> 'ports': >> >> >>>>>>> >>> >>> > >> ['bond1.100']}, 'storage': {'iface': u'bond1', >> >> >>>>>>> >>> >>> > >> 'addr': >> >> >>>>>>> >>> >>> > >> '10.10.10.6', >> >> >>>>>>> >>> >>> > >> 'bridged': False, 'ipv6addrs': >> >> >>>>>>> >>> >>> > >> ['fe80::210:18ff:fecd:daac/64'], >> >> >>>>>>> >>> >>> > >> 'mtu': >> >> >>>>>>> >>> >>> > >> '9000', 'bootproto4': 'none', 'netmask': >> >> >>>>>>> >>> >>> > >> '255.255.255.0', >> >> >>>>>>> >>> >>> > >> 'ipv4addrs': [' >> >> >>>>>>> >>> >>> > >> 10.10.10.6/24' <http://10.10.10.6/24%27>], >> >> >>>>>>> >>> >>> > >> 'interface': >> >> >>>>>>> >>> >>> > >> u'bond1', >> >> >>>>>>> >>> >>> > >> 'ipv6gateway': '::', 'gateway': ''}, 'VMNetwork': >> >> >>>>>>> >>> >>> > >> {'iface': >> >> >>>>>>> >>> >>> > >> 'VMNetwork', >> >> >>>>>>> >>> >>> > >> 'addr': '', 'cfg': {'DEFROUTE': 'no', 'HOTPLUG': >> >> >>>>>>> >>> >>> > >> 'no', >> >> >>>>>>> >>> >>> > >> 'MTU': >> >> >>>>>>> >>> >>> > >> '1500', >> >> >>>>>>> >>> >>> > >> 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': >> >> >>>>>>> >>> >>> > >> 'none', >> >> >>>>>>> >>> >>> > >> 'STP': >> >> >>>>>>> >>> >>> > >> 'off', >> >> >>>>>>> >>> >>> > >> 'DEVICE': 'VMNetwork', 'TYPE': 'Bridge', >> 'ONBOOT': >> >> >>>>>>> >>> >>> > >> 'no'}, >> >> >>>>>>> >>> >>> > >> 'bridged': >> >> >>>>>>> >>> >>> > >> True, >> >> >>>>>>> >>> >>> > >> 'ipv6addrs': ['fe80::62eb:69ff:fe20:b46c/64'], >> >> >>>>>>> >>> >>> > >> 'gateway': >> >> >>>>>>> >>> >>> > >> '', >> >> >>>>>>> >>> >>> > >> 'bootproto4': >> >> >>>>>>> >>> >>> > >> 'none', 'netmask': '', 'stp': 'off', >> 'ipv4addrs': [], >> >> >>>>>>> >>> >>> > >> 'mtu': >> >> >>>>>>> >>> >>> > >> '1500', >> >> >>>>>>> >>> >>> > >> 'ipv6gateway': '::', 'ports': ['bond0.36']}}, >> >> >>>>>>> >>> >>> > >> 'bridges': >> >> >>>>>>> >>> >>> > >> {'Internal': >> >> >>>>>>> >>> >>> > >> {'addr': '', 'cfg': {'DEFROUTE': 'no', 'HOTPLUG': >> >> >>>>>>> >>> >>> > >> 'no', >> >> >>>>>>> >>> >>> > >> 'MTU': >> >> >>>>>>> >>> >>> > >> '9000', >> >> >>>>>>> >>> >>> > >> 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': >> >> >>>>>>> >>> >>> > >> 'none', >> >> >>>>>>> >>> >>> > >> 'STP': >> >> >>>>>>> >>> >>> > >> 'off', >> >> >>>>>>> >>> >>> > >> 'DEVICE': 'Internal', 'TYPE': 'Bridge', 'ONBOOT': >> >> >>>>>>> >>> >>> > >> 'no'}, >> >> >>>>>>> >>> >>> > >> 'ipv6addrs': >> >> >>>>>>> >>> >>> > >> ['fe80::210:18ff:fecd:daac/64'], 'mtu': '9000', >> >> >>>>>>> >>> >>> > >> 'netmask': '', >> >> >>>>>>> >>> >>> > >> 'stp': >> >> >>>>>>> >>> >>> > >> 'off', 'ipv4addrs': [], 'ipv6gateway': '::', >> >> >>>>>>> >>> >>> > >> 'gateway': >> >> >>>>>>> >>> >>> > >> '', >> >> >>>>>>> >>> >>> > >> 'opts': >> >> >>>>>>> >>> >>> > >> {'topology_change_detected': '0', >> >> >>>>>>> >>> >>> > >> 'multicast_last_member_count': >> >> >>>>>>> >>> >>> > >> '2', >> >> >>>>>>> >>> >>> > >> 'hash_elasticity': '4', >> >> >>>>>>> >>> >>> > >> 'multicast_query_response_interval': >> >> >>>>>>> >>> >>> > >> '999', >> >> >>>>>>> >>> >>> > >> 'multicast_snooping': '1', >> >> >>>>>>> >>> >>> > >> 'multicast_startup_query_interval': >> >> >>>>>>> >>> >>> > >> '3124', >> >> >>>>>>> >>> >>> > >> 'hello_timer': '31', >> 'multicast_querier_interval': >> >> >>>>>>> >>> >>> > >> '25496', >> >> >>>>>>> >>> >>> > >> 'max_age': >> >> >>>>>>> >>> >>> > >> '1999', 'hash_max': '512', 'stp_state': '0', >> >> >>>>>>> >>> >>> > >> 'root_id': >> >> >>>>>>> >>> >>> > >> '8000.001018cddaac', 'priority': '32768', >> >> >>>>>>> >>> >>> > >> 'multicast_membership_interval': >> >> >>>>>>> >>> >>> > >> '25996', 'root_path_cost': '0', 'root_port': '0', >> >> >>>>>>> >>> >>> > >> 'multicast_querier': >> >> >>>>>>> >>> >>> > >> '0', >> >> >>>>>>> >>> >>> > >> 'multicast_startup_query_count': '2', >> 'hello_time': >> >> >>>>>>> >>> >>> > >> '199', >> >> >>>>>>> >>> >>> > >> 'topology_change': '0', 'bridge_id': >> >> >>>>>>> >>> >>> > >> '8000.001018cddaac', >> >> >>>>>>> >>> >>> > >> 'topology_change_timer': '0', 'ageing_time': >> '29995', >> >> >>>>>>> >>> >>> > >> 'gc_timer': >> >> >>>>>>> >>> >>> > >> '31', >> >> >>>>>>> >>> >>> > >> 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', >> >> >>>>>>> >>> >>> > >> 'multicast_query_interval': '12498', >> >> >>>>>>> >>> >>> > >> 'multicast_last_member_interval': >> >> >>>>>>> >>> >>> > >> '99', 'multicast_router': '1', 'forward_delay': >> '0'}, >> >> >>>>>>> >>> >>> > >> 'ports': >> >> >>>>>>> >>> >>> > >> ['bond1.100']}, 'VMNetwork': {'addr': '', 'cfg': >> >> >>>>>>> >>> >>> > >> {'DEFROUTE': >> >> >>>>>>> >>> >>> > >> 'no', >> >> >>>>>>> >>> >>> > >> 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', >> >> >>>>>>> >>> >>> > >> 'NM_CONTROLLED': >> >> >>>>>>> >>> >>> > >> 'no', >> >> >>>>>>> >>> >>> > >> 'BOOTPROTO': 'none', 'STP': 'off', 'DEVICE': >> >> >>>>>>> >>> >>> > >> 'VMNetwork', >> >> >>>>>>> >>> >>> > >> 'TYPE': >> >> >>>>>>> >>> >>> > >> 'Bridge', >> >> >>>>>>> >>> >>> > >> 'ONBOOT': 'no'}, 'ipv6addrs': >> >> >>>>>>> >>> >>> > >> ['fe80::62eb:69ff:fe20:b46c/64'], >> >> >>>>>>> >>> >>> > >> 'mtu': >> >> >>>>>>> >>> >>> > >> '1500', 'netmask': '', 'stp': 'off', >> 'ipv4addrs': [], >> >> >>>>>>> >>> >>> > >> 'ipv6gateway': >> >> >>>>>>> >>> >>> > >> '::', >> >> >>>>>>> >>> >>> > >> 'gateway': '', 'opts': >> {'topology_change_detected': >> >> >>>>>>> >>> >>> > >> '0', >> >> >>>>>>> >>> >>> > >> 'multicast_last_member_count': '2', >> >> >>>>>>> >>> >>> > >> 'hash_elasticity': >> >> >>>>>>> >>> >>> > >> '4', >> >> >>>>>>> >>> >>> > >> 'multicast_query_response_interval': '999', >> >> >>>>>>> >>> >>> > >> 'multicast_snooping': >> >> >>>>>>> >>> >>> > >> '1', >> >> >>>>>>> >>> >>> > >> 'multicast_startup_query_interval': '3124', >> >> >>>>>>> >>> >>> > >> 'hello_timer': >> >> >>>>>>> >>> >>> > >> '131', >> >> >>>>>>> >>> >>> > >> 'multicast_querier_interval': '25496', 'max_age': >> >> >>>>>>> >>> >>> > >> '1999', >> >> >>>>>>> >>> >>> > >> 'hash_max': >> >> >>>>>>> >>> >>> > >> '512', 'stp_state': '0', 'root_id': >> >> >>>>>>> >>> >>> > >> '8000.60eb6920b46c', >> >> >>>>>>> >>> >>> > >> 'priority': >> >> >>>>>>> >>> >>> > >> '32768', 'multicast_membership_interval': >> '25996', >> >> >>>>>>> >>> >>> > >> 'root_path_cost': >> >> >>>>>>> >>> >>> > >> '0', >> >> >>>>>>> >>> >>> > >> 'root_port': '0', 'multicast_querier': '0', >> >> >>>>>>> >>> >>> > >> 'multicast_startup_query_count': '2', >> 'hello_time': >> >> >>>>>>> >>> >>> > >> '199', >> >> >>>>>>> >>> >>> > >> 'topology_change': '0', 'bridge_id': >> >> >>>>>>> >>> >>> > >> '8000.60eb6920b46c', >> >> >>>>>>> >>> >>> > >> 'topology_change_timer': '0', 'ageing_time': >> '29995', >> >> >>>>>>> >>> >>> > >> 'gc_timer': >> >> >>>>>>> >>> >>> > >> '31', >> >> >>>>>>> >>> >>> > >> 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', >> >> >>>>>>> >>> >>> > >> 'multicast_query_interval': '12498', >> >> >>>>>>> >>> >>> > >> 'multicast_last_member_interval': >> >> >>>>>>> >>> >>> > >> '99', 'multicast_router': '1', 'forward_delay': >> '0'}, >> >> >>>>>>> >>> >>> > >> 'ports': >> >> >>>>>>> >>> >>> > >> ['bond0.36']}}, 'uuid': >> >> >>>>>>> >>> >>> > >> '44454C4C-4C00-1057-8053-B7C04F504E31', >> >> >>>>>>> >>> >>> > >> 'lastClientIface': 'bond1', 'nics': {'eth3': >> >> >>>>>>> >>> >>> > >> {'permhwaddr': >> >> >>>>>>> >>> >>> > >> '00:10:18:cd:da:ae', 'addr': '', 'cfg': {'SLAVE': >> >> >>>>>>> >>> >>> > >> 'yes', >> >> >>>>>>> >>> >>> > >> 'NM_CONTROLLED': >> >> >>>>>>> >>> >>> > >> 'no', 'MTU': '9000', 'HWADDR': >> '00:10:18:cd:da:ae', >> >> >>>>>>> >>> >>> > >> 'MASTER': >> >> >>>>>>> >>> >>> > >> 'bond1', >> >> >>>>>>> >>> >>> > >> 'DEVICE': 'eth3', 'ONBOOT': 'no'}, 'ipv6addrs': >> [], >> >> >>>>>>> >>> >>> > >> 'mtu': >> >> >>>>>>> >>> >>> > >> '9000', >> >> >>>>>>> >>> >>> > >> 'netmask': '', 'ipv4addrs': [], 'hwaddr': >> >> >>>>>>> >>> >>> > >> '00:10:18:cd:da:ac', >> >> >>>>>>> >>> >>> > >> 'speed': >> >> >>>>>>> >>> >>> > >> 1000}, 'eth2': {'permhwaddr': >> '00:10:18:cd:da:ac', >> >> >>>>>>> >>> >>> > >> 'addr': '', >> >> >>>>>>> >>> >>> > >> 'cfg': >> >> >>>>>>> >>> >>> > >> {'SLAVE': 'yes', 'NM_CONTROLLED': 'no', 'MTU': >> >> >>>>>>> >>> >>> > >> '9000', >> >> >>>>>>> >>> >>> > >> 'HWADDR': >> >> >>>>>>> >>> >>> > >> '00:10:18:cd:da:ac', 'MASTER': 'bond1', 'DEVICE': >> >> >>>>>>> >>> >>> > >> 'eth2', >> >> >>>>>>> >>> >>> > >> 'ONBOOT': >> >> >>>>>>> >>> >>> > >> 'no'}, >> >> >>>>>>> >>> >>> > >> 'ipv6addrs': [], 'mtu': '9000', 'netmask': '', >> >> >>>>>>> >>> >>> > >> 'ipv4addrs': [], >> >> >>>>>>> >>> >>> > >> 'hwaddr': >> >> >>>>>>> >>> >>> > >> '00:10:18:cd:da:ac', 'speed': 1000}, 'eth1': >> >> >>>>>>> >>> >>> > >> {'permhwaddr': >> >> >>>>>>> >>> >>> > >> '60:eb:69:20:b4:6d', 'addr': '', 'cfg': {'SLAVE': >> >> >>>>>>> >>> >>> > >> 'yes', >> >> >>>>>>> >>> >>> > >> 'NM_CONTROLLED': >> >> >>>>>>> >>> >>> > >> 'no', 'MTU': '1500', 'HWADDR': >> '60:eb:69:20:b4:6d', >> >> >>>>>>> >>> >>> > >> 'MASTER': >> >> >>>>>>> >>> >>> > >> 'bond0', >> >> >>>>>>> >>> >>> > >> 'DEVICE': 'eth1', 'ONBOOT': 'yes'}, 'ipv6addrs': >> [], >> >> >>>>>>> >>> >>> > >> 'mtu': >> >> >>>>>>> >>> >>> > >> '1500', >> >> >>>>>>> >>> >>> > >> 'netmask': '', 'ipv4addrs': [], 'hwaddr': >> >> >>>>>>> >>> >>> > >> '60:eb:69:20:b4:6c', >> >> >>>>>>> >>> >>> > >> 'speed': >> >> >>>>>>> >>> >>> > >> 1000}, 'eth0': {'permhwaddr': >> '60:eb:69:20:b4:6c', >> >> >>>>>>> >>> >>> > >> 'addr': '', >> >> >>>>>>> >>> >>> > >> 'cfg': >> >> >>>>>>> >>> >>> > >> {'SLAVE': 'yes', 'NM_CONTROLLED': 'no', 'MTU': >> >> >>>>>>> >>> >>> > >> '1500', >> >> >>>>>>> >>> >>> > >> 'HWADDR': >> >> >>>>>>> >>> >>> > >> '60:eb:69:20:b4:6c', 'MASTER': 'bond0', 'DEVICE': >> >> >>>>>>> >>> >>> > >> 'eth0', >> >> >>>>>>> >>> >>> > >> 'ONBOOT': >> >> >>>>>>> >>> >>> > >> 'yes'}, >> >> >>>>>>> >>> >>> > >> 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', >> >> >>>>>>> >>> >>> > >> 'ipv4addrs': [], >> >> >>>>>>> >>> >>> > >> 'hwaddr': >> >> >>>>>>> >>> >>> > >> '60:eb:69:20:b4:6c', 'speed': 1000}}, >> >> >>>>>>> >>> >>> > >> 'software_revision': '1', >> >> >>>>>>> >>> >>> > >> 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', >> '3.4', >> >> >>>>>>> >>> >>> > >> '3.5'], >> >> >>>>>>> >>> >>> > >> 'cpuFlags': >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> >> >>>>>>> >>> >>> > >> >> u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,dca,sse4_1,sse4_2,popcnt,aes,lahf_lm,tpr_shadow,vnmi,flexpriority,ept,vpid,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270', >> >> >>>>>>> >>> >>> > >> 'ISCSIInitiatorName': >> >> >>>>>>> >>> >>> > >> 'iqn.1994-05.com.redhat:32151ce183c8', >> >> >>>>>>> >>> >>> > >> 'netConfigDirty': 'False', 'supportedENGINEs': >> >> >>>>>>> >>> >>> > >> ['3.0', >> >> >>>>>>> >>> >>> > >> '3.1', >> >> >>>>>>> >>> >>> > >> '3.2', >> >> >>>>>>> >>> >>> > >> '3.3', >> >> >>>>>>> >>> >>> > >> '3.4', '3.5'], 'autoNumaBalancing': 2, >> 'reservedMem': >> >> >>>>>>> >>> >>> > >> '321', >> >> >>>>>>> >>> >>> > >> 'bondings': >> >> >>>>>>> >>> >>> > >> {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', >> >> >>>>>>> >>> >>> > >> 'netmask': '', >> >> >>>>>>> >>> >>> > >> 'slaves': >> >> >>>>>>> >>> >>> > >> [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': >> {'addr': >> >> >>>>>>> >>> >>> > >> '', >> >> >>>>>>> >>> >>> > >> 'cfg': >> >> >>>>>>> >>> >>> > >> {'HOTPLUG': 'no', 'MTU': '1500', 'NM_CONTROLLED': >> >> >>>>>>> >>> >>> > >> 'no', >> >> >>>>>>> >>> >>> > >> 'BONDING_OPTS': >> >> >>>>>>> >>> >>> > >> 'mode=4 miimon=100', 'DEVICE': 'bond0', 'ONBOOT': >> >> >>>>>>> >>> >>> > >> 'yes'}, >> >> >>>>>>> >>> >>> > >> 'ipv6addrs': >> >> >>>>>>> >>> >>> > >> ['fe80::62eb:69ff:fe20:b46c/64'], 'mtu': '1500', >> >> >>>>>>> >>> >>> > >> 'netmask': '', >> >> >>>>>>> >>> >>> > >> 'ipv4addrs': [], 'hwaddr': '60:eb:69:20:b4:6c', >> >> >>>>>>> >>> >>> > >> 'slaves': >> >> >>>>>>> >>> >>> > >> ['eth0', >> >> >>>>>>> >>> >>> > >> 'eth1'], >> >> >>>>>>> >>> >>> > >> 'opts': {'miimon': '100', 'mode': '4'}}, 'bond1': >> >> >>>>>>> >>> >>> > >> {'addr': >> >> >>>>>>> >>> >>> > >> '10.10.10.6', >> >> >>>>>>> >>> >>> > >> 'cfg': {'DEFROUTE': 'no', 'IPADDR': '10.10.10.6', >> >> >>>>>>> >>> >>> > >> 'HOTPLUG': >> >> >>>>>>> >>> >>> > >> 'no', >> >> >>>>>>> >>> >>> > >> 'MTU': >> >> >>>>>>> >>> >>> > >> '9000', 'NM_CONTROLLED': 'no', 'NETMASK': >> >> >>>>>>> >>> >>> > >> '255.255.255.0', >> >> >>>>>>> >>> >>> > >> 'BOOTPROTO': >> >> >>>>>>> >>> >>> > >> 'none', 'BONDING_OPTS': 'mode=4 miimon=100', >> >> >>>>>>> >>> >>> > >> 'DEVICE': >> >> >>>>>>> >>> >>> > >> 'bond1', >> >> >>>>>>> >>> >>> > >> 'ONBOOT': >> >> >>>>>>> >>> >>> > >> 'no'}, 'ipv6addrs': >> ['fe80::210:18ff:fecd:daac/64'], >> >> >>>>>>> >>> >>> > >> 'mtu': >> >> >>>>>>> >>> >>> > >> '9000', >> >> >>>>>>> >>> >>> > >> 'netmask': '255.255.255.0', 'ipv4addrs': >> >> >>>>>>> >>> >>> > >> ['10.10.10.6/24' >> >> >>>>>>> >>> >>> > >> <http://10.10.10.6/24%27>], 'hwaddr': >> >> >>>>>>> >>> >>> > >> '00:10:18:cd:da:ac', >> >> >>>>>>> >>> >> > ... [Mensaje recortado]