<div dir="ltr">Hi,<div> I'm new to oVirt and recently built a 10 node ovirt 3.5 DC with shared storage using gluster configured as distributed-replicated (replication = 2). Shortly after 7 of the 10 nodes dropped, one at a time over a few hours, into "Non Operational" state. Attempting to activate one of these nodes gives the error: "Failed to connect Host ovirt-node260 to Storage Pool LADC-TBX". Attempting to put the node into Maintenance eaves the node stuck in "Preparing For maintenance". <div><br><div>When I rebooted one of the nodes I see this in the nodes event list:</div></div></div><div><br></div><div>"Host ovirt-node269 reports about one of the Active Storage Domains as Problematic."</div><div><br></div><div>I see many of these errors in the vdsm log from the failed nodes:</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<p class="p1"><span class="s1">Thread-10000::ERROR::2015-08-12 10:01:17,748::__init__::506::jsonrpc.JsonRpcServer::(_serveRequest) Internal server error</span></p>
<p class="p1"><span class="s1">Traceback (most recent call last):</span></p>
<p class="p1"><span class="s1"> File "/usr/lib/python2.6/site-packages/yajsonrpc/__init__.py", line 501, in _serveRequest</span></p>
<p class="p1"><span class="s1"> res = method(**params)</span></p>
<p class="p1"><span class="s1"> File "/usr/share/vdsm/rpc/Bridge.py", line 267, in _dynamicMethod</span></p>
<p class="p1"><span class="s1"> result = fn(*methodArgs)</span></p>
<p class="p1"><span class="s1"> File "/usr/share/vdsm/API.py", line 1330, in getStats</span></p>
<p class="p1"><span class="s1"> stats.update(self._cif.mom.getKsmStats())</span></p>
<p class="p1"><span class="s1"> File "/usr/share/vdsm/momIF.py", line 60, in getKsmStats</span></p>
<p class="p1"><span class="s1"> stats = self._mom.getStatistics()['host']</span></p>
<p class="p1"><span class="s1"> File "/usr/lib/python2.6/site-packages/mom/MOMFuncs.py", line 75, in getStatistics</span></p>
<p class="p1"><span class="s1"> host_stats = self.threads['host_monitor'].interrogate().statistics[-1]</span></p>
<p class="p1"><span class="s1">AttributeError: 'NoneType' object has no attribute 'statistics'</span></p></blockquote><div>Any help here is appreciated.</div><div><br></div><div> -- Chris</div><div><br></div></div>