log from messages

Feb  3 08:27:53 ovirtnode3 ovs-vsctl: ovs|00001|db_ctl_base|ERR|unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)
Feb  3 08:27:53 ovirtnode3 journal: vdsm vds ERROR Executing commands failed: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No su
ch file or directory)#012Traceback (most recent call last):#012  File "/usr/share/vdsm/API.py", line 1531, in setupNetworks#012    supervdsm.getProxy().setup
Networks(networks, bondings, options)#012  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in __call__#012    return callMethod()#012  Fi
le "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in <lambda>#012    **kwargs)#012  File "<string>", line 2, in setupNetworks#012  File "/usr
/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod#012    raise convert_to_error(kind, result)#012ConfigNetworkError: (21, 'Executing co
mmands failed: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)')

Log from vdsm.log

Reactor thread::INFO::2017-02-03 08:32:24,638::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepted connection from ::1:53082
Reactor thread::DEBUG::2017-02-03 08:32:24,643::protocoldetector::91::ProtocolDetector.Detector::(__init__) Using required_size=11
Reactor thread::INFO::2017-02-03 08:32:24,643::protocoldetector::127::ProtocolDetector.Detector::(handle_read) Detected protocol xml from ::1:53082
Reactor thread::DEBUG::2017-02-03 08:32:24,643::bindingxmlrpc::1317::XmlDetector::(handle_socket) xml over http detected from ('::1', 53082)
BindingXMLRPC::INFO::2017-02-03 08:32:24,643::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request handler for ::1:53082
Thread-20::INFO::2017-02-03 08:32:24,644::xmlrpc::83::vds.XMLRPCServer::(_process_requests) Request handler for ::1:53082 started
Thread-20::DEBUG::2017-02-03 08:32:24,644::bindingxmlrpc::1263::vds::(wrapper) client [::1]::call getAllVmStats with () {}
Thread-20::DEBUG::2017-02-03 08:32:24,644::bindingxmlrpc::1270::vds::(wrapper) return getAllVmStats with {'status': {'message': 'Done', 'code': 0}, 'statsLis
t': (suppressed)}
Thread-20::INFO::2017-02-03 08:32:24,644::bindingxmlrpc::1297::vds::(wrapper) RPC call getAllVmStats finished (code=0) in 0.00 seconds
Thread-20::INFO::2017-02-03 08:32:24,645::xmlrpc::91::vds.XMLRPCServer::(_process_requests) Request handler for ::1:53082 stopped
periodic/0::DEBUG::2017-02-03 08:32:24,907::sampling::508::virt.sampling.VMBulkSampler::(__call__) sampled timestamp 4347942.15 elapsed 0.010 acquired True d
omains all
periodic/2::DEBUG::2017-02-03 08:32:24,908::task::599::storage.TaskManager.Task::(_updateState) (Task='bb07f89c-6165-42bc-902f-1b7701b813fa') moving from sta
te init -> state preparing
periodic/2::INFO::2017-02-03 08:32:24,908::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None)
periodic/2::INFO::2017-02-03 08:32:24,908::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
periodic/2::DEBUG::2017-02-03 08:32:24,909::task::1195::storage.TaskManager.Task::(prepare) (Task='bb07f89c-6165-42bc-902f-1b7701b813fa') finished: {}
periodic/2::DEBUG::2017-02-03 08:32:24,909::task::599::storage.TaskManager.Task::(_updateState) (Task='bb07f89c-6165-42bc-902f-1b7701b813fa') moving from sta
te preparing -> state finished
periodic/2::DEBUG::2017-02-03 08:32:24,909::resourceManager::910::storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
periodic/2::DEBUG::2017-02-03 08:32:24,909::resourceManager::947::storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
periodic/2::DEBUG::2017-02-03 08:32:24,909::task::997::storage.TaskManager.Task::(_decref) (Task='bb07f89c-6165-42bc-902f-1b7701b813fa') ref 0 aborting False
jsonrpc/4::DEBUG::2017-02-03 08:32:26,653::__init__::532::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getAllVmStats' in bridge with {}
jsonrpc/4::DEBUG::2017-02-03 08:32:26,653::__init__::557::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getAllVmStats' in bridge with (suppressed)
jsonrpc/4::INFO::2017-02-03 08:32:26,653::__init__::515::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getAllVmStats succeeded in 0.00 seconds
jsonrpc/6::DEBUG::2017-02-03 08:32:27,706::__init__::532::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {}
jsonrpc/6::DEBUG::2017-02-03 08:32:27,706::__init__::532::jsonrpc.JsonRpcServer::(_handle_request) Calling 'Host.getStats' in bridge with {}
jsonrpc/6::DEBUG::2017-02-03 08:32:27,707::task::599::storage.TaskManager.Task::(_updateState) (Task='9285329e-485b-4165-8bdc-4f411f394356') moving from state init -> state preparing
jsonrpc/6::INFO::2017-02-03 08:32:27,707::logUtils::49::dispatcher::(wrapper) Run and protect: repoStats(options=None)
jsonrpc/6::INFO::2017-02-03 08:32:27,707::logUtils::52::dispatcher::(wrapper) Run and protect: repoStats, Return response: {}
jsonrpc/6::DEBUG::2017-02-03 08:32:27,707::task::1195::storage.TaskManager.Task::(prepare) (Task='9285329e-485b-4165-8bdc-4f411f394356') finished: {}
jsonrpc/6::DEBUG::2017-02-03 08:32:27,707::task::599::storage.TaskManager.Task::(_updateState) (Task='9285329e-485b-4165-8bdc-4f411f394356') moving from state preparing -> state finished
jsonrpc/6::DEBUG::2017-02-03 08:32:27,708::resourceManager::910::storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {}
jsonrpc/6::DEBUG::2017-02-03 08:32:27,708::resourceManager::947::storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {}
jsonrpc/6::DEBUG::2017-02-03 08:32:27,708::task::997::storage.TaskManager.Task::(_decref) (Task='9285329e-485b-4165-8bdc-4f411f394356') ref 0 aborting False
jsonrpc/6::DEBUG::2017-02-03 08:32:27,712::__init__::557::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getStats' in bridge with {'cpuStatistics': {'28': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.87'}, '29': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '24': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '25': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '26': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '27': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '20': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '21': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '22': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '23': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '46': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '47': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '44': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '45': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '42': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '43': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '40': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '41': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '1': {'cpuUser': '0.13', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.80'}, '0': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '3': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.86'}, '2': {'cpuUser': '0.07', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.93'}, '5': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '4': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.07', 'cpuIdle': '99.93'}, '7': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '6': {'cpuUser': '0.20', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '99.80'}, '9': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '8': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '39': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '38': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '11': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '10': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '13': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '12': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '15': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '14': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '17': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '16': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '19': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '18': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '31': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '30': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '37': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '36': {'cpuUser': '0.00', 'nodeIndex': 1, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '35': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '34': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '33': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}, '32': {'cpuUser': '0.00', 'nodeIndex': 0, 'cpuSys': '0.00', 'cpuIdle': '100.00'}}, 'numaNodeMemFree': {'1': {'memPercent': 3, 'memFree': '63780'}, '0': {'memPercent': 4, 'memFree': '63426'}}, 'memShared': 0, 'thpState': 'always', 'ksmMergeAcrossNodes': True, 'vmCount': 0, 'memUsed': '2', 'storageDomains': {}, 'incomingVmMigrations': 0, 'network': {'ib0': {'txErrors': '0', 'state': 'down', 'sampleTime': 1486090944.903051, 'name': 'ib0', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'lo': {'txErrors': '0', 'state': 'up', 'sampleTime': 1486090944.903051, 'name': 'lo', 'tx': '56174388', 'txDropped': '0', 'rx': '56174388', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'eno2': {'txErrors': '0', 'state': 'down', 'sampleTime': 1486090944.903051, 'name': 'eno2', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}, 'eno1': {'txErrors': '0', 'state': 'up', 'sampleTime': 1486090944.903051, 'name': 'eno1', 'tx': '26032823', 'txDropped': '0', 'rx': '174722400', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '5624'}, ';vdsmdummy;': {'txErrors': '0', 'state': 'down', 'sampleTime': 1486090944.903051, 'name': ';vdsmdummy;', 'tx': '0', 'txDropped': '0', 'rx': '0', 'rxErrors': '0', 'speed': '1000', 'rxDropped': '0'}}, 'txDropped': '0', 'anonHugePages': '34', 'ksmPages': 100, 'elapsedTime': '287.81', 'cpuLoad': '0.07', 'cpuSys': '0.00', 'diskStats': {'/var/log': {'free': '15072'}, '/var/log/core': {'free': '15072'}, '/var/run/vdsm/': {'free': '64418'}, '/tmp': {'free': '914756'}}, 'cpuUserVdsmd': '0.13', 'netConfigDirty': 'True', 'memCommitted': 0, 'ksmState': False, 'vmMigrating': 0, 'ksmCpu': 0, 'memAvailable': 127200, 'bootTime': '1486037670', 'haStats': {'active': False, 'configured': False, 'score': 0, 'localMaintenance': False, 'globalMaintenance': False}, 'momStatus': 'active', 'rxDropped': '5624', 'outgoingVmMigrations': 0, 'swapTotal': 4095, 'swapFree': 4095, 'dateTime': '2017-02-03T03:02:27 GMT', 'cpuUser': '0.02', 'memFree': 127456, 'cpuIdle': '99.98', 'vmActive': 0, 'v2vJobs': {}, 'cpuSysVdsmd': '0.07'}
jsonrpc/6::INFO::2017-02-03 08:32:27,713::__init__::515::jsonrpc.JsonRpcServer::(_serveRequest) RPC call Host.getStats succeeded in 0.00 seconds
Reactor thread::INFO::2017-02-03 08:32:39,661::protocoldetector::72::ProtocolDetector.AcceptorImpl::(handle_accept) Accepted connection from ::1:53084

supervdsm.log


  File "/usr/share/vdsm/supervdsmServer", line 95, in wrapper
    res = func(*args, **kwargs)
  File "/usr/share/vdsm/supervdsmServer", line 143, in umount
    timeout=timeout)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 281, in _umount
    _runcmd(cmd, timeout)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/mount.py", line 297, in _runcmd
    raise MountError(rc, ";".join((out, err)))
MountError: (32, ';umount: /rhev/data-center/mnt/10.1.235.6:_iso: mountpoint not found\n')
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:26:59,495::supervdsmServer::93::SuperVdsm.ServerCallback::(wrapper) call hbaRescan with () {}
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:26:59,495::commands::69::storage.HBA::(execCmd) /usr/bin/taskset --cpu-list 0-47 /usr/libexec/vdsm/fc-scan (cwd N
one)
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:26:59,579::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper) return hbaRescan with None
MainThread::DEBUG::2017-02-03 08:27:26,367::supervdsmServer::311::SuperVdsm.Server::(main) Terminated normally
MainThread::DEBUG::2017-02-03 08:27:36,148::__init__::47::blivet::(register_device_format) registered device format class DeviceFormat as None
MainThread::DEBUG::2017-02-03 08:27:36,149::__init__::47::blivet::(register_device_format) registered device format class BIOSBoot as biosboot
MainThread::DEBUG::2017-02-03 08:27:36,150::__init__::47::blivet::(register_device_format) registered device format class DiskLabel as disklabel
MainThread::DEBUG::2017-02-03 08:27:36,150::__init__::47::blivet::(register_device_format) registered device format class DMRaidMember as dmraidmember
MainThread::DEBUG::2017-02-03 08:27:36,151::__init__::47::blivet::(register_device_format) registered device format class Ext2FS as ext2
MainThread::DEBUG::2017-02-03 08:27:36,152::__init__::47::blivet::(register_device_format) registered device format class Ext3FS as ext3
MainThread::DEBUG::2017-02-03 08:27:36,152::__init__::47::blivet::(register_device_format) registered device format class Ext4FS as ext4
MainThread::DEBUG::2017-02-03 08:27:36,152::__init__::47::blivet::(register_device_format) registered device format class FATFS as vfat
MainThread::DEBUG::2017-02-03 08:27:36,152::__init__::47::blivet::(register_device_format) registered device format class EFIFS as efi
MainThread::DEBUG::2017-02-03 08:27:36,152::__init__::47::blivet::(register_device_format) registered device format class BTRFS as btrfs
MainThread::DEBUG::2017-02-03 08:27:36,153::__init__::47::blivet::(register_device_format) registered device format class GFS2 as gfs2
MainThread::DEBUG::2017-02-03 08:27:36,153::__init__::47::blivet::(register_device_format) registered device format class JFS as jfs
MainThread::DEBUG::2017-02-03 08:27:36,153::__init__::47::blivet::(register_device_format) registered device format class ReiserFS as reiserfs
MainThread::DEBUG::2017-02-03 08:27:36,153::__init__::47::blivet::(register_device_format) registered device format class XFS as xfs
MainThread::DEBUG::2017-02-03 08:27:36,153::__init__::47::blivet::(register_device_format) registered device format class HFS as hfs
MainThread::DEBUG::2017-02-03 08:27:36,153::__init__::47::blivet::(register_device_format) registered device format class AppleBootstrapFS as appleboot
MainThread::DEBUG::2017-02-03 08:27:36,154::__init__::47::blivet::(register_device_format) registered device format class HFSPlus as hfs+
MainThread::DEBUG::2017-02-03 08:27:36,154::__init__::47::blivet::(register_device_format) registered device format class MacEFIFS as macefi
MainThread::DEBUG::2017-02-03 08:27:36,154::__init__::47::blivet::(register_device_format) registered device format class NTFS as ntfs
MainThread::DEBUG::2017-02-03 08:27:36,154::__init__::47::blivet::(register_device_format) registered device format class NFS as nfs
MainThread::DEBUG::2017-02-03 08:27:36,154::__init__::47::blivet::(register_device_format) registered device format class NFSv4 as nfs4
MainThread::DEBUG::2017-02-03 08:27:36,154::__init__::47::blivet::(register_device_format) registered device format class Iso9660FS as iso9660
MainThread::DEBUG::2017-02-03 08:27:36,154::__init__::47::blivet::(register_device_format) registered device format class NoDevFS as nodev
MainThread::DEBUG::2017-02-03 08:27:36,155::__init__::47::blivet::(register_device_format) registered device format class DevPtsFS as devpts
MainThread::DEBUG::2017-02-03 08:27:36,155::__init__::47::blivet::(register_device_format) registered device format class ProcFS as proc
MainThread::DEBUG::2017-02-03 08:27:36,155::__init__::47::blivet::(register_device_format) registered device format class SysFS as sysfs
MainThread::DEBUG::2017-02-03 08:27:36,155::__init__::47::blivet::(register_device_format) registered device format class TmpFS as tmpfs
MainThread::DEBUG::2017-02-03 08:27:36,155::__init__::47::blivet::(register_device_format) registered device format class BindFS as bind
MainThread::DEBUG::2017-02-03 08:27:36,155::__init__::47::blivet::(register_device_format) registered device format class SELinuxFS as selinuxfs
MainThread::DEBUG::2017-02-03 08:27:36,155::__init__::47::blivet::(register_device_format) registered device format class USBFS as usbfs
MainThread::DEBUG::2017-02-03 08:27:36,155::__init__::47::blivet::(register_device_format) registered device format class EFIVarFS as efivarfs
MainThread::DEBUG::2017-02-03 08:27:36,179::__init__::47::blivet::(register_device_format) registered device format class LUKS as luks
MainThread::DEBUG::2017-02-03 08:27:36,181::__init__::47::blivet::(register_device_format) registered device format class LVMPhysicalVolume as lvmpv
MainThread::DEBUG::2017-02-03 08:27:36,181::__init__::47::blivet::(register_device_format) registered device format class MDRaidMember as mdmember
MainThread::DEBUG::2017-02-03 08:27:36,181::__init__::47::blivet::(register_device_format) registered device format class MultipathMember as multipath_member
MainThread::DEBUG::2017-02-03 08:27:36,181::__init__::47::blivet::(register_device_format) registered device format class PPCPRePBoot as prepboot
MainThread::DEBUG::2017-02-03 08:27:36,182::__init__::47::blivet::(register_device_format) registered device format class SwapSpace as swap
MainThread::DEBUG::2017-02-03 08:27:36,216::storage_log::69::blivet::(log_exception_info) IGNORED:        Caught exception, continuing.
MainThread::DEBUG::2017-02-03 08:27:36,216::storage_log::72::blivet::(log_exception_info) IGNORED:        Problem description: failed to get initiator name from iscsi firmware
MainThread::DEBUG::2017-02-03 08:27:36,216::storage_log::73::blivet::(log_exception_info) IGNORED:        Begin exception details.
MainThread::DEBUG::2017-02-03 08:27:36,216::storage_log::76::blivet::(log_exception_info) IGNORED:            Traceback (most recent call last):
MainThread::DEBUG::2017-02-03 08:27:36,216::storage_log::76::blivet::(log_exception_info) IGNORED:              File "/usr/lib/python2.7/site-packages/blivet
/iscsi.py", line 146, in __init__
MainThread::DEBUG::2017-02-03 08:27:36,216::storage_log::76::blivet::(log_exception_info) IGNORED:                initiatorname = libiscsi.get_firmware_initi
ator_name()
MainThread::DEBUG::2017-02-03 08:27:36,216::storage_log::76::blivet::(log_exception_info) IGNORED:            IOError: Unknown error
MainThread::DEBUG::2017-02-03 08:27:36,216::storage_log::77::blivet::(log_exception_info) IGNORED:        End exception details.
MainThread::DEBUG::2017-02-03 08:27:36,227::supervdsmServer::271::SuperVdsm.Server::(main) Making sure I'm root - SuperVdsm
MainThread::DEBUG::2017-02-03 08:27:36,227::supervdsmServer::280::SuperVdsm.Server::(main) Parsing cmd args
MainThread::DEBUG::2017-02-03 08:27:36,227::supervdsmServer::283::SuperVdsm.Server::(main) Cleaning old socket /var/run/vdsm/svdsm.sock
MainThread::DEBUG::2017-02-03 08:27:36,227::supervdsmServer::287::SuperVdsm.Server::(main) Setting up keep alive thread
MainThread::DEBUG::2017-02-03 08:27:36,228::supervdsmServer::293::SuperVdsm.Server::(main) Creating remote object manager
MainThread::DEBUG::2017-02-03 08:27:36,228::fileUtils::246::storage.fileUtils::(chown) Changing owner for /var/run/vdsm/svdsm.sock, to (36:36)
MainThread::DEBUG::2017-02-03 08:27:36,228::supervdsmServer::303::SuperVdsm.Server::(main) Started serving super vdsm object
sourceRoute::DEBUG::2017-02-03 08:27:36,229::sourceroutethread::79::root::(_subscribeToInotifyLoop) sourceRouteThread.subscribeToInotifyLoop started
restore-net::INFO::2017-02-03 08:27:37,160::vdsm-restore-net-config::465::root::(restore) networks already restored. doing nothing.
MainProcess|hsm/init::DEBUG::2017-02-03 08:27:39,601::supervdsmServer::93::SuperVdsm.ServerCallback::(wrapper) call hbaRescan with () {}
MainProcess|hsm/init::DEBUG::2017-02-03 08:27:39,602::commands::69::storage.HBA::(execCmd) /usr/bin/taskset --cpu-list 0-47 /usr/libexec/vdsm/fc-scan (cwd No
ne)
MainProcess|hsm/init::DEBUG::2017-02-03 08:27:39,691::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper) return hbaRescan with None
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,355::supervdsmServer::93::SuperVdsm.ServerCallback::(wrapper) call network_caps with () {}
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,360::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /sbin/ip route show to 0.0.0.0/0 table
main (cwd None)
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,364::commands::93::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,408::libvirtconnection::160::root::(get) trying to connect libvirt
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,450::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /usr/sbin/tc qdisc show (cwd None)
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,453::commands::93::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,454::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /bin/systemctl --no-pager list-unit-fil
es (cwd None)
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,530::commands::93::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,530::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /bin/systemctl status openvswitch.servi
ce (cwd None)
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,561::commands::93::root::(execCmd) FAILED: <err> = ''; <rc> = 3
MainProcess|jsonrpc/4::DEBUG::2017-02-03 08:27:48,561::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper) return network_caps with {'bridges': {}, 'bo
ndings': {}, 'nameservers': ['10.1.1.8'], 'nics': {'ib0': {'ipv6autoconf': True, 'addr': '', 'ipv6gateway': '::', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '4
092', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '80:00:02:08:fe:80:00:00:00:00:00:00:0c:c4:7a:ff:ff:0f:d8:85', 's
peed': 0, 'gateway': ''}, 'eno1': {'ipv6autoconf': False, 'addr': '10.1.235.5', 'ipv6gateway': '::', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4
': False, 'netmask': '255.255.0.0', 'ipv4defaultroute': True, 'ipv4addrs': ['10.1.235.5/16'], 'hwaddr': '0c:c4:7a:0f:6b:2a', 'speed': 1000, 'gateway': '10.1.
0.1'}, 'eno2': {'ipv6autoconf': True, 'addr': '', 'ipv6gateway': '::', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4
defaultroute': False, 'ipv4addrs': [], 'hwaddr': '0c:c4:7a:0f:6b:2b', 'speed': 0, 'gateway': ''}}, 'supportsIPv6': True, 'vlans': {}, 'networks': {}}
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,690::supervdsmServer::93::SuperVdsm.ServerCallback::(wrapper) call setupNetworks with ({}, {u'connectivityC
heck': u'true', u'connectivityTimeout': 120}) {}
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,691::api::247::root::(setupNetworks) Setting up network according to configuration: networks:{u'ovirtmgmt': {u'ipv6autoconf': False, u'nic': u'eno1', u'ipaddr': u'10.1.235.5', u'netmask': u'255.255.0.0', u'mtu': 1500, u'switch': u'ovs', u'dhcpv6': False, u'STP': u'no', u'bridged': u'true', u'gateway': u'10.1.0.1', u'defaultRoute': True}}, bondings:{}, options:{u'connectivityCheck': u'true', u'connectivityTimeout': 120}
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,691::api::252::root::(setupNetworks) Validating configuration
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,694::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /sbin/ip route show to 0.0.0.0/0 table main (cwd None)
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,698::commands::93::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,771::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /usr/sbin/tc qdisc show (cwd None)
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,774::commands::93::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,774::vsctl::57::root::(commit) Executing commands: /usr/bin/ovs-vsctl --oneline --format=json -- list Bridge -- list Port -- list Interface
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,774::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /usr/bin/ovs-vsctl --oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None)
MainProcess|jsonrpc/5::DEBUG::2017-02-03 08:27:53,807::commands::93::root::(execCmd) FAILED: <err> = 'ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)\n'; <rc> = 1
MainProcess|jsonrpc/5::ERROR::2017-02-03 08:27:53,808::supervdsmServer::97::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks
Traceback (most recent call last):
  File "/usr/share/vdsm/supervdsmServer", line 95, in wrapper
    res = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/network/api.py", line 254, in setupNetworks
    netswitch.validate(networks, bondings)
  File "/usr/lib/python2.7/site-packages/vdsm/network/netswitch.py", line 121, in validate
    ovs_switch.validate_network_setup(ovs_nets, ovs_bonds)
  File "/usr/lib/python2.7/site-packages/vdsm/network/ovs/switch.py", line 40, in validate_network_setup
    ovs_networks = info.create_netinfo(info.OvsInfo())['networks']
  File "/usr/lib/python2.7/site-packages/vdsm/network/ovs/info.py", line 73, in __init__
    ovs_db = OvsDB(driver.create())
  File "/usr/lib/python2.7/site-packages/vdsm/network/ovs/info.py", line 64, in __init__
    transaction.add(ifaces_command)
  File "/usr/lib/python2.7/site-packages/vdsm/network/ovs/driver/__init__.py", line 54, in __exit__
    self.result = self.commit()
  File "/usr/lib/python2.7/site-packages/vdsm/network/ovs/driver/vsctl.py", line 63, in commit
    'Executing commands failed: %s' % '\n'.join(err))
ConfigNetworkError: (21, 'Executing commands failed: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)')
MainProcess|Thread-2::DEBUG::2017-02-03 08:27:54,440::supervdsmServer::93::SuperVdsm.ServerCallback::(wrapper) call ksmTune with () {}
MainProcess|Thread-2::DEBUG::2017-02-03 08:27:54,440::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper) return ksmTune with None
MainProcess|jsonrpc/0::DEBUG::2017-02-03 08:30:00,918::supervdsmServer::93::SuperVdsm.ServerCallback::(wrapper) call network_caps with () {}
MainProcess|jsonrpc/0::DEBUG::2017-02-03 08:30:00,922::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /sbin/ip route show to 0.0.0.0/0 table main (cwd None)
MainProcess|jsonrpc/0::DEBUG::2017-02-03 08:30:00,927::commands::93::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|jsonrpc/0::DEBUG::2017-02-03 08:30:01,013::commands::69::root::(execCmd) /usr/bin/taskset --cpu-list 0-47 /usr/sbin/tc qdisc show (cwd None)
MainProcess|jsonrpc/0::DEBUG::2017-02-03 08:30:01,016::commands::93::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0
MainProcess|jsonrpc/0::DEBUG::2017-02-03 08:30:01,017::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper) return network_caps with {'bridges': {}, 'bondings': {}, 'nameservers': ['10.1.1.8'], 'nics': {'ib0': {'ipv6autoconf': True, 'addr': '', 'ipv6gateway': '::', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '4092', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '80:00:02:08:fe:80:00:00:00:00:00:00:0c:c4:7a:ff:ff:0f:d8:85', 'speed': 0, 'gateway': ''}, 'eno1': {'ipv6autoconf': False, 'addr': '10.1.235.5', 'ipv6gateway': '::', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '255.255.0.0', 'ipv4defaultroute': True, 'ipv4addrs': ['10.1.235.5/16'], 'hwaddr': '0c:c4:7a:0f:6b:2a', 'speed': 1000, 'gateway': '10.1.0.1'}, 'eno2': {'ipv6autoconf': True, 'addr': '', 'ipv6gateway': '::', 'dhcpv6': False, 'ipv6addrs': [], 'mtu': '1500', 'dhcpv4': False, 'netmask': '', 'ipv4defaultroute': False, 'ipv4addrs': [], 'hwaddr': '0c:c4:7a:0f:6b:2b', 'speed': 0, 'gateway': ''}}, 'supportsIPv6': True, 'vlans': {}, 'networks': {}}
MainProcess|jsonrpc/1::DEBUG::2017-02-03 08:30:01,036::supervdsmServer::93::SuperVdsm.ServerCallback::(wrapper) call getHardwareInfo with () {}
MainProcess|jsonrpc/1::DEBUG::2017-02-03 08:30:01,066::supervdsmServer::100::SuperVdsm.ServerCallback::(wrapper) return getHardwareInfo with {'systemProductName': 'X9DRT', 'systemSerialNumber': '0123456789', 'systemFamily': 'To be filled by O.E.M.', 'systemVersion': '0123456789', 'systemUUID': '00000000-0000-0000-0000-0CC47A0F6B2A', 'systemManufacturer': 'Supermicro'}
(END)

On Fri, Feb 3, 2017 at 12:05 AM, Edward Haas <ehaas@redhat.com> wrote:

Hello Shalabh,

Please provide the logs from your node:

- messages

- vdsm/vdsm.log, vdsm/supervdsm.log

It may be that you are missing openvswitch installed, although VDSM should not require it for its operation.

Thanks,

Edy.

On Thu, Feb 2, 2017 at 2:10 PM, Shalabh Goel <shalabhgoel13@gmail.com> wrote:

HI,

I am getting the following error on my node after rebooting it.

VDSM ovirtnode2 command HostSetupNetworksVDS failed: Executing commands failed: ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)

To solve this, I am trying to restart ovsdb-server using the following command,

ovsdb-server --remote=punix:*/var/run/openvswitch/db.sock* --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key--certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach

But I am getting the following error.

ovsdb-server: /var/run/openvswitch/ovsdb-server.pid.tmp: create failed (No such file or directory)

How to restart the ovsdb-server?? Also ovirtmgmt network is missing from my node. It happened after I rebooted my node after it got upgraded to Ovirt 4.1

--

Shalabh Goel

_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

--

Shalabh Goel