<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 1, 2016 at 5:16 PM, Cristian Mammoli <span dir="ltr"><<a href="mailto:c.mammoli@apra.it" target="_blank">c.mammoli@apra.it</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Here it is: <a href="http://cloud.apra.it/index.php/s/4cdcde8cafdb7a1c2c2374b02dce118e" rel="noreferrer" target="_blank">http://cloud.apra.it/index.php<wbr>/s/4cdcde8cafdb7a1c2c2374b02dc<wbr>e118e</a><br>
<br>
I tarred all the agent.log on both servers.<br>
<br>
The engine was running on kvm01 and got shutdown on kvm01 around 10:35 AM on 29 November. But I think that's not the problem, it is supposed to shut down if the host can't reach the gateway. Probably the nic problem was already there but got triggered on reboot<br>
<br>
Btw I kept digging: I extracted the ovf from which vm.conf is generated:<br>
<br>
ovirt_hosted_engine_ha.lib.ovf<wbr>.ovf_store.OVFStore::(getEngin<wbr>eVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/blockSD/<wbr>2c3585cc-b7bc-4881-85b3-aa6514<wbr>991a26/images/9c5e2121-f1a3-<wbr>4886-964c-c74fdfbbb3c1/ff76505<wbr>5-09c5-4b05-9cc7-5277b15c5d08<br>
<br>
# tar xvf /rhev/data-center/mnt/blockSD/<wbr>2c3585cc-b7bc-4881-85b3-aa6514<wbr>991a26/images/9c5e2121-f1a3-<wbr>4886-964c-c74fdfbbb3c1/ff76505<wbr>5-09c5-4b05-9cc7-5277b15c5d08<br>
497f5e4a-0c76-441a-b72e-724d70<wbr>92d07e.ovf<br>
info.json<br>
<br>
In the ovf file there is no Nic section...<br>
<br></blockquote><div><br></div><div>Ciao Cristian,</div><div>do you see any interface for the engine VM in the engine admin portal?</div><div><br></div><div>Could you please execute this on the engine VM and share its output?</div><div> sudo -u postgres psql engine -c "select * from vm_device where type='interface' and vm_id='<span style="font-size:12.8px">497f5e4a-0c76-441a-b72e-7</span><wbr style="font-size:12.8px"><span style="font-size:12.8px">24d7092d07e</span>'"<br></div><div> sudo -u postgres psql engine -c "select * from vms where vm_guid='<span style="font-size:12.8px">497f5e4a-0c76-441a-b72e-7</span><wbr style="font-size:12.8px"><span style="font-size:12.8px">24d7092d07e</span>'"<br></div><div><br></div><div>thanks</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I uploaded the ovf on the same share as the logs<br>
<br>
Ty<div class="gmail-HOEnZb"><div class="gmail-h5"><br>
<br>
Il 01/12/2016 15:26, Yedidyah Bar David ha scritto:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
On Thu, Dec 1, 2016 at 1:08 PM, Cristian Mammoli <<a href="mailto:c.mammoli@apra.it" target="_blank">c.mammoli@apra.it</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Hi, I upgraded an oVirt installation a month ago to the latest 3.6.7. Before<br>
it was 3.6.0 if I remember correctly.<br>
Everything went fine so far for a month or so.<br>
<br>
A couple of days ago the the default gateway got rebooted and the physical<br>
server hosting the HE decided to shut down the vm because it could not ping<br>
the gateway.<br>
The other host restarted the hevm but it now has *no nic*.<br>
As a workaround I attached a virtio nic via virsh but every time the vm gets<br>
restarted the nic get lost<br>
<br>
After a bit of troubleshooting and digging this is what I found:<br>
<br>
This is the /var/run/ovirt-hosted-engine-h<wbr>a/vm.conf which, as far as I<br>
understand, gets extracted from the HE storage domain<br>
<br>
emulatedMachine=pc<br>
vmId=497f5e4a-0c76-441a-b72e-7<wbr>24d7092d07e<br>
smp=2<br>
memSize=6144<br>
spiceSecureChannels=smain,sdis<wbr>play,sinputs,scursor,splayback<wbr>,srecord,ssmartcard,susbredir<br>
vmName=HostedEngine<br>
display=vnc<br>
devices={index:0,iface:virtio,<wbr>format:raw,bootOrder:1,address<wbr>:{slot:0x06,bus:0x00,domain:<wbr>0x0000,type:pci,function:0x0},<wbr>volumeID:bb3218ba-cbe9-4cd0-<wbr>b50b-931deae992f7,imageID:<wbr>d65b82e2-2ad1-<br>
4f4f-bfad-0277c37f2808,readonl<wbr>y:false,domainID:2c3585cc-<wbr>b7bc-4881-85b3-aa6514991a26,<wbr>deviceId:d65b82e2-2ad1-4f4f-<wbr>bfad-0277c37f2808,poolID:<wbr>00000000-0000-0000-0000-<wbr>000000000000,device:disk,<br>
shared:exclusive,propagateErro<wbr>rs:off,type:disk}<br>
devices={index:2,iface:ide,sha<wbr>red:false,readonly:true,device<wbr>Id:8c3179ac-b322-4f5c-9449-<wbr>c52e3665e0ae,address:{controll<wbr>er:0,target:0,unit:0,bus:1,<wbr>type:drive},device:cdrom,path:<wbr>,type:disk}<br>
devices={device:cirrus,alias:v<wbr>ideo0,type:video,deviceId:a994<wbr>68b6-02d4-4a77-8f94-e5df806030<wbr>f6,address:{slot:0x02,bus:<wbr>0x00,domain:0x0000,type:pci,<wbr>function:0x0}}<br>
devices={device:virtio-serial,<wbr>type:controller,deviceId:b7580<wbr>676-19fb-462f-a61e-677b65ad920<wbr>a,address:{slot:0x03,bus:0x00,<wbr>domain:0x0000,type:pci,<wbr>function:0x0}}<br>
devices={device:usb,type:contr<wbr>oller,deviceId:c63092b3-7bd8-<wbr>4b54-bcd3-51f34dce478a,address<wbr>:{slot:0x01,bus:0x00,domain:<wbr>0x0000,type:pci,function:0x2}}<br>
devices={device:ide,type:contr<wbr>oller,deviceId:c77c2c01-6ccc-<wbr>404b-b8d6-5a7f0631a52f,address<wbr>:{slot:0x01,bus:0x00,domain:<wbr>0x0000,type:pci,function:0x1}}<br>
<br>
As you can see there is no nic, and there is no nic in the qemu-kvm<br>
command-line:<br>
qemu 23290 1 14 00:23 ? 01:44:26 /usr/libexec/qemu-kvm -name<br>
HostedEngine -S -machine pc-i440fx-rhel7.2.0,accel=kvm,<wbr>usb=off -cpu<br>
qemu64,-svm -m 6144 -realtime mlock=off -s<br>
mp 2,sockets=2,cores=1,threads=1 -uuid 497f5e4a-0c76-441a-b72e-724d70<wbr>92d07e<br>
-smbios type=1,manufacturer=oVirt,prod<wbr>uct=oVirt<br>
Node,version=7-2.1511.el7.cent<wbr>os.2.10,serial=4C4C4544-004B-<wbr>571<br>
0-8044-B9C04F5A3732,uuid=497f5<wbr>e4a-0c76-441a-b72e-724d7092d07<wbr>e<br>
-no-user-config -nodefaults -chardev<br>
socket,id=charmonitor,path=/va<wbr>r/lib/libvirt/qemu/domain-Host<wbr>edEngine/monitor.sock,serve<br>
r,nowait -mon chardev=charmonitor,id=monitor<wbr>,mode=control -rtc<br>
base=2016-11-30T23:23:26,drift<wbr>fix=slew -global<br>
kvm-pit.lost_tick_policy=disca<wbr>rd -no-hpet -no-reboot -boot strict=on -device<br>
piix3-usb-uhci,id=usb,bus=pci.<wbr>0,addr=0x1.0x2 -device<br>
virtio-serial-pci,id=virtio-se<wbr>rial0,max_ports=16,bus=pci.0,<wbr>addr=0x3 -drive<br>
file=/var/run/vdsm/storage/2c3<wbr>585cc-b7bc-4881-85b3-aa6514<br>
991a26/d65b82e2-2ad1-4f4f-bfad<wbr>-0277c37f2808/bb3218ba-cbe9-<wbr>4cd0-b50b-931deae992f7,if=<wbr>none,id=drive-virtio-disk0,for<wbr>mat=raw,serial=d65b82e2-2ad1-<wbr>4f4f-bfad-0277c37f2808,cache=<wbr>none,werror=st<br>
op,rerror=stop,aio=native -device<br>
virtio-blk-pci,scsi=off,bus=pc<wbr>i.0,addr=0x6,drive=drive-virti<wbr>o-disk0,id=virtio-disk0,bootin<wbr>dex=1<br>
-drive if=none,id=drive-ide0-1-0,read<wbr>only=on,format=raw<br>
-device ide-cd,bus=ide.1,unit=0,drive=<wbr>drive-ide0-1-0,id=ide0-1-0 -chardev<br>
socket,id=charchannel0,path=/v<wbr>ar/lib/libvirt/qemu/channels/4<wbr>97f5e4a-0c76-441a-b72e-724d709<wbr>2d07e.com.redhat.rhevm<br>
.vdsm,server,nowait -device<br>
virtserialport,bus=virtio-seri<wbr>al0.0,nr=1,chardev=charchannel<wbr>0,id=channel0,name=com.redhat.<wbr>rhevm.vdsm<br>
-chardev socket,id=charchannel1,path=/v<wbr>ar/lib/libvirt/qem<br>
u/channels/497f5e4a-0c76-441a-<wbr>b72e-724d7092d07e.org.qemu.gue<wbr>st_agent.0,server,nowait<br>
-device<br>
virtserialport,bus=virtio-seri<wbr>al0.0,nr=2,chardev=charchannel<wbr>1,id=channel1,name=org.qemu.<wbr>guest<br>
_agent.0 -chardev<br>
socket,id=charchannel2,path=/v<wbr>ar/lib/libvirt/qemu/channels/4<wbr>97f5e4a-0c76-441a-b72e-724d709<wbr>2d07e.org.ovirt.hosted-engine-<wbr>setup.0,server,nowait<br>
-device virtserialport,bus<br>
=virtio-serial0.0,nr=3,chardev<wbr>=charchannel2,id=channel2,<wbr>name=org.ovirt.hosted-engine-<wbr>setup.0<br>
-vnc 0:0,password -device cirrus-vga,id=video0,bus=pci.0<wbr>,addr=0x2 -msg<br>
timestamp=on<br>
<br>
I extracted the vm.conf from the storage domain and the nic is there:<br>
mId=497f5e4a-0c76-441a-b72e-72<wbr>4d7092d07e<br>
memSize=6144<br>
display=vnc<br>
devices={index:2,iface:ide,add<wbr>ress:{ controller:0, target:0,unit:0, bus:1,<br>
type:drive},specParams:{},read<wbr>only:true,deviceId:857b98b3-<wbr>cf43-4c2d-8061-e7f105234a65,<wbr>path:,device:cdrom,shared<br>
:false,type:disk}<br>
devices={index:0,iface:virtio,<wbr>format:raw,poolID:00000000-000<wbr>0-0000-0000-000000000000,volum<wbr>eID:bb3218ba-cbe9-4cd0-b50b-<wbr>931deae992f7,imageID:d65b82e2-<wbr>2ad1-4f4f-bfad-0277c37f2808,<wbr>specParams<br>
:{},readonly:false,domainID:2c<wbr>3585cc-b7bc-4881-85b3-aa651499<wbr>1a26,optional:false,deviceId:<wbr>d65b82e2-2ad1-4f4f-bfad-<wbr>0277c37f2808,address:{bus:<wbr>0x00,<br>
slot:0x06, domain:0x0000, type:pci, funct<br>
ion:0x0},device:disk,shared:ex<wbr>clusive,propagateErrors:off,ty<wbr>pe:disk,bootOrder:1}<br>
devices={device:scsi,model:vir<wbr>tio-scsi,type:controller}<br>
devices={nicModel:pv,macAddr:0<wbr>0:16:3e:7d:d8:27,linkActive:tr<wbr>ue,network:ovirtmgmt,filter:vd<wbr>sm-no-mac-spoofing,specParams:<wbr>{},deviceId:5be8a089-9f51-<wbr>46dc-a8bd-28422985aa35,<wbr>address:{bus:0x00<br>
, slot:0x03, domain:0x0000, type:pci,<br>
function:0x0},device:bridge,ty<wbr>pe:interface}<br>
devices={device:console,specPa<wbr>rams:{},type:console,deviceId:<wbr>1644f556-a4ff-4c93-8945-<wbr>5aa165de2a85,alias:console0}<br>
vmName=HostedEngine<br>
spiceSecureChannels=smain,sdis<wbr>play,sinputs,scursor,splayback<wbr>,srecord,ssmartcard,susbredir<br>
smp=2<br>
cpuType=SandyBridge<br>
emulatedMachine=pc<br>
<br>
The local vm.conf gets continuosly overwritten but for some reason the nic<br>
line gets lost in the process.<br>
</blockquote>
Can you please check/share /var/log/ovirt-hosted-engine-h<wbr>a/agent.log?<br>
Preferably all of it (including backups)? Thanks.<br>
</blockquote>
<br>
-- <br></div></div><span class="gmail-HOEnZb"><font color="#888888">
Mammoli Cristian<br>
System administrator<br>
T. <a href="tel:%2B39%200731%2022911" value="+39073122911" target="_blank">+39 0731 22911</a><br>
Via Brodolini 6 | 60035 Jesi (an)</font></span><div class="gmail-HOEnZb"><div class="gmail-h5"><br>
<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
</div></div></blockquote></div><br></div></div>