Re: [Users] Panic on FreeBSD guest with >1 CPU
27 nov 2012 kl. 16.01 skrev :
27 nov 2012 kl. 15.59 skrev Itamar Heim:
On 11/27/2012 09:56 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 15.42 skrev Itamar Heim:
On 11/27/2012 08:28 AM, Karli Sjöberg wrote:
Hey all!
Since recently patching our hosts, I´ve been having trouble running FreeBSD guests with more than one virtual core or socket. I have managed take a screenshot of how it looks like when it panics when booting kernel, right after ACPI: http://i47.tinypic.com/2u90qrr.png
I´ve tried this with similar results using 8.2-RELEASE, 8.3-RELEASE, 9.0-RELEASE and 9-STABLE.
If I edit the guest to have only one virtual core or socket, it boots up without issue.
Since noticing this, I´ve tried updating the packages one more time, thinking maybe it had already been fixed but no, it remains. These are the package versions I´m using: # rpm -qa | egrep '(kernel|libvirt|qemu|vdsm|seabios)' | sort -d ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.2-4.fc17.x86_64 kernel-3.6.3-1.fc17.x86_64 kernel-3.6.7-4.fc17.x86_64 << This is the one that´s running libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
Do you have any insights as to what the problem might be?
Best Regards Karli Sjöberg _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
the first suspect would be qemu-kvm and maybe the bios, can you please downgrade it to the previously
Oh the fun of downgrading...pass:) But it just so happens that we have another ovirt system running, apart from the production system, that may be less patched. I´ll check and see if there´s any difference there. Is there any data you wish me to share, like logs or something while testing? Or just yay/nay?
if we identify the offending package, and versions, easier to report the regression and ask to fix it
So yay/nay it is! Thanks.
I´ve now tested to create a new FreeBSD server with dual cores in our experiment/test system and it worked, no problemo. oVirt test system - good: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.3.4-5.fc17.x86_64 libvirt-0.9.11.6-1.fc17.x86_64 libvirt-client-0.9.11.6-1.fc17.x86_64 libvirt-daemon-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.6-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.6-1.fc17.x86_64 libvirt-python-0.9.11.6-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.0-1.fc17.x86_64 seabios-bin-1.7.0-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch oVirt prod system - bad: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.7-4.fc17.x86_64 libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
On 11/28/2012 01:52 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 16.01 skrev :
27 nov 2012 kl. 15.59 skrev Itamar Heim:
On 11/27/2012 09:56 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 15.42 skrev Itamar Heim:
On 11/27/2012 08:28 AM, Karli Sjöberg wrote:
Hey all!
Since recently patching our hosts, I´ve been having trouble running FreeBSD guests with more than one virtual core or socket. I have managed take a screenshot of how it looks like when it panics when booting kernel, right after ACPI: http://i47.tinypic.com/2u90qrr.png
I´ve tried this with similar results using 8.2-RELEASE, 8.3-RELEASE, 9.0-RELEASE and 9-STABLE.
If I edit the guest to have only one virtual core or socket, it boots up without issue.
Since noticing this, I´ve tried updating the packages one more time, thinking maybe it had already been fixed but no, it remains. These are the package versions I´m using: # rpm -qa | egrep '(kernel|libvirt|qemu|vdsm|seabios)' | sort -d ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.2-4.fc17.x86_64 kernel-3.6.3-1.fc17.x86_64 kernel-3.6.7-4.fc17.x86_64 << This is the one that´s running libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
Do you have any insights as to what the problem might be?
Best Regards Karli Sjöberg _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
the first suspect would be qemu-kvm and maybe the bios, can you please downgrade it to the previously
Oh the fun of downgrading...pass:) But it just so happens that we have another ovirt system running, apart from the production system, that may be less patched. I´ll check and see if there´s any difference there. Is there any data you wish me to share, like logs or something while testing? Or just yay/nay?
if we identify the offending package, and versions, easier to report the regression and ask to fix it
So yay/nay it is! Thanks.
I´ve now tested to create a new FreeBSD server with dual cores in our experiment/test system and it worked, no problemo.
oVirt test system - good: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.3.4-5.fc17.x86_64 libvirt-0.9.11.6-1.fc17.x86_64 libvirt-client-0.9.11.6-1.fc17.x86_64 libvirt-daemon-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.6-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.6-1.fc17.x86_64 libvirt-python-0.9.11.6-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.0-1.fc17.x86_64 seabios-bin-1.7.0-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
oVirt prod system - bad: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.7-4.fc17.x86_64 libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
so seems like you have different: kernel seabios libvirt can you please upgrade them one by one to find the culprit (I'd do in this order for simplicity of rollback: seabios, libvirt, kernel) thanks, Itamar
28 nov 2012 kl. 09.19 skrev Itamar Heim:
On 11/28/2012 01:52 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 16.01 skrev :
27 nov 2012 kl. 15.59 skrev Itamar Heim:
On 11/27/2012 09:56 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 15.42 skrev Itamar Heim:
On 11/27/2012 08:28 AM, Karli Sjöberg wrote: > Hey all! > > Since recently patching our hosts, I´ve been having trouble running FreeBSD guests with more than one virtual core or socket. I have managed take a screenshot of how it looks like when it panics when booting kernel, right after ACPI: > http://i47.tinypic.com/2u90qrr.png > > I´ve tried this with similar results using 8.2-RELEASE, 8.3-RELEASE, 9.0-RELEASE and 9-STABLE. > > If I edit the guest to have only one virtual core or socket, it boots up without issue. > > Since noticing this, I´ve tried updating the packages one more time, thinking maybe it had already been fixed but no, it remains. These are the package versions I´m using: > # rpm -qa | egrep '(kernel|libvirt|qemu|vdsm|seabios)' | sort -d > ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch > kernel-3.6.2-4.fc17.x86_64 > kernel-3.6.3-1.fc17.x86_64 > kernel-3.6.7-4.fc17.x86_64 << This is the one that´s running > libvirt-0.9.11.7-1.fc17.x86_64 > libvirt-client-0.9.11.7-1.fc17.x86_64 > libvirt-daemon-0.9.11.7-1.fc17.x86_64 > libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 > libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 > libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 > libvirt-python-0.9.11.7-1.fc17.x86_64 > qemu-common-1.0.1-2.fc17.x86_64 > qemu-img-1.0.1-2.fc17.x86_64 > qemu-kvm-1.0.1-2.fc17.x86_64 > qemu-kvm-tools-1.0.1-2.fc17.x86_64 > qemu-system-x86-1.0.1-2.fc17.x86_64 > seabios-1.7.1-1.fc17.x86_64 > seabios-bin-1.7.1-1.fc17.noarch > vdsm-4.10.0-10.fc17.x86_64 > vdsm-cli-4.10.0-10.fc17.noarch > vdsm-python-4.10.0-10.fc17.x86_64 > vdsm-xmlrpc-4.10.0-10.fc17.noarch > > Do you have any insights as to what the problem might be? > > Best Regards > Karli Sjöberg > _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users >
the first suspect would be qemu-kvm and maybe the bios, can you please downgrade it to the previously
Oh the fun of downgrading...pass:) But it just so happens that we have another ovirt system running, apart from the production system, that may be less patched. I´ll check and see if there´s any difference there. Is there any data you wish me to share, like logs or something while testing? Or just yay/nay?
if we identify the offending package, and versions, easier to report the regression and ask to fix it
So yay/nay it is! Thanks.
I´ve now tested to create a new FreeBSD server with dual cores in our experiment/test system and it worked, no problemo.
oVirt test system - good: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.3.4-5.fc17.x86_64 libvirt-0.9.11.6-1.fc17.x86_64 libvirt-client-0.9.11.6-1.fc17.x86_64 libvirt-daemon-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.6-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.6-1.fc17.x86_64 libvirt-python-0.9.11.6-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.0-1.fc17.x86_64 seabios-bin-1.7.0-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
oVirt prod system - bad: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.7-4.fc17.x86_64 libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
so seems like you have different: kernel seabios libvirt
can you please upgrade them one by one to find the culprit (I'd do in this order for simplicity of rollback: seabios, libvirt, kernel)
Done. Kernel is the culprit! We have two hosts in the test system, so what I did was to first update seabios; test, then update libvirt; test, kernel; test on host1, where all was well until after booting new kernel. So I wanted to know for sure that it was only the kernel's blame, so on host2 only kernel was updated and afterwards the issue started appearing. So definitely, kernel. PS. As I expected, downgrading the packages again failed horribly. I friggin' hate trying to downgrade, even for just a few packages, I opt-out and rather just reinstall the whole thing to save me the headache:)
thanks, Itamar
On 11/28/2012 08:39 AM, Karli Sjöberg wrote:
28 nov 2012 kl. 09.19 skrev Itamar Heim:
On 11/28/2012 01:52 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 16.01 skrev :
27 nov 2012 kl. 15.59 skrev Itamar Heim:
On 11/27/2012 09:56 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 15.42 skrev Itamar Heim:
> On 11/27/2012 08:28 AM, Karli Sjöberg wrote: >> Hey all! >> >> Since recently patching our hosts, I´ve been having trouble running FreeBSD guests with more than one virtual core or socket. I have managed take a screenshot of how it looks like when it panics when booting kernel, right after ACPI: >> http://i47.tinypic.com/2u90qrr.png >> >> I´ve tried this with similar results using 8.2-RELEASE, 8.3-RELEASE, 9.0-RELEASE and 9-STABLE. >> >> If I edit the guest to have only one virtual core or socket, it boots up without issue. >> >> Since noticing this, I´ve tried updating the packages one more time, thinking maybe it had already been fixed but no, it remains. These are the package versions I´m using: >> # rpm -qa | egrep '(kernel|libvirt|qemu|vdsm|seabios)' | sort -d >> ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch >> kernel-3.6.2-4.fc17.x86_64 >> kernel-3.6.3-1.fc17.x86_64 >> kernel-3.6.7-4.fc17.x86_64 << This is the one that´s running >> libvirt-0.9.11.7-1.fc17.x86_64 >> libvirt-client-0.9.11.7-1.fc17.x86_64 >> libvirt-daemon-0.9.11.7-1.fc17.x86_64 >> libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 >> libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 >> libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 >> libvirt-python-0.9.11.7-1.fc17.x86_64 >> qemu-common-1.0.1-2.fc17.x86_64 >> qemu-img-1.0.1-2.fc17.x86_64 >> qemu-kvm-1.0.1-2.fc17.x86_64 >> qemu-kvm-tools-1.0.1-2.fc17.x86_64 >> qemu-system-x86-1.0.1-2.fc17.x86_64 >> seabios-1.7.1-1.fc17.x86_64 >> seabios-bin-1.7.1-1.fc17.noarch >> vdsm-4.10.0-10.fc17.x86_64 >> vdsm-cli-4.10.0-10.fc17.noarch >> vdsm-python-4.10.0-10.fc17.x86_64 >> vdsm-xmlrpc-4.10.0-10.fc17.noarch >> >> Do you have any insights as to what the problem might be? >> >> Best Regards >> Karli Sjöberg >> _______________________________________________ >> Users mailing list >> Users@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> > > the first suspect would be qemu-kvm and maybe the bios, can you please downgrade it to the previously
Oh the fun of downgrading...pass:) But it just so happens that we have another ovirt system running, apart from the production system, that may be less patched. I´ll check and see if there´s any difference there. Is there any data you wish me to share, like logs or something while testing? Or just yay/nay?
if we identify the offending package, and versions, easier to report the regression and ask to fix it
So yay/nay it is! Thanks.
I´ve now tested to create a new FreeBSD server with dual cores in our experiment/test system and it worked, no problemo.
oVirt test system - good: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.3.4-5.fc17.x86_64 libvirt-0.9.11.6-1.fc17.x86_64 libvirt-client-0.9.11.6-1.fc17.x86_64 libvirt-daemon-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.6-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.6-1.fc17.x86_64 libvirt-python-0.9.11.6-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.0-1.fc17.x86_64 seabios-bin-1.7.0-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
oVirt prod system - bad: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.7-4.fc17.x86_64 libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
so seems like you have different: kernel seabios libvirt
can you please upgrade them one by one to find the culprit (I'd do in this order for simplicity of rollback: seabios, libvirt, kernel)
Done. Kernel is the culprit!
We have two hosts in the test system, so what I did was to first update seabios; test, then update libvirt; test, kernel; test on host1, where all was well until after booting new kernel. So I wanted to know for sure that it was only the kernel's blame, so on host2 only kernel was updated and afterwards the issue started appearing. So definitely, kernel.
PS. As I expected, downgrading the packages again failed horribly. I friggin' hate trying to downgrade, even for just a few packages, I opt-out and rather just reinstall the whole thing to save me the headache:)
please open a bug on fedora kernel, and paste here. thanks well, i guess next question is does fedora 18 kernel solves the issue...
28 nov 2012 kl. 17.07 skrev Itamar Heim:
On 11/28/2012 08:39 AM, Karli Sjöberg wrote:
28 nov 2012 kl. 09.19 skrev Itamar Heim:
On 11/28/2012 01:52 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 16.01 skrev :
27 nov 2012 kl. 15.59 skrev Itamar Heim:
On 11/27/2012 09:56 AM, Karli Sjöberg wrote: > > 27 nov 2012 kl. 15.42 skrev Itamar Heim: > >> On 11/27/2012 08:28 AM, Karli Sjöberg wrote: >>> Hey all! >>> >>> Since recently patching our hosts, I´ve been having trouble running FreeBSD guests with more than one virtual core or socket. I have managed take a screenshot of how it looks like when it panics when booting kernel, right after ACPI: >>> http://i47.tinypic.com/2u90qrr.png >>> >>> I´ve tried this with similar results using 8.2-RELEASE, 8.3-RELEASE, 9.0-RELEASE and 9-STABLE. >>> >>> If I edit the guest to have only one virtual core or socket, it boots up without issue. >>> >>> Since noticing this, I´ve tried updating the packages one more time, thinking maybe it had already been fixed but no, it remains. These are the package versions I´m using: >>> # rpm -qa | egrep '(kernel|libvirt|qemu|vdsm|seabios)' | sort -d >>> ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch >>> kernel-3.6.2-4.fc17.x86_64 >>> kernel-3.6.3-1.fc17.x86_64 >>> kernel-3.6.7-4.fc17.x86_64 << This is the one that´s running >>> libvirt-0.9.11.7-1.fc17.x86_64 >>> libvirt-client-0.9.11.7-1.fc17.x86_64 >>> libvirt-daemon-0.9.11.7-1.fc17.x86_64 >>> libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 >>> libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 >>> libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 >>> libvirt-python-0.9.11.7-1.fc17.x86_64 >>> qemu-common-1.0.1-2.fc17.x86_64 >>> qemu-img-1.0.1-2.fc17.x86_64 >>> qemu-kvm-1.0.1-2.fc17.x86_64 >>> qemu-kvm-tools-1.0.1-2.fc17.x86_64 >>> qemu-system-x86-1.0.1-2.fc17.x86_64 >>> seabios-1.7.1-1.fc17.x86_64 >>> seabios-bin-1.7.1-1.fc17.noarch >>> vdsm-4.10.0-10.fc17.x86_64 >>> vdsm-cli-4.10.0-10.fc17.noarch >>> vdsm-python-4.10.0-10.fc17.x86_64 >>> vdsm-xmlrpc-4.10.0-10.fc17.noarch >>> >>> Do you have any insights as to what the problem might be? >>> >>> Best Regards >>> Karli Sjöberg >>> _______________________________________________ >>> Users mailing list >>> Users@ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/users >>> >> >> the first suspect would be qemu-kvm and maybe the bios, can you please downgrade it to the previously > > Oh the fun of downgrading...pass:) But it just so happens that we have another ovirt system running, apart from the production system, that may be less patched. I´ll check and see if there´s any difference there. Is there any data you wish me to share, like logs or something while testing? Or just yay/nay?
if we identify the offending package, and versions, easier to report the regression and ask to fix it
So yay/nay it is! Thanks.
I´ve now tested to create a new FreeBSD server with dual cores in our experiment/test system and it worked, no problemo.
oVirt test system - good: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.3.4-5.fc17.x86_64 libvirt-0.9.11.6-1.fc17.x86_64 libvirt-client-0.9.11.6-1.fc17.x86_64 libvirt-daemon-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.6-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.6-1.fc17.x86_64 libvirt-python-0.9.11.6-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.0-1.fc17.x86_64 seabios-bin-1.7.0-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
oVirt prod system - bad: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.7-4.fc17.x86_64 libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
so seems like you have different: kernel seabios libvirt
can you please upgrade them one by one to find the culprit (I'd do in this order for simplicity of rollback: seabios, libvirt, kernel)
Done. Kernel is the culprit!
We have two hosts in the test system, so what I did was to first update seabios; test, then update libvirt; test, kernel; test on host1, where all was well until after booting new kernel. So I wanted to know for sure that it was only the kernel's blame, so on host2 only kernel was updated and afterwards the issue started appearing. So definitely, kernel.
PS. As I expected, downgrading the packages again failed horribly. I friggin' hate trying to downgrade, even for just a few packages, I opt-out and rather just reinstall the whole thing to save me the headache:)
please open a bug on fedora kernel, and paste here. thanks well, i guess next question is does fedora 18 kernel solves the issue...
On 11/29/2012 01:52 AM, Karli Sjöberg wrote:
28 nov 2012 kl. 17.07 skrev Itamar Heim:
On 11/28/2012 08:39 AM, Karli Sjöberg wrote:
28 nov 2012 kl. 09.19 skrev Itamar Heim:
On 11/28/2012 01:52 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 16.01 skrev :
27 nov 2012 kl. 15.59 skrev Itamar Heim:
> On 11/27/2012 09:56 AM, Karli Sjöberg wrote: >> >> 27 nov 2012 kl. 15.42 skrev Itamar Heim: >> >>> On 11/27/2012 08:28 AM, Karli Sjöberg wrote: >>>> Hey all! >>>> >>>> Since recently patching our hosts, I´ve been having trouble running FreeBSD guests with more than one virtual core or socket. I have managed take a screenshot of how it looks like when it panics when booting kernel, right after ACPI: >>>> http://i47.tinypic.com/2u90qrr.png >>>> >>>> I´ve tried this with similar results using 8.2-RELEASE, 8.3-RELEASE, 9.0-RELEASE and 9-STABLE. >>>> >>>> If I edit the guest to have only one virtual core or socket, it boots up without issue. >>>> >>>> Since noticing this, I´ve tried updating the packages one more time, thinking maybe it had already been fixed but no, it remains. These are the package versions I´m using: >>>> # rpm -qa | egrep '(kernel|libvirt|qemu|vdsm|seabios)' | sort -d >>>> ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch >>>> kernel-3.6.2-4.fc17.x86_64 >>>> kernel-3.6.3-1.fc17.x86_64 >>>> kernel-3.6.7-4.fc17.x86_64 << This is the one that´s running >>>> libvirt-0.9.11.7-1.fc17.x86_64 >>>> libvirt-client-0.9.11.7-1.fc17.x86_64 >>>> libvirt-daemon-0.9.11.7-1.fc17.x86_64 >>>> libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 >>>> libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 >>>> libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 >>>> libvirt-python-0.9.11.7-1.fc17.x86_64 >>>> qemu-common-1.0.1-2.fc17.x86_64 >>>> qemu-img-1.0.1-2.fc17.x86_64 >>>> qemu-kvm-1.0.1-2.fc17.x86_64 >>>> qemu-kvm-tools-1.0.1-2.fc17.x86_64 >>>> qemu-system-x86-1.0.1-2.fc17.x86_64 >>>> seabios-1.7.1-1.fc17.x86_64 >>>> seabios-bin-1.7.1-1.fc17.noarch >>>> vdsm-4.10.0-10.fc17.x86_64 >>>> vdsm-cli-4.10.0-10.fc17.noarch >>>> vdsm-python-4.10.0-10.fc17.x86_64 >>>> vdsm-xmlrpc-4.10.0-10.fc17.noarch >>>> >>>> Do you have any insights as to what the problem might be? >>>> >>>> Best Regards >>>> Karli Sjöberg >>>> _______________________________________________ >>>> Users mailing list >>>> Users@ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/users >>>> >>> >>> the first suspect would be qemu-kvm and maybe the bios, can you please downgrade it to the previously >> >> Oh the fun of downgrading...pass:) But it just so happens that we have another ovirt system running, apart from the production system, that may be less patched. I´ll check and see if there´s any difference there. Is there any data you wish me to share, like logs or something while testing? Or just yay/nay? > > if we identify the offending package, and versions, easier to report the regression and ask to fix it
So yay/nay it is! Thanks.
I´ve now tested to create a new FreeBSD server with dual cores in our experiment/test system and it worked, no problemo.
oVirt test system - good: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.3.4-5.fc17.x86_64 libvirt-0.9.11.6-1.fc17.x86_64 libvirt-client-0.9.11.6-1.fc17.x86_64 libvirt-daemon-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.6-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.6-1.fc17.x86_64 libvirt-python-0.9.11.6-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.0-1.fc17.x86_64 seabios-bin-1.7.0-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
oVirt prod system - bad: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.7-4.fc17.x86_64 libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
so seems like you have different: kernel seabios libvirt
can you please upgrade them one by one to find the culprit (I'd do in this order for simplicity of rollback: seabios, libvirt, kernel)
Done. Kernel is the culprit!
We have two hosts in the test system, so what I did was to first update seabios; test, then update libvirt; test, kernel; test on host1, where all was well until after booting new kernel. So I wanted to know for sure that it was only the kernel's blame, so on host2 only kernel was updated and afterwards the issue started appearing. So definitely, kernel.
PS. As I expected, downgrading the packages again failed horribly. I friggin' hate trying to downgrade, even for just a few packages, I opt-out and rather just reinstall the whole thing to save me the headache:)
please open a bug on fedora kernel, and paste here. thanks well, i guess next question is does fedora 18 kernel solves the issue...
thanks. dor/karen - is there anyone who can take a look at this fedora regression causing SMP vm's to panic with a kernel upgrade? thanks, Itamar
On 11/29/2012 09:55 AM, Itamar Heim wrote:
On 11/29/2012 01:52 AM, Karli Sjöberg wrote:
28 nov 2012 kl. 17.07 skrev Itamar Heim:
On 11/28/2012 08:39 AM, Karli Sjöberg wrote:
28 nov 2012 kl. 09.19 skrev Itamar Heim:
On 11/28/2012 01:52 AM, Karli Sjöberg wrote:
27 nov 2012 kl. 16.01 skrev :
> > 27 nov 2012 kl. 15.59 skrev Itamar Heim: > >> On 11/27/2012 09:56 AM, Karli Sjöberg wrote: >>> >>> 27 nov 2012 kl. 15.42 skrev Itamar Heim: >>> >>>> On 11/27/2012 08:28 AM, Karli Sjöberg wrote: >>>>> Hey all! >>>>> >>>>> Since recently patching our hosts, I´ve been having trouble >>>>> running FreeBSD guests with more than one virtual core or >>>>> socket. I have managed take a screenshot of how it looks like >>>>> when it panics when booting kernel, right after ACPI: >>>>> http://i47.tinypic.com/2u90qrr.png >>>>> >>>>> I´ve tried this with similar results using 8.2-RELEASE, >>>>> 8.3-RELEASE, 9.0-RELEASE and 9-STABLE. >>>>> >>>>> If I edit the guest to have only one virtual core or socket, >>>>> it boots up without issue. >>>>> >>>>> Since noticing this, I´ve tried updating the packages one >>>>> more time, thinking maybe it had already been fixed but no, >>>>> it remains. These are the package versions I´m using: >>>>> # rpm -qa | egrep '(kernel|libvirt|qemu|vdsm|seabios)' | sort -d >>>>> ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch >>>>> kernel-3.6.2-4.fc17.x86_64 >>>>> kernel-3.6.3-1.fc17.x86_64 >>>>> kernel-3.6.7-4.fc17.x86_64 << This is the one that´s running >>>>> libvirt-0.9.11.7-1.fc17.x86_64 >>>>> libvirt-client-0.9.11.7-1.fc17.x86_64 >>>>> libvirt-daemon-0.9.11.7-1.fc17.x86_64 >>>>> libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 >>>>> libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 >>>>> libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 >>>>> libvirt-python-0.9.11.7-1.fc17.x86_64 >>>>> qemu-common-1.0.1-2.fc17.x86_64 >>>>> qemu-img-1.0.1-2.fc17.x86_64 >>>>> qemu-kvm-1.0.1-2.fc17.x86_64 >>>>> qemu-kvm-tools-1.0.1-2.fc17.x86_64 >>>>> qemu-system-x86-1.0.1-2.fc17.x86_64 >>>>> seabios-1.7.1-1.fc17.x86_64 >>>>> seabios-bin-1.7.1-1.fc17.noarch >>>>> vdsm-4.10.0-10.fc17.x86_64 >>>>> vdsm-cli-4.10.0-10.fc17.noarch >>>>> vdsm-python-4.10.0-10.fc17.x86_64 >>>>> vdsm-xmlrpc-4.10.0-10.fc17.noarch >>>>> >>>>> Do you have any insights as to what the problem might be? >>>>> >>>>> Best Regards >>>>> Karli Sjöberg >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users@ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>> >>>> >>>> the first suspect would be qemu-kvm and maybe the bios, can >>>> you please downgrade it to the previously >>> >>> Oh the fun of downgrading...pass:) But it just so happens that >>> we have another ovirt system running, apart from the production >>> system, that may be less patched. I´ll check and see if there´s >>> any difference there. Is there any data you wish me to share, >>> like logs or something while testing? Or just yay/nay? >> >> if we identify the offending package, and versions, easier to >> report the regression and ask to fix it > > So yay/nay it is! Thanks. >
I´ve now tested to create a new FreeBSD server with dual cores in our experiment/test system and it worked, no problemo.
oVirt test system - good: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.3.4-5.fc17.x86_64 libvirt-0.9.11.6-1.fc17.x86_64 libvirt-client-0.9.11.6-1.fc17.x86_64 libvirt-daemon-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.6-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.6-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.6-1.fc17.x86_64 libvirt-python-0.9.11.6-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.0-1.fc17.x86_64 seabios-bin-1.7.0-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
oVirt prod system - bad: ipxe-roms-qemu-20120328-1.gitaac9718.fc17.noarch kernel-3.6.7-4.fc17.x86_64 libvirt-0.9.11.7-1.fc17.x86_64 libvirt-client-0.9.11.7-1.fc17.x86_64 libvirt-daemon-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-network-0.9.11.7-1.fc17.x86_64 libvirt-daemon-config-nwfilter-0.9.11.7-1.fc17.x86_64 libvirt-lock-sanlock-0.9.11.7-1.fc17.x86_64 libvirt-python-0.9.11.7-1.fc17.x86_64 qemu-common-1.0.1-2.fc17.x86_64 qemu-img-1.0.1-2.fc17.x86_64 qemu-kvm-1.0.1-2.fc17.x86_64 qemu-kvm-tools-1.0.1-2.fc17.x86_64 qemu-system-x86-1.0.1-2.fc17.x86_64 seabios-1.7.1-1.fc17.x86_64 seabios-bin-1.7.1-1.fc17.noarch vdsm-4.10.0-10.fc17.x86_64 vdsm-cli-4.10.0-10.fc17.noarch vdsm-python-4.10.0-10.fc17.x86_64 vdsm-xmlrpc-4.10.0-10.fc17.noarch
so seems like you have different: kernel seabios libvirt
can you please upgrade them one by one to find the culprit (I'd do in this order for simplicity of rollback: seabios, libvirt, kernel)
Done. Kernel is the culprit!
We have two hosts in the test system, so what I did was to first update seabios; test, then update libvirt; test, kernel; test on host1, where all was well until after booting new kernel. So I wanted to know for sure that it was only the kernel's blame, so on host2 only kernel was updated and afterwards the issue started appearing. So definitely, kernel.
PS. As I expected, downgrading the packages again failed horribly. I friggin' hate trying to downgrade, even for just a few packages, I opt-out and rather just reinstall the whole thing to save me the headache:)
please open a bug on fedora kernel, and paste here. thanks well, i guess next question is does fedora 18 kernel solves the issue...
thanks. dor/karen - is there anyone who can take a look at this fedora regression causing SMP vm's to panic with a kernel upgrade?
Thanks for the report, I assigned it to Marcelo (CCed)
thanks, Itamar
participants (3)
-
Dor Laor -
Itamar Heim -
Karli Sjöberg