5:30 a.m.
Hi list!
Sorry fort the previous E-Mail... problem on my Outlook... :(
Here again...
After a "war-week" I finally got a systemd-script to put the host in
"maintenance" when a shutdown will started.
Now the problem is, that the automatically migration of the VM does NOT
work...
I see in the Web console the host will "Preparing for maintenance" and the
VM will start the migration, then the host is in "maintenance" and a couple of
seconds later the VM will be killed on the other host...
In the Log of the engine I see:
2015-09-23 11:14:17,165 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(org.ovirt.thread.pool-8-thread-30) [683624fe] Correlation ID: 52938d4d, Job ID:
c0efe5b9-0bc3-4c81-9ee7-63ddf90a6afc, Call Stack: null, Custom Event ID: -1, Message: Mig
ration failed while Host is in 'preparing for maintenance' state.
Consider manual intervention: stopping/migrating Vms as Host's state will not
turn to maintenance while VMs are still running on it.(VM: TestVM, Source: vmhost06,
Destination: vmhost03).
2015-09-23 11:14:17,165 INFO [org.ovirt.engine.core.bll.InternalMigrateVmCommand]
(org.ovirt.thread.pool-8-thread-30) [683624fe] Lock freed to object EngineLock
[exclusiveLocks= key: aabf6e76-8387-4441-a328-6a7dc32e2b4d value: VM
, sharedLocks= ]
(see http://pastebin.com/3Ca8W3vE)
On the host I see these two errors:
libvirtEventLoop::ERROR::2015-09-23
11:14:14,690::task::866::Storage.TaskManager.Task::(_setError)
Task=`2670e82a-c9c7-4da6-b6f6-cff7bce25da1`::Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 873, in _run
return fn(*args, **kargs)
File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
res = f(*args, **kwargs)
File "/usr/share/vdsm/storage/hsm.py", line 3209, in inappropriateDevices
fails = supervdsm.getProxy().rmAppropriateRules(thiefId)
File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
**kwargs)
File "<string>", line 2, in rmAppropriateRules
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 755, in
_callmethod
self._connect()
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 742, in
_connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib64/python2.7/multiprocessing/connection.py", line 173, in
Client
c = SocketClient(address)
File "/usr/lib64/python2.7/multiprocessing/connection.py", line 308, in
SocketClient
s.connect(address)
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: [Errno 2] No such file or directory
libvirtEventLoop::ERROR::2015-09-23
11:14:14,696::dispatcher::79::Storage.Dispatcher::(wrapper) [Errno 2] No such file or
directory
Traceback (most recent call last):
File "/usr/share/vdsm/storage/dispatcher.py", line 71, in wrapper
result = ctask.prepare(func, *args, **kwargs)
File "/usr/share/vdsm/storage/task.py", line 103, in wrapper
return m(self, *a, **kw)
File "/usr/share/vdsm/storage/task.py", line 1179, in prepare
raise self.error
error: [Errno 2] No such file or directory
libvirtEventLoop::DEBUG::2015-09-23 11:14:14,697::vm::2799::vm.Vm::(setDownStatus)
vmId=`aabf6e76-8387-4441-a328-6a7dc32e2b4d`::Changed state to Down: User shut down from
within the guest (code=7)
libvirtEventLoop::DEBUG::2015-09-23 11:14:14,698::sampling::425::vm.Vm::(stop)
vmId=`aabf6e76-8387-4441-a328-6a7dc32e2b4d`::Stop statistics collection
Thread-891::ERROR::2015-09-23 11:14:14,704::migration::161::vm.Vm::(_recover)
vmId=`aabf6e76-8387-4441-a328-6a7dc32e2b4d`::'NoneType' object has no attribute
'XMLDesc'
Thread-891::WARNING::2015-09-23 11:14:14,712::vm::1966::vm.Vm::(_set_lastStatus)
vmId=`aabf6e76-8387-4441-a328-6a7dc32e2b4d`::trying to set state to Up when already Down
Thread-891::ERROR::2015-09-23 11:14:14,712::migration::260::vm.Vm::(run)
vmId=`aabf6e76-8387-4441-a328-6a7dc32e2b4d`::Failed to migrate
Traceback (most recent call last):
File "/usr/share/vdsm/virt/migration.py", line 231, in run
self._setupRemoteMachineParams()
File "/usr/share/vdsm/virt/migration.py", line 132, in
_setupRemoteMachineParams
self._machineParams['_srcDomXML'] = self._vm._dom.XMLDesc(0)
AttributeError: 'NoneType' object has no attribute 'XMLDesc'
Can someone help me finding the problem?
Thanks
Mit freundlichen Grüßen
Luca Bertoncello
--
Besuchen Sie unsere Webauftritte:
www.queo.biz Agentur für Markenführung und Kommunikation
www.queoflow.com IT-Consulting und Individualsoftwareentwicklung
Luca Bertoncello
Administrator
Telefon: +49 351 21 30 38 0
Fax: +49 351 21 30 38 99
E-Mail: l.bertoncello(a)queo-group.com
queo GmbH
Tharandter Str. 13
01159 Dresden
Sitz der Gesellschaft: Dresden
Handelsregistereintrag: Amtsgericht Dresden HRB 22352
Geschäftsführer: Rüdiger Henke, André Pinkert
USt-IdNr.: DE234220077
8:41 a.m.
On 23/09/15 10:30, Luca Bertoncello wrote:
Hi list!
Sorry fort the previous E-Mail... problem on my Outlook... :(
Here again...
After a "war-week" I finally got a systemd-script to put the host in
"maintenance" when a shutdown will started.
Now the problem is, that the automatically migration of the VM does NOT
work...
I see in the Web console the host will "Preparing for maintenance" and the
VM will start the migration, then the host is in "maintenance" and a couple of
seconds later the VM will be killed on the other host...
Did the host you are
running the script on shut down before the
migration completed?
If you put the host in maintenance from the GUI, does it successfully
migrate off all VMs?
Alex
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
.
8:54 a.m.
Hi Alex
Did the host you are running the script on shut down before the
migration completed?
Apparently yes...
If you put the host in maintenance from the GUI, does it successfully
migrate off all VMs?
Yes, this happens!
Now, I think, the problem is that the system first unmount /run and then call my script,
so that libvirt has no more possibility to successfully migrate the VMs...
Can someone suggest me a way to call my script as FIRST script on shutdown/reboot and to
block the shutdown/reboot until my script complete?
This will solve the problem...
Of course, I can try with a wrapper for /sbin/shutdown and /sbin/reboot, but this is not a
nice solution...
Thanks
Mit freundlichen Grüßen
Luca Bertoncello
--
Besuchen Sie unsere Webauftritte:
www.queo.biz Agentur für Markenführung und Kommunikation
www.queoflow.com IT-Consulting und Individualsoftwareentwicklung
Luca Bertoncello
Administrator
Telefon: +49 351 21 30 38 0
Fax: +49 351 21 30 38 99
E-Mail: l.bertoncello(a)queo-group.com
queo GmbH
Tharandter Str. 13
01159 Dresden
Sitz der Gesellschaft: Dresden
Handelsregistereintrag: Amtsgericht Dresden HRB 22352
Geschäftsführer: Rüdiger Henke, André Pinkert
USt-IdNr.: DE234220077
9:03 a.m.
On 23/09/15 13:54, Luca Bertoncello wrote:
Hi Alex
> Did the host you are running the script on shut down before the
> migration completed?
Apparently yes...
Thought so. The migration then cannot continue, obviously.
> If you put the host in maintenance from the GUI, does it successfully
> migrate off all VMs?
Yes, this happens!
Now, I think, the problem is that the system first unmount /run and then call my script,
so that libvirt has no more possibility to successfully migrate the VMs...
Can someone suggest me a way to call my script as FIRST script on shutdown/reboot and to
block the shutdown/reboot until my script complete?
This will solve the problem...
Add your systemd script as a "requires" entry in the systemd script
responsible for shutting down the system?
Of course, I can try with a wrapper for /sbin/shutdown and /sbin/reboot, but this is not
a nice solution...
Why don't you manage this from another machine, not from the hosts? Just
have a script call the API to initiate maintenance, wait for the
migration to complete, then call the API to shut down the host?
Or is it really too hard to do this from the GUI? I don't understand why
you have such a hard requirement to be able to do this from the hosts -
the whole point of Ovirt is that you don't have to manage your hosts on
an individual basis!
Alex
Thanks
Mit freundlichen Grüßen
Luca Bertoncello
--
Besuchen Sie unsere Webauftritte:
www.queo.biz Agentur für Markenführung und Kommunikation
www.queoflow.com IT-Consulting und Individualsoftwareentwicklung
Luca Bertoncello
Administrator
Telefon: +49 351 21 30 38 0
Fax: +49 351 21 30 38 99
E-Mail: l.bertoncello(a)queo-group.com
queo GmbH
Tharandter Str. 13
01159 Dresden
Sitz der Gesellschaft: Dresden
Handelsregistereintrag: Amtsgericht Dresden HRB 22352
Geschäftsführer: Rüdiger Henke, André Pinkert
USt-IdNr.: DE234220077
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
.
9:11 a.m.
Hi Alex
Thought so. The migration then cannot continue, obviously.
Obviously... :(
> Can someone suggest me a way to call my script as FIRST script
on
shutdown/reboot and to block the shutdown/reboot until my script
complete?
> This will solve the problem...
Add your systemd script as a "requires" entry in the systemd script
responsible for shutting down the system?
What do you mean? Could you please explain, maybe with an example?
> Of course, I can try with a wrapper for /sbin/shutdown and
/sbin/reboot,
but this is not a nice solution...
Why don't you manage this from another machine, not from the hosts? Just
have a script call the API to initiate maintenance, wait for the migration to
complete, then call the API to shut down the host?
Well, the problem is, that the host MUST be able to shutdown and migrate the VMs if for
example we have a black-out...
The UPS send a signal and NUT start the shutdown.
In this case, the VMs have to migrate.
Or is it really too hard to do this from the GUI? I don't
understand why you
have such a hard requirement to be able to do this from the hosts - the
whole point of Ovirt is that you don't have to manage your hosts on an
individual basis!
Well, the problem is just one: if someone other Admin has to perform some works on the
host that require a reboot, and he MUST log into a GUI to put the host in maintenance
before the shutdown, we have a higher fail-possibility, if he forgot that.
And, of course, automatically shutdown cannot log into the GUI...
So, the best solution is, that the host itself, BEFORE the shutdown begins, send oVirt a
signal and let oVirt put the host in Maintenance, migrating all VMs, and THEN, proceed
with the shutdown...
I think, it MUST be possible, but I can't find how...
Thanks
Mit freundlichen Grüßen
Luca Bertoncello
--
Besuchen Sie unsere Webauftritte:
www.queo.biz Agentur für Markenführung und Kommunikation
www.queoflow.com IT-Consulting und Individualsoftwareentwicklung
Luca Bertoncello
Administrator
Telefon: +49 351 21 30 38 0
Fax: +49 351 21 30 38 99
E-Mail: l.bertoncello(a)queo-group.com
queo GmbH
Tharandter Str. 13
01159 Dresden
Sitz der Gesellschaft: Dresden
Handelsregistereintrag: Amtsgericht Dresden HRB 22352
Geschäftsführer: Rüdiger Henke, André Pinkert
USt-IdNr.: DE234220077
9:22 a.m.
On 23/09/15 14:11, Luca Bertoncello wrote:
Hi Alex
> Thought so. The migration then cannot continue, obviously.
Obviously... :(
>> Can someone suggest me a way to call my script as FIRST script on
> shutdown/reboot and to block the shutdown/reboot until my script
> complete?
>> This will solve the problem...
> Add your systemd script as a "requires" entry in the systemd script
> responsible for shutting down the system?
What do you mean? Could you please explain, maybe with an example?
I've not done anything with systemd myself but it should be entirely
possible. Ask systemd people.
>> Of course, I can try with a wrapper for /sbin/shutdown and /sbin/reboot,
> but this is not a nice solution...
>
> Why don't you manage this from another machine, not from the hosts? Just
> have a script call the API to initiate maintenance, wait for the migration to
> complete, then call the API to shut down the host?
OK, change the shutdown script on the host that NUT calls (ie *not* the
systemd stuff) to do what I said above. That would work. You will have
to be careful with your timing to ensure that migration can finish
before your UPS runs out of juice though. NUT normally only starts the
shutdown process on LOWBATT from the UPS.
> Or is it really too hard to do this from the GUI? I don't understand why you
> have such a hard requirement to be able to do this from the hosts - the
> whole point of Ovirt is that you don't have to manage your hosts on an
> individual basis!
Well, the problem is just one: if someone other Admin has to perform some works on the
host that require a reboot, and he MUST log into a GUI to put the host in maintenance
before the shutdown, we have a higher fail-possibility, if he forgot that.
And, of course, automatically shutdown cannot log into the GUI...
No, but it can have access to the API. You'd have to mess around with
the systemd scripts to do this bit. As above, I think you'd be better
off asking the systemd people than this list - as this is now getting
quite offtopic for here.
Alex
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
.
3383
days inactive
3383
days old
5 comments
2 participants
participants (2)
-
Alex Crow -
Luca Bertoncello