unicode sandwich in otopi/engine-setup
by Yedidyah Bar David
Hi all,
This is in a sense a continuation of the thread "Why filetransaction
needs to encode the content to utf-8?", but I decided that a new
thread is better.
I started to systematically convert the code to use a unicode
sandwich. I admit it was harder than I expected, and made me think
somewhat differently about the move to python3, and about how
reasonable (or not) it is to develop in the common subset of python2
and python3 vs ditching python2 and moving fully to python3. It seems
like at least parts of our (integration team) code will still have to
run in python2 also in oVirt 4.4, so I guess we'll not have much
choice :-)
Current patches are only for otopi and engine-setup, and are by no
means thorough - I didn't check each and every open() call and similar
ones. But it's enough for getting engine-setup finish successfully on
both python2 and python3 (EL7 and Fedora 29), with some utf-8 inserted
in relevant places of the input (for the plugins already handled).
I didn't bother trying non-utf-8 encodings. Perhaps I should, but it's
not completely clear to me what's the best approach [2].
Currently, you must have both otopi and engine updated to get things
working. If there is demand, I might spend some time
splitting/rebasing/etc to make it possible to update just one of them
and only later the other, but not sure it's worth it.
I don't mind splitting/squashing if it makes reviews simpler, but I
think the patches are ok as-is. These are the bottom patches of each
stack:
otopi: https://gerrit.ovirt.org/102085
engine-setup: https://gerrit.ovirt.org/102934
[1] http://python-future.org/unicode_literals.html
[2] https://stackoverflow.com/questions/4012571/python-which-encoding-is-used...
Thanks and best regards,
--
Didi
5 years, 3 months
Re: Certificate checking on TLS migrations to an IP address
by Milan Zamazal
Daniel P. Berrangé <berrange(a)redhat.com> writes:
> On Wed, Sep 04, 2019 at 03:38:25PM +0200, Milan Zamazal wrote:
>> Hi, I'm trying to add TLS migrations to oVirt, but I've hit a problem
>> with certificate checking.
>
>>
>> oVirt uses the destination host IP address, rather than the host name,
>> in the migration URI passed to virDomainMigrateToURI3. One reason for
>> doing that is that a separate migration network may be used for
>> migrations, while the host name resolves to the management network
>> interface.
>>
>> But it causes a problem with certificate checking. The destination IP
>> address is checked against the name, which is a host name, given in the
>> destination certificate. That means there is mismatch and the migration
>> fails. I don't think it'd be a very good idea to avoid the problem by
>> putting IP addresses into server certificates.
>
> In fact that is *exactly* what you should be doing.
>
> Traditionally certificates were created with the 'common name' field
> holding the fully qualified DNS based hostname for the server.
>
> This was long known to be a problem because it is very common for
> servers to have multiple DNS names, or for clients to use the
> unqualified hostname, or use the IP address(es).
The problem with putting IP addresses into certificates is that the
certificate must be updated each time an IP address changes, is added or
is removed. Doing this in oVirt would be complicated and error-prone.
While host names are stable, host networks and the related IP addresses
may change.
> Thus, the "Subject alt name" extension was created. This allows
> certificates to be created containing multiple hostnames and
> multiple IP addresses. The certificate will be validated correctly
> if any one of those data items matches. When 'subject alt name' is
> present in a certificate, the 'common name' field should be completely
> ignored by compliant TLS clients, so you are free to put whatever
> you want in the common name - hostname or IP address or blah...
We can switch to using Subject Alt Name and we have a patch for that now
based on your advice, but it doesn't solve the problem with tracking IP
address changes and updating the corresponding certificates whenever a
change occurs.
> If you look at our docs, we updated them to illustrate how to
> issue certs containing hostnames + IP addresses:
>
> https://libvirt.org/remote.html#Remote_TLS_server_certificates
>
>>
>> Is there any way to make TLS migrations working under these
>> circumstances? For instance, SPICE remote-viewer allows the client to
>> specify the certificate subject to expect on the host when connecting to
>> it using an IP address. Can (or could) libvirt do something similar?
Would it be possible? We have host names in the certificates under our
control and we know which host name to expect in the certificate
regardless the IP address used for the given connection. Checking the
certificate against a given host name would solve the problem easily and
robustly for us.
>> Or is there any other mechanism to handle this problem?
Thanks,
Milan
5 years, 3 months
Not getting desired output for ovirt backup script.
by smidhunraj＠gmail.com
I am trying to run the code of
https://github.com/wefixit-AT/oVirtBackup/blob/master/backup.py
I am getting output as ovirtsdk.infrastructure.brokers.VMSnapshot object at 0x7fe9e59371d0
for the snippet print vm.snapshots.list(config.get_snapshot_description())
Is this the expected output.
When i run the whole code ..
The log i get is ...
2019-09-23 12:34:47,263: Start backup for: test2-sep20__vm25-183
2019-09-23 12:34:48,978: Snapshot creation started ...
2019-09-23 12:34:48,978: Snapshot created
2019-09-23 12:34:59,926: !!! No snapshot found !!!
2019-09-23 12:34:59,926: All backups done
2019-09-23 12:34:59,926: Backup failured for:
2019-09-23 12:34:59,927: test2-sep20__vm25-183
2019-09-23 12:34:59,927: Some errors occured during the backup, please check the log file
Could you guys please help me.
5 years, 3 months
master CQ: random failures for add_master_storage_domain
by Dusan Fodor
Hello all,
lately i witnessed multiple failures for add_master_storage_domain test,
which were not related to changes themselves, nor any infra issue. One
example can be found here [1].
After investigation with huge help of Milan, issue is that Host falls from
up state to whatever-but-not-up suddenly.
1. add_storage_domain picks a random host that is in up state
2. meantime engine starts fence action for it, so probably something
gone bad with the host; the fence action fails with:
*[org.ovirt.engine.core.bll.pm.FenceProxyLocator]
(EE-ManagedThreadFactory-engineScheduled-Thread-38) [6692895f] Can not run
fence action on host 'lago-basic-suite-master-host-0', no suitable proxy
host was found.*
3. test fails on not being able to attach the domain to non-up
host: *[org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
(default task-1) [] Operation Failed: [Cannot add storage server connection
when Host status is not up]*
For better orientation in failed job's engine log [1], fence action for
host fails at
:46:12,842-04
engine learns it cannot connect storage to host at
:46:16,105-04
The test itself add_master_storage_domain starts at ~ :46:13,753 (according
to lago log).
Could you please check this?
Thanks
[1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15829
[2]
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/15829/arti...
5 years, 3 months
FYI: RAM needed to compile the engine
by Fedor Gavrilov
Hi,
This is all out of curiosity and just FYI.
I noticed that most recent engine master on CentOS 7.7 now requires about 8 GB of memory to compile. Otherwise it will fail during GWT part. 6 GB - not good enough.
Strange thing is about a month ago it worked perfectly fine for me on 4 GB. Also, I am running it on VM, so old rule applies: it is 8 gigs starting memory, not max memory.
I wonder if someone had similar experience.
Fedor
5 years, 3 months
