November 2018 - Devel - oVirt List Archives

ovirt-engine has been tagged (ovirt-engine-4.2.8)
by Tal Nisan 27 Nov '18

27 Nov '18

1 0

Re: splitting tests for stdci v2
by Nir Soffer 27 Nov '18

27 Nov '18

On Mon, Nov 26, 2018 at 3:00 PM Marcin Sobczyk <msobczyk(a)redhat.com> wrote: > Hi, > > I'm currently working on paralleling our stdci v2. > > I've already extracted 'linters' stage, more patches (and more > substages) are on the way. > > This part i.e. : > > if git diff-tree --no-commit-id --name-only -r HEAD | egrep --quiet > 'vdsm.spec.in|Makefile.am|automation' ; then > ./automation/build-artifacts.sh" > ... > > seems to be an excellent candidate for extraction to a separate substage. > > The question is - how should we proceed with tests? I can create > substage for each of: > > tox -e "tests,{storage,lib,network,virt}" > > But the original 'check-patch' combined the coverage reports into one - > we would lose that. > My long term goal is to get rid of all the ugly bash code in the makefile, and run everything via tox, but as first step I think we can split the work by running: make tests In the tests substage, instead of "make check" today. Does it change anything about coverage? Theoretically we can split also to storage/network/virt/infra jobs but I think this will consume too many resources and harm other projects sharing the slaves. > There is a possibility that we could work on something that gathers > coverage data from multiple sources (tests, OST) as a completely > separate jenkins job or smth, but that will be a bigger effort. > What do you think about it? > > Marcin > > > > >

3 3

OST: Warning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
by Yedidyah Bar David 27 Nov '18

27 Nov '18

Hi all, I noticed this warning in OST's lago log, e.g. [1]: 2018-11-27 07:32:44,684::log_utils.py::start_log_task::655::nose::INFO:: # detach_vm_network_from_host_0: [0m [0m 2018-11-27 07:32:50,360::network_utils_v4.py::<genexpr>::119::py.warnings::WARNING:: * Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal 2018-11-27 07:32:50,360::log_utils.py::end_log_task::670::nose::INFO:: # detach_vm_network_from_host_0: [32mSuccess [0m (in 0:00:05) This seems to be a result of OST:basic-suite-master/test_utils/network_utils_v4.py:get_network_attachment , which does: network = next(net for net in dc.networks_service().list() if net.name == network_name) Apparently, dc.networks_service() returns a str, not unicode. Is this normal? Expected? Should the caller (OST) convert to unicode if needed, or the SDK? If so, [2] should fix (pushed just now, check-patch should finish in an hour or so, only then we can know for sure). Adding Ondra for considering the SDK side. For reference: to debug this on my own OST machine, I did this (after running the basic suite): python import ovirtsdk4 as sdk4 engine = sdk4.Connection(url='https://%s/ovirt-engine/api' % '192.168.201.4', username='admin@internal', password='123', insecure=True, debug=True).system_service() dc = engine.data_centers_service().data_center_service(engine.data_centers_service().list(search='name={}'.format('test-dc'))[0].id) [(net.name, type(net.name)) for net in dc.networks_service().list()] Which returned: [('ovirtmgmt', <type 'str'>), ('VM Network with a very long name and \xd7\xa2\xd7\x91\xd7\xa8\xd7\x99\xd7\xaa', <type 'str'>), ('Labeled_Network', <type 'str'>), ('Migration_Net', <type 'str'>)] Best regards, [1] https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11697/artifa… [2] https://gerrit.ovirt.org/95768 -- Didi

1 0

oVirt impersonation and sessions
by Anastasiya Ruzhanskaya 26 Nov '18

26 Nov '18

Hello everyone! I wanted to find out how the impersonation technique used in oVirt works? I know from libvirt developers, that oVirt opens one connection only for multiple clients. How does this work? Also I found out in source code that in ActionParameterBase class the sessionId field is marked transient but, for example, for GWT rpc message, which goes to the server and says what action will be made (shut down, pause vm) this is the only field in all sent information which says what the session is. Where is the session sent instead? There was also a field with session id in https headers, but this was related to cookie so I am not completely sure if this can help to identify the current user.

2 2

Build fail with "# /usr/sbin/groupadd -g 1000 mock" - no test was run
by Nir Soffer 26 Nov '18

26 Nov '18

I have seen this issue in the recent days several times. Here are some examples: - https://jenkins.ovirt.org/job/vdsm_master_check-patch-fc28-x86_64/2269/cons… - https://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/26243/cons… - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/91/console - https://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/26187/cons… There are too issues: - No test ran, so we need to retrigger manually - Gerrit report build FAILURE instead of build ERROR I reported the second issue many times in the last 5 years, no progress yet. Nir

1 0

UI test failing on a vdsm change?
by Dan Kenigsberg 26 Nov '18

26 Nov '18

I just noticed that a vdsm change to gluster tests https://gerrit.ovirt.org/#/c/95596/ failed in the change queue, on WebDriverException in _init_browser connecting to hub https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11365/testRe… The failure is clearly unrelated to the patch; maybe one of you can explain why the test fails?

3 5

Vdsm: Failing test_no_match
by Milan Zamazal 26 Nov '18

26 Nov '18

Hi, test_no_match fails on CI most of the time (but not always) in https://gerrit.ovirt.org/95518: _____________________________ test_no_match[qcow2] _____________________________ 11:30:16 11:30:16 tmpdir = local('/var/tmp/vdsm/test_no_match_qcow2_0'), image_format = 'qcow2' 11:30:16 11:30:16 def test_no_match(tmpdir, image_format): 11:30:16 path = str(tmpdir.join('test.' + image_format)) 11:30:16 op = qemuimg.create(path, '1m', image_format) 11:30:16 op.run() 11:30:16 qemuio.write_pattern(path, image_format, pattern=2) 11:30:16 with pytest.raises(qemuio.VerificationError): 11:30:16 > qemuio.verify_pattern(path, image_format, pattern=4) 11:30:16 11:30:16 storage/qemuio_test.py:59: 11:30:16 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 11:30:16 11:30:16 path = '/var/tmp/vdsm/test_no_match_qcow2_0/test.qcow2', format = 'qcow2' 11:30:16 offset = 512, len = 1024, pattern = 4 11:30:16 11:30:16 def verify_pattern(path, format, offset=512, len=1024, pattern=5): 11:30:16 read_cmd = 'read -P %d -s 0 -l %d %d %d' % (pattern, len, offset, len) 11:30:16 cmd = ['qemu-io', '-f', format, '-c', read_cmd, path] 11:30:16 rc, out, err = commands.execCmd(cmd, raw=True) 11:30:16 if rc != 0 or err != b"": 11:30:16 > raise cmdutils.Error(cmd, rc, out, err) 11:30:16 E Error: Command ['qemu-io', '-f', 'qcow2', '-c', 'read -P 4 -s 0 -l 1024 512 1024', '/var/tmp/vdsm/test_no_match_qcow2_0/test.qcow2'] failed with rc=1 out='Pattern verification failed at offset 512, 1024 bytes\nread 1024/1024 bytes at offset 512\n1 KiB, 1 ops; 0.0002 sec (3.756 MiB/sec and 3846.1538 ops/sec)\n' err='' 11:30:16 11:30:16 storage/qemuio.py:50: Error (Similarly for raw.) You can see the complete test run log here (or in other CI runs of the patch): https://jenkins.ovirt.org/job/vdsm_master_check-patch-fc28-x86_64/2040/cons… It fails on both Fedora and CentOS. It may or may not be related to the fact that QEMU 2.11 is used in the failed runs. Any idea what could be wrong? Thanks, Milan

3 10

vdsm has been tagged (v4.20.44)
by Milan Zamazal 26 Nov '18

26 Nov '18

1 0

[VDSM] TestMount.testSymlinkMount failing
by Germano Veit Michel 26 Nov '18

26 Nov '18

Hi, I have seen CI failing with some "random" errors which makes submitting a series of patches a bit annoying, as some patch always fails with some error, then next run another patch fails. One of these problems is TestMount.testSymlinkMount. Looks like its getting EBUSY when doing a rmdir. So probably something still using it? =================================== FAILURES =================================== __________________________ TestMount.testSymlinkMount __________________________ args = (<storage.mount_test.TestMount testMethod=testSymlinkMount>,) kwargs = {} @wraps(f) def wrapper(*args, **kwargs): if cond: raise SkipTest(reason) > return f(*args, **kwargs) testValidation.py:274: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ testValidation.py:333: in wrapper return f(*args, **kwargs) storage/mount_test.py:167: in testSymlinkMount m.umount() /usr/lib64/python2.7/contextlib.py:35: in __exit__ self.gen.throw(type, value, traceback) testlib.py:218: in namedTemporaryDir shutil.rmtree(tmpDir) /usr/lib64/python2.7/shutil.py:261: in rmtree rmtree(fullname, ignore_errors, onerror) /usr/lib64/python2.7/shutil.py:270: in rmtree onerror(os.rmdir, path, sys.exc_info()) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ path = '/var/tmp/tmpOYMiHx/mountpoint', ignore_errors = False onerror = <function onerror at 0x7fd68afc9938> def rmtree(path, ignore_errors=False, onerror=None): """Recursively delete a directory tree. If ignore_errors is set, errors are ignored; otherwise, if onerror is set, it is called to handle the error with arguments (func, path, exc_info) where func is os.listdir, os.remove, or os.rmdir; path is the argument to that function that caused it to fail; and exc_info is a tuple returned by sys.exc_info(). If ignore_errors is false and onerror is None, an exception is raised. """ if ignore_errors: def onerror(*args): pass elif onerror is None: def onerror(*args): raise try: if os.path.islink(path): # symlinks to directories are forbidden, see bug #1669 raise OSError("Cannot call rmtree on a symbolic link") except OSError: onerror(os.path.islink, path, sys.exc_info()) # can't continue even if onerror hook returns return names = [] try: names = os.listdir(path) except os.error, err: onerror(os.listdir, path, sys.exc_info()) for name in names: fullname = os.path.join(path, name) try: mode = os.lstat(fullname).st_mode except os.error: mode = 0 if stat.S_ISDIR(mode): rmtree(fullname, ignore_errors, onerror) else: try: os.remove(fullname) except os.error, err: onerror(os.remove, fullname, sys.exc_info()) try: > os.rmdir(path) E OSError: [Errno 16] Device or resource busy: '/var/tmp/tmpOYMiHx/mountpoint' /usr/lib64/python2.7/shutil.py:268: OSError ----------------------------- Captured stderr call ----------------------------- 2018-11-07 00:41:55,518 DEBUG (MainThread) [storage.Misc.excCmd] /usr/bin/taskset --cpu-list 0-1 /sbin/mkfs.ext2 -F /var/tmp/tmpOYMiHx/backing.img (cwd None) (commands:68) 2018-11-07 00:41:55,561 DEBUG (MainThread) [storage.Misc.excCmd] SUCCESS: <err> = 'mke2fs 1.44.2 (14-May-2018)\n'; <rc> = 0 (commands:89) 2018-11-07 00:41:55,562 INFO (MainThread) [storage.Mount] mounting /var/tmp/tmpOYMiHx/link_to_image at /var/tmp/tmpOYMiHx/mountpoint (mount:204) 2018-11-07 00:41:55,563 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /usr/bin/mount -o loop /var/tmp/tmpOYMiHx/link_to_image /var/tmp/tmpOYMiHx/mountpoint (cwd None) (commands:68) 2018-11-07 00:41:55,598 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:89) 2018-11-07 00:41:55,599 DEBUG (MainThread) [storage.Mount] /var/tmp/tmpOYMiHx/mountpoint mounted: 0.03 seconds (utils:454) 2018-11-07 00:41:55,599 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /sbin/udevadm settle --timeout=5 (cwd None) (commands:68) 2018-11-07 00:41:55,624 DEBUG (MainThread) [root] SUCCESS: <err> = ''; <rc> = 0 (commands:89) 2018-11-07 00:41:55,625 DEBUG (MainThread) [storage.Mount] Waiting for udev mount events: 0.03 seconds (utils:454) 2018-11-07 00:41:55,628 INFO (MainThread) [storage.Mount] unmounting /var/tmp/tmpOYMiHx/mountpoint (mount:212) 2018-11-07 00:41:55,629 DEBUG (MainThread) [root] /usr/bin/taskset --cpu-list 0-1 /usr/bin/umount /var/tmp/tmpOYMiHx/mountpoint (cwd None) (commands:68) 2018-11-07 00:41:55,658 DEBUG (MainThread) [root] FAILED: <err> = 'umount: /var/tmp/tmpOYMiHx/mountpoint: target is busy.\n'; <rc> = 32 (commands:89) Full logs here: https://jenkins.ovirt.org/job/vdsm_master_check-patch-fc28-x86_64/1811/arti…

2 5

OST fails: python2-pbr requires git-core
by Yedidyah Bar David 25 Nov '18

25 Nov '18

Hi all, https://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-test… fails with: 11:04:30 + yum install --nogpgcheck --downloaddir=/dev/shm -y ntp net-snmp ovirt-engine ovirt-log-collector 'ovirt-engine-extension-aaa-ldap*' otopi-debug-plugins cronie 11:04:30 Error: Package: python2-pbr-4.2.0-2.el7.noarch (alocalsync) 11:04:30 Requires: git-core 11:04:30 + ret=1 Any idea? Thanks and best regards, -- Didi

2 4