the restart was not on the SPM so it should not have caused this task to
not be cleaned,
i think that the task was just still running.
On Thu, Jul 5, 2018 at 10:44 AM, Tal Nisan <tnisan(a)redhat.com> wrote:
The task is stuck and never cleared after the restart or we just have
to
wait for it to be finished and cleared?
On Thu, Jul 5, 2018 at 12:30 PM, Dafna Ron <dron(a)redhat.com> wrote:
> Jira opened:
https://ovirt-jira.atlassian.net/browse/OVIRT-2286
>
> Tal, can you help fix the test?
>
> thanks,
> Dafna
>
>
> On Wed, Jul 4, 2018 at 7:41 PM, Dafna Ron <dron(a)redhat.com> wrote:
>
>> The vdsm restart is a test in basic_sanity.py
>> The task that is stuck is downloadImageFromStream
>>
>> 2018-07-04 10:12:52,659-04 WARN [org.ovirt.engine.core.bll.sto
>> rage.domain.DeactivateStorageDomainWithOvfUpdateCommand] (default
>> task-1) [ce1c28ba-1550-457f-b5e3-ad051488f897] There are running tasks:
>> 'AsyncTask:{commandId='0b18c13f-0fce-
>> 4303-8f85-ae5a2b051991',
rootCommandId='0b18c13f-0fce-4303-8f85-ae5a2b051991',
>> storagePoolId='8dd2fe5a-9dca-42e2-8593-1de1b18b4887',
>> taskId='f2af86fb-dbbb-430c-afd9-2f25131583b1',
>> vdsmTaskId='45ee0fc8-830d-47d5-9c4a-6d4ed72ae6a1', stepId=
>> 'null', taskType='downloadImageFromStream',
status='running'}
>>
>> The task succeeded but after the deactivate storage domain attempt:
>>
>> 2018-07-04 10:13:01,579-04 INFO [org.ovirt.engine.core.bll.Ser
>> ialChildCommandsExecutionCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-98)
>> [1c611686] Command 'ProcessOvfUpdateForStorageDomain' id:
>> '6342fce4-96ff-4ae3-8b40-8155a
>> 5509761' child commands '[0b18c13f-0fce-4303-8f85-ae5a2b051991,
>> 8e16826f-93dd-442a-8a74-13d14222d45e]' executions were completed,
>> status 'SUCCEEDED'
>>
>>
>> we should have a solution in ost for locked objects failing jobs... will
>> open a Jira to follow
>>
>>
>>
>> On Wed, Jul 4, 2018 at 6:16 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>>
>>> On Wed, Jul 4, 2018 at 6:46 PM Dafna Ron <dron(a)redhat.com> wrote:
>>>
>>>> The actual test has failed with error:
>>>>
>>>> 2018-07-04 10:12:52,665-04 ERROR [org.ovirt.engine.api.restapi.
>>>> resource.AbstractBackendResource] (default task-1) [] Operation
>>>> Failed: [Cannot deactivate Storage while there are running tasks on this
>>>> Storage.
>>>>
>>>> -Please wait until tasks will finish and try again.]
>>>>
>>>> However, there is a problem with vdsm on host-1. it restarts which may
>>>> cause the issue with the running tasks.
>>>>
>>>
>>> Who restarted vdsm? why?
>>>
>>>
>>>> 2018-07-04 10:08:46,603-0400 INFO (ioprocess/5191) [IOProcessClient]
>>>> shutdown requested (__init__:108)
>>>> 2018-07-04 10:08:46,604-0400 INFO (MainThread) [storage.udev]
>>>> Stopping multipath event listener (udev:149)
>>>> 2018-07-04 10:08:46,604-0400 INFO (MainThread) [vdsm.api] FINISH
>>>> prepareForShutdown return=None from=internal,
task_id=bf3b87e4-febf-4cb6-8bfa-5840fc926b49
>>>> (api:52)
>>>> 2018-07-04 10:08:46,605-0400 INFO (MainThread) [vds] Stopping threads
>>>> (vdsmd:160)
>>>> 2018-07-04 10:08:46,605-0400 INFO (MainThread) [vds] Exiting
>>>> (vdsmd:171)
>>>> 2018-07-04 10:10:00,145-0400 INFO (MainThread) [vds] (PID: 14034) I
>>>> am the actual vdsm 4.20.32-1.el7 lago-basic-suite-4-2-host-1
>>>> (3.10.0-862.2.3.el7.x86_64) (vdsmd:149)
>>>> 2018-07-04 10:10:00,146-0400 INFO (MainThread) [vds] VDSM will run
>>>> with cpu affinity: frozenset([1]) (vdsmd:262)
>>>> 2018-07-04 10:10:00,151-0400 INFO (MainThread) [storage.HSM] START
>>>> HSM init (hsm:366)
>>>> 2018-07-04 10:10:00,154-0400 INFO (MainThread) [storage.HSM] Creating
>>>> data-center mount directory '/rhev/data-center/mnt' (hsm:373)
>>>> 2018-07-04 10:10:00,154-0400 INFO (MainThread) [storage.fileUtils]
>>>> Creating directory: /rhev/data-center/mnt mode: None (fileUtils:197)
>>>> 2018-07-04 10:10:00,265-0400 INFO (MainThread) [storage.HSM]
>>>> Unlinking file '/rhev/data-center/8dd2fe5a-9d
>>>> ca-42e2-8593-1de1b18b4887/44eba8db-3a9c-4fbe-ba33-a039fcd561e1'
>>>> (hsm:523)
>>>> 2018-07-04 10:10:00,266-0400 INFO (MainThread) [storage.HSM]
>>>> Unlinking file '/rhev/data-center/8dd2fe5a-9d
>>>> ca-42e2-8593-1de1b18b4887/mastersd' (hsm:523)
>>>> 2018-07-04 10:10:00,266-0400 INFO (MainThread) [storage.HSM]
>>>> Unlinking file '/rhev/data-center/8dd2fe5a-9d
>>>> ca-42e2-8593-1de1b18b4887/c7980a1e-91ef-4095-82eb-37ec03da9b3f'
>>>> (hsm:523)
>>>> 2018-07-04 10:10:00,267-0400 INFO (MainThread) [storage.HSM]
>>>> Unlinking file '/rhev/data-center/8dd2fe5a-9d
>>>> ca-42e2-8593-1de1b18b4887/4fc62763-d8a5-4c36-8687-91870a92ff05'
>>>> (hsm:523)
>>>> 2018-07-04 10:10:00,267-0400 INFO (MainThread) [storage.HSM]
>>>> Unlinking file '/rhev/data-center/8dd2fe5a-9d
>>>> ca-42e2-8593-1de1b18b4887/02363608-01b9-4176-b7a1-e9ee235f792a'
>>>> (hsm:523)
>>>> 2018-07-04 10:10:00,267-0400 INFO (MainThread) [storage.udev]
>>>> Registering multipath event monitor <vdsm.storage.mpathhealth.Monitor
>>>> object at 0x7f9aa4548150> (udev:182)
>>>> 2018-07-04 10:10:00,267-0400 INFO (MainThread) [storage.udev]
>>>> Starting multipath event listener (udev:116)
>>>> 2018-07-04 10:10:00,298-0400 INFO (MainThread) [storage.check]
>>>> Starting check service (check:91)
>>>> 2018-07-04 10:10:00,303-0400 INFO (MainThread) [storage.Dispatcher]
>>>> Starting StorageDispatcher... (di
>>>>
>>>> On Wed, Jul 4, 2018 at 4:12 PM, Greg Sheremeta
<gshereme(a)redhat.com>
>>>> wrote:
>>>>
>>>>> """
>>>>> Error: Fault reason is "Operation Failed". Fault detail is
"[Cannot
>>>>> deactivate Storage while there are running tasks on this Storage.
>>>>> -Please wait until tasks will finish and try again.]". HTTP
response
>>>>> code is 409.
>>>>> """
>>>>>
>>>>>
>>>>> On Wed, Jul 4, 2018 at 10:56 AM Ehud Yonasi
<eyonasi(a)redhat.com>
>>>>> wrote:
>>>>>
>>>>>> Link to job:
>>>>>>
http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/2571/
>>>>>>
>>>>>> Suspected error:
>>>>>>
>>>>>> Error occured, aborting
>>>>>> Traceback (most recent call last):
>>>>>> File
"/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 362, in do_run
>>>>>> self.cli_plugins[args.ovirtverb].do_run(args)
>>>>>> File
"/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run
>>>>>> self._do_run(**vars(args))
>>>>>> File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 505, in wrapper
>>>>>> return func(*args, **kwargs)
>>>>>> File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 516, in wrapper
>>>>>> return func(*args, prefix=prefix, **kwargs)
>>>>>> File
"/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 99, in
do_ovirt_runtest
>>>>>> raise RuntimeError('Some tests failed')
>>>>>> RuntimeError: Some tests failed
>>>>>> + res=1
>>>>>> + cd -
>>>>>>
/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests
>>>>>> + return 1
>>>>>> + failed=true
>>>>>> + env_collect
/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ovirt-system-tests/test_logs/basic-suite-4.2/post-007_sd_reattach.py
>>>>>>
>>>>>> _______________________________________________
>>>>>> Infra mailing list -- infra(a)ovirt.org
>>>>>> To unsubscribe send an email to infra-leave(a)ovirt.org
>>>>>> Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
>>>>>> oVirt Code of Conduct:
https://www.ovirt.org/communit
>>>>>> y/about/community-guidelines/
>>>>>> List Archives:
https://lists.ovirt.org/archiv
>>>>>>
es/list/infra(a)ovirt.org/message/JVGNSGB3HDB3E7Y7CTQR2FCJ3KEXMLFW/
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> GREG SHEREMETA
>>>>>
>>>>> SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
>>>>>
>>>>> Red Hat NA
>>>>>
>>>>> <
https://www.redhat.com/>
>>>>>
>>>>> gshereme(a)redhat.com IRC: gshereme
>>>>> <
https://red.ht/sig>
>>>>>
>>>>> _______________________________________________
>>>>> Infra mailing list -- infra(a)ovirt.org
>>>>> To unsubscribe send an email to infra-leave(a)ovirt.org
>>>>> Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
>>>>> oVirt Code of Conduct:
https://www.ovirt.org/communit
>>>>> y/about/community-guidelines/
>>>>> List Archives:
https://lists.ovirt.org/archiv
>>>>> es/list/infra(a)ovirt.org/message/GRWM4DDAGDV52QM3AWMKLWJP7MNI7S5O/
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Infra mailing list -- infra(a)ovirt.org
>>>> To unsubscribe send an email to infra-leave(a)ovirt.org
>>>> Privacy Statement:
https://www.ovirt.org/site/privacy-policy/
>>>> oVirt Code of Conduct:
https://www.ovirt.org/communit
>>>> y/about/community-guidelines/
>>>> List Archives:
https://lists.ovirt.org/archiv
>>>> es/list/infra(a)ovirt.org/message/EPWWZ3WCO43XEDB53XPJYZL4PGTKOYDQ/
>>>>
>>>
>>
>