On Tue, Feb 21, 2017 at 11:47 AM, Fred Rolland <frolland@redhat.com> wrote:
Add before the command (with your db password): PGPASSWORD=engine

for example:
PGPASSWORD=engine /usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh -T

PGPASSWORD=engine /usr/share/ovirt-engine/setup/dbutils/unlock_entity.sh -t disk -u engine -q


From taskcleaner, if I use the "-T" option I get error

[root@ovmgr1 ovirt-engine]# PGPASSWORD=my_pwd /usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh -d engine -u engine -T
 t
ERROR:  column "job_id" does not exist
LINE 1: ...created_at,status,return_value,return_value_class,job_id,ste...
                                                             ^
FATAL: Cannot execute sql command: --command=SELECT command_id,command_type,root_command_id,command_parameters,command_params_class,created_at,status,return_value,return_value_class,job_id,step_id,executed FROM GetAllCommandsWithRunningTasks();

I see the function GetAllCommandsWithRunningTasks as defined only in /usr/share/ovirt-engine/setup/dbutils/taskcleaner_sp_3_5.sql
and it seems it makes query on commands_entities, but if I directly go inside db, the table doesn't contain indeed a job_id column....
 
I'm on 4.1 upgraded from 4.0.6

engine=# \d command_entities
                          Table "public.command_entities"
        Column         |           Type           |            Modifiers            
-----------------------+--------------------------+---------------------------------
 command_id            | uuid                     | not null
 command_type          | integer                  | not null
 root_command_id       | uuid                     | 
 command_parameters    | text                     | 
 command_params_class  | character varying(256)   | 
 created_at            | timestamp with time zone | 
 status                | character varying(20)    | default NULL::character varying
 callback_enabled      | boolean                  | default false
 callback_notified     | boolean                  | default false
 return_value          | text                     | 
 return_value_class    | character varying(256)   | 
 executed              | boolean                  | default false
 user_id               | uuid                     | 
 parent_command_id     | uuid                     | 
 data                  | text                     | 
 engine_session_seq_id | bigint                   | 
 command_context       | text                     | 
Indexes:
    "pk_command_entities" PRIMARY KEY, btree (command_id)
    "idx_root_command_id" btree (root_command_id) WHERE root_command_id IS NOT NULL
Referenced by:
    TABLE "command_assoc_entities" CONSTRAINT "fk_coco_command_assoc_entity" FOREIGN KEY (command_id) REFERENCES comm
and_entities(command_id) ON DELETE CASCADE

engine=# 

Anyway after unlocking the disk and retrying the move, I get the same error while creating auto snapshot... the first problem on host (that is a different host from the chosen yesterday) seems 

MetaDataKeyNotFoundError: Meta Data key not found error: ("Missing metadata key: 'DOMAIN': found: {'NONE': 


2017-02-21 11:38:58,985 INFO  (jsonrpc/0) [dispatcher] Run and protect: createVolume(sdUUID=u'900b1853-e192-4661-a0f9-7c7c396f6f49', spUUID=u'588237b8-0031-02f6-035d-000000000136', imgUUID=u'f0b5a0e4-ee5d-44a7-ba07-08285791368a', size=u'461708984320', volFormat=4, preallocate=2, diskType=2, volUUID=u'c39c3d9f-dde8-45ab-b4a9-7c3b45c6391d', desc=u'', srcImgUUID=u'f0b5a0e4-ee5d-44a7-ba07-08285791368a', srcVolUUID=u'7ed43974-1039-4a68-a8b3-321e7594fe4c', initialSize=None) (logUtils:49)
2017-02-21 11:38:58,987 INFO  (jsonrpc/0) [IOProcessClient] Starting client ioprocess-6269 (__init__:330)
2017-02-21 11:38:59,006 INFO  (ioprocess/32170) [IOProcess] Starting ioprocess (__init__:452)
2017-02-21 11:38:59,040 INFO  (jsonrpc/0) [dispatcher] Run and protect: createVolume, Return response: None (logUtils:52)
2017-02-21 11:38:59,053 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Volume.create succeeded in 0.07 seconds (__init__:515)
2017-02-21 11:38:59,054 INFO  (tasks/9) [storage.ThreadPool.WorkerThread] START task 08d7797a-af46-489f-ada0-c70bf4359366 (cmd=<bound method Task.commit of <storage.task.Task instance at 0x7f8eb84aacf8>>, args=None) (threadPool:208)
2017-02-21 11:38:59,150 WARN  (tasks/9) [storage.ResourceManager] Resource factory failed to create resource '01_img_900b1853-e192-4661-a0f9-7c7c396f6f49.f0b5a0e4-ee5d-44a7-ba07-08285791368a'. Canceling request. (resourceManager:542)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/resourceManager.py", line 538, in registerResource
    obj = namespaceObj.factory.createResource(name, lockType)
  File "/usr/share/vdsm/storage/resourceFactories.py", line 190, in createResource
    lockType)
  File "/usr/share/vdsm/storage/resourceFactories.py", line 119, in __getResourceCandidatesList
    imgUUID=resourceName)
  File "/usr/share/vdsm/storage/image.py", line 220, in getChain
    if srcVol.isLeaf():
  File "/usr/share/vdsm/storage/volume.py", line 1261, in isLeaf
    return self._manifest.isLeaf()
  File "/usr/share/vdsm/storage/volume.py", line 133, in isLeaf
    return self.getVolType() == sc.type2name(sc.LEAF_VOL)
  File "/usr/share/vdsm/storage/volume.py", line 129, in getVolType
    self.voltype = self.getMetaParam(sc.VOLTYPE)
  File "/usr/share/vdsm/storage/volume.py", line 113, in getMetaParam
    meta = self.getMetadata()
  File "/usr/share/vdsm/storage/blockVolume.py", line 115, in getMetadata
    md = VolumeMetadata.from_lines(lines)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/volumemetadata.py", line 103, in from_lines
    "Missing metadata key: %s: found: %s" % (e, md))
MetaDataKeyNotFoundError: Meta Data key not found error: ("Missing metadata key: 'DOMAIN': found: {'NONE': '######################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################'}",)
2017-02-21 11:38:59,183 WARN  (tasks/9) [storage.ResourceManager.Request] (ResName='01_img_900b1853-e192-4661-a0f9-7c7c396f6f49.f0b5a0e4-ee5d-44a7-ba07-08285791368a', ReqID='571de7e4-6da2-4d40-911e-defb0eb735d8') Tried to cancel a processed request (resourceManager:183)
2017-02-21 11:38:59,184 ERROR (tasks/9) [storage.TaskManager.Task] (Task='08d7797a-af46-489f-ada0-c70bf4359366') Unexpected error (task:870)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 877, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/storage/task.py", line 333, in run
    return self.cmd(*self.argslist, **self.argsdict)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/share/vdsm/storage/sp.py", line 1924, in createVolume
    with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE):
  File "/usr/share/vdsm/storage/resourceManager.py", line 1025, in acquireResource
    return _manager.acquireResource(namespace, name, lockType, timeout=timeout)
  File "/usr/share/vdsm/storage/resourceManager.py", line 473, in acquireResource
    raise se.ResourceAcqusitionFailed()
ResourceAcqusitionFailed: Could not acquire resource. Probably resource factory threw an exception.: ()