Re: [Users] cant remove disks from iscsi domain

Thread-60725::DEBUG::2013-08-22 10:37:45,549::lvm::485::OperationMutex::(_invali datevgs) Operation 'lvm invalidate operation' released the operation mutex Thread-60725::WARNING::2013-08-22 10:37:45,549::blockSD::931::Storage.StorageDom ain::(rmDCVolLinks) Can't unlink /rhev/data-center/mnt/blockSD/d786e2d5-05ab-4da 6-95fc-1af791a3c113/images/5344ca63-302a-43de-9193-da7937fbdfad/dfefc573-de85-40 85-8900-da271affe831. [Errno 2] No such file or directory: '/rhev/data-center/mn t/blockSD/d786e2d5-05ab-4da6-95fc-1af791a3c113/images/5344ca63-302a-43de-9193-da 7937fbdfad/dfefc573-de85-4085-8900-da271affe831' Thread-60725::WARNING::2013-08-22 10:37:45,549::blockSD::931::Storage.StorageDom ain::(rmDCVolLinks) Can't unlink /rhev/data-center/mnt/blockSD/d786e2d5-05ab-4da 6-95fc-1af791a3c113/images/5344ca63-302a-43de-9193-da7937fbdfad/c6cd6d1d-b70f-43 5d-bdc7-713b445a2326. [Errno 2] No such file or directory: '/rhev/data-center/mn t/blockSD/d786e2d5-05ab-4da6-95fc-1af791a3c113/images/5344ca63-302a-43de-9193-da 7937fbdfad/c6cd6d1d-b70f-435d-bdc7-713b445a2326' Thread-60725::DEBUG::2013-08-22 10:37:45,549::blockSD::934::Storage.StorageDomai n::(rmDCVolLinks) removed: [] Thread-60725::ERROR::2013-08-22 10:37:45,549::task::833::TaskManager.Task::(_set Error) Task=`83867bdc-48cd-4ba0-b453-6f8abbace13e`::Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 840, in _run return fn(*args, **kargs) File "/usr/share/vdsm/logUtils.py", line 42, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/storage/hsm.py", line 1460, in deleteImage dom.deleteImage(sdUUID, imgUUID, volsByImg) File "/usr/share/vdsm/storage/blockSD.py", line 957, in deleteImage self.rmDCImgDir(imgUUID, volsImgs) File "/usr/share/vdsm/storage/blockSD.py", line 943, in rmDCImgDir self.log.warning("Can't rmdir %s. %s", imgPath, exc_info=True) File "/usr/lib64/python2.6/logging/__init__.py", line 1068, in warning self._log(WARNING, msg, args, **kwargs) File "/usr/lib64/python2.6/logging/__init__.py", line 1173, in _log self.handle(record) File "/usr/lib64/python2.6/logging/__init__.py", line 1183, in handle self.callHandlers(record) File "/usr/lib64/python2.6/logging/__init__.py", line 1220, in callHandlers hdlr.handle(record) File "/usr/lib64/python2.6/logging/__init__.py", line 679, in handle self.emit(record) File "/usr/lib64/python2.6/logging/handlers.py", line 780, in emit msg = self.format(record) File "/usr/lib64/python2.6/logging/__init__.py", line 654, in format return fmt.format(record) File "/usr/lib64/python2.6/logging/__init__.py", line 436, in format record.message = record.getMessage() File "/usr/lib64/python2.6/logging/__init__.py", line 306, in getMessage msg = msg % self.args TypeError: not enough arguments for format string

On 08/21/2013 04:10 PM, Dafna Ron wrote: > there is a is an exception in the log related to a quota calculation > > 2013-08-21 17:52:32,694 ERROR > [org.ovirt.engine.core.utils.timer.SchedulerUtilQu artzImpl] > (DefaultQuartzScheduler_Worker-7) failed to invoke sceduled method upd > ateQuotaCache: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown > Source) > [:1.7.0 _25] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > sorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) > [rt.jar:1.7.0_25] > at > org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java: > 60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > [quartz.jar:] > at > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.j > ava:557) [quartz.jar:] > Caused by: org.springframework.jdbc.BadSqlGrammarException: > PreparedStatementCal lback; bad SQL grammar [select * from > calculateallstorageusage()]; nested excep tion is > org.postgresql.util.PSQLException: ERROR: column "quota_limitation.quota > _id" must appear in the GROUP BY clause or be used in an aggregate > function > Where: PL/pgSQL function "calculateallstorageusage" line 3 at > RETURN QUERY > > > in any case this is a bug. > I'm adding Doron to this mail, perhaps this was reported in the past and > already solved in later versions. > if not it should be reported and fixed. > > Dafna > > If I'm not mistaken, it looks like this bug: https://bugzilla.redhat.com/show_bug.cgi?id=905891 Greg > > > > On 08/21/2013 05:26 PM, Yura Demchenko wrote: >> 21.08.2013 19:18, Dafna Ron пишет: >>> from the logs it appears to be a quota issue. >>> do you have quota enabled? >>> >> Yes, quota in "enforced" mode. But VMs/disks in question belongs to >> "unlimited" quota (defined quota policy with no limits on >> storage/cpu/ram) >> >>> >>> On 08/21/2013 03:20 PM, Yuriy Demchenko wrote: >>>> Hi, >>>> >>>> I've recently encountered a problem with removing disks from iscsi >>>> domain in my test lab - just cant remove any. >>>> Remove operation fails with message "User admin@internal failed to >>>> initiate removing of disk pg-slave1_opt from domain iscsi-store" and >>>> re-elections of SPM. After that - disk is marked as "illegal" in >>>> ovirt >>>> webinterface, however, it is _in fact_ removed from storage - >>>> lvdisplay doesn't show it and free space is updated correctly. >>>> And this happens with just about every disk/vm i try to remove. >>>> >>>> ovirt 3.2.2-el6 >>>> centos 6.4 >>>> vdsm-4.10.3-17.el6 >>>> lvm2-2.02.98-9.el6 >>>> >>>> Any tips? logs in attach >>>> >>>> >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users(a)ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>> >> > >