[Users] error using cli on ovirt 3.1
Juan Hernandez
jhernand at redhat.com
Wed Feb 6 16:34:31 UTC 2013
On 02/06/2013 05:09 PM, Jim Kinney wrote:
> bad admin: the pki restore did not include some vital parts (.keystore !)
>
> SPM host is live now but others won't activate. Seems the database
> restore dropped a function:
Make sure you have the "postgresql-contrib" package installed, then try
this:
# psql -U postgres engine
engine# create extension "uuid-ossp";
After that restart the engine.
>
> 2013-02-06 11:06:55,650 ERROR
> [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-3-thread-49)
> [58e7592f] Command org.ovirt.engine.core.bll.ActivateVdsCommand throw
> exception: org.springframework.jdbc.BadSqlGrammarException:
> CallableStatementCallback; bad SQL grammar [{call
> insert_entity_snapshot(?, ?, ?, ?, ?, ?, ?, ?)}]; nested exception is
> org.postgresql.util.PSQLException: ERROR: function uuid_generate_v1()
> does not exist
> Hint: No function matches the given name and argument types. You
> might need to add explicit type casts.
> Where: PL/pgSQL function "insert_entity_snapshot" line 4 at SQL statement
> at
> org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:97)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:952) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:985)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:368)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:342)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:164)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:124)
> [engine-dal.jar:]
> at
> org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:37)
> [engine-dal.jar:]
> at
> org.ovirt.engine.core.dao.BusinessEntitySnapshotDAODbFacadeImpl.save(BusinessEntitySnapshotDAODbFacadeImpl.java:84)
> [engine-dal.jar:]
> at
> org.ovirt.engine.core.bll.context.DefaultCompensationContext.stateChanged(DefaultCompensationContext.java:138)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.ActivateVdsCommand$1.runInTransaction(ActivateVdsCommand.java:60)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.ActivateVdsCommand$1.runInTransaction(ActivateVdsCommand.java:53)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204)
> [engine-utils.jar:]
> at
> org.ovirt.engine.core.bll.ActivateVdsCommand.executeCommand(ActivateVdsCommand.java:53)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.CommandBase.ExecuteWithoutTransaction(CommandBase.java:804)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:896)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1203)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204)
> [engine-utils.jar:]
> at
> org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142)
> [engine-utils.jar:]
> at
> org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109)
> [engine-utils.jar:]
> at
> org.ovirt.engine.core.bll.CommandBase.Execute(CommandBase.java:911)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:268) [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommands(MultipleActionsRunner.java:182)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:162)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:84)
> [engine-bll.jar:]
> at
> org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:64)
> [engine-utils.jar:]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [rt.jar:1.7.0_09-icedtea]
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> [rt.jar:1.7.0_09-icedtea]
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> [rt.jar:1.7.0_09-icedtea]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> [rt.jar:1.7.0_09-icedtea]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [rt.jar:1.7.0_09-icedtea]
> at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]
> Caused by: org.postgresql.util.PSQLException: ERROR: function
> uuid_generate_v1() does not exist
> Hint: No function matches the given name and argument types. You
> might need to add explicit type casts.
> Where: PL/pgSQL function "insert_entity_snapshot" line 4 at SQL statement
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
> [postgresql-jdbc.jar:]
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795)
> [postgresql-jdbc.jar:]
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
> [postgresql-jdbc.jar:]
> at
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
> [postgresql-jdbc.jar:]
> at
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
> [postgresql-jdbc.jar:]
> at
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:360)
> [postgresql-jdbc.jar:]
> at
> org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297)
> at
> org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404)
> at
> org.springframework.jdbc.core.JdbcTemplate$5.doInCallableStatement(JdbcTemplate.java:987)
> [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
> at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:936) [spring-jdbc-2.5.6.SEC02.jar:2.5.6.SEC02]
>
>
> On Wed, Feb 6, 2013 at 10:46 AM, Jim Kinney <jim.kinney at gmail.com
> <mailto:jim.kinney at gmail.com>> wrote:
>
>
>
> On Wed, Feb 6, 2013 at 10:18 AM, Jim Kinney <jim.kinney at gmail.com
> <mailto:jim.kinney at gmail.com>> wrote:
>
> progress. Restored pki files from backup. Still had to reset
> AdminPassword. Able to login to the gui. All hosts are
> "unresponsive". The SPM host is just totally locked (but fine
> from it's console - idle).
>
> Tried to reinstall one of the hosts and got a new error message:
>
> Error:
>
> vmhost5:
>
> * size must be between 0 and 50
>
>
> vmhost2 is locked as SPM host and nothing seems to allow it to be
> reset. Found the following in engine.log:
>
> 2013-02-06 10:42:51,809 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand]
> (QuartzScheduler_Worker-64) XML RPC error in command
> GetCapabilitiesVDS ( Vds: vmhost2 ), the error was:
> java.util.concurrent.ExecutionException:
> java.lang.reflect.InvocationTargetException,
> SunCertPathBuilderException: unable to find valid certification path
> to requested target
> 2013-02-06 10:42:51,818 ERROR
> [org.ovirt.engine.core.engineencryptutils.EncryptionUtils]
> (QuartzScheduler_Worker-57) Failed to decryptData must start with zero
> 2013-02-06 10:42:52,513 ERROR
> [org.ovirt.engine.core.engineencryptutils.EncryptionUtils]
> (QuartzScheduler_Worker-62) Failed to decryptData must start with zero
> 2013-02-06 10:42:53,245 ERROR
> [org.ovirt.engine.core.engineencryptutils.EncryptionUtils]
> (QuartzScheduler_Worker-60) Failed to decryptData must start with zero
>
> This repeats every few seconds.
>
> None of the vmhosts have the usual vdsm running profiles. They all have:
>
> 19343 ? S< 0:00 /bin/bash -e /usr/share/vdsm/respawn
> --minlifetime 10 --daemon --masterpid /var/run/vdsm/respawn.pid
> /usr/share/vdsm/vdsm
> 19346 ? S<l 0:02 /usr/bin/python /usr/share/vdsm/vdsm
> 19366 ? S< 0:00 /usr/bin/sudo -n /usr/bin/python
> /usr/share/vdsm/supervdsmServer.py
> 414b69cd-383b-48af-a82a-7f1d042608e3 19346
> 19367 ? S<l 0:00 /usr/bin/python
> /usr/share/vdsm/supervdsmServer.py
> 414b69cd-383b-48af-a82a-7f1d042608e3 19346
>
>
> Expected to see may [vdsmd] entries in ps ax output.
>
>
>
> On Wed, Feb 6, 2013 at 9:42 AM, Juan Hernandez
> <jhernand at redhat.com <mailto:jhernand at redhat.com>> wrote:
>
> On 02/06/2013 03:32 PM, Jim Kinney wrote:
>
> The pki folder is likely to be a problem but the backups
> folder is
> populated. Is there a way to remove client certs from
> hosts to restore
> access with a host add process?
>
>
> If you don't have the pki folder you have two problems.
> First is that some data in the database is encrypted, namely
> the AdminPassword. That you can solve with "engine-config -s
> AdminPassword=interactive". Second is the certificates of
> the hosts, the easy way to solve that is to re-install them
> (from the ovirt-engine GUI, no need to re-install the
> operating system) that will generate new certificates.
>
>
> On Feb 6, 2013 9:24 AM, "Juan Hernandez"
> <jhernand at redhat.com <mailto:jhernand at redhat.com>
> <mailto:jhernand at redhat.com
> <mailto:jhernand at redhat.com>>> wrote:
>
> On 02/06/2013 03:02 PM, Jim Kinney wrote:
>
> as things stand now:
>
> I manually reinstalled 3.1, then dropped the
> engine database and
> restored from the backup. There were some
> errors at the end.
> Even though
> I used all the same passwords, the
> admin at internal account was not
> working. Used engine-config -s
> LocalAdminPassword='*****' to
> fix. On log
> in, everything is down, offline, unreachable.
> No hosts can be
> contacted.
> No storage is connected. Can't add a new host.
>
> crud.
>
> I copied the database backup and removed all
> the db creation part
> leaving just the data "copy into..." section
> (that was fun). Ran
> engine-cleanup then engine-setup then tried to
> restore just the
> data.
>
> no joy there either.
>
> The system is CentOS 6.3 as are the hosts. This
> ran wonderfully
> until I
> goofed trying to get the cli and sdk updated.
> Without the database
> working, I have no way to know what vm is what
> in the ISCSI LVM
> storage
> system to even export to another platform.
>
> So I'm assuming my next step is panic (or total
> reinstall from bare
> iron?). I'm setting this up at work and today
> is my last day as I'm
> moving to a new job at a totally different
> organization. I'd hate to
> walk out and lose all the windows VMs and
> templates that were
> built over
> the last 2 months.
>
>
> Do you still have the original backup of the
> database and the
> contents of the original /etc/pki/ovirt-engine
> directory? With those
> two things it is possible to recover.
>
> I would suggest the following procedure:
>
> 1. Make a clean installation of 3.1, exactly the
> same version that
> you had before trying to update (make a backup of
> the database and
> of the /etc/pki/ovirt-engine directory before, just
> in case). During
> this installation use the answers that you used
> during the initial
> installation (specially the passwords).
>
> 2. Stop the engine, then drop and recover the
> database as you
> already did.
>
> 3. Restore the contents of the
> /etc/pki/ovirt-engine directory.
>
> 4. Start the engine.
>
> You should be able to log in with the same
> credentials that you used
> in the original installation.
>
>
> On Wed, Feb 6, 2013 at 8:43 AM, Jim Kinney
> <jim.kinney at gmail.com <mailto:jim.kinney at gmail.com>
> <mailto:jim.kinney at gmail.com
> <mailto:jim.kinney at gmail.com>>
> <mailto:jim.kinney at gmail.com
> <mailto:jim.kinney at gmail.com>
> <mailto:jim.kinney at gmail.com
> <mailto:jim.kinney at gmail.com>>>__> wrote:
>
> added 3.2 lines to dre ovirt yum repo (and
> disabled 3.1 -
> probably
> not good) and did engine-upgrade.
>
> Process choked at opening the CA cert and
> proceeded to
> "rollback".
> Didn't actually roll back as 3.1 repo was
> disabled.
>
> System still has 3.2 installed. Did yum
> update to pull in the
> cli/sdk 3.2 (wish I had done that first!).
>
> Engine starts but fails to open CA to run
> gui. found
> following in log:
>
> 2013-02-05 14:02:40,825 ERROR
> [org.ovirt.engine.core.
> engineencryptutils.____EncryptionUtils]
> (MSC service thread
> 1-16) Can't
> load keystore from file
> "/etc/pki/ovirt-engine/.____keystore".
>
> IOException: DerInputStream.getLength():
> lengthTag=109, too
> big.
> 2013-02-05 14:02:40,826 ERROR
>
>
> [org.ovirt.engine.core.____engineencryptutils.____EncryptionUtils]
> (MSC
>
> service thread 1-16) Failed to decrypt
> java.io.IOException:
> DerInputStream.getLength(): lengthTag=109,
> too big.
> 2013-02-05 14:02:40,827 ERROR
>
>
> [org.ovirt.engine.core.dal.____dbbroker.generic.____DBConfigUtils]
> (MSC
>
> service thread 1-16) Failed to decrypt
> value for property
> TruststorePass will be used encrypted value
> 2013-02-05 14:02:40,829 WARN
>
> [org.ovirt.engine.core.utils.____ConfigUtilsBase] (MSC
>
> service thread
> 1-16) Could not find enum value for
> option: CertificatePassword
> 2013-02-05 14:02:40,830 ERROR
>
>
> [org.ovirt.engine.core.____engineencryptutils.____EncryptionUtils]
> (MSC
>
> service thread 1-16) Can't load keystore
> from file
> "/etc/pki/ovirt-engine/.____keystore".
> IOException:
>
> DerInputStream.getLength(): lengthTag=109,
> too big.
> 2013-02-05 14:02:40,830 ERROR
>
>
> [org.ovirt.engine.core.____engineencryptutils.____EncryptionUtils]
> (MSC
>
> service thread 1-16) Failed to decrypt
> java.io.IOException:
> DerInputStream.getLength(): lengthTag=109,
> too big.
> 2013-02-05 14:02:40,831 ERROR
>
>
> [org.ovirt.engine.core.dal.____dbbroker.generic.____DBConfigUtils]
> (MSC
>
> service thread 1-16) Failed to decrypt
> value for property
> LocalAdminPassword will be used encrypted
> value
> 2013-02-05 14:02:40,833 ERROR
>
>
> [org.ovirt.engine.core.____engineencryptutils.____EncryptionUtils]
> (MSC
>
> service thread 1-16) Can't load keystore
> from file
> "/etc/pki/ovirt-engine/.____keystore".
> IOException:
>
> DerInputStream.getLength(): lengthTag=109,
> too big.
> 2013-02-05 14:02:40,834 ERROR
>
>
> [org.ovirt.engine.core.____engineencryptutils.____EncryptionUtils]
> (MSC
>
> service thread 1-16) Failed to decrypt
> java.io.IOException:
> DerInputStream.getLength(): lengthTag=109,
> too big.
>
>
> On Tue, Feb 5, 2013 at 6:11 AM, Michael
> Pasternak
> <mpastern at redhat.com
> <mailto:mpastern at redhat.com> <mailto:mpastern at redhat.com
> <mailto:mpastern at redhat.com>>
> <mailto:mpastern at redhat.com
> <mailto:mpastern at redhat.com> <mailto:mpastern at redhat.com
> <mailto:mpastern at redhat.com>>>> wrote:
>
>
> Hi Jim,
>
> On 02/04/2013 08:33 PM, Jim Kinney wrote:
> > I'm trying to setup a way to
> restart a large group
> of windows
> vms on a schedule. I'm getting a
> connection failure
> that seems
> related to the use of https but I'm
> not sure.
> >
> > error: __init__() got an unexpected
> keyword argument
> 'source_address'
>
> This error is caused by running
> ovirt-sdk on a older
> version of
> python (less then python27),
> please upgrade your sdk/cli with one
> shipped in 3.2 (it's
> backward compatible to 3.1).
>
> >
> > I ran:
> > ovirt-shell -A <path to server
> cert/certfile
> exported from
> browser> -c
> >
> > and my .ovirtshellrc is:
> >
> > [ovirt-shell]
> > username = "admin at internal"
> > url = https://my.internal.url/api
> > #insecure = False
> > #filter = False
> > #timeout = -1
> > password = **********************
> >
> >
> > I tried putting the ca_cert = <path
> to cert> but
> that clearly
> was not allowed in .ovirtshellrc
>
> not related, but supported in 3.2 cli.
>
> >
> > ideas?
> > --
> > --
> > James P. Kinney III
> > ////
> > ////Every time you stop a school,
> you will have to
> build a
> jail. What you gain at one end you
> lose at the other.
> It's like
> feeding a dog on his own tail. It
> won't fatten
> > the dog.
> > - Speech 11/23/1900 Mark Twain
> > ////
> > http://electjimkinney.org
> >
> http://heretothereideas.__blog__spot.com/
> <http://blogspot.com/>
> <http://heretothereideas.__blogspot.com/
> <http://heretothereideas.blogspot.com/>>
> > ////
> >
> >
> >
> >
> ___________________________________________________
>
> > Users mailing list
> > Users at ovirt.org
> <mailto:Users at ovirt.org> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>>
> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org> <mailto:Users at ovirt.org
> <mailto:Users at ovirt.org>>>
> >
> http://lists.ovirt.org/____mailman/listinfo/users
> <http://lists.ovirt.org/__mailman/listinfo/users>
>
>
> <http://lists.ovirt.org/__mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>>
>
>
> --
>
> Michael Pasternak
> RedHat, ENG-Virtualization R&D
>
>
>
>
> --
> --
> James P. Kinney III
> ////
> ////Every time you stop a school, you will
> have to build a
> jail.
> What you gain at one end you lose at the
> other. It's like
> feeding a
> dog on his own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
> ////
> http://electjimkinney.org
> http://heretothereideas.__blog__spot.com/
> <http://blogspot.com/>
>
> <http://heretothereideas.__blogspot.com/
> <http://heretothereideas.blogspot.com/>>
> ////
>
>
>
>
> --
> --
> James P. Kinney III
> ////
> ////Every time you stop a school, you will have
> to build a jail.
> What
> you gain at one end you lose at the other. It's
> like feeding a
> dog on
> his own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
> ////
> http://electjimkinney.org
> http://heretothereideas.__blog__spot.com/
> <http://blogspot.com/>
> <http://heretothereideas.__blogspot.com/
> <http://heretothereideas.blogspot.com/>>
> ////
>
>
> ___________________________________________________
>
> Users mailing list
> Users at ovirt.org <mailto:Users at ovirt.org>
> <mailto:Users at ovirt.org <mailto:Users at ovirt.org>>
> http://lists.ovirt.org/____mailman/listinfo/users
> <http://lists.ovirt.org/__mailman/listinfo/users>
>
>
> <http://lists.ovirt.org/__mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>>
>
>
>
> --
> Dirección Comercial: C/Jose Bardasano Baos, 9,
> Edif. Gorbea 3,
> planta 3ºD, 28016 Madrid, Spain
> Inscrita en el Reg. Mercantil de Madrid – C.I.F.
> B82657941 - Red Hat
> S.L.
>
>
>
> _________________________________________________
> Users mailing list
> Users at ovirt.org <mailto:Users at ovirt.org>
> http://lists.ovirt.org/__mailman/listinfo/users
> <http://lists.ovirt.org/mailman/listinfo/users>
>
>
>
> --
> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea
> 3, planta 3ºD, 28016 Madrid, Spain
> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 -
> Red Hat S.L.
>
>
>
>
> --
> --
> James P. Kinney III
> ////
> ////Every time you stop a school, you will have to build a jail.
> What you gain at one end you lose at the other. It's like
> feeding a dog on his own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
> ////
> http://electjimkinney.org
> http://heretothereideas.blogspot.com/
> ////
>
>
>
>
> --
> --
> James P. Kinney III
> ////
> ////Every time you stop a school, you will have to build a jail.
> What you gain at one end you lose at the other. It's like feeding a
> dog on his own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
> ////
> http://electjimkinney.org
> http://heretothereideas.blogspot.com/
> ////
>
>
>
>
> --
> --
> James P. Kinney III
> ////
> ////Every time you stop a school, you will have to build a jail. What
> you gain at one end you lose at the other. It's like feeding a dog on
> his own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
> ////
> http://electjimkinney.org
> http://heretothereideas.blogspot.com/
> ////
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
--
Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3ºD, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
More information about the Users
mailing list