[Users] I am using the CentOS build from the 12th. I the engine is dieing with the following error.
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files I restarted ovirt-engine and ran lsof -p 3557 I see hundreds of the same open file. java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore Is anyone else seeing this? Thanks Robert
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here: 1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem? http://gerrit.ovirt.org/#/c/5423 2. You may be having a problem with the file descriptors limit due to other issues. During setup we should increase the file descriptor limit from 1024 (the default) to 65535, but we do that for the "jboss" user instead of for "ovirt". The fix for this issue is in review: http://gerrit.ovirt.org/#/c/5375 The workaround for this is to edit the /etc/security/limits.conf file and replace "jboss" with "ovirt", then restart the engine.
On 06/18/2012 04:47 AM, Juan Hernandez wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
http://gerrit.ovirt.org/#/c/5423 I have been using rpm's but am attempting to compile the code.
2. You may be having a problem with the file descriptors limit due to other issues. During setup we should increase the file descriptor limit from 1024 (the default) to 65535, but we do that for the "jboss" user instead of for "ovirt". The fix for this issue is in review:
http://gerrit.ovirt.org/#/c/5375
The workaround for this is to edit the /etc/security/limits.conf file and replace "jboss" with "ovirt", then restart the engine. I made that change but the web interface still dies after about 15 min or so with the same number of open files and the same errror. I rebooted the system so I am sure the change should have take place.
Thanks Robert
On 06/20/2012 04:09 AM, Robert Middleswarth wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
http://gerrit.ovirt.org/#/c/5423 I have been using rpm's but am attempting to compile the code.
2. You may be having a problem with the file descriptors limit due to other issues. During setup we should increase the file descriptor limit from 1024 (the default) to 65535, but we do that for the "jboss" user instead of for "ovirt". The fix for this issue is in review:
http://gerrit.ovirt.org/#/c/5375
The workaround for this is to edit the /etc/security/limits.conf file and replace "jboss" with "ovirt", then restart the engine. I made that change but the web interface still dies after about 15 min or so with the same number of open files and the same errror. I rebooted
On 06/18/2012 04:47 AM, Juan Hernandez wrote: the system so I am sure the change should have take place.
did you try with this patch applied? http://gerrit.ovirt.org/#/c/5423/
On 06/20/2012 01:36 AM, Itamar Heim wrote:
On 06/20/2012 04:09 AM, Robert Middleswarth wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
http://gerrit.ovirt.org/#/c/5423 I have been using rpm's but am attempting to compile the code.
2. You may be having a problem with the file descriptors limit due to other issues. During setup we should increase the file descriptor limit from 1024 (the default) to 65535, but we do that for the "jboss" user instead of for "ovirt". The fix for this issue is in review:
http://gerrit.ovirt.org/#/c/5375
The workaround for this is to edit the /etc/security/limits.conf file and replace "jboss" with "ovirt", then restart the engine. I made that change but the web interface still dies after about 15 min or so with the same number of open files and the same errror. I rebooted
On 06/18/2012 04:47 AM, Juan Hernandez wrote: the system so I am sure the change should have take place.
did you try with this patch applied? http://gerrit.ovirt.org/#/c/5423/ Not yet. Haven't been able to get a build to complete yet. On my 3rd try.
Thanks Robert
On 06/18/2012 04:47 AM, Juan Hernandez wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
I am still tiring to get a working build environment setup. However dreyou@gmail.com added into his repo and my problem went away after the change was applied. I recommend that this fix gets cherry-picked into the 3.1 tree
2. You may be having a problem with the file descriptors limit due to other issues. During setup we should increase the file descriptor limit from 1024 (the default) to 65535, but we do that for the "jboss" user instead of for "ovirt". The fix for this issue is in review:
http://gerrit.ovirt.org/#/c/5375
The workaround for this is to edit the /etc/security/limits.conf file and replace "jboss" with "ovirt", then restart the engine.
Thanks Robert
On 06/21/2012 07:20 PM, Robert Middleswarth wrote:
On 06/18/2012 04:47 AM, Juan Hernandez wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
I am still tiring to get a working build environment setup. However dreyou@gmail.com added into his repo and my problem went away after the change was applied. I recommend that this fix gets cherry-picked into the 3.1 tree
Ofer, can we do this?
2. You may be having a problem with the file descriptors limit due to other issues. During setup we should increase the file descriptor limit from 1024 (the default) to 65535, but we do that for the "jboss" user instead of for "ovirt". The fix for this issue is in review:
http://gerrit.ovirt.org/#/c/5375
The workaround for this is to edit the /etc/security/limits.conf file and replace "jboss" with "ovirt", then restart the engine.
Thanks Robert
On 06/21/2012 08:20 PM, Robert Middleswarth wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
I am still tiring to get a working build environment setup. However dreyou@gmail.com added into his repo and my problem went away after the change was applied. I recommend that this fix gets cherry-picked into the 3.1 tree
agree. is there an ovirt bug blocking the 3.1 blockers to track it?
On 06/21/2012 08:25 PM, Itamar Heim wrote:
On 06/21/2012 08:20 PM, Robert Middleswarth wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
I am still tiring to get a working build environment setup. However dreyou@gmail.com added into his repo and my problem went away after the change was applied. I recommend that this fix gets cherry-picked into the 3.1 tree
agree. is there an ovirt bug blocking the 3.1 blockers to track it?
Here is the bug: https://bugzilla.redhat.com/834399 Should it block "Tracker: oVirt 3.1 release"?
On 06/21/2012 09:47 PM, Juan Hernandez wrote:
On 06/21/2012 08:25 PM, Itamar Heim wrote:
On 06/21/2012 08:20 PM, Robert Middleswarth wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
I am still tiring to get a working build environment setup. However dreyou@gmail.com added into his repo and my problem went away after the change was applied. I recommend that this fix gets cherry-picked into the 3.1 tree
agree. is there an ovirt bug blocking the 3.1 blockers to track it?
Here is the bug:
https://bugzilla.redhat.com/834399
Should it block "Tracker: oVirt 3.1 release"?
we can debate if it is a blocker or not, but i don't see a reason not to cherry pick it to the 3.1 branch for the build, hence to flag it for tracking under the blocker bug
----- Original Message -----
On 06/21/2012 08:25 PM, Itamar Heim wrote:
On 06/21/2012 08:20 PM, Robert Middleswarth wrote:
On 06/18/2012 04:25 AM, Robert Middleswarth wrote:
2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File "/usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File "/usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files
I restarted ovirt-engine and ran lsof -p 3557
I see hundreds of the same open file.
java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore
Is anyone else seeing this?
There are two issues that might be affecting you here:
1. We may have a leak of file descriptors on the logic that opens the keystore. Can you take a look at the following change? Can you apply it and let us know if it fixes your problem?
I am still tiring to get a working build environment setup. However dreyou@gmail.com added into his repo and my problem went away after the change was applied. I recommend that this fix gets cherry-picked into the 3.1 tree
agree. is there an ovirt bug blocking the 3.1 blockers to track it?
Here is the bug:
https://bugzilla.redhat.com/834399
Should it block "Tracker: oVirt 3.1 release"?
If it's blocking 3.1 from working properly (sounds to me like it is), so yes.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (4)
-
Itamar Heim -
Juan Hernandez -
Ofer Schreiber -
Robert Middleswarth