2:27 a.m.
Hi,
I wrote this sample code and the resident memory of the process is
increasing gradually over time. What could be the reason ? I don't see any
obvious leaks in my program. Could it be that the API is not
freeing/deleting memory ?
Regards,
Navin
package ovcollector;
import java.util.*;
import java.io.*;
import java.lang.management.ManagementFactory;
import org.apache.http.client.*;
import org.apache.http.params.*;
import org.ovirt.engine.sdk.Api;
import org.apache.http.pool.*;
import org.ovirt.engine.sdk.decorators.Host;
import org.ovirt.engine.sdk.decorators.HostStatistic;
import org.ovirt.engine.sdk.decorators.StorageDomain;
import org.ovirt.engine.sdk.decorators.VM;
import org.ovirt.engine.sdk.decorators.VMDisk;
import org.ovirt.engine.sdk.decorators.VMDiskStatistic;
import org.ovirt.engine.sdk.decorators.VMStatistic;
import org.ovirt.engine.sdk.decorators.VMs;
import org.ovirt.engine.sdk.entities.CPU;
import org.ovirt.engine.sdk.entities.Value;
import org.ovirt.engine.sdk.exceptions.ServerException;
import org.ovirt.engine.sdk.exceptions.UnsecuredConnectionAttemptError;
import org.ovirt.engine.*;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTable;
import java.awt.Dimension;
import java.awt.GridLayout;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
public class collectHosts extends JPanel {
/**
* @param args
*/
public static String[] columnNames = {"Host Name",
"Host uuid",
"Cpu Cores", "Cpu Sockets",
"Memory.total",
"Memory.used",
"Memory.free",
"Memory.shared","Memory.buffers","Memory.cached","swap.total","swap.free","swap.used","swap.cached","ksm.cpu.current",
"cpu.current.user","cpu.current.system","cpu.current.idle","cpu.load.avg.5m"
};
public static Object[][] data =
{
{"blankvmname", "blankuuid", new Double(0),new Double(0),new
Double(0),new
Double(0),new Double(0),new Double(0),new Double(0), new Double(0),
new Double(0),new Double(0),new Double(0),new Double(0),new Double(0),new
Double(0),new Double(0),new Double(0),new Double(0)},
{"blankvmname", "blankuuid", new Double(0),new Double(0),new
Double(0),new
Double(0),new Double(0),new Double(0),new Double(0), new Double(0),
new Double(0),new Double(0),new Double(0),new Double(0),new Double(0),new
Double(0),new Double(0),new Double(0),new Double(0)}
};
public collectHosts() {
super(new GridLayout(1,0));
final JTable table = new JTable(data, columnNames);
table.setPreferredScrollableViewportSize(new Dimension(500, 70));
table.setFillsViewportHeight(true);
//Create the scroll pane and add the table to it.
JScrollPane scrollPane = new JScrollPane(table);
//Add the scroll pane to this panel.
add(scrollPane);
}
/**
* Create the GUI and show it. For thread safety,
* this method should be invoked from the
* event-dispatching thread.
*/
private static void createAndShowGUI() {
//Create and set up the window.
JFrame frame = new JFrame("Hosts");
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
//Create and set up the content pane.
collectHosts newContentPane = new collectHosts();
newContentPane.setOpaque(true); //content panes must be opaque
frame.setContentPane(newContentPane);
//Display the window.
frame.pack();
frame.setVisible(true);
}
private static final String URL = "http://somehost:80/api";
public static void main(String[] args) throws
ClientProtocolException,ServerException,
UnsecuredConnectionAttemptError,IOException {
// TODO Auto-generated method stub
System.out.println("Hosts");
while (true)
{
Api api = new Api(URL,"SOMEUSERNAME","SOMEPASSWORD");
System.out.println(ManagementFactory.getRuntimeMXBean().getName()) ;
List<Host> hlist = api.getHosts().list();
Integer i=0,j=0;
for(Host obj: hlist)
{
j=0;
System.out.println(obj.getName()+ ":" + obj.getId() +":"+
obj.getMemory()+":" + obj.getCpu().getTopology().getCores() +":"+
obj.getCpu().getTopology().getSockets());
data[i][j++]=obj.getName();
data[i][j++]=obj.getId();
data[i][j++]=obj.getCpu().getTopology().getCores();
data[i][j++]=obj.getCpu().getTopology().getSockets();
List<HostStatistic> lhs=obj.getStatistics().list();
for(HostStatistic obj1: lhs)
{
System.out.println(obj1.getName());
List<Value> vll=obj1.getValues().getValues();
for(Value vl:vll)
{
System.out.println(vl.getDatum());
data[i][j++]=vl.getDatum();
}
}
i++;
}
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
/*
javax.swing.SwingUtilities.invokeLater(new Runnable() {
public void run() {
createAndShowGUI();
}
});
List<StorageDomain> list_SD=api.getStorageDomains().list();
Integer i=0;
for(StorageDomain obj: list_SD)
{
System.out.println(obj.getName()+ " | " + obj.getUsed() + " | " +
obj.getAvailable() + " | " + obj.getType());
}
*/
}
}
7:09 a.m.
Hi Michael,
On Tue, Jan 29, 2013 at 2:42 PM, Michael Pasternak <mpastern(a)redhat.com>wrote:
Hi Navin,
On 01/29/2013 10:27 AM, navin p wrote:
> while (true)
> {
> Api api = new Api(URL,"SOMEUSERNAME","SOMEPASSWORD");
i'd suggest you moving SDK proxy initiation out of this (endless) loop.
I tried that but it didn't help. The Resident memory was increasing in the
top
-p output
Regards,
Navin
8:41 a.m.
Navin,
It's quite hard to see from your piece of code what exactly causing this,
also i can't reproduce this on my environment running very same sdk code,
can you please run your app with some profiling tool attached and send me
the output?
On 01/29/2013 03:09 PM, navin p wrote:
Hi Michael,
On Tue, Jan 29, 2013 at 2:42 PM, Michael Pasternak <mpastern(a)redhat.com
<mailto:mpastern@redhat.com>> wrote:
Hi Navin,
On 01/29/2013 10:27 AM, navin p wrote:
> while (true)
> {
> Api api = new Api(URL,"SOMEUSERNAME","SOMEPASSWORD");
i'd suggest you moving SDK proxy initiation out of this (endless) loop.
I tried that but it didn't help. The Resident memory was increasing in the top -p
output
Regards,
Navin
--
Michael Pasternak
RedHat, ENG-Virtualization R&D
4:50 p.m.
On 01/29/2013 10:27 AM, navin p wrote:
Hi,
I wrote this sample code and the resident memory of the process is
increasing gradually over time. What could be the reason ? I don't see
any obvious leaks in my program. Could it be that the API is not
freeing/deleting memory ?
By monitoring the program, it seems that the failure is due to constant
threads creation by invoking the "new API()" call:
Exception in thread "main" java.lang.OutOfMemoryError: unable to create
new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:691)
at
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.createPoolingClientConnectionManager(ConnectionsPoolBuilder.java:182)
at
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.createDefaultHttpClient(ConnectionsPoolBuilder.java:160)
at
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.build(ConnectionsPoolBuilder.java:234)
at org.ovirt.engine.sdk.Api.<init>(Api.java:82)
at collectHosts.main(collectHosts.java:102)
By pulling the API instantiation outside of the loop, problem solved,
since only a single thread is created to monitor the idle/expired
connections.
Michael, wouldn't you suggest adding some sort of API.shutdown() method
in order to release resources used by it including the connection
monitor and any other live connections if exists?
Regards,
Navin
package ovcollector;
import java.util.*;
import java.io.*;
import java.lang.management.ManagementFactory;
import org.apache.http.client.*;
import org.apache.http.params.*;
import org.ovirt.engine.sdk.Api;
import org.apache.http.pool.*;
import org.ovirt.engine.sdk.decorators.Host;
import org.ovirt.engine.sdk.decorators.HostStatistic;
import org.ovirt.engine.sdk.decorators.StorageDomain;
import org.ovirt.engine.sdk.decorators.VM;
import org.ovirt.engine.sdk.decorators.VMDisk;
import org.ovirt.engine.sdk.decorators.VMDiskStatistic;
import org.ovirt.engine.sdk.decorators.VMStatistic;
import org.ovirt.engine.sdk.decorators.VMs;
import org.ovirt.engine.sdk.entities.CPU;
import org.ovirt.engine.sdk.entities.Value;
import org.ovirt.engine.sdk.exceptions.ServerException;
import org.ovirt.engine.sdk.exceptions.UnsecuredConnectionAttemptError;
import org.ovirt.engine.*;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTable;
import java.awt.Dimension;
import java.awt.GridLayout;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
public class collectHosts extends JPanel {
/**
* @param args
*/
public static String[] columnNames = {"Host Name",
"Host uuid",
"Cpu Cores", "Cpu Sockets",
"Memory.total",
"Memory.used",
"Memory.free",
"Memory.shared","Memory.buffers","Memory.cached","swap.total","swap.free","swap.used","swap.cached","ksm.cpu.current",
"cpu.current.user","cpu.current.system","cpu.current.idle","cpu.load.avg.5m"
};
public static Object[][] data =
{
{"blankvmname", "blankuuid", new Double(0),new Double(0),new
Double(0),new Double(0),new Double(0),new Double(0),new Double(0), new
Double(0),
new Double(0),new Double(0),new Double(0),new Double(0),new
Double(0),new Double(0),new Double(0),new Double(0),new Double(0)},
{"blankvmname", "blankuuid", new Double(0),new Double(0),new
Double(0),new Double(0),new Double(0),new Double(0),new Double(0), new
Double(0),
new Double(0),new Double(0),new Double(0),new Double(0),new
Double(0),new Double(0),new Double(0),new Double(0),new Double(0)}
};
public collectHosts() {
super(new GridLayout(1,0));
final JTable table = new JTable(data, columnNames);
table.setPreferredScrollableViewportSize(new Dimension(500, 70));
table.setFillsViewportHeight(true);
//Create the scroll pane and add the table to it.
JScrollPane scrollPane = new JScrollPane(table);
//Add the scroll pane to this panel.
add(scrollPane);
}
/**
* Create the GUI and show it. For thread safety,
* this method should be invoked from the
* event-dispatching thread.
*/
private static void createAndShowGUI() {
//Create and set up the window.
JFrame frame = new JFrame("Hosts");
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
//Create and set up the content pane.
collectHosts newContentPane = new collectHosts();
newContentPane.setOpaque(true); //content panes must be opaque
frame.setContentPane(newContentPane);
//Display the window.
frame.pack();
frame.setVisible(true);
}
private static final String URL = "http://somehost:80/api";
public static void main(String[] args) throws
ClientProtocolException,ServerException,
UnsecuredConnectionAttemptError,IOException {
// TODO Auto-generated method stub
System.out.println("Hosts");
while (true)
{
Api api = new Api(URL,"SOMEUSERNAME","SOMEPASSWORD");
System.out.println(ManagementFactory.getRuntimeMXBean().getName()) ;
List<Host> hlist = api.getHosts().list();
Integer i=0,j=0;
for(Host obj: hlist)
{
j=0;
System.out.println(obj.getName()+ ":" + obj.getId() +":"+
obj.getMemory()+":" + obj.getCpu().getTopology().getCores() +":"+
obj.getCpu().getTopology().getSockets());
data[i][j++]=obj.getName();
data[i][j++]=obj.getId();
data[i][j++]=obj.getCpu().getTopology().getCores();
data[i][j++]=obj.getCpu().getTopology().getSockets();
List<HostStatistic> lhs=obj.getStatistics().list();
for(HostStatistic obj1: lhs)
{
System.out.println(obj1.getName());
List<Value> vll=obj1.getValues().getValues();
for(Value vl:vll)
{
System.out.println(vl.getDatum());
data[i][j++]=vl.getDatum();
}
}
i++;
}
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
/*
javax.swing.SwingUtilities.invokeLater(new Runnable() {
public void run() {
createAndShowGUI();
}
});
List<StorageDomain> list_SD=api.getStorageDomains().list();
Integer i=0;
for(StorageDomain obj: list_SD)
{
System.out.println(obj.getName()+ " | " + obj.getUsed() + " | " +
obj.getAvailable() + " | " + obj.getType());
}
*/
}
}
_______________________________________________
Engine-devel mailing list
Engine-devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
2:56 a.m.
On 01/31/2013 12:50 AM, Moti Asayag wrote:
--
Michael Pasternak
RedHat, ENG-Virtualization R&D
On 01/29/2013 10:27 AM, navin p wrote:
> > Hi,
> > I wrote this sample code and the resident memory of the process is
> > increasing gradually over time. What could be the reason ? I don't see
> > any obvious leaks in my program. Could it be that the API is not
> > freeing/deleting memory ?
By monitoring the program, it seems that the failure is due to constant
threads creation by invoking the "new API()" call:
Exception in thread "main" java.lang.OutOfMemoryError: unable to create
new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:691)
at
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.createPoolingClientConnectionManager(ConnectionsPoolBuilder.java:182)
at
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.createDefaultHttpClient(ConnectionsPoolBuilder.java:160)
at
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.build(ConnectionsPoolBuilder.java:234)
at org.ovirt.engine.sdk.Api.<init>(Api.java:82)
at collectHosts.main(collectHosts.java:102)
By pulling the API instantiation outside of the loop, problem solved,
since only a single thread is created to monitor the idle/expired
connections.
Thanks Moti,
I already suggested navin to take SDK proxy initiation out of his while loop.
Michael, wouldn't you suggest adding some sort of API.shutdown() method
in order to release resources used by it including the connection
monitor and any other live connections if exists?
no need for that, in SDK i have dedicated thread (watchdog) for that.
3:30 a.m.
On 02/03/2013 10:56 AM, Michael Pasternak wrote:
On 01/31/2013 12:50 AM, Moti Asayag wrote:
> On 01/29/2013 10:27 AM, navin p wrote:
>>> Hi,
>>> I wrote this sample code and the resident memory of the process is
>>> increasing gradually over time. What could be the reason ? I don't see
>>> any obvious leaks in my program. Could it be that the API is not
>>> freeing/deleting memory ?
> By monitoring the program, it seems that the failure is due to constant
> threads creation by invoking the "new API()" call:
>
> Exception in thread "main" java.lang.OutOfMemoryError: unable to create
> new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:691)
> at
>
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.createPoolingClientConnectionManager(ConnectionsPoolBuilder.java:182)
> at
>
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.createDefaultHttpClient(ConnectionsPoolBuilder.java:160)
> at
>
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.build(ConnectionsPoolBuilder.java:234)
> at org.ovirt.engine.sdk.Api.<init>(Api.java:82)
> at collectHosts.main(collectHosts.java:102)
>
>
> By pulling the API instantiation outside of the loop, problem solved,
> since only a single thread is created to monitor the idle/expired
> connections.
Thanks Moti,
I already suggested navin to take SDK proxy initiation out of his while loop.
>
>
> Michael, wouldn't you suggest adding some sort of API.shutdown() method
> in order to release resources used by it including the connection
> monitor and any other live connections if exists?
no need for that, in SDK i have dedicated thread (watchdog) for that.
But what if you wish to instantiate several API classes? each of them
will leave a detached (daemon) thread for monitoring the connections.
I suggested to perform inside the API.shutdown() release of any resource
used by it, including the watchdog thread. Else there is a sort of
thread-leak: when you have no longer reference to the API object, yet
the thread used to clean connections open by it is still running.
>
3:56 a.m.
On 02/03/2013 11:30 AM, Moti Asayag wrote:
On 02/03/2013 10:56 AM, Michael Pasternak wrote:
> On 01/31/2013 12:50 AM, Moti Asayag wrote:
>> On 01/29/2013 10:27 AM, navin p wrote:
>>>> Hi,
>>>> I wrote this sample code and the resident memory of the process is
>>>> increasing gradually over time. What could be the reason ? I don't
see
>>>> any obvious leaks in my program. Could it be that the API is not
>>>> freeing/deleting memory ?
>> By monitoring the program, it seems that the failure is due to constant
>> threads creation by invoking the "new API()" call:
>>
>> Exception in thread "main" java.lang.OutOfMemoryError: unable to
create
>> new native thread
>> at java.lang.Thread.start0(Native Method)
>> at java.lang.Thread.start(Thread.java:691)
>> at
>>
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.createPoolingClientConnectionManager(ConnectionsPoolBuilder.java:182)
>> at
>>
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.createDefaultHttpClient(ConnectionsPoolBuilder.java:160)
>> at
>>
org.ovirt.engine.sdk.web.ConnectionsPoolBuilder.build(ConnectionsPoolBuilder.java:234)
>> at org.ovirt.engine.sdk.Api.<init>(Api.java:82)
>> at collectHosts.main(collectHosts.java:102)
>>
>>
>> By pulling the API instantiation outside of the loop, problem solved,
>> since only a single thread is created to monitor the idle/expired
>> connections.
>
> Thanks Moti,
>
> I already suggested navin to take SDK proxy initiation out of his while loop.
>
>>
>>
>> Michael, wouldn't you suggest adding some sort of API.shutdown() method
>> in order to release resources used by it including the connection
>> monitor and any other live connections if exists?
>
> no need for that, in SDK i have dedicated thread (watchdog) for that.
>
But what if you wish to instantiate several API classes? each of them
will leave a detached (daemon) thread for monitoring the connections.
I suggested to perform inside the API.shutdown() release of any resource
used by it, including the watchdog thread. Else there is a sort of
thread-leak: when you have no longer reference to the API object, yet
the thread used to clean connections open by it is still running.
Good point Moti (in terms of self cleanup), but since SDK is used as infrastructure
and not re-initiated/closed during consuming application lifetime, i'm not concerned
about this,
but again this is definitely something that can be implemented in the one
of next releases.
>>
>
>
--
Michael Pasternak
RedHat, ENG-Virtualization R&D
4097
days inactive
4102
days old
7 comments
3 participants
participants (3)
-
Michael Pasternak -
Moti Asayag -
navin p