Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long. So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3] Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test. - So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java - run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar I hope this would serve all of us well, please review and add your benchmarks. PS - this will not run in the CI atm. [1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version): # Run complete. Total time: 00:09:06 Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s Process finished with exit code 0
On 03/23/2017 11:05 AM, Roy Golan wrote:
Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3]
Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s
Process finished with exit code 0
Very interesting Roy, thanks. What was the root cause of the host edit taking minutes?
On Mar 23, 2017 12:16 PM, "Juan Hernández" <jhernand@redhat.com> wrote: On 03/23/2017 11:05 AM, Roy Golan wrote:
Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3]
Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s
Process finished with exit code 0
Very interesting Roy, thanks. Indeed! What was the root cause of the host edit taking minutes? No. Juan - your original thought that the encrypt / decrypt are there due to way too many queries was right. In the env there were around 1800 calls to get all the hosts due to network checks that ran there. We just wanted to check whether the encryption and decryption has any impact in addition, and Roy looked into it and saw it doesn't impact much. _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Hi, I am just wondering whether you tried Roman's Hystrix integration to see what command was so slow? Martin On Thu, Mar 23, 2017 at 11:05 AM, Roy Golan <rgolan@redhat.com> wrote:
Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3]
Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s
Process finished with exit code 0
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Thu, Mar 23, 2017, 12:25 PM Martin Sivak <msivak@redhat.com> wrote:
Hi,
I am just wondering whether you tried Roman's Hystrix integration to see what command was so slow?
Martin
More on that in a different thread. Using only the debug logs it was quite easy to see that it was 13000+ queries to the db instead of one.
On Thu, Mar 23, 2017 at 11:05 AM, Roy Golan <rgolan@redhat.com> wrote:
Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3]
Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s
Process finished with exit code 0
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--Apple-Mail=_6187776C-BF3B-44A1-8637-9137E966A914 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii
On 23 Mar 2017, at 11:37, Roy Golan <rgolan@redhat.com> wrote: =20 =20 =20 On Thu, Mar 23, 2017, 12:25 PM Martin Sivak <msivak@redhat.com = <mailto:msivak@redhat.com>> wrote: Hi, =20 I am just wondering whether you tried Roman's Hystrix integration to see what command was so slow? =20 Martin =20 More on that in a different thread. Using only the debug logs it was = quite easy to see that it was 13000+ queries to the db instead of one.
=20 On Thu, Mar 23, 2017 at 11:05 AM, Roy Golan <rgolan@redhat.com = <mailto:rgolan@redhat.com>> wrote:
Lately we came across an interesting case where = multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that = was raised which we wanted to eliminate was that the decryption we =
which thread? perform on a
fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the = jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the = output as an example[3]
Although The JMH project recommends to create a separate project I = find it would be less trivial to people to contribute benchmarks let alone = just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ = <http://openjdk.java.net/projects/code-tools/jmh/> [2] https://gerrit.ovirt.org/74537 <https://gerrit.ovirt.org/74537> = microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score = error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 = 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 = 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 = 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 = 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 = 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 = 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 = 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 = 0.001 s
Process finished with exit code 0
_______________________________________________ Devel mailing list Devel@ovirt.org <mailto:Devel@ovirt.org> http://lists.ovirt.org/mailman/listinfo/devel = <http://lists.ovirt.org/mailman/listinfo/devel>
Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--Apple-Mail=_6187776C-BF3B-44A1-8637-9137E966A914 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 23 Mar 2017, at 11:37, Roy Golan <<a href="mailto:rgolan@redhat.com" class="">rgolan@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><br class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Thu, Mar 23, 2017, 12:25 PM Martin Sivak <<a href="mailto:msivak@redhat.com" class="">msivak@redhat.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br class="gmail_msg"> <br class="gmail_msg"> I am just wondering whether you tried Roman's Hystrix integration to<br class="gmail_msg"> see what command was so slow?<br class="gmail_msg"><br class="gmail_msg"> Martin<br class="gmail_msg"></blockquote></div><div class=""><br class=""></div><div class="">More on that in a different thread. Using only the debug logs it was quite easy to see that it was 13000+ queries to the db instead of one.</div></div></blockquote><div><br class=""></div>which thread?</div><div><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br class="gmail_msg"> On Thu, Mar 23, 2017 at 11:05 AM, Roy Golan <<a href="mailto:rgolan@redhat.com" class="gmail_msg" target="_blank">rgolan@redhat.com</a>> wrote:<br class="gmail_msg"> > Lately we came across an interesting case where multi-host+mult-networks<br class="gmail_msg"> > resulted editing a host to conclude in minutes. One assumption that was<br class="gmail_msg"> > raised which we wanted to eliminate was that the decryption we perform on a<br class="gmail_msg"> > fence agent password might be taking too long.<br class="gmail_msg"> ><br class="gmail_msg"> > So these days it's an easy task thanks to JMH[1], supplied by the jdk<br class="gmail_msg"> > itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output<br class="gmail_msg"> > as an example[3]<br class="gmail_msg"> ><br class="gmail_msg"> > Although The JMH project recommends to create a separate project I find it<br class="gmail_msg"> > would be less trivial to people to contribute benchmarks let alone just<br class="gmail_msg"> > playing around with current code they want to test.<br class="gmail_msg"> ><br class="gmail_msg"> > - So, (when it will be merged) you add your benchmark under<br class="gmail_msg"> > backend/manager/modules/benchmarks/MyBenchmark.java<br class="gmail_msg"> ><br class="gmail_msg"> > - run it from intellij using the jmh plugin exactly like a unit-test<br class="gmail_msg"> > OR<br class="gmail_msg"> > - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks<br class="gmail_msg"> > OR<br class="gmail_msg"> > - java -jar benchmarks.jar<br class="gmail_msg"> ><br class="gmail_msg"> > I hope this would serve all of us well, please review and add your<br class="gmail_msg"> > benchmarks.<br class="gmail_msg"> ><br class="gmail_msg"> > PS - this will not run in the CI atm.<br class="gmail_msg"> ><br class="gmail_msg"> > [1] <a href="http://openjdk.java.net/projects/code-tools/jmh/" rel="noreferrer" class="gmail_msg" target="_blank">http://openjdk.java.net/projects/code-tools/jmh/</a><br class="gmail_msg"> > [2] <a href="https://gerrit.ovirt.org/74537" rel="noreferrer" class="gmail_msg" target="_blank">https://gerrit.ovirt.org/74537</a> microbenchmarks: Introduce<br class="gmail_msg"> > microbenchmarks using JMH<br class="gmail_msg"> > [3] DecryptionBenchmark output (short version):<br class="gmail_msg"> ><br class="gmail_msg"> > # Run complete. Total time: 00:09:06<br class="gmail_msg"> ><br class="gmail_msg"> > Benchmark Mode Samples Score Score error<br class="gmail_msg"> > Units<br class="gmail_msg"> > b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270<br class="gmail_msg"> > ops/s<br class="gmail_msg"> > b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667<br class="gmail_msg"> > ops/s<br class="gmail_msg"> > b.DecryptionBenchmark.decryption avgt 50 0.010 0.000<br class="gmail_msg"> > s/op<br class="gmail_msg"> > b.DecryptionBenchmark.encryption avgt 50 0.004 0.000<br class="gmail_msg"> > s/op<br class="gmail_msg"> > b.DecryptionBenchmark.decryption sample 5544 0.010 0.000<br class="gmail_msg"> > s/op<br class="gmail_msg"> > b.DecryptionBenchmark.encryption sample 13067 0.004 0.000<br class="gmail_msg"> > s/op<br class="gmail_msg"> > b.DecryptionBenchmark.decryption ss 50 0.014 0.001<br class="gmail_msg"> > s<br class="gmail_msg"> > b.DecryptionBenchmark.encryption ss 50 0.009 0.001<br class="gmail_msg"> > s<br class="gmail_msg"> ><br class="gmail_msg"> > Process finished with exit code 0<br class="gmail_msg"> ><br class="gmail_msg"> ><br class="gmail_msg"> > _______________________________________________<br class="gmail_msg"> > Devel mailing list<br class="gmail_msg"> > <a href="mailto:Devel@ovirt.org" class="gmail_msg" target="_blank">Devel@ovirt.org</a><br class="gmail_msg"> > <a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" class="gmail_msg" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br class="gmail_msg"> </blockquote></div> _______________________________________________<br class="">Devel mailing list<br class=""><a href="mailto:Devel@ovirt.org" class="">Devel@ovirt.org</a><br class="">http://lists.ovirt.org/mailman/listinfo/devel</div></blockquote></div><br class=""></body></html> --Apple-Mail=_6187776C-BF3B-44A1-8637-9137E966A914--
very cool! In terms of scale that sounds great for unit test | benchmarks, assume we have some loaded DB. we can create some benchmarks for launching vms for example, what do you think will be the challenge ? On Thu, Mar 23, 2017 at 12:25 PM Martin Sivak <msivak@redhat.com> wrote:
Hi,
I am just wondering whether you tried Roman's Hystrix integration to see what command was so slow?
Martin
On Thu, Mar 23, 2017 at 11:05 AM, Roy Golan <rgolan@redhat.com> wrote:
Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3]
Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s
Process finished with exit code 0
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
+1, any plans to get this into CI and perhaps using statistics (standard deviation) to give us warnings? I still want to build something like BrowserLab [4], and I'm currently working on selenium + phantomjs in OST. Greg [4] https://code.facebook.com/posts/1368798036482517/browserlab-automated-regres... On Thu, Mar 23, 2017 at 6:40 AM, Eldad Marciano <emarcian@redhat.com> wrote:
very cool! In terms of scale that sounds great for unit test | benchmarks, assume we have some loaded DB. we can create some benchmarks for launching vms for example, what do you think will be the challenge ?
On Thu, Mar 23, 2017 at 12:25 PM Martin Sivak <msivak@redhat.com> wrote:
Hi,
I am just wondering whether you tried Roman's Hystrix integration to see what command was so slow?
Martin
On Thu, Mar 23, 2017 at 11:05 AM, Roy Golan <rgolan@redhat.com> wrote:
Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3]
Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s
Process finished with exit code 0
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Greg Sheremeta, MBA Red Hat, Inc. Sr. Software Engineer gshereme@redhat.com
On Thu, Mar 23, 2017 at 3:17 PM Greg Sheremeta <gshereme@redhat.com> wrote:
+1, any plans to get this into CI and perhaps using statistics (standard deviation) to give us warnings?
Still didn't work the details of producing reports from that. There is a jenkins-jmh plugin that can help https://github.com/blackboard/jmh-jenkins . If anyone has experience with something similar I'd be happy to hear about it.
I still want to build something like BrowserLab [4], and I'm currently
working on
selenium + phantomjs in OST.
I'm very interested in this work. Thanks.
Greg
[4] https://code.facebook.com/posts/1368798036482517/browserlab-automated-regres...
On Thu, Mar 23, 2017 at 6:40 AM, Eldad Marciano <emarcian@redhat.com> wrote:
very cool! In terms of scale that sounds great for unit test | benchmarks, assume we have some loaded DB. we can create some benchmarks for launching vms for example, what do you think will be the challenge ?
On Thu, Mar 23, 2017 at 12:25 PM Martin Sivak <msivak@redhat.com> wrote:
Hi,
I am just wondering whether you tried Roman's Hystrix integration to see what command was so slow?
Martin
On Thu, Mar 23, 2017 at 11:05 AM, Roy Golan <rgolan@redhat.com> wrote:
Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3]
Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s
Process finished with exit code 0
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Greg Sheremeta, MBA Red Hat, Inc. Sr. Software Engineer gshereme@redhat.com
On Thu, Mar 23, 2017, 12:40 PM Eldad Marciano <emarcian@redhat.com> wrote:
very cool! In terms of scale that sounds great for unit test | benchmarks, assume we have some loaded DB. we can create some benchmarks for launching vms for example, what do you think will be the challenge ?
It is mainly for small chunks of code, without involving integration, like db, if I understand your proposal correctly, otherwise the deviation will be too big.
On Thu, Mar 23, 2017 at 12:25 PM Martin Sivak <msivak@redhat.com> wrote:
Hi,
I am just wondering whether you tried Roman's Hystrix integration to see what command was so slow?
Martin
On Thu, Mar 23, 2017 at 11:05 AM, Roy Golan <rgolan@redhat.com> wrote:
Lately we came across an interesting case where multi-host+mult-networks resulted editing a host to conclude in minutes. One assumption that was raised which we wanted to eliminate was that the decryption we perform on a fence agent password might be taking too long.
So these days it's an easy task thanks to JMH[1], supplied by the jdk itself. I kickstarted [2] and added a 'DecryptionBenchmark', see the output as an example[3]
Although The JMH project recommends to create a separate project I find it would be less trivial to people to contribute benchmarks let alone just playing around with current code they want to test.
- So, (when it will be merged) you add your benchmark under backend/manager/modules/benchmarks/MyBenchmark.java
- run it from intellij using the jmh plugin exactly like a unit-test OR - mvn test -P benchmarks -pl org.ovirt.engine:benchmarks OR - java -jar benchmarks.jar
I hope this would serve all of us well, please review and add your benchmarks.
PS - this will not run in the CI atm.
[1] http://openjdk.java.net/projects/code-tools/jmh/ [2] https://gerrit.ovirt.org/74537 microbenchmarks: Introduce microbenchmarks using JMH [3] DecryptionBenchmark output (short version):
# Run complete. Total time: 00:09:06
Benchmark Mode Samples Score Score error Units b.DecryptionBenchmark.decryption thrpt 50 101.258 1.270 ops/s b.DecryptionBenchmark.encryption thrpt 50 238.587 4.667 ops/s b.DecryptionBenchmark.decryption avgt 50 0.010 0.000 s/op b.DecryptionBenchmark.encryption avgt 50 0.004 0.000 s/op b.DecryptionBenchmark.decryption sample 5544 0.010 0.000 s/op b.DecryptionBenchmark.encryption sample 13067 0.004 0.000 s/op b.DecryptionBenchmark.decryption ss 50 0.014 0.001 s b.DecryptionBenchmark.encryption ss 50 0.009 0.001 s
Process finished with exit code 0
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (7)
-
Eldad Marciano -
Greg Sheremeta -
Juan Hernández -
Martin Sivak -
Michal Skrivanek -
Oved Ourfali -
Roy Golan