October 2017 - Infra - oVirt List Archives

[JIRA] (OVIRT-1701) jenkins jobs try to collect find-bugs data even on non engine jobs
by Barak Korren (oVirt JIRA) 16 Oct '17

16 Oct '17

This is a multi-part message in MIME format... ------------=_1508158834-18385-127 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1701?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1701: -------------------------------- Resolution: Won't Fix Status: Done (was: To Do) > jenkins jobs try to collect find-bugs data even on non engine jobs > ------------------------------------------------------------------ > > Key: OVIRT-1701 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1701 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: infra > > Example_ > http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co… > Shows the failure: > *09:44:19* Archiving artifacts*09:44:19* Recording test > results*09:44:20* [ovirt-wgt_master_check-patch-fc26-x86_64] $ > /bin/bash -xe /tmp/CommandInterpreterCondition8048067474184520875.sh*09:44:20* > + find exported-artifacts/find-bugs/ -name '*.xml'*09:44:20* + grep -e > '.*'*09:44:20* find: 'exported-artifacts/find-bugs/': No such file or > directory*09:44:20* Run condition [Execute Shell] preventing perform > for step [[Publish FindBugs analysis results]] > Jobs shouldn't try to collect something which is not supposed to exist. > -- > SANDRO BONAZZOLA > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > Red Hat EMEA <https://www.redhat.com/> > <https://red.ht/sig> > TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > <http://www.teraplan.it/redhat-osd-2017/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065) ------------=_1508158834-18385-127 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1701?page=com.atlassian.jira.… ]</pre> <h3>Barak Korren updated OVIRT-1701:</h3> <pre>Resolution: Won't Fix Status: Done (was: To Do)</pre> <blockquote><h3>jenkins jobs try to collect find-bugs data even on non engine jobs</h3> <pre> Key: OVIRT-1701 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1701 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: sbonazzo Assignee: infra</pre> <p>Example_ <a href="http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co…">http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co…</a> Shows the failure: <strong>09:44:19</strong> Archiving artifacts*09:44:19* Recording test results*09:44:20* [ovirt-wgt_master_check-patch-fc26-x86_64] $ /bin/bash -xe /tmp/CommandInterpreterCondition8048067474184520875.sh*09:44:20* + find exported-artifacts/find-bugs/ -name ‘*.xml'*09:44:20* + grep -e '.*'*09:44:20* find: 'exported-artifacts/find-bugs/’: No such file or directory*09:44:20* Run condition [Execute Shell] preventing perform for step [[Publish FindBugs analysis results]] Jobs shouldn't try to collect something which is not supposed to exist. — SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <<a href="https://www.redhat.com/">https://www.redhat.com/</a>> <<a href="https://red.ht/sig">https://red.ht/sig</a>> TRIED. TESTED. TRUSTED. <<a href="https://redhat.com/trusted">https://redhat.com/trusted</a>> <<a href="http://www.teraplan.it/redhat-osd-2017/">http://www.teraplan.it/redhat-osd-2017/</a>></p></blockquote> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYLyz…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1508158834-18385-127--

1 0

[JIRA] (OVIRT-1701) jenkins jobs try to collect find-bugs data even on non engine jobs
by Barak Korren (oVirt JIRA) 16 Oct '17

16 Oct '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1701?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1701: ------------------------------------- What you are seeing is a shell 'find' expression looking for findbugs files and skipping the collection when they are not found. So essentially findbugs reporst are *not* collected when they are not there. Closing this as notabug. > jenkins jobs try to collect find-bugs data even on non engine jobs > ------------------------------------------------------------------ > > Key: OVIRT-1701 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1701 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: infra > > Example_ > http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co… > Shows the failure: > *09:44:19* Archiving artifacts*09:44:19* Recording test > results*09:44:20* [ovirt-wgt_master_check-patch-fc26-x86_64] $ > /bin/bash -xe /tmp/CommandInterpreterCondition8048067474184520875.sh*09:44:20* > + find exported-artifacts/find-bugs/ -name '*.xml'*09:44:20* + grep -e > '.*'*09:44:20* find: 'exported-artifacts/find-bugs/': No such file or > directory*09:44:20* Run condition [Execute Shell] preventing perform > for step [[Publish FindBugs analysis results]] > Jobs shouldn't try to collect something which is not supposed to exist. > -- > SANDRO BONAZZOLA > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > Red Hat EMEA <https://www.redhat.com/> > <https://red.ht/sig> > TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > <http://www.teraplan.it/redhat-osd-2017/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)

1 0

[JIRA] (OVIRT-1704) Add a job to update/modify secrets file
by Daniel Belenky (oVirt JIRA) 16 Oct '17

16 Oct '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1704?page=com.atlassian.jira.… ] Daniel Belenky updated OVIRT-1704: ---------------------------------- Epic Link: OVIRT-400 > Add a job to update/modify secrets file > --------------------------------------- > > Key: OVIRT-1704 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1704 > Project: oVirt - virtualization made easy > Issue Type: New Feature > Reporter: Daniel Belenky > Assignee: infra > > We need to have a proper way to update/modify the secrets file which is stored in our Jenkins server. > The alternative is having to add a way to download then update locally and upload again. > If we choose to allow downloading the file, we need to make it convenient and safe so that users which are unauthorized won't be able to download it. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)

1 0

[JIRA] (OVIRT-1704) Add a job to update/modify secrets file
by Daniel Belenky (oVirt JIRA) 16 Oct '17

16 Oct '17

This is a multi-part message in MIME format... ------------=_1508156621-23410-229 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Daniel Belenky created OVIRT-1704: ------------------------------------- Summary: Add a job to update/modify secrets file Key: OVIRT-1704 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1704 Project: oVirt - virtualization made easy Issue Type: New Feature Reporter: Daniel Belenky Assignee: infra We need to have a proper way to update/modify the secrets file which is stored in our Jenkins server. The alternative is having to add a way to download then update locally and upload again. If we choose to allow downloading the file, we need to make it convenient and safe so that users which are unauthorized won't be able to download it. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065) ------------=_1508156621-23410-229 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <h3>Daniel Belenky created OVIRT-1704:</h3> <pre> Summary: Add a job to update/modify secrets file Key: OVIRT-1704 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1704 Project: oVirt - virtualization made easy Issue Type: New Feature Reporter: Daniel Belenky Assignee: infra</pre> <p>We need to have a proper way to update/modify the secrets file which is stored in our Jenkins server. The alternative is having to add a way to download then update locally and upload again. If we choose to allow downloading the file, we need to make it convenient and safe so that users which are unauthorized won't be able to download it.</p> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYLyz…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1508156621-23410-229--

1 0

[JIRA] (OVIRT-1701) jenkins jobs try to collect find-bugs data even on non engine jobs
by Daniel Belenky (oVirt JIRA) 16 Oct '17

16 Oct '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1701?page=com.atlassian.jira.… ] Daniel Belenky commented on OVIRT-1701: --------------------------------------- [~eedri] That is what it does now. It checks if there is a directory called 'find-bugs' and if so - it collects the XMLs and parse them. The message you see in the job is from the publisher. Unfortunately, it's not supported to set find-bugs as a conditional publisher with the current version of JJB. We'll have to wait for JJB 2 which is currently in its beta state to be released. > jenkins jobs try to collect find-bugs data even on non engine jobs > ------------------------------------------------------------------ > > Key: OVIRT-1701 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1701 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: infra > > Example_ > http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co… > Shows the failure: > *09:44:19* Archiving artifacts*09:44:19* Recording test > results*09:44:20* [ovirt-wgt_master_check-patch-fc26-x86_64] $ > /bin/bash -xe /tmp/CommandInterpreterCondition8048067474184520875.sh*09:44:20* > + find exported-artifacts/find-bugs/ -name '*.xml'*09:44:20* + grep -e > '.*'*09:44:20* find: 'exported-artifacts/find-bugs/': No such file or > directory*09:44:20* Run condition [Execute Shell] preventing perform > for step [[Publish FindBugs analysis results]] > Jobs shouldn't try to collect something which is not supposed to exist. > -- > SANDRO BONAZZOLA > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > Red Hat EMEA <https://www.redhat.com/> > <https://red.ht/sig> > TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > <http://www.teraplan.it/redhat-osd-2017/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)

1 0

[JIRA] (OVIRT-1702) Jenkins STD-CI are running in dirty mock chroots
by Barak Korren (oVirt JIRA) 16 Oct '17

16 Oct '17

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1702?page=com.atlassian.jira.… ] Barak Korren commented on OVIRT-1702: ------------------------------------- Not sure what is $PWD for the git command in the log above. If you are creating files in the source directory, it is not part of the chroot, it is bind mounted. We do have git clean running on the clone directories, but it has its own rules. > Jenkins STD-CI are running in dirty mock chroots > ------------------------------------------------ > > Key: OVIRT-1702 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1702 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: infra > > Example: > http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co… > *09:44:06* + ./automation/build-artifacts.sh*09:44:06* + [[ -d > exported-artifacts ]]*09:44:06* + mkdir -p > exported-artifacts*09:44:06* + [[ -d tmp.repos/SOURCES ]]*09:44:06* + > mkdir -p tmp.repos/SOURCES*09:44:06* + git clone > git://anongit.freedesktop.org/spice/spice-nsis*09:44:06* fatal: > destination path 'spice-nsis' already exists and is not an empty > directory. > jobs should run in clean chroots. > -- > SANDRO BONAZZOLA > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > Red Hat EMEA <https://www.redhat.com/> > <https://red.ht/sig> > TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > <http://www.teraplan.it/redhat-osd-2017/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)

1 0

[JIRA] (OVIRT-1703) vm0152.workers-phx.ovirt.org ran out of file descriptors
by Daniel Belenky (oVirt JIRA) 16 Oct '17

16 Oct '17

Daniel Belenky created OVIRT-1703: ------------------------------------- Summary: vm0152.workers-phx.ovirt.org ran out of file descriptors Key: OVIRT-1703 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1703 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Daniel Belenky Assignee: infra Link to the failed job: [http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3161/console|…] I've opened this ticket to track this issue and see if we see it again sometime. I wasn't able to reboot the slave from the terminal, had to reboot it from the engine. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)

1 0

[JIRA] (OVIRT-1703) vm0152.workers-phx.ovirt.org ran out of file descriptors
by Daniel Belenky (oVirt JIRA) 16 Oct '17

16 Oct '17

This is a multi-part message in MIME format... ------------=_1508154442-11027-355 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1703?page=com.atlassian.jira.… ] Daniel Belenky updated OVIRT-1703: ---------------------------------- Epic Link: OVIRT-403 > vm0152.workers-phx.ovirt.org ran out of file descriptors > -------------------------------------------------------- > > Key: OVIRT-1703 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1703 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: Daniel Belenky > Assignee: infra > > Link to the failed job: [http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3161/console|…] > I've opened this ticket to track this issue and see if we see it again sometime. I wasn't able to reboot the slave from the terminal, had to reboot it from the engine. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065) ------------=_1508154442-11027-355 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1703?page=com.atlassian.jira.… ]</pre> <h3>Daniel Belenky updated OVIRT-1703:</h3> <pre>Epic Link: OVIRT-403</pre> <blockquote><h3>vm0152.workers-phx.ovirt.org ran out of file descriptors</h3> <pre> Key: OVIRT-1703 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1703 Project: oVirt - virtualization made easy Issue Type: Bug Reporter: Daniel Belenky Assignee: infra</pre> <p>Link to the failed job: [<a href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3161/console">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3161/console</a>|here] I've opened this ticket to track this issue and see if we see it again sometime. I wasn't able to reboot the slave from the terminal, had to reboot it from the engine.</p></blockquote> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYLyz…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1508154442-11027-355--

1 0

[CQ]: 82528, 9 (ovirt-engine) failed "ovirt-master" system tests, but isn't the failure root cause
by oVirt Jenkins 16 Oct '17

16 Oct '17

A system test invoked by the "ovirt-master" change queue including change 82528,9 (ovirt-engine) failed. However, this change seems not to be the root cause for this failure. Change 82615,5 (ovirt-engine) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 82615,5 (ovirt-engine) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://gerrit.ovirt.org/#/c/82528/9 For further details about the change that seems to be the root cause behind the testing failures see: https://gerrit.ovirt.org/#/c/82615/5 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3179/

1 0

[JIRA] (OVIRT-1702) Jenkins STD-CI are running in dirty mock chroots
by sbonazzo (oVirt JIRA) 16 Oct '17

16 Oct '17

This is a multi-part message in MIME format... ------------=_1508152529-14419-305 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1702?page=com.atlassian.jira.… ] sbonazzo commented on OVIRT-1702: --------------------------------- 2017-10-16 13:09 GMT+02:00 eyal edri (oVirt JIRA) < Yes, I'm sure it hasn't been created during the same instance of the job. As a workaround I changed the code for deleting the directory if found there but yes, having a dirty chroot is a risk. -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> <http://www.teraplan.it/redhat-osd-2017/> > Jenkins STD-CI are running in dirty mock chroots > ------------------------------------------------ > > Key: OVIRT-1702 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1702 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: sbonazzo > Assignee: infra > > Example: > http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co… > *09:44:06* + ./automation/build-artifacts.sh*09:44:06* + [[ -d > exported-artifacts ]]*09:44:06* + mkdir -p > exported-artifacts*09:44:06* + [[ -d tmp.repos/SOURCES ]]*09:44:06* + > mkdir -p tmp.repos/SOURCES*09:44:06* + git clone > git://anongit.freedesktop.org/spice/spice-nsis*09:44:06* fatal: > destination path 'spice-nsis' already exists and is not an empty > directory. > jobs should run in clean chroots. > -- > SANDRO BONAZZOLA > ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D > Red Hat EMEA <https://www.redhat.com/> > <https://red.ht/sig> > TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > <http://www.teraplan.it/redhat-osd-2017/> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065) ------------=_1508152529-14419-305 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1702?page=com.atlassian.jira.… ]</pre> <h3>sbonazzo commented on OVIRT-1702:</h3> <p>2017-10-16 13:09 GMT+02:00 eyal edri (oVirt JIRA) <</p> <p>Yes, I'm sure it hasn't been created during the same instance of the job. As a workaround I changed the code for deleting the directory if found there but yes, having a dirty chroot is a risk.</p> <p>—</p> <p>SANDRO BONAZZOLA</p> <p>ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D</p> <p>Red Hat EMEA <<a href="https://www.redhat.com/">https://www.redhat.com/</a>> <<a href="https://red.ht/sig">https://red.ht/sig</a>> TRIED. TESTED. TRUSTED. <<a href="https://redhat.com/trusted">https://redhat.com/trusted</a>> <<a href="http://www.teraplan.it/redhat-osd-2017/">http://www.teraplan.it/redhat-osd-2017/</a>></p> <blockquote><h3>Jenkins STD-CI are running in dirty mock chroots</h3> <pre> Key: OVIRT-1702 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1702 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: sbonazzo Assignee: infra</pre> <p>Example: <a href="http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co…">http://jenkins.ovirt.org/job/ovirt-wgt_master_check-patch-fc26-x86_64/11/co…</a> <strong>09:44:06</strong> + ./automation/build-artifacts.sh*09:44:06* + [[ -d exported-artifacts ]]*09:44:06* + mkdir -p exported-artifacts*09:44:06* + [[ -d tmp.repos/SOURCES ]]*09:44:06* + mkdir -p tmp.repos/SOURCES*09:44:06* + git clone git://anongit.freedesktop.org/spice/spice-nsis*09:44:06* fatal: destination path ‘spice-nsis’ already exists and is not an empty directory. jobs should run in clean chroots. — SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <<a href="https://www.redhat.com/">https://www.redhat.com/</a>> <<a href="https://red.ht/sig">https://red.ht/sig</a>> TRIED. TESTED. TRUSTED. <<a href="https://redhat.com/trusted">https://redhat.com/trusted</a>> <<a href="http://www.teraplan.it/redhat-osd-2017/">http://www.teraplan.it/redhat-osd-2017/</a>></p></blockquote> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100065)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYLyz…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1508152529-14419-305--

1 0