11:49 a.m.
Didi, Marin, any update on the patch?
On Sun, Sep 16, 2018 at 11:09 AM Yedidyah Bar David <didi(a)redhat.com> wrote:
On Sun, Sep 16, 2018 at 12:53 PM Yedidyah Bar David
<didi(a)redhat.com>
wrote:
>
> On Fri, Sep 14, 2018 at 6:06 PM Martin Perina <mperina(a)redhat.com>
wrote:
> >
> >
> >
> > On Fri, Sep 14, 2018 at 4:51 PM, Ravi Shankar Nori <rnori(a)redhat.com>
wrote:
> >>
> >> I see the same errors on my dev env. From the logs attached by Andrej
the response received by otopi has a bunch of null chars before the actual
response CONFIRM DEPLOY_PROCEED=yes
> >>
> >>
> >>
> >> 2018-09-14 15:49:23,018+0200 DEBUG otopi.plugins.otopi.dialog.machine
dialog.__logString:204 DIALOG:SEND ### Response is CONFIRM
DEPLOY_PROCEED=yes|no or ABORT DEPLOY_PROCEED
> >>
> >> ^@^@^@^@^@^@^@^@^@CONFIRM DEPLOY_PROCEED=yes
> >
> >
> > Didi/Sandro, could you please take a look? Below error seems like some
issue in otopi, where an error is raised when handling binary input:
>
> Not sure the issue is "binary input" in general, but simply illegal
> input. The prompt expects, as it says, one of these 3 replies:
>
> CONFIRM DEPLOY_PROCEED=yes
> CONFIRM DEPLOY_PROCEED=no
> ABORT DEPLOY_PROCEED
>
> Instead, judging from the file supplied by Andrej, it gets from the
engine:
> <7169 null bytes>CONFIRM DEPLOY_PROCEED=yes
>
> So either the engine now sends, for some reason, 7169 null bytes, in
> this response, or there is some low-level change causing this to be
> eventually supplied to otopi - a change in apache-sshd, openssh, some
> library, the kernel, no idea.
>
> Well, thinking a bit, I have a wild guess: Perhaps it's related to the
> patch introduced recently to change the tar blocking?
https://gerrit.ovirt.org/94357
I am leaving soon, perhaps someone can try the manual job with the
result of the check-patch job for above patch, to see if it fixes.
Otherwise I'll do this tomorrow.
>
> >
> >
> > 2018-09-14 15:49:23,032+0200 DEBUG otopi.context
context._executeMethod:143 method exception
> > Traceback (most recent call last):
> > File "/usr/lib/python2.7/site-packages/otopi/context.py", line
133,
in _executeMethod
> > method['method']()
> > File
"/tmp/ovirt-O6CfS4aUHI/otopi-plugins/ovirt-host-deploy/core/misc.py", line
87, in _confirm
> > prompt=True,
> > File
"/tmp/ovirt-O6CfS4aUHI/otopi-plugins/otopi/dialog/machine.py",
line 478, in confirm
> > code=opcode,
> >
> >
> >>
> >> On Fri, Sep 14, 2018 at 10:44 AM, Dafna Ron <dron(a)redhat.com> wrote:
> >>>
> >>> if you run it with mock you would remove any environmental
conditions that can effect the outcome so I recommend using mock
> >>>
> >>>
> >>> On Fri, Sep 14, 2018 at 3:32 PM, Martin Perina
<mperina(a)redhat.com>
wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Sep 14, 2018 at 3:49 PM, Dafna Ron <dron(a)redhat.com>
wrote:
> >>>>>
> >>>>> did you use mock to reproduce?
> >>>>
> >>>>
> >>>> No, just run_suite under myself
> >>>>>
> >>>>>
> >>>>> On Fri, Sep 14, 2018 at 2:39 PM, Martin Perina
<mperina(a)redhat.com>
wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> the problem is that we haven't fetched the temporary
host-deploy
log from /tmp directory, so we don't know which string that host-deploy
process sent to engine is causing that issue. I tried to reproduce on my
local machine, but I was unable to reproduce it, 002_bootstrap phase
finished successfully (other phases are still running).
> >>>>>>
> >>>>>> So if anyone is able to reproduce, please try to fetch
host-deploy log from /tmp directory after the error is raised and share it.
> >>>>>>
> >>>>>> Thanks
> >>>>>>
> >>>>>> Martin
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Sep 14, 2018 at 1:52 PM, Dafna Ron
<dron(a)redhat.com>
wrote:
> >>>>>>>
> >>>>>>> Full logs can be found here:
> >>>>>>>
> >>>>>>>
https://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_cha...
> >>>>>>>
> >>>>>>> On Fri, Sep 14, 2018 at 12:48 PM, Dafna Ron
<dron(a)redhat.com>
wrote:
> >>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> The previous regression was resolved and we now
have a new
regression.
> >>>>>>>>
> >>>>>>>> I don't think that the reported change is
related so can
someone from ovirt-engine take a look?
> >>>>>>>>
> >>>>>>>> The failure is add host on the upgrade suite.
> >>>>>>>>
> >>>>>>>> Please note that we have not had an engine-ovirt
build for over
10 days due to several consecutive regressions and I would ask you to stop
merging until we can stabilize the project and have a new package of engine.
> >>>>>>>>
> >>>>>>>> error:
> >>>>>>>>
> >>>>>>>> 2018-09-14 05:51:07,670-04 INFO
[org.ovirt.engine.core.uutils.ssh.SSHDialog]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] SSH execute
'root@lago-upgrade-from-release-suite-master-host-0' 'umask 0077;
MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)";
trap
"chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr
\"${MYTMP}\" >
/dev/null 2>&1" 0; tar -b1 --warning=no-timestamp -C "${MYTMP}" -x
&&
"${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine
DIALOG/customization=bool:True'
> >>>>>>>> 2018-09-14 05:51:08,550-04 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(VdsDeploy) [5c91fcbd] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing
Host lago-upgrade-from-release-suite-master-host-0. Stage: Initializing.
> >>>>>>>> 2018-09-14 05:51:08,565-04 INFO
[org.ovirt.engine.core.utils.transaction.TransactionSupport] (VdsDeploy)
[5c91fcbd] transaction rolled back
> >>>>>>>> 2018-09-14 05:51:08,574-04 ERROR
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [5c91fcbd]
Error during deploy dialog
> >>>>>>>> 2018-09-14 05:51:08,578-04 ERROR
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Error during host
lago-upgrade-from-release-suite-master-host-0 install
> >>>>>>>> 2018-09-14 05:51:08,586-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] EVENT_ID:
VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during
installation of Host lago-upgrade-from-release-suite-master-host-0:
CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00; nested
exception is org.postgresql.util.PSQLException: ERROR: invalid byte
sequence for encoding "UTF8": 0x00.
> >>>>>>>> 2018-09-14 05:51:08,586-04 ERROR
[org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Error during host
lago-upgrade-from-release-suite-master-host-0 install, preferring first
exception: CallableStatementCallback; SQL [{call insertauditlog(?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?)}ERROR: invalid byte sequence for encoding "UTF8": 0x00;
nested exception is org.postgresql.util.PSQLException: ERROR: invalid byte
sequence for encoding "UTF8": 0x00
> >>>>>>>> 2018-09-14 05:51:08,586-04 ERROR
[org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Host installation
failed for host 'e475e93a-63b3-4573-b242-162c2ed864f0',
'lago-upgrade-from-release-suite-master-host-0': CallableStatementCallback;
SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte
sequence for encoding "UTF8": 0x00; nested exception is
org.postgresql.util.PSQLException: ERROR: invalid byte sequence for
encoding "UTF8": 0x00
> >>>>>>>> 2018-09-14 05:51:08,615-04 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] START,
SetVdsStatusVDSCommand(HostName =
lago-upgrade-from-release-suite-master-host-0,
SetVdsStatusVDSCommandParameters:{hostId='e475e93a-63b3-4573-b242-162c2ed864f0',
status='InstallFailed', nonOperationalReason='NONE',
stopSpmFailureLogged='false', maintenanceReason='null'}), log id:
146cdc08
> >>>>>>>> 2018-09-14 05:51:08,626-04 INFO
[org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] FINISH,
SetVdsStatusVDSCommand, return: , log id: 146cdc08
> >>>>>>>> 2018-09-14 05:51:08,639-04 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] EVENT_ID:
VDS_INSTALL_FAILED(505), Host lago-upgrade-from-release-suite-master-host-0
installation failed. CallableStatementCallback; SQL [{call
insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}ERROR: invalid byte sequence for
encoding "UTF8": 0x00; nested exception is
org.postgresql.util.PSQLException: ERROR: invalid byte sequence for
encoding "UTF8": 0x00.
> >>>>>>>> 2018-09-14 05:51:08,652-04 INFO
[org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand]
(EE-ManagedThreadFactory-engine-Thread-1) [5c91fcbd] Lock freed to object
'EngineLock:{exclusiveLocks='[e475e93a-63b3-4573-b242-162c2ed864f0=VDS]',
sharedLocks=''}'
> >>>>>>>> 2018-09-14 05:51:37,996-04 INFO
[org.ovirt.engine.core.bll.quota.QuotaManager]
(EE-ManagedThreadFactory-engineScheduled-Thread-44) [] Quota Cache updated.
(19 msec)
> >>>>>>>> (END)
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Dafna
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Martin Perina
> >>>>>> Associate Manager, Software Engineering
> >>>>>> Red Hat Czech s.r.o.
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Martin Perina
> >>>> Associate Manager, Software Engineering
> >>>> Red Hat Czech s.r.o.
> >>>
> >>>
> >>
> >
> >
> >
> > --
> > Martin Perina
> > Associate Manager, Software Engineering
> > Red Hat Czech s.r.o.
>
>
>
> --
> Didi
--
Didi