[ovirt-devel] execCmd() and storing stdout and stderr in log file

[Richard W.M. Jones]

On Thu, Jul 14, 2016 at 03:07:29PM +0300, Nir Soffer wrote:
> This should work with execCmd, since the special subprocess.STDOUT
> parameter is handled in subprocess.Popen, and cpopen.CPopen inherit
> this code. However this is not tested with cpopen, so it may be broken.
> 
> But merging stdout and stderr is likely to break v2v output parser, and vdsm
> log is not the place for virt-v2v debug logs.
> 
> I understand that the issue is keeping virt-v2v debug logs (using --verbose?),
> and the logs are spread in stdout and stderr. Did you discuss this issue
> with Richard?

FWIW we just faced this same issue with virt-p2v (which runs virt-v2v
as a remote subprocess).  The resolution was to write a wrapper script
that separates out the stdout & stderr.  Stdout is displayed to the
user.  At the same time, stdout + stderr contains all the debugging
information and that is sent to a log file.  You can find the code
here:

https://github.com/libguestfs/libguestfs/blob/master/p2v/conversion.c#L959

There were also some corresponding changes to virt-v2v itself to more
sensibly apportion the output between stdout & stderr, but those are
all included in RHEL 7.3 (not 7.2).

The only problem was that if virt-v2v fails, the final error gets sent
to stderr (hence to the debugging log file) and not displayed to the
user, so on error the wrapper script has to dump out the last 50 lines
of the log file.

We also wrote a simple ANSI colour parser so that the full colourized
output of 'virt-v2v --colours' shows up in virt-p2v (ie. what comes
via stdout and is displayed to the user).  The code is:

https://github.com/libguestfs/libguestfs/blob/master/p2v/gui.c#L1782

> I would use tee to write stdout and stderr to an import log file, without
> changing the code checking import progress.
> 
> Maybe virt-v2v can add a --logfile option appending to given log file?

Tricky to implement.  In any case, starting with RHEL 7.3, you can
achieve the same thing using the ' >> log | tee -a log ' technique
used by the wrapper script, see link above.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v