[Kimchi-devel] RFC: Add serial as a valid graphics type

[Zhou Zheng Sheng]

on 2014/10/16 17:00, Zhou Zheng Sheng wrote:
> on 2014/10/16 01:43, Aline Manera wrote:
>>
>> On 10/15/2014 03:15 AM, Zhou Zheng Sheng wrote:
>>> on 2014/10/15 12:10, Zhou Zheng Sheng wrote:
>>>> on 2014/10/14 21:45, Aline Manera wrote:
>>>>> On 10/14/2014 07:04 AM, Yu Xin Huo wrote:
>>>>>> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/sect-Virtualization-Troubleshooting_-Troubleshooting_with_serial_consoles.html
>>>>>>
>>>>>>
>>>>>>
>>>>>> Per my understanding:
>>>>>>
>>>>>> 1. To enable the serial console, it needs configuration on guest OS.
>>>>>>
>>>>>> /        You must configure the virtualized operating system to output
>>>>>> information to the virtual serial port.//
>>>>>> //        The serial port is called //|ttyS0|//on Linux or
>>>>>> //|COM1|//on Windows./
>>>>>>
>>>>>>      It need to modify the |/boot/grub/grub.conf| file on guest OS and
>>>>>> reboot guest.
>>>>>>
>>>>>>      So VM must have OS already installed, seems like it needs the
>>>>>> guest to be running.
>>>>> Prior to do that guest OS configuration, the guest must have the serial
>>>>> console set on its XML.
>>>>>
>>>>> <serial type='pty'>
>>>>>    <target port='0'/>
>>>>> </serial>
>>>>> <console type='pty'>
>>>>>    <target type='serial' port='0'/>
>>>>> </console>
>>>>>
>>>> I found that Fedora can scan and use ttyS0 as terminal
>>>> automatically, so save the user from manually editing grub config. This
>>>> is not often used, but it's handy when troubleshooting, or when the
>>>> network is too slow to support an VNC/Spice session.
>>>>
>>>>> So the idea is providing a way to user to choose between spice, vnc or
>>>>> serial and Kimchi will setup the guest XML accordingly.
>>>>>
>>>> However I don't think spice/vnc and serial are exclusive options. Spice
>>>> and VNC are alternative to each other but serial should be a complement,
>>>> mainly for troubleshooting. If a newly created guest doesn't have
>>>> spice/vnc, and only contains serial port, it may not be able to install
>>>> OS successfully, because not all Linux installation programme support
>>>> serial console mode. Since the user has to modify kernel argument before
>>>> using a serial port, if without vnc/spice, how can user edit the kernel
>>>> argument.
>>>>
>>>> It's true the real servers does not have display and keyboard, but the
>>>> admin can use KVM (Keyboard Video Mouse facilities) to connect to the
>>>> server video display, or use something like IMM on IBM X servers.
>>>>
>>>> I notice that by default, guest created by virt-manager contains a
>>>> serial port. I think we can follow the same idea. We could always
>>>> provide vnc/spice and serial port. vnc/spice is for main use, serial
>>>> port is for troubleshooting.
>>>>
>>>>> For reference:
>>>>> -
>>>>> http://rwmj.wordpress.com/2011/07/08/setting-up-a-serial-console-in-qemu-and-libvirt/
>>>>>
>>>>>
>>>>> - http://libvirt.org/formatdomain.html#elementsConsole
>>>>>
>>>>>> 2. The serial console can be accessed with 'virsh console' or 'use
>>>>>> |virt-manager| to display the virtual text console'.
>>>>>>
>>>>>>       So the target of this work item is to add a 'web command line
>>>>>> console' in kimchi UI to access the 'serial console'?
>>>>>>
>>>>>> ||||
>>>>> We could try to find an AJAX text console to enable serial console view
>>>>> direct on Kimchi (the same virt-manager does)
>>>>> So if the guest is configured with serial console and user clicks on
>>>>> "Connect" we open the text console in a new tab.
>>>>> But unless we find this AJAX text console done in an open source
>>>>> project, I don't expect to have it for 1.4 release.
>>>>>
>>>> I found that in websockify source code, there is a wspty. It is a
>>>> terminal implemented in HTML and js. I was once in oVirt community, at
>>>> that time I was able to build a serial console solution using this
>>>> wspty. However I found that there some bugs, some keys are not
>>>> interpreted correctly. Maybe we can have a look at this wspty? If it's
>>>> good we can submit bug fixes to websockify.
>>>>
>>>> The console solution once I implemented was like this.
>>>>    guest serial port -> qemu char devices -> host pty
>>>>
>>>> Then I made a proxy to open this pty and forward the data to an HTTP
>>>> long connecion.
>>>>    host pty -> proxy ---(HTTP)--> wspty
>>>>
>>>> wspty is using websocket, I changed it a little bit to use raw HTTP
>>>> instead.
>>>>
>>>> For this part, I have a prototype repository on github, maybe you can
>>>> have a look. There are instructions on how to setup a guest and using a
>>>> proxy and wspty to access it's serial port.
>>>>
>>>>    https://github.com/edwardbadboy/ptyforward/blob/master/README
>>> Ha ha, I just found in this repository, I also wrote an websocket proxy
>>> to forward pty to websocket. The websocket and web server was based on
>>> ws4py + WSGI + gevent, in just 113 lines of Python. This repository is
>>> too old I almost forget it.
>>
>> Glad to know it!
>> Do you want to test this integration into Kimchi? And check how it will
>> work?
>> If we don't find a lot of bugs we could do it for 1.4
>>
> 
> I'm interested on this. Had a little investigation, ws4py provides
> Cherrypy plugin and tool to add websocket support. It'll be very nice if
> we can provide websocket in current Cherrypy server. It solves to
> authentication, authorization and security problem. On Cherrypy offical
> site, it also says websocket is supported by using ws4py.
> 
> A problem is that ws4py is not packaged in Fedora and RHEL, maybe we'll
> have to ship the library in our source code. That raise another problem.
> The license of ws4py is an ad-hoc one. We may have to verify if it is
> compatible with LGPL3.
> 

Had read the license of ws4py, it does not restrict redistributions and
modification, as long as we keep the license message for that piece of
code. Looks a good choice, because it can be integrated into cherrypy
easily, so we don't have the authentication problem as websockify. If we
can use this library, we can easily use virDomain.openConsole() API to
get the console stream and forward to websocket.

>>>> If we are to use websocket, the nature solution would be as follow
>>>>    host pty -> websockify ---(websocket)--> wspty
>>>>
>>>> There is a potential security risk. If we use websockify to transfer
>>>> serial port data, we are exposing the serial port to public without any
>>>> authentication. We have the same problem in VNC/Spice, that's why we add
>>>> ticket. The VNC ticket is using the password mechanism built in the VNC
>>>> protocol. However when using serial port, there is no such protocol, it
>>>> just transfers raw data. A solution is that we use the websocket
>>>> facility supported by cherrypy and forward the serial port data
>>>> ourselves, so that we can do authentication as usual.

I did a little more investigation. websockify does not support
forwarding a file like object (such as /dev/pts/4). Though it support
forwarding a Unix socket, the socket path is defined in the command line
once, and it does not support loading the socket part in target config
file dynamically. So a workaround maybe we firstly use socat to forward
the file/socket to/from 127.0.0.1:some_port, then add a token in target
config file to ask websockify forward this some_port.

Qemu and libvirt support defining a network backed chardev, so we can
use the following XML for the VMs we create to define a serial port.

    <serial type='tcp'>
      <source mode='bind' host='127.0.0.1' service='0'/>
      <protocol type='raw'/>
      <target port='0'/>
    </serial>

Then use the following qmp command to retrieve the allocated local port.

  virsh qemu-monitor-command domain_name --cmd \
    '{"execute": "query-chardev"}'

At last we can add a new token in websockify config dir according to the
returned port information.

Another method is that we can do a monkey patch of the
"websockify.websocketproxy", and replace the new_websocket_client()
method to make it load pty and Unix socket dynamically. In the proxy()
method, it uses select.select() to forward between target and client
socket, and unfortunately it seems virStream Python binding object does
not provide a FD for use to select(). So we can not use
"virDomain.openConsole()" but to parse the guest XML in Kimchi, and find
the back-end of the first serial device ourselves.