[Kimchi-devel] [PATCH V4] [Wok 0/4] Implement User Request Logger

[Lucio Correia]

Changes in V4:
* Update log removal interval to 6 hours

Changes in V3:
* Replaced cron by cherrypy's BackgroundTask
* Created RequestLogger class
* Removed --req-log config option
* Small fixes

Changes in V2:
* applied review from gingerbase patch
* added download option
* added crontab for deleting generated downloads
* log login/logout requests

User requests log is saved by default at log/wok-req.log. The maximum file
size is 3 MB and there is rotation with log/wok-req.log.1, which means it will
use at most 6 MB of disk space. Suggestions here are appreciated.

The log format is JSON, since:
* the web user interface will provide a log utility to easily search and filter
log results.
* there is a download function which generates a log-like text file based on
search results.

Request log results can be searched using the following parameters:
* app: filter by application that received the request (wok, kimchi, etc.)
* user: filter by user that performed the request
* req: filter by request type: POST, DELETE, PUT. GET requests are not logged.
* date: filter by request date in format YYYY-MM-DD
* download: generate a log-like text file for download

Sample JSON response:

dev at u1510:~$ curl -u dev -H "Content-Type: application/json" -H "Accept: application/json" "http://localhost:8010/logs?download=True" -X GET -d '{}'
Enter host password for user 'dev':
{
  "records":[
    {
      "app":"wok",
      "req":"POST",
      "user":"dev",
      "time":"14:39:59",
      "date":"2016-02-25",
      "message":"User 'dev' logout"
    },
    (...truncated...)
    {
      "app":"gingerbase",
      "req":"POST",
      "user":"dev",
      "time":"14:42:31",
      "date":"2016-02-25",
      "message":"Host software update"
    }
  ],
  "uri":"/data/logs/tmpbnOzP2.txt"
}

"uri" value can then be used to download log file:

dev at u1510:~$ curl -u dev -H "Content-Type: application/json" -H "Accept: application/json" "http://localhost:8010/data/logs/tmpbnOzP2.txt" -X GET -d '{}'
Enter host password for user 'dev':
[2016-02-25 14:39:59] POST   wok         dev: User 'dev' logout
[2016-02-25 14:40:13] POST   gingerbase  dev: Create host debug report 'test'
[2016-02-25 14:40:58] PUT    gingerbase  dev: Update host debug report 'test0'
[2016-02-25 14:41:10] DELETE gingerbase  dev: Delete host debug report 'test0'
[2016-02-25 14:41:55] POST   gingerbase  dev: Enable host software repository 'archive.canonical.com-wily-partner'
[2016-02-25 14:42:06] POST   gingerbase  dev: Disable host software repository 'archive.canonical.com-wily-partner'
[2016-02-25 14:42:31] POST   gingerbase  dev: Host software update

Lucio Correia (4):
  Add User Request Logger
  Log user requests
  Implement User Request Logger API
  Fix tests

 docs/API/logs.md        |  27 +++++++
 src/wok/config.py.in    |  10 +++
 src/wok/control/base.py |  81 +++++++++++++++++++--
 src/wok/control/logs.py |  43 +++++++++++
 src/wok/i18n.py         |   4 +-
 src/wok/model/logs.py   |  31 ++++++++
 src/wok/reqlogger.py    | 184 ++++++++++++++++++++++++++++++++++++++++++++++++
 src/wok/root.py         |  28 ++++++++
 src/wok/server.py       |   6 ++
 src/wok/utils.py        |  32 +++++++++
 tests/utils.py          |   1 +
 11 files changed, 440 insertions(+), 7 deletions(-)
 create mode 100644 docs/API/logs.md
 create mode 100644 src/wok/control/logs.py
 create mode 100644 src/wok/model/logs.py
 create mode 100644 src/wok/reqlogger.py

-- 
1.9.1