[ovirt-users] audit_log table performance tuning

[Juan Hernández]

On 04/23/2016 07:17 AM, Oved Ourfali wrote:
> 
> On Apr 23, 2016 7:46 AM, "Marina Kalinin" <mkalinin at redhat.com
> <mailto:mkalinin at redhat.com>> wrote:
>>
>> Hi all,
>>
>> So far I created this solution for immediate remedy:
>> https://access.redhat.com/solutions/721423
>>
>> I created this general RFE, that would help in our situation:
>> https://bugzilla.redhat.com/show_bug.cgi?id=1329793
>>
>> However, this RFE is not all what I have in my mind.
> 
> I also don't think we should have such tool as mentioned in the RFE.
> 
>> I am thinking if there is anyway we can limit the number of identical
> records in audit_log?
> 
> Identical records might imply some issue, so I don't think they should
> be limited. What we can add configuration for is maybe the frequency in
> which the data is purged, which is 30 days currently, afair.
> 
>> Or, as Oved suggested, something to do with RestAPI and CFME to reduce
> the amount of logging?
>> BTW, does current version of CFME already contains this feature:
>> http://old.ovirt.org/Features/RESTSessionManagement
>> ?
> 
> CFME uses the session. However, I think that we don't reuse the session
> among unrelated requests.
> Not 100% sure about that.
> 
> Juan?
> 

CFME does use the persistent authentication mechanism. It creates new
sessions (which triggers the audit log message) in the following situations:

1. When CFME starts it creates a new session to poll RHEV-M events. This
session is reused as long as CFME is running.

2. When a refresh of the inventory is performed. Each refresh creates a
new session.

3. When performing actions like start or shutdown a VM. Each of these
actions creates a new session.

So unless there is a high level of activity in the environment (starting
virtual machines, stopping them, etc) there shouldn't be a high amount
of activity in the logs.

In systems where high activity is detected I'd suggest to configure the
RHEVM-M web server log file so that we will have a clear idea of what is
creating sessions. This can be achieved adding the following line to the
/etc/httpd/conf.d/ssl.conf file:

  CustomLog logs/api_log "%h %t \"%r\" User-Agent=\"%{User-Agent}i\"
JSESSIONID=\"%{JSESSIONID}C\" Prefer=\"%{Prefer}i\" %>s"

Then restart the web server:

  # service httpd restart

The content of that "api_log" file would help us what is really sending
requests that trigger the login/logout audit messages.

Note that the additional log file consumes resources, both CPU and disk
space, so it should be disabled once the relevant information is captured.

>>
>> Thank you,
>> Marina.
>>
>> ________________________________
>>>
>>>
>>>
>>> On Sun, Apr 17, 2016 at 9:33 AM, Oved Ourfali <oourfali at redhat.com
> <mailto:oourfali at redhat.com>> wrote:
>>>>
>>>> Juan - we should try to reduce this number consumed by CFME, if
> possible.
>>>> CC-ing Eli for DB related tips.
>>>
>>>
>>> Reminds me of https://gerrit.ovirt.org/#/c/55743/ .
>>
>> This commit is great, however not relevant for my case, since
> event_notification_hist is empty.
>>>
>>> Y.
>>>>
>>>>
>>>> On Fri, Apr 15, 2016 at 1:12 AM, Marina Kalinin <mkalinin at redhat.com
> <mailto:mkalinin at redhat.com>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Any suggestions or maybe already available features in the pipeline
> for tuning the database, and specifically the audit_log table?
>>>>>
>>>>> The problem today is that with multiple applications accessing the
> engine through the RestAPI, especially deployments with CloudForms,
> create huge amount of login records in the audit_table. Which, in turns,
> consumes most of the available memory on the machine running the engine
> and the database and results in a terrible performance of engine and
> inaccessible Web UI.
>>>>>
>>>>> The solution today is to delete those records from the table [1]:
>>>>> => delete from audit_log where message like '%logged%';
>>>>>
>>>>>
>>>>> Are there any current tunings we can apply to the database?
>>>>> And if not - do we have any RFEs on limiting the records entered to
> the database or a way to delete/filter those records somehow from the WebUI?
>>>>> All I could find was RFE#1120659 [2], but it does not describe the
> exact issue.
>>>>>
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Marina.
>>>>>
>>>>>
>>>>> [1] https://access.redhat.com/solutions/2110011
>>>>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1120659
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>
>>
>>
>>
>> --
>> --
>>  mku
> 
> 
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
> 


-- 
Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3ºD, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.