Log importing during deleting old logs

@theyosh wrote:

Hi,

I was wondering what will happen if we import logfiles with the python script import_logs.py and there is also a scheduled task running that is cleaning old visitor logs.

The cleanup of old logfiles takes about a week to finish here. We have about 1 billion records in our table piwik_log_link_visit_action. This is for 45 days. We could reduce another 10 days, but not more.

So for now, we are importing logfiles, archiving them, and once a month a cleanup. When the cleanup is running we wait/delay importing and archiving.

So is it possible to continue import new logfiles while the cleanup schedule is still running? This will speed up our archiving a lot, and customers don't have to wait for a week once a month.

Posts: 1

Participants: 1

Read full topic