Seeking help to understand Tracker configuration for imports using import_logs.py

@kwisarts wrote:

I’m trying to understand how the tracking_requests_require_authentication_when_custom_timestamp_newer_than configuration option plays into log-analytics’ import_logs.py.

E.g.

root@org-matomo:/var/www/matomo/core/Tracker# sudo -u www-data python /var/www/matomo/misc/log-analytics/import_logs.py --url=https://domain.com --recorders=2 --idsite=4 --enable-http-errors --tracker-endpoint=/piwik.php?queuedtracking=0 /home/matomo/logs/*domain.com.access.log-20200301

(This might be related to Lots of invalid requests)

In a nutshell, I was astonished that I had to modify the value to make imports of logs ranging from N-8 to N-1 (yesterday) possible. I don’t have any recollection to having to do this previously (I also didn’t find any documentation on it, but had to look into Core\Tracker\Request to find that it was necessary).

Here’s the snippet:

        // If timestamp in the past, token_auth is required
        $timeFromNow = $this->timestamp - $cdt;
        $isTimestampRecent = $timeFromNow < $this->customTimestampDoesNotRequireTokenauthWhenNewerThan;

        if (!$isTimestampRecent) {
            if (!$this->isAuthenticated()) {
                $message = sprintf("Custom timestamp is %s seconds old, requires &token_auth because it is bigger than %s...", $timeFromNow, $this->customTimestampDoesNotRequireTokenauthWhenNewerThan);
                Common::printDebug($message);
                Common::printDebug("WARN: Tracker API 'cdt' was used with invalid token_auth");
                throw new InvalidRequestParameterException($message);
            }
        }

The other thing is naming. Looking at the snippet here above, you can see that it starts by calculating $timeFromNow, the age of the request to import. We then determine whether a record is recent by determining whether its age is younger (smaller) than the value set through tracking_requests_require_authentication_when_custom_timestamp_newer_than.

IMHO, the variable is badly named. It should be named tracking_requests_require_authentication_when_custom_timestamp_older_than because a request’s timestamp is compared to that value and authentication is required when the timestamp of the request is more than tracking_requests_require_authentication_when_custom_timestamp_newer_than in the past. Or am I getting this wrong?

Value in global.php.ini/config.php.ini:

tracking_requests_require_authentication_when_custom_timestamp_newer_than = 86400;

Posts: 1

Participants: 1

Read full topic