MariaDB | Slow Query | High CPU Load

@tpimpao wrote:

Hi there,
I have struggled with a problem that occurs at the peak of the day’s highest load, causing a high CPU load on the database server.

My database server:

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
Thread(s) per core:    2
Core(s) per socket:    8
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 45
Model name:            Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
Stepping:              7
CPU MHz:               2200.000
CPU max MHz:           2200.0000
CPU min MHz:           1200.0000
BogoMIPS:              4389.13
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              20480K
NUMA node0 CPU(s):     0-7,16-23
NUMA node1 CPU(s):     8-15,24-31

MemTotal:       65764440 kB
MemFree:         4493980 kB
MemAvailable:    4999644 kB
Buffers:              16 kB
Cached:           539544 kB
SwapCached:      2550824 kB
Active:         52374964 kB
Inactive:        7595748 kB
Active(anon):   51917680 kB
Inactive(anon):  7528420 kB
Active(file):     457284 kB
Inactive(file):    67328 kB
Unevictable:        8648 kB
Mlocked:            8648 kB
SwapTotal:      16777212 kB
SwapFree:       12554256 kB
Dirty:                52 kB
Writeback:             0 kB
AnonPages:      58823428 kB
Mapped:            30216 kB
Shmem:              8600 kB
Slab:             630112 kB
SReclaimable:     474008 kB
SUnreclaim:       156104 kB
KernelStack:       13600 kB
PageTables:       128872 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    49659432 kB
Committed_AS:   58100260 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      425568 kB
VmallocChunk:   34325395452 kB
HardwareCorrupted:     0 kB
AnonHugePages:  13486080 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      226988 kB
DirectMap2M:     8126464 kB
DirectMap1G:    58720256 kB

I have about 20 million page views per month, so, my dedicated database server are complaint with Matomo requirements.

I’m using MariaDB 10.4.10 and this is my.cnf file

[mysqld]
datadir=/DB/mysql
socket=/DB/mysql/mysql.sock
tmpdir=/TEMPDB
bind-address=0.0.0.0
general_log_file=/var/log/mariadb/mariadb_acess.log
long_query_time = 10
slow_query_log
slow_query_log_file =/var/log/mariadb/slow-query.log
log_error=/var/log/mariadb/mariadb.err
innodb_flush_method=O_DIRECT
innodb_file_per_table=ON
innodb_flush_log_at_trx_commit = 2
innodb_log_group_home_dir = /TEMPDB

innodb_buffer_pool_size=51G
innodb_buffer_pool_instances=51
innodb_print_all_deadlocks=1
performance_schema=ON
max_allowed_packet=268435456
max_connections = 400
wait_timeout = 14400
interactive_timeout = 14400

[client]
port=3306
socket=/DB/mysql/mysql.sock

[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid

In the highest peak of the day i see that MySQL get allot slows querys similar to below:

# User@Host: matomo[matomo] @ server1 [xxx.xxx.xxx.xxx]
# Thread_id: 2001714  Schema: piwik  QC_hit: No
# Query_time: 16.680956  Lock_time: 0.000157  Rows_sent: 1  Rows_examined: 206832
# Rows_affected: 0  Bytes_sent: 5354
SET timestamp=1582018281;
SELECT visit_last_action_time, visit_first_action_time, idvisitor, idvisit, user_id, visit_exit_idaction_url, visit_exit_idaction_name, visitor_returning, visitor_days_since_first, visitor_days_since_order, visitor_count_visits, visit_goal_buyer, location_country, location_region, location_city, location_latitude, location_longitude, referer_name, referer_keyword, referer_type, idsite, visit_entry_idaction_url, visit_total_actions, visit_total_interactions, visit_total_searches, referer_url, config_device_brand, config_device_model, config_device_type, visit_total_events, visit_total_time, location_ip, location_browser_lang, campaign_content, campaign_id, campaign_keyword, campaign_medium, campaign_name, campaign_source, last_idlink_va, custom_dimension_1, custom_dimension_2, custom_dimension_3, custom_dimension_4, custom_dimension_5, custom_var_k1, custom_var_v1, custom_var_k2, custom_var_v2, custom_var_k3, custom_var_v3, custom_var_k4, custom_var_v4, custom_var_k5, custom_var_v5   FROM piwik_log_visit WHERE idsite = '23' AND visit_last_action_time <= '2020-02-18 10:01:04' AND idvisitor = 'S><FF><DF>J\n-<93>'
                ORDER BY visit_last_action_time DESC
                LIMIT 1;

I don’t have any problems running this query’s directly, but i have a lot of them in the state “Sending Data” when i run the SHOW FULL PROCESSLIST;

High Load TOP

top - 09:46:47 up 331 days, 6:56, 1 user, load average: 42.14, 43.14, 42.75
Tasks: 485 total, 3 running, 482 sleeping, 0 stopped, 0 zombie
%Cpu(s): 90.6 us, 0.9 sy, 0.0 ni, 8.5 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
KiB Mem : 65764440 total, 4386816 free, 60103412 used, 1274212 buff/cache
KiB Swap: 16777212 total, 12578576 free, 4198636 used. 4996896 avail Mem

What i’m missing?

Thanks!
TP

Posts: 1

Participants: 1

Read full topic