CPU spike

[Devika7733]

Describe the bug

Hi Team,

I am experiencing a CPU spike when loading Gatus on Kubernetes with Postgres as the backend. I have added the following config:

storage:
path: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_SERVER}:5432/${POSTGRES_DB}?sslmode=require
type: postgres
caching: true
maximum-number-of-results: 180000
maximum-number-of-events: 200

DB config is: Burstable, B1ms, 1 vCores, 2 GiB RAM, 64 GiB storage

However, I am still seeing a spike after 5–6 hours. I am using version v5.35.0. I have nearly 524 endpoints for monitoring.

What do you see?

Spike in CPU usage even after giving maximum-number-of-results and maximum-number-of-events values. Cpu is getting increased after 4-5 hours then continue so for a long period.

What do you expect to see?

No response

List the steps that must be taken to reproduce this issue

No response

Version

v0.35.0

Additional information

Application is deployed on k8 and Using azure DB with config : Burstable, B1ms, 1 vCores, 2 GiB RAM, 64 GiB storage

[TwiN]

Is it Gatus causing a spike in CPU usage, or postgres? I'm asking because the metric name seems to indicate it might be the latter.

Also, how does it look in v5.34.x?

There was recently a change made to improve postgres performance, so I'm wondering if it's related.

cc: @ozanonurtek, if you're interested

[Devika7733]

@TwiN and @ozanonurtek , I have tested versions V5.34 and V5.35, and both are consuming more CPU on the database (Postgres). I have a total of 552 endpoints. What I am observing is that the application runs for 3–4 hours without any issues, but afterward, the CPU usage increases rapidly. During peak CPU usage, the system executes nearly 14 SELECT queries and are given below for your references.

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, success, errors, connected, status, dns_rcode, certificate_expiration, domain_expiration, hostname, ip, duration, timestamp
FROM endpoint_results
WHERE endpoint_id = $1
ORDER BY endpoint_result_id DESC -- Normally, we'd sort by timestamp, but sorting by endpoint_result_id is faster
LIMIT $2 OFFSET $3
"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, success, errors, connected, status, dns_rcode, certificate_expiration, domain_expiration, hostname, ip, duration, timestamp
FROM endpoint_results
WHERE endpoint_id = $1
ORDER BY endpoint_result_id DESC -- Normally, we'd sort by timestamp, but sorting by endpoint_result_id is faster
LIMIT $2 OFFSET $3
"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, success, errors, connected, status, dns_rcode, certificate_expiration, domain_expiration, hostname, ip, duration, timestamp
FROM endpoint_results
WHERE endpoint_id = $1
ORDER BY endpoint_result_id DESC -- Normally, we'd sort by timestamp, but sorting by endpoint_result_id is faster
LIMIT $2 OFFSET $3
"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

"SELECT endpoint_result_id, condition, success
FROM endpoint_result_conditions
WHERE endpoint_result_id IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50)"

SELECT COUNT(1) FROM endpoint_results WHERE endpoint_id = $1

Below is my storage configuration:

        _storage:
          path: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_SERVER}:5432/${POSTGRES_DB}?sslmode=require
          type: postgres
          caching: false
          maximum-number-of-results: 386640
          maximum-number-of-events: 60
        metrics: true
        concurrency: 350
        global_defaults: &global_defaults
          interval: 1m
          client:
            insecure: true
            timeout: 5s_