Skip to content

[BUG]: high memory and disk usage with ddtrace #17063

@chandan-pe

Description

@chandan-pe

Tracer Version(s)

1.20.8,4.2.3

Python Version(s)

3.10.11

Pip Version(s)

23.3.2

Bug Report

Python web app with falcon==3.1.3, gunicorn==20.1.0 with ddtrace enabled, deployed in a k8s env is causing high total memory usages and disk usages but same is not true without dd trace.
Without ddtrace the Working set memory is same as total Memory used but when dd trace is enabled the memory and disk usage gradually increases and the memory reaches the Memory limit but the Working set remains constant and after sometime pod restarts. And the disk grows with it reaches the limit too we have 40gb of disk usage in a datadog tracing enabled pod.
We have also upgraded the ddtrace version but nothing helped.
1.20.8 => 4.2.3.

No anomaly without ddtrace.

Image Image

Reproduction Code

We did nothing extra just followed the doc https://docs.datadoghq.com/tracing/trace_collection/dd_libraries/python/

CMD ddtrace-run gunicorn -c gunicorn.conf main:api

Error Logs

No response

Libraries in Use

requirement.txt
falcon==3.1.3
gunicorn==20.1.0
mysql-connector-python==8.4.0
sqlalchemy==2.0.8
xlrd==1.2.0
requests==2.26.0
mysqlclient==2.2.6
alembic==1.0.11
neo4j==1.7.4
boto3==1.21.5
psycopg2-binary==2.9.10
marshmallow==3.5.1
kafka-python==2.0.1
pymongo==3.13.0
cachetools==4.1.1
dnspython==1.16.0
redis==3.5.3
statsd==3.3.0
botocore==1.24.44
urllib3==1.26.5
protobuf==3.17.3
dpath==2.1.2
func-timeout==4.3.5
wrapt==1.16.0
ddtrace==4.2.3

Operating System

python:3.10.11 image

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions