SNOW-2920022: SessionManager leaks memory due to defaultdict usage for sessions map

[timgraham]

While developing https://github.com/Snowflake-Labs/django-snowflake and after fixing #2725, a Django test reveals that SessionManager.__init__() leaks memory due to defaultdict usage for the SessionManager._sessions_map:

======================================================================
FAIL: test_select_related_memory_leak (select_related.tests.SelectRelatedTests.test_select_related_memory_leak)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/tests/select_related/tests.py", line 149, in test_select_related_memory_leak
    self.assert_no_memory_leaks()
  File "/home/tim/code/django/tests/select_related/tests.py", line 73, in assert_no_memory_leaks
    self.assertEqual(gc.garbage, [])
AssertionError: Lists differ: [<snowflake.connector.session_manager.Sess[330 chars] {})] != []

First list contains 5 additional elements.
First extra element 0:
<snowflake.connector.session_manager.SessionManager object at 0x7bb6daff8dd0>

+ []
- [<snowflake.connector.session_manager.SessionManager object at 0x7bb6daff8dd0>,
-  <cell at 0x7bb6daffb6a0: SessionManager object at 0x7bb6daff8dd0>,
-  (<cell at 0x7bb6daffb6a0: SessionManager object at 0x7bb6daff8dd0>,),
-  <function SessionManager.__init__.<locals>.<lambda> at 0x7bb6dad16160>,
-  defaultdict(<function SessionManager.__init__.<locals>.<lambda> at 0x7bb6dad16160>, {})]

I'm not sure how to fix this offhand.

[sfc-gh-sghosh]

Hi @timgraham ,

Thanks for raising the issue and the PR Could you mention the environment details, Python version, Snowflake Python connector version, and a minimal repro code.

Are these and the previous issue #2725 related, and the same PR will resolve both ?

[timgraham]

The issue isn't environment or Python version-specific. The latest connector version is affected. The reproducer is to run the Django test.

This isn't a duplicate report. As I wrote in the description, this issue remains "after fixing #2725."

[sfc-gh-fpawlowski]

Hi @timgraham,
Thank you a lot for both memory findings!

Would removing defaultdict help - like in this draft PR #2741 ?

[timgraham]

No, in that case, the test fails like this:

======================================================================
FAIL: test_select_related_memory_leak (select_related.tests.SelectRelatedTests.test_select_related_memory_leak)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/tests/select_related/tests.py", line 148, in test_select_related_memory_leak
    self.assert_no_memory_leaks()
  File "/home/tim/code/django/tests/select_related/tests.py", line 72, in assert_no_memory_leaks
    self.assertEqual(gc.garbage, [])
AssertionError: Lists differ: [<snowflake.connector.session_manager.Sess[129 chars]50>}] != []

First list contains 3 additional elements.
First extra element 0:
<snowflake.connector.session_manager.SessionManager object at 0x7cb7caf20650>

+ []
- [<snowflake.connector.session_manager.SessionManager object at 0x7cb7caf20650>,
-  {},
-  {'_manager': <snowflake.connector.session_manager.SessionManager object at 0x7cb7caf20650>}]