Status of v0.43.0

[amotl]

Hi there,

thanks a stack for wrapping up many pending improvements previously bundled per GH-215 and releasing them with v0.43.0. Three categories of downstream outcomes materialized: Successful applications, details that need more love, and spots where things go south.

I haven't evaluated the failure incidents any closer so far. Please inspect CI/GHA and decide at your disposal if it's advisable to yank the package from PyPI, especially if you receive more such signals as we go.

With kind regards,
Andreas.

Success

Those spots can now successfully remove corresponding monkeypatches that ship with the dialect package today.

• Dependencies: Update to sqlalchemy-cratedb 0.43.0, remove monkeypatches pyveci/supertask#232

Refinements needed

Those need more love as we go.

• Dependencies: Update to sqlalchemy-cratedb next, remove monkeypatches langchain-cratedb#115
  Requirement: Compiler: Forwarding calls to PGCompiler.visit_on_conflict_do_update #276

CI failures

An enumeration where things go south on nightly scheduled CI runs.

• Scheduled CI run #27739496576 @ cratedb-fivetran-destination: 2 test failures
  crate==2.2.1, sqlalchemy-cratedb==0.43.0, sqlalchemy==2.0.51

• Scheduled CI run #27736731406 @ cratedb-toolkit: 4 test failures
  crate==2.2.1, sqlalchemy-cratedb==0.43.0, sqlalchemy==1.4.54

[bgunebakan]

Hi,
Thank you reporting this. I'm investigating the issue, Tests are already failing with current sqlalchemy-cratedb version (0.42.0.dev2), so looks like it's not a new issue or introduced with new version. They are also failing with latest cratedb server.

https://github.com/crate/cratedb-fivetran-destination/actions/runs/27748808489/job/82093762034?pr=192

[amotl]

Q&A

Tests are already failing with current sqlalchemy-cratedb version (0.42.0.dev2), so looks like it's not a new issue or introduced with new version.

All right, then this has a different root cause. Thanks for investigating!

Observations

It looks like it's neither a new issue nor introduced with sqlalchemy-cratedb 0.43.0.

Indeed: From the CI outcome we can see that the scheduled run on main also went south last night, independently from the PR submitted by Dependabot/Renovate. In this case, it is more likely that CrateDB nightly is causing this. It can easily be proofed or refuted by running the test suite against a GA release or against the previous nightly release of CrateDB.

-- https://github.com/crate/cratedb-fivetran-destination/actions/workflows/tests.yml

Evaluations

While the chance seems low, indeed such coincidences happen from time to time: While there seems to be solid circumstantial evidence that a certain ecosystem package upgrade is causing an issue, it is sometimes indeed CrateDB nightly that makes the test suite go south, by chance happening at the same time.

In this spirit, please don't change the design of the various test harnesses floating around too much. They are exactly made like this to catch such regressions early. 🙏

• A less aggressive dependabot cratedb-examples#1779
• Test examples against released versions cratedb-examples#1780

NB: Of course, the root cause could be anything else. My message was just meant to report about previous experiences to support your explorations.

[amotl]

Explorations

Tests are also failing with latest CrateDB server.

Oh, that's strange. Scheduled CI runs against CrateDB nightly provide stable and successful results until yesterday.

-- https://github.com/crate/cratedb-fivetran-destination/actions/workflows/tests.yml

[amotl]

Thoughts

To rule out the other moving target, you may want to trigger another CI run while pinning to previous crate-python 2.1.2?

[amotl]

One more sample

We are observing problems on another spot, in ctk this time, while using the newest crate==2.2.1 and sqlalchemy-cratedb==0.43.0 packages but with sqlalchemy==1.4.54 this time.

FAILED tests/adapter/test_pymongo.py::test_pymongo_insert_one_single -
crate.client.exceptions.ProgrammingError: SQLParseException[line 1:49: no viable alternative at input 'VALUES (%']

-- Scheduled CI run #27736731406

References

• Version 2.2.0 has a negative impact on SQLAchemy dialect's parameter handling and results crate-python#806 (comment)

[amotl]

^^ It didn't help to downgrade to sqlalchemy-cratedb==0.42.*, we also needed to downgrade to crate-python==2.1.*.

• Harmonize drivers to crate<2.2 and sqlalchemy-cratedb<0.43 cratedb-toolkit#843
• Harmonize drivers to crate<2.2 cratedb-fivetran-destination#193

[amotl]

We've added a more concise report to the tracker of the upstream project, as we think it might be the root cause for the hiccups we've observed.

• Version 2.2.1 fails test cases in two downstream projects crate-python#815

[amotl]

That's another two failures coming from a downstream test suite when updating to sqlalchemy-cratedb 0.42 or 0.43.

• fix(deps): update dependency sqlalchemy-cratedb to <0.44 meltano-tap-cratedb#48

FAILED tests/test_core.py::test_temporal_datatypes - AssertionError: assert {'column_timestamp': '1918-02-03T13:00:01.000000+00:00'} == {'column_timestamp': -1638097199000}
  
  Differing items:
  {'column_timestamp': '1918-02-03T13:00:01.000000+00:00'} != {'column_timestamp': -1638097199000}
  
  Full diff:
    {
  -     'column_timestamp': -1638097199000,
  +     'column_timestamp': '1918-02-03T13:00:01.000000+00:00',
    }

FAILED tests/test_core.py::test_decimal - AssertionError: assert 'number' in ['integer', 'null']