After upgrading awswrangler, I started getting errors when reading values of UUID columns (datatype 1111) in Postgres.
The error happens with awswrangler-3.16.1 (and pyarrow-24.0.0), but does not happen with awswrangler-3.16.0 (and pyarrow-23.0.1).
import awswrangler as wr
import pg8000
con = pg8000.connect(
user="test_user",
password="???",
host="localhost",
port=5432,
database="test_database"
)
sql = "SELECT uuid_column FROM test.uuid_test_table LIMIT 10"
try:
df = wr.postgresql.read_sql_query(sql=sql, con=con)
print("Query succeeded:")
print(df)
except Exception as e:
print(f"Error occurred: {type(e).__name__}: {e}")
finally:
con.close()
Traceback (most recent call last):
File "/.../reproduce_uuid_issue.py", line 24, in <module>
df = wr.postgresql.read_sql_query(sql=sql, con=con)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/test_proj/lib/python3.11/site-packages/awswrangler/_utils.py", line 126, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/test_proj/lib/python3.11/site-packages/awswrangler/postgresql.py", line 360, in read_sql_query
return _db_utils.read_sql_query(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/test_proj/lib/python3.11/site-packages/awswrangler/_databases.py", line 330, in read_sql_query
return _fetch_all_results(
^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/test_proj/lib/python3.11/site-packages/awswrangler/_databases.py", line 260, in _fetch_all_results
return _records2df(
^^^^^^^^^^^^
File "/opt/miniconda3/envs/test_proj/lib/python3.11/site-packages/awswrangler/_databases.py", line 182, in _records2df
df = table.to_pandas( # Creating Pandas DataFrame
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/array.pxi", line 1071, in pyarrow.lib._PandasConvertible.to_pandas
File "pyarrow/table.pxi", line 5146, in pyarrow.lib.Table._to_pandas
File "/opt/miniconda3/envs/test_proj/lib/python3.11/site-packages/pyarrow/pandas_compat.py", line 827, in table_to_dataframe
blocks = [
^
File "/opt/miniconda3/envs/test_proj/lib/python3.11/site-packages/pyarrow/pandas_compat.py", line 828, in <listcomp>
_reconstruct_block(
File "/opt/miniconda3/envs/test_proj/lib/python3.11/site-packages/pyarrow/pandas_compat.py", line 773, in _reconstruct_block
pandas_dtype = extension_columns[name]
~~~~~~~~~~~~~~~~~^^^^^^
KeyError: 'uuid_column'
Describe the bug
After upgrading awswrangler, I started getting errors when reading values of UUID columns (datatype 1111) in Postgres.
The error happens with awswrangler-3.16.1 (and pyarrow-24.0.0), but does not happen with awswrangler-3.16.0 (and pyarrow-23.0.1).
How to Reproduce
Example code:
ERROR:
Error occurred: KeyError: 'uuid_column'
Full traceback:
Expected behavior
No response
Your project
No response
Screenshots
No response
OS
Mac, Linux
Python version
3.11
AWS SDK for pandas version
3.16.1
Additional context
No response