Skip to content

BUG: to_sql with ADBC driver fails when schema exists without tables #59841

Description

@WillAyd

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

This is hard to create a reproducer for given sqlite does not have schemas and is not affected by the issue. However, with the proper database setup running code like:

with adbc_driver_postgresql.dbapi.connect() as conn:
    df.to_sql("table", schema="schema_with_no_tables")

Will produce the following traceback:

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/util/_decorators.py:333, in deprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
    327 if len(args) > num_allow_args:
    328     warnings.warn(
    329         msg.format(arguments=_format_argument_list(allow_args)),
    330         FutureWarning,
    331         stacklevel=find_stack_level(),
    332     )
--> 333 return func(*args, **kwargs)

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/core/generic.py:3087, in NDFrame.to_sql(self, name, con, schema, if_exists, index, index_label, chunksize, dtype, method)
   2889 """
   2890 Write records stored in a DataFrame to a SQL database.
   2891 
   (...)
   3083 [(1,), (None,), (2,)]
   3084 """  # noqa: E501
   3085 from pandas.io import sql
-> 3087 return sql.to_sql(
   3088     self,
   3089     name,
   3090     con,
   3091     schema=schema,
   3092     if_exists=if_exists,
   3093     index=index,
   3094     index_label=index_label,
   3095     chunksize=chunksize,
   3096     dtype=dtype,
   3097     method=method,
   3098 )

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:842, in to_sql(frame, name, con, schema, if_exists, index, index_label, chunksize, dtype, method, engine, **engine_kwargs)
    837     raise NotImplementedError(
    838         "'frame' argument should be either a Series or a DataFrame"
    839     )
    841 with pandasSQL_builder(con, schema=schema, need_transaction=True) as pandas_sql:
--> 842     return pandas_sql.to_sql(
    843         frame,
    844         name,
    845         if_exists=if_exists,
    846         index=index,
    847         index_label=index_label,
    848         schema=schema,
    849         chunksize=chunksize,
    850         dtype=dtype,
    851         method=method,
    852         engine=engine,
    853         **engine_kwargs,
    854     )

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:2386, in ADBCDatabase.to_sql(self, frame, name, if_exists, index, index_label, schema, chunksize, dtype, method, engine, **engine_kwargs)
   2381 # pandas if_exists="append" will still create the
   2382 # table if it does not exist; ADBC is more explicit with append/create
   2383 # as applicable modes, so the semantics get blurred across
   2384 # the libraries
   2385 mode = "create"
-> 2386 if self.has_table(name, schema):
   2387     if if_exists == "fail":
   2388         raise ValueError(f"Table '{table_name}' already exists.")

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:2422, in ADBCDatabase.has_table(self, name, schema)
   2419 if not schema_record:
   2420     continue
-> 2422 for table_record in schema_record["db_schema_tables"]:
   2423     if table_record["table_name"] == name:
   2424         return True

TypeError: 'NoneType' object is not iterable

Issue Description

The pandas code does not handle well the case where schema_record["db_schema_tables"] may be None

Expected Behavior

No error

Installed Versions

2.2.2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BugIO SQLto_sql, read_sql, read_sql_query

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions