Skip to content

Leverage DuckDB? #256

@cholmes

Description

@cholmes

I've been building geoparquet tools that are basically just thin python wrappers over DuckDB. But I love gpq's installation ease and general speed, and ideally many of the operations I'm building there could be done in gpq.

So was wondering what you think of having a DuckDB dependency, and then using that to offer a lot more. It'd essentially pull in GDAL/OGR as well, as DuckDB does an amazing job of packaging GDAL. So then 'convert' could easily offer the full range of OGR formats, and could add all kinds of spatial predicates, do partitioning, spatial ordering, etc. etc. And then things like my checking of best practices could be in GPQ, alongside the core validation that's already there (perhaps a separate command, or at least make clear what's required and what's recommended).

Totally fine if that's too big / too much / is more of a hassle than I'm imagining (likely). But if it could come here I'd likely be excited to try to contribute (with lots of AI assistance). But I also understand if we want to keep this 'pure'. Or perhaps we could have a 'sister' project in go that brings in DuckDB.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions