Skip to content

[chord] add log for arrow flight read#354

Open
JNSimba wants to merge 4 commits intoapache:masterfrom
JNSimba:addlog-foradbc
Open

[chord] add log for arrow flight read#354
JNSimba wants to merge 4 commits intoapache:masterfrom
JNSimba:addlog-foradbc

Conversation

@JNSimba
Copy link
Copy Markdown
Member

@JNSimba JNSimba commented Apr 1, 2026

Proposed changes

Issue Number: close #xxx

Problem Summary:

Describe the overview of changes.

Checklist(Required)

  1. Does it affect the original behavior: (Yes/No/I Don't know)
  2. Has unit tests been added: (Yes/No/No Need)
  3. Has document been added or modified: (Yes/No/No Need)
  4. Does it need to update dependencies: (Yes/No)
  5. Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds additional logging around Arrow Flight SQL reads in the Spark Doris connector to help observe query submission timing and per-batch/partition read progress.

Changes:

  • Log per-batch load metrics (rows + elapsed time) when reading Arrow batches.
  • Track and log per-partition totals (batch count and total rows) when closing the reader.
  • Log query submission latency for Flight SQL execution.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants