-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
fix(logging): add API response timestamp and fix request timestamp timing #1300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
fix(logging): add API response timestamp and fix request timestamp timing #1300
Conversation
Summary of ChangesHello @sususu98, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the logging system by introducing more accurate timestamping for both incoming requests and their corresponding API responses. By capturing these events closer to their actual occurrence, the system gains the ability to precisely measure and analyze the latency of backend responses, which is crucial for debugging and performance monitoring, especially in error scenarios. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request effectively adds timestamps to request and API response logs, which is a great step for monitoring backend latency. The implementation correctly captures the request timestamp at the beginning of the middleware chain. However, there are a couple of issues that undermine the feature's accuracy. For streaming responses, the API response timestamp is recorded at the end of the request, not when the first response chunk arrives. Additionally, for some logging paths, the request timestamp falls back to the log write time instead of using the correctly captured arrival time. I've added specific comments with suggestions on how to address these points.
| if requestTimestamp.IsZero() { | ||
| requestTimestamp = time.Now() | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This fallback to time.Now() for a zero requestTimestamp contradicts one of the main goals of this PR, which is to 'Fix REQUEST INFO timestamp to capture when request first arrives (not at log write time)'. The LogRequest method calls this with a zero timestamp, causing the log to record the log write time instead of the request arrival time. To ensure the request arrival time is always used, you should consider updating the LogRequest method in the RequestLogger interface to accept a requestTimestamp and update its callers accordingly, or deprecate it in favor of LogRequestWithOptions.
internal/logging/request_logger.go
Outdated
| return nil | ||
| } | ||
| w.apiResponse = bytes.Clone(apiResponse) | ||
| w.apiResponseTimestamp = time.Now() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For streaming responses, this sets the apiResponseTimestamp to the time when the entire response has been processed and Finalize is called, not when the first chunk of the upstream API response arrived. This is inconsistent with the non-streaming implementation and the goal of measuring backend latency.
The timestamp is correctly captured for the first chunk in sdk/api/handlers/handlers.go's appendAPIResponse function and stored in the gin.Context. However, it's not being utilized here.
To fix this, you should:
- Modify the
StreamingLogWriterinterface'sWriteAPIResponsemethod to accept atime.Timeparameter:WriteAPIResponse(apiResponse []byte, timestamp time.Time) error. - Update this
FileStreamingLogWriter.WriteAPIResponseimplementation to use the passed timestamp instead oftime.Now(). - Update
NoOpStreamingLogWriter.WriteAPIResponseto match the new interface. - In
internal/api/middleware/response_writer.go'sFinalizemethod, extract the timestamp from the context usingw.extractAPIResponseTimestamp(c)and pass it when callingw.streamWriter.WriteAPIResponse().
| ts, isExist := c.Get("API_RESPONSE_TIMESTAMP") | ||
| if !isExist { | ||
| return time.Time{} | ||
| } | ||
| if t, ok := ts.(time.Time); ok { | ||
| return t | ||
| } | ||
| return time.Time{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
…ming Previously: - REQUEST INFO timestamp was captured at log write time (not request arrival) - API RESPONSE had NO timestamp at all This fix: - Captures REQUEST INFO timestamp when request first arrives - Adds API RESPONSE timestamp when upstream response arrives Changes: - Add Timestamp field to RequestInfo, set at middleware initialization - Set API_RESPONSE_TIMESTAMP in appendAPIResponse() and gemini handler - Pass timestamps through logging chain to writeNonStreamingLog() - Add timestamp output to API RESPONSE section This enables accurate measurement of backend response latency in error logs.
3b0238a to
d87c704
Compare
…ps required - Add firstChunkTimestamp field to ResponseWriterWrapper for sync capture - Capture TTFB in Write() and WriteString() before async channel send - Add SetFirstChunkTimestamp() to StreamingLogWriter interface - Make requestTimestamp/apiResponseTimestamp required in LogRequest() - Remove timestamp capture from WriteAPIResponse() (now via setter) - Fix Gemini handler to set API_RESPONSE_TIMESTAMP before writing response This ensures accurate TTFB measurement for all streaming API formats (OpenAI, Gemini, Claude) by capturing timestamp synchronously when the first response chunk arrives, not when the stream finalizes.
d87c704 to
295f34d
Compare
Summary
Fixes #1299
Changes
request_logging.goTimestampfield toRequestInfo, set at middleware initializationresponse_writer.goextractAPIResponseTimestamp(), pass timestamps through logging chainrequest_logger.goLogRequestWithOptions()andwriteAPISection()to accept/output timestampshandlers.goAPI_RESPONSE_TIMESTAMPinappendAPIResponse()on first responsegemini-cli_handlers.goAPI_RESPONSE_TIMESTAMPin Gemini CLI handlerExample Output