fix(store): run GC after squashing history to prevent loose object accumulation

[ThanhNguyxn]

Summary

Fixes #1104 - GitStore local repo fills tmpfs due to accumulating loose git objects.

Problem

When using GitStore with tmpfs, the local repository grows unbounded because:

• History is squashed to a single commit via rewriteHeadAsSingleCommit
• But unreachable objects are never garbage collected
• This causes ~16,305 loose objects to accumulate (~63.7MB)
• Eventually fills tmpfs and breaks the service

Solution

Added Prune() and RepackObjects() calls after squashing history in commitAndPushLocked(). This:

• Prunes unreachable objects immediately after creating orphan commits
• Repacks remaining objects into packfiles for better storage efficiency

Changes

• internal/store/gitstore.go: Added GC calls after rewriteHeadAsSingleCommit

Testing

• Build passes
• No test files for store package (existing behavior)

[gemini-code-assist]

Summary of Changes

Hello @ThanhNguyxn, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves an issue where the GitStore's local repository, particularly when utilizing tmpfs, would experience continuous growth due to the unchecked accumulation of unreachable Git objects after history squashing. The solution implements immediate garbage collection and object repacking following each history rewrite, thereby maintaining efficient storage and preventing potential service disruptions caused by tmpfs filling up.

Highlights

• Git Garbage Collection: Introduced explicit calls to repo.Prune() and repo.RepackObjects() within the commitAndPushLocked() function after history squashing.
• Resource Management: This change prevents the accumulation of loose Git objects, which previously led to unbounded repository growth and tmpfs exhaustion, ensuring more efficient storage utilization.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses an important issue of loose git objects accumulating in tmpfs by introducing garbage collection after squashing git history. The approach is sound, but my review identifies a significant improvement: the new code currently ignores errors from the garbage collection operations. I've suggested handling these errors to ensure the fix is robust and reliably prevents the tmpfs exhaustion problem.

[gemini-code-assist]

The errors returned by repo.Prune and repo.RepackObjects are being ignored. If these operations fail, loose objects will continue to accumulate, which would lead to the same tmpfs exhaustion issue this pull request is intended to fix. To make the fix more robust, these errors should be handled by returning them, which is consistent with the error handling in the rest of the function.

	if err := repo.Prune(git.PruneOptions{OnlyObjectsOlderThan: time.Now()}); err != nil {
		return fmt.Errorf("git token store: prune objects: %w", err)
	}
	if err := repo.RepackObjects(&git.RepackConfig{}); err != nil {
		return fmt.Errorf("git token store: repack objects: %w", err)
	}

[ThanhNguyxn]

@gemini-code-assist I intentionally ignored these errors for the following reasons:

1. GC is an optimization, not a core operation. If Prune/RepackObjects fails, the commit and push have already succeeded. Returning an error here would falsely indicate that the save operation failed, potentially causing the caller to retry or assume data wasn't persisted.

2. Consistency with existing error handling patterns. The codebase already ignores non-critical errors in similar contexts (e.g., lines 177-188 ignore various pull errors like ErrUnstagedChanges, ErrNonFastForwardUpdate). GC failures are similarly non-critical.

3. Eventual consistency. If GC fails once, the next commitAndPushLocked call will attempt it again. The loose objects will eventually be cleaned up.

4. Avoid cascading failures. Returning an error from GC could trigger retry logic that would create more loose objects, making the problem worse.

The fix is designed to be best-effort: we try to clean up after squashing, but don't fail the entire operation if cleanup doesn't succeed.

[gemini-code-assist]

Thanks for providing such a detailed explanation, @ThanhNguyxn! Your reasoning for not returning errors from repo.Prune() and repo.RepackObjects() is well-articulated, especially concerning the potential for cascading failures and the nature of GC as an optimization rather than a core operation.

I agree that the primary commitAndPushLocked operation should not fail if the garbage collection steps encounter a transient issue. However, given that the core problem this PR addresses is the tmpfs filling up due to accumulating loose objects, completely ignoring these errors (_ = ...) means we lose visibility if the GC process itself is consistently failing.

While the commit and push might succeed, if Prune() or RepackObjects() continuously fail for some underlying reason (e.g., permissions, resource contention), the tmpfs exhaustion issue could re-emerge without any indication. "Eventual consistency" relies on the GC eventually succeeding, but if there's a persistent problem, it won't.

Suggestion:

Instead of completely ignoring the errors, I recommend logging them. This would provide crucial observability into the health of the GC process without blocking the main operation. For example:

	errPrune := repo.Prune(git.PruneOptions{OnlyObjectsOlderThan: time.Now()})
	if errPrune != nil {
		// Log the error, but don't return it as the primary operation succeeded.
		s.logger.Errorf("Failed to prune Git repository: %v", errPrune)
	}

	errRepack := repo.RepackObjects(&git.RepackConfig{})
	if errRepack != nil {
		// Log the error, but don't return it as the primary operation succeeded.
		s.logger.Errorf("Failed to repack Git repository: %v", errRepack)
	}

This approach maintains your desired behavior of not failing the commitAndPushLocked operation, while also ensuring that potential issues with the garbage collection that could lead to a regression of the tmpfs problem are visible in the logs. This improves the maintainability and reliability of the fix.

[ThanhNguyxn]

@gemini-code-assist Good point. I've updated the code to log warnings on GC failures instead of silently ignoring them. This provides visibility into potential issues while keeping the main operation non-blocking.

if errPrune := repo.Prune(git.PruneOptions{OnlyObjectsOlderThan: time.Now()}); errPrune != nil {
    log.WithError(errPrune).Warn("git token store: failed to prune objects")
}
if errRepack := repo.RepackObjects(&git.RepackConfig{}); errRepack != nil {
    log.WithError(errRepack).Warn("git token store: failed to repack objects")
}

This follows the same logging pattern used in objectstore.go and postgresstore.go.

[gemini-code-assist]

That's an excellent update, @ThanhNguyxn! Logging the GC failures as warnings with log.WithError(...).Warn(...) is a perfect solution. It addresses the observability concern by providing visibility into potential issues without blocking the primary operation, aligning perfectly with your initial reasoning. The consistency with existing logging patterns in objectstore.go and postgresstore.go is also a great touch for maintainability. Well done!