Skip to content

[Continuous Learning 5/14] Candidate merge and deduplication#77

Open
xinhuagu wants to merge 1 commit into
mainfrom
feat/issue-56-candidate-merge-dedup
Open

[Continuous Learning 5/14] Candidate merge and deduplication#77
xinhuagu wants to merge 1 commit into
mainfrom
feat/issue-56-candidate-merge-dedup

Conversation

@xinhuagu

@xinhuagu xinhuagu commented Feb 22, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a new LearningCandidate model and enums for candidate lifecycle
  • add CandidateStore with JSONL+HMAC persistence, upsert-merge path, and write-time dedup guard
  • add CandidateSimilarity and tests for similarity boundaries, dedup behavior, corruption tolerance, and 10k-scale merge baseline

Issue

Closes #56

Summary by CodeRabbit

  • New Features

    • New observation storage system with multi-category support, lifecycle state management, and intelligent similarity-based deduplication.
    • Persistent storage with automatic merging of similar observations and configurable matching thresholds.
    • Validation and integrity support for stored observations.
  • Tests

    • Comprehensive test coverage for storage operations, deduplication logic, and similarity scoring.

@coderabbitai

coderabbitai Bot commented Feb 22, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

This pull request introduces a complete candidate storage and management system for learning signals, including two enums for candidate classification and lifecycle tracking, a LearningCandidate record with merge capabilities, a similarity scoring utility for candidate deduplication, and a persistent CandidateStore with HMAC-signed JSONL storage and upsert logic. Comprehensive tests validate similarity computation and merge behavior.

Changes

Cohort / File(s) Summary
Core Data Model
aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateKind.java, aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateState.java
New enums defining candidate classification (ERROR_RECOVERY, PREFERENCE, WORKFLOW, ANTI_PATTERN, SKILL_SEED) and lifecycle states (SHADOW, PROMOTED, REJECTED).
Learning Candidate Record
aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java
New record type encapsulating learning signals with validation, signable payload generation, and merge logic for combining candidates with evidence aggregation and timestamp bounds.
Similarity & Persistence
aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateSimilarity.java, aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java
Similarity utility computing blended scores via Jaccard on content/tags with category and toolTag constraints; store managing in-memory candidates with file persistence, HMAC signing, recent-window merge/dedup logic, and thread-safe upsert.
Test Coverage
aceclaw-memory/src/test/java/dev/aceclaw/memory/CandidateSimilarityTest.java, aceclaw-memory/src/test/java/dev/aceclaw/memory/CandidateStoreTest.java
Unit tests validating similarity boundary cases, store merge/upsert behavior, tamper detection, recent-window filtering, and 10k-entry performance.

Sequence Diagram

sequenceDiagram
    participant Client
    participant CandidateStore
    participant CandidateSimilarity
    participant FileSystem
    participant MemorySigner

    Client->>CandidateStore: upsert(CandidateObservation)
    CandidateStore->>CandidateSimilarity: score(newCandidate, recentCandidate)
    CandidateSimilarity-->>CandidateStore: similarity score
    
    alt Similarity >= Threshold
        CandidateStore->>CandidateStore: mergeWith(existing, new)
        Note over CandidateStore: Update evidence counts,<br/>merge sourceRefs,<br/>aggregate timestamp bounds
    else Similarity < Threshold
        CandidateStore->>CandidateStore: append new candidate
    end
    
    CandidateStore->>MemorySigner: sign(candidatePayload)
    MemorySigner-->>CandidateStore: hmac signature
    CandidateStore->>FileSystem: write/append candidates.jsonl
    FileSystem-->>CandidateStore: write complete
    CandidateStore-->>Client: return LearningCandidate
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related issues


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error, 1 warning)

Check name Status Explanation Resolution
Block Major Correctness And Security Risks ❌ Error CandidateSimilarity.score() method lacks null-guards for parameters a and b, risking NullPointerException when accessing methods like .category() and .toolTag(). Add Objects.requireNonNull(a, "a") and Objects.requireNonNull(b, "b") at the start of the score() method to prevent NPEs on null inputs.
Docstring Coverage ⚠️ Warning Docstring coverage is 6.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title '[Continuous Learning 5/14] Candidate merge and deduplication' directly and clearly matches the PR's main objective of implementing candidate merge and deduplication functionality.
Linked Issues check ✅ Passed The PR implements all core requirements from issue #56: similarity computation with jaccard-based text matching [CandidateSimilarity], merge/upsert behavior with evidence accumulation [CandidateStore.mergeWith, upsert], pre-write dedup guard [CandidateStore], and performance validation at 10k scale [CandidateStoreTest]. Comprehensive tests validate similarity boundaries, merge behavior, and performance baselines.
Out of Scope Changes check ✅ Passed All changes are directly aligned with issue #56 scope: new models (LearningCandidate, CandidateKind, CandidateState), similarity/merge logic (CandidateSimilarity, CandidateStore), and comprehensive tests. No unrelated modifications detected.
✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/issue-56-candidate-merge-dedup

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps

greptile-apps Bot commented Feb 22, 2026

Copy link
Copy Markdown

Greptile Summary

Implements candidate lifecycle infrastructure for continuous learning: LearningCandidate record tracks evidence with HMAC signing, CandidateStore provides JSONL persistence with write-time deduplication via Jaccard similarity (0.50 threshold, 30-day window), and CandidateSimilarity enables merge decisions using tokenized content/tag comparison. Tests cover similarity boundaries, corruption tolerance, and 10k-scale merge performance.

Key additions:

  • CandidateKind and CandidateState enums for candidate categorization and lifecycle
  • HMAC-SHA256 signing for tamper detection (reuses existing MemorySigner)
  • Upsert-merge path: similar candidates within recent window merge evidence counts and weighted scores
  • Atomic file writes with .tmp + atomic move pattern for crash safety

Issue found:

  • LearningCandidate.mergeWith() line 85: reuses old HMAC after merge, but signature covers fields that changed (evidenceCount, successCount, failureCount, lastSeenAt, sourceRefs). The CandidateStore.upsert() re-signs after merge, but the intermediate merged record has an invalid signature.

Confidence Score: 4/5

  • Safe to merge with one logic issue that doesn't affect runtime behavior
  • The HMAC reuse in mergeWith() is incorrect in principle, but CandidateStore re-signs the result immediately, so the bug has no runtime impact. Defensive coding is solid throughout (null guards, bounds checks, atomic file ops). Tests are comprehensive (similarity, dedup, corruption, 10k perf).
  • LearningCandidate.java line 85 - HMAC should be null after merge

Important Files Changed

Filename Overview
aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateSimilarity.java Jaccard similarity implementation with null safety and proper tokenization
aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java Record with HMAC signing, merge logic, and comprehensive validation - HMAC reuse issue in merge
aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java JSONL persistence with HMAC verification and merge/dedup - missing tag null guard in CandidateObservation

Class Diagram

%%{init: {'theme': 'neutral'}}%%
classDiagram
    class CandidateStore {
        -CopyOnWriteArrayList~LearningCandidate~ candidates
        -MemorySigner signer
        -ReentrantLock fileLock
        -Duration recentWindow
        -double mergeThreshold
        +upsert(CandidateObservation) LearningCandidate
        +load() void
        +all() List~LearningCandidate~
        -findMergeIndex(LearningCandidate) int
        -sign(LearningCandidate) LearningCandidate
        -loadOrCreateKey() byte[]
    }
    
    class LearningCandidate {
        +String id
        +Category category
        +CandidateKind kind
        +CandidateState state
        +String content
        +String toolTag
        +List~String~ tags
        +double score
        +int evidenceCount
        +Instant firstSeenAt
        +Instant lastSeenAt
        +String hmac
        +mergeWith(LearningCandidate) LearningCandidate
        +signablePayload() String
    }
    
    class CandidateSimilarity {
        +score(LearningCandidate, LearningCandidate) double
        +jaccard(String, String) double
        -tokenize(String) Set~String~
    }
    
    class CandidateObservation {
        +Category category
        +CandidateKind kind
        +String content
        +String toolTag
        +List~String~ tags
        +double score
        +int successDelta
        +int failureDelta
        +String sourceRef
        +Instant occurredAt
    }
    
    class CandidateKind {
        <<enumeration>>
        ERROR_RECOVERY
        PREFERENCE
        WORKFLOW
        ANTI_PATTERN
        SKILL_SEED
    }
    
    class CandidateState {
        <<enumeration>>
        SHADOW
        PROMOTED
        REJECTED
    }
    
    CandidateStore --> LearningCandidate : stores/manages
    CandidateStore --> CandidateObservation : receives
    CandidateStore ..> CandidateSimilarity : uses for merge decision
    LearningCandidate --> CandidateKind : has
    LearningCandidate --> CandidateState : has
    CandidateObservation --> CandidateKind : has
Loading

Last reviewed commit: a2516c2

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

firstSeenAt.isBefore(incoming.firstSeenAt) ? firstSeenAt : incoming.firstSeenAt,
lastSeenAt.isAfter(incoming.lastSeenAt) ? lastSeenAt : incoming.lastSeenAt,
List.copyOf(mergedSources),
hmac

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HMAC from old candidate reused after merge, but signablePayload() includes all fields that changed (evidenceCount, successCount, failureCount, lastSeenAt, sourceRefs). Merged candidate has invalid signature.

Suggested change
hmac
null
Prompt To Fix With AI
This is a comment left during a code review.
Path: aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java
Line: 85

Comment:
HMAC from old candidate reused after merge, but `signablePayload()` includes all fields that changed (evidenceCount, successCount, failureCount, lastSeenAt, sourceRefs). Merged candidate has invalid signature.

```suggestion
                null
```

How can I resolve this? If you propose a fix, please make it concise.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java (1)

83-104: Ordering drifts after upserts.

Candidates are sorted by lastSeenAt on load, but merges/appends don’t preserve that order. If consumers rely on recency order (or you want to short‑circuit recent-window scans), consider re-sorting after upsert or documenting that ordering is undefined. As per coding guidelines, aceclaw-memory/**: Verify HMAC signing is maintained, check for memory tier ordering issues, and ensure workspace isolation.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java` around
lines 83 - 104, The upsert method updates candidates but doesn’t preserve the
sorted-by-lastSeenAt invariant; after merging or appending you should resort the
in-memory list by lastSeenAt (newest first) and then ensure the stored entries
remain HMAC-signed and workspace-isolated: after calling sign(...) and before
rewriteFile()/append(...), insert a step to update the candidates collection
ordering (e.g., Collections.sort or equivalent comparator on
LearningCandidate.getLastSeenAt), then call rewriteFile() when you replaced the
file or append() when adding (but ensure append preserves the on-disk order if
you rely on it), and verify sign(...) is applied to the candidate returned by
mergeWith(...) and to any newly created candidate from
createUnsignedCandidate(...) so HMAC remains valid and workspace isolation is
maintained.
aceclaw-memory/src/test/java/dev/aceclaw/memory/CandidateStoreTest.java (1)

80-114: Perf assertion is extremely lenient.

A 10,000 ms ceiling is unlikely to catch regressions; consider tightening the budget or making it configurable to reflect a meaningful baseline.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@aceclaw-memory/src/test/java/dev/aceclaw/memory/CandidateStoreTest.java`
around lines 80 - 114, The perf assertion in
mergeBaselineAtTenThousandCandidates is too loose (asserting elapsedMs <
10_000); update the test to use a tighter, meaningful budget or make it
configurable: replace the hardcoded 10_000 with a named constant (e.g.,
PERF_MS_LIMIT) or read a test-only env/jvm property (e.g., "perf.ms.limit") and
assert elapsedMs < PERF_MS_LIMIT; adjust the constant value to a realistic
baseline (for example 1000ms or whatever your CI baseline supports) and
document/update any test fixtures if needed. Ensure changes are made in the
mergeBaselineAtTenThousandCandidates test and references to
CandidateStore.upsert/elapsedMs remain unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateSimilarity.java`:
- Around line 3-5: Add explicit null-guards for the score parameters in
CandidateSimilarity by calling Objects.requireNonNull on the parameters (e.g., a
and b) before any dereference (before using a.category(), a.toolTag(),
a.content(), a.tags() and likewise for b) so the downstream jaccard() calls
never receive nulls; also add import java.util.Objects if missing and keep the
requireNonNull calls at the top of the method that performs these comparisons
(referencing the CandidateSimilarity method that invokes jaccard()).

In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java`:
- Around line 263-270: The record canonical constructor for CandidateObservation
currently only null-checks tags but does not defensively copy it; update the
constructor (the CandidateObservation record) to call
Objects.requireNonNull(tags, "tags") and then replace the incoming list with an
immutable defensive copy (e.g., tags = List.copyOf(tags) or new
ArrayList<>(tags) as appropriate) before assigning to the record component; also
consider validating that the list contains no null elements if your conventions
require it.

In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java`:
- Around line 61-86: Add a null-guard at the start of
LearningCandidate.mergeWith(LearningCandidate incoming) (e.g.,
Objects.requireNonNull(incoming) or an early return) to avoid accessing incoming
when it is null, and ensure the returned merged LearningCandidate clears the
stale signature by passing null for the hmac field (since
score/evidence/success/failure/firstSeenAt/lastSeenAt/sourceRefs affect
signablePayload()). Reference mergeWith, signablePayload, and hmac when making
these changes.
- Around line 34-46: The record constructor for LearningCandidate currently
null-checks tags and sourceRefs before copying; change this to defensively
null-guard by assigning tags = tags != null ? List.copyOf(tags) : List.of() and
sourceRefs = sourceRefs != null ? List.copyOf(sourceRefs) : List.of(); in
addition, update mergeWith(...) to null-guard the incoming parameter before
dereferencing it and ensure the new LearningCandidate created in mergeWith does
not reuse the old hmac—recompute or clear the hmac appropriately (i.e., do not
pass the original hmac unchanged when other payload fields like scores, counts,
timestamps, or sourceRefs change).

---

Nitpick comments:
In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java`:
- Around line 83-104: The upsert method updates candidates but doesn’t preserve
the sorted-by-lastSeenAt invariant; after merging or appending you should resort
the in-memory list by lastSeenAt (newest first) and then ensure the stored
entries remain HMAC-signed and workspace-isolated: after calling sign(...) and
before rewriteFile()/append(...), insert a step to update the candidates
collection ordering (e.g., Collections.sort or equivalent comparator on
LearningCandidate.getLastSeenAt), then call rewriteFile() when you replaced the
file or append() when adding (but ensure append preserves the on-disk order if
you rely on it), and verify sign(...) is applied to the candidate returned by
mergeWith(...) and to any newly created candidate from
createUnsignedCandidate(...) so HMAC remains valid and workspace isolation is
maintained.

In `@aceclaw-memory/src/test/java/dev/aceclaw/memory/CandidateStoreTest.java`:
- Around line 80-114: The perf assertion in mergeBaselineAtTenThousandCandidates
is too loose (asserting elapsedMs < 10_000); update the test to use a tighter,
meaningful budget or make it configurable: replace the hardcoded 10_000 with a
named constant (e.g., PERF_MS_LIMIT) or read a test-only env/jvm property (e.g.,
"perf.ms.limit") and assert elapsedMs < PERF_MS_LIMIT; adjust the constant value
to a realistic baseline (for example 1000ms or whatever your CI baseline
supports) and document/update any test fixtures if needed. Ensure changes are
made in the mergeBaselineAtTenThousandCandidates test and references to
CandidateStore.upsert/elapsedMs remain unchanged.

Comment on lines +3 to +5
import java.util.HashSet;
import java.util.Locale;
import java.util.Set;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

find . -name "CandidateSimilarity.java" -type f

Repository: xinhuagu/AceClaw

Length of output: 134


🏁 Script executed:

head -100 aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateSimilarity.java

Repository: xinhuagu/AceClaw

Length of output: 1732


Add null-guards for score parameters.

Parameters a and b are dereferenced immediately (via .category(), .toolTag(), .content(), and .tags()) and passed to downstream jaccard() calls without null checks. Add explicit Objects.requireNonNull() guards per coding guidelines.

💡 Proposed fix
import java.util.HashSet;
import java.util.Locale;
+import java.util.Objects;
import java.util.Set;

public static double score(LearningCandidate a, LearningCandidate b) {
+   Objects.requireNonNull(a, "a");
+   Objects.requireNonNull(b, "b");
    if (a.category() != b.category()) return 0.0;
    if (!a.toolTag().equalsIgnoreCase(b.toolTag())) return 0.0;
    double content = jaccard(a.content(), b.content());
    double tags = jaccard(String.join(" ", a.tags()), String.join(" ", b.tags()));
    return (0.7 * content) + (0.3 * tags);
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateSimilarity.java`
around lines 3 - 5, Add explicit null-guards for the score parameters in
CandidateSimilarity by calling Objects.requireNonNull on the parameters (e.g., a
and b) before any dereference (before using a.category(), a.toolTag(),
a.content(), a.tags() and likewise for b) so the downstream jaccard() calls
never receive nulls; also add import java.util.Objects if missing and keep the
requireNonNull calls at the top of the method that performs these comparisons
(referencing the CandidateSimilarity method that invokes jaccard()).

Comment on lines +263 to +270
public CandidateObservation {
Objects.requireNonNull(category, "category");
Objects.requireNonNull(kind, "kind");
Objects.requireNonNull(content, "content");
Objects.requireNonNull(tags, "tags");
if (score < 0.0 || score > 1.0) {
throw new IllegalArgumentException("score must be in [0.0, 1.0], got: " + score);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cd aceclaw-memory && find . -name "CandidateStore.java" -type f

Repository: xinhuagu/AceClaw

Length of output: 114


🏁 Script executed:

cd aceclaw-memory && sed -n '250,280p' src/main/java/dev/aceclaw/memory/CandidateStore.java

Repository: xinhuagu/AceClaw

Length of output: 845


Null-guard and defensively copy tags list field in CandidateObservation record constructor.

The tags field is currently only validated for non-null but not defensively copied. Per coding guidelines, all List fields in record constructors must be null-guarded and defensively copied to prevent NPEs from downstream mutations.

Proposed fix
-            Objects.requireNonNull(tags, "tags");
+            tags = tags != null ? List.copyOf(tags) : List.of();
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public CandidateObservation {
Objects.requireNonNull(category, "category");
Objects.requireNonNull(kind, "kind");
Objects.requireNonNull(content, "content");
Objects.requireNonNull(tags, "tags");
if (score < 0.0 || score > 1.0) {
throw new IllegalArgumentException("score must be in [0.0, 1.0], got: " + score);
}
public CandidateObservation {
Objects.requireNonNull(category, "category");
Objects.requireNonNull(kind, "kind");
Objects.requireNonNull(content, "content");
tags = tags != null ? List.copyOf(tags) : List.of();
if (score < 0.0 || score > 1.0) {
throw new IllegalArgumentException("score must be in [0.0, 1.0], got: " + score);
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java` around
lines 263 - 270, The record canonical constructor for CandidateObservation
currently only null-checks tags but does not defensively copy it; update the
constructor (the CandidateObservation record) to call
Objects.requireNonNull(tags, "tags") and then replace the incoming list with an
immutable defensive copy (e.g., tags = List.copyOf(tags) or new
ArrayList<>(tags) as appropriate) before assigning to the record component; also
consider validating that the list contains no null elements if your conventions
require it.

Comment on lines +34 to +46
public LearningCandidate {
Objects.requireNonNull(id, "id");
Objects.requireNonNull(category, "category");
Objects.requireNonNull(kind, "kind");
Objects.requireNonNull(state, "state");
Objects.requireNonNull(content, "content");
Objects.requireNonNull(toolTag, "toolTag");
Objects.requireNonNull(tags, "tags");
Objects.requireNonNull(firstSeenAt, "firstSeenAt");
Objects.requireNonNull(lastSeenAt, "lastSeenAt");
Objects.requireNonNull(sourceRefs, "sourceRefs");
tags = List.copyOf(tags);
sourceRefs = List.copyOf(sourceRefs);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

find . -name "LearningCandidate.java" -type f

Repository: xinhuagu/AceClaw

Length of output: 132


🏁 Script executed:

cat -n ./aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java | head -100

Repository: xinhuagu/AceClaw

Length of output: 3942


🏁 Script executed:

# Check if there are any tests or other usages that might rely on current behavior
rg "LearningCandidate" --type java -A 3 -B 3 | grep -E "(null|mergeWith)" | head -20

Repository: xinhuagu/AceClaw

Length of output: 689


🏁 Script executed:

# Check if incoming parameter is used without null-checks in mergeWith
sed -n '61,87p' ./aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java | grep -E "incoming\."

Repository: xinhuagu/AceClaw

Length of output: 575


Null-guard list fields in constructor and fix stale HMAC in mergeWith.

The record constructor currently calls Objects.requireNonNull() on tags and sourceRefs before copying them. Per the coding guideline, these should be defensively null-guarded instead: tags = tags != null ? List.copyOf(tags) : List.of().

Additionally, the mergeWith() method creates a new LearningCandidate with a significantly different payload (merged scores, counts, timestamps, and sourceRefs), but reuses the original hmac. This causes HMAC verification to fail. The incoming parameter must also be null-guarded before dereferencing.

💡 Proposed fixes

Constructor (lines 41, 44-46):

-        Objects.requireNonNull(tags, "tags");
         Objects.requireNonNull(firstSeenAt, "firstSeenAt");
         Objects.requireNonNull(lastSeenAt, "lastSeenAt");
-        Objects.requireNonNull(sourceRefs, "sourceRefs");
-        tags = List.copyOf(tags);
-        sourceRefs = List.copyOf(sourceRefs);
+        tags = tags != null ? List.copyOf(tags) : List.of();
+        sourceRefs = sourceRefs != null ? List.copyOf(sourceRefs) : List.of();

mergeWith (lines 61, 85):

     public LearningCandidate mergeWith(LearningCandidate incoming) {
+        Objects.requireNonNull(incoming, "incoming");
         int mergedEvidence = evidenceCount + incoming.evidenceCount;
         ...
-                hmac
+                null
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public LearningCandidate {
Objects.requireNonNull(id, "id");
Objects.requireNonNull(category, "category");
Objects.requireNonNull(kind, "kind");
Objects.requireNonNull(state, "state");
Objects.requireNonNull(content, "content");
Objects.requireNonNull(toolTag, "toolTag");
Objects.requireNonNull(tags, "tags");
Objects.requireNonNull(firstSeenAt, "firstSeenAt");
Objects.requireNonNull(lastSeenAt, "lastSeenAt");
Objects.requireNonNull(sourceRefs, "sourceRefs");
tags = List.copyOf(tags);
sourceRefs = List.copyOf(sourceRefs);
public LearningCandidate {
Objects.requireNonNull(id, "id");
Objects.requireNonNull(category, "category");
Objects.requireNonNull(kind, "kind");
Objects.requireNonNull(state, "state");
Objects.requireNonNull(content, "content");
Objects.requireNonNull(toolTag, "toolTag");
Objects.requireNonNull(firstSeenAt, "firstSeenAt");
Objects.requireNonNull(lastSeenAt, "lastSeenAt");
tags = tags != null ? List.copyOf(tags) : List.of();
sourceRefs = sourceRefs != null ? List.copyOf(sourceRefs) : List.of();
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java`
around lines 34 - 46, The record constructor for LearningCandidate currently
null-checks tags and sourceRefs before copying; change this to defensively
null-guard by assigning tags = tags != null ? List.copyOf(tags) : List.of() and
sourceRefs = sourceRefs != null ? List.copyOf(sourceRefs) : List.of(); in
addition, update mergeWith(...) to null-guard the incoming parameter before
dereferencing it and ensure the new LearningCandidate created in mergeWith does
not reuse the old hmac—recompute or clear the hmac appropriately (i.e., do not
pass the original hmac unchanged when other payload fields like scores, counts,
timestamps, or sourceRefs change).

Comment on lines +61 to +86
public LearningCandidate mergeWith(LearningCandidate incoming) {
int mergedEvidence = evidenceCount + incoming.evidenceCount;
double mergedScore = ((score * evidenceCount) + (incoming.score * incoming.evidenceCount))
/ Math.max(1, mergedEvidence);
var mergedSources = new LinkedHashSet<>(sourceRefs);
mergedSources.addAll(incoming.sourceRefs);
while (mergedSources.size() > MAX_SOURCE_REFS) {
mergedSources.remove(mergedSources.iterator().next());
}
return new LearningCandidate(
id,
category,
kind,
state,
content,
toolTag,
tags,
mergedScore,
mergedEvidence,
successCount + incoming.successCount,
failureCount + incoming.failureCount,
firstSeenAt.isBefore(incoming.firstSeenAt) ? firstSeenAt : incoming.firstSeenAt,
lastSeenAt.isAfter(incoming.lastSeenAt) ? lastSeenAt : incoming.lastSeenAt,
List.copyOf(mergedSources),
hmac
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

find . -type f -name "*.java" | grep -i "aceclaw-memory" | head -20

Repository: xinhuagu/AceClaw

Length of output: 1454


🏁 Script executed:

find . -path "*aceclaw-memory*" -name "LearningCandidate.java"

Repository: xinhuagu/AceClaw

Length of output: 132


🏁 Script executed:

wc -l aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java

Repository: xinhuagu/AceClaw

Length of output: 133


🏁 Script executed:

cat -n aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java

Repository: xinhuagu/AceClaw

Length of output: 3942


🏁 Script executed:

rg "mergeWith" aceclaw-memory --type java -B 2 -A 2

Repository: xinhuagu/AceClaw

Length of output: 1236


🏁 Script executed:

rg -A 5 -B 5 "hmac" aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java

Repository: xinhuagu/AceClaw

Length of output: 693


🏁 Script executed:

rg "MemorySigner\|hmac" aceclaw-memory --type java | head -20

Repository: xinhuagu/AceClaw

Length of output: 42


🏁 Script executed:

cat aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java | head -100

Repository: xinhuagu/AceClaw

Length of output: 3678


🏁 Script executed:

rg "sign\(" aceclaw-memory/src/main/java/dev/aceclaw/memory/CandidateStore.java -B 2 -A 5

Repository: xinhuagu/AceClaw

Length of output: 986


🏁 Script executed:

rg "mergeWith" aceclaw-memory --type java -A 10 -B 2

Repository: xinhuagu/AceClaw

Length of output: 2925


🏁 Script executed:

rg "new LearningCandidate\(" aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java

Repository: xinhuagu/AceClaw

Length of output: 97


Add null-guard for incoming and clear stale HMAC.

The mergeWith() method modifies payload fields (score, evidenceCount, successCount, failureCount, firstSeenAt, lastSeenAt, sourceRefs) that are all part of signablePayload(), making the old HMAC invalid. Setting it to null prevents callers from accidentally persisting a record with a stale signature. Additionally, add a null-guard for the incoming parameter which is accessed multiple times without validation.

Proposed fix
     public LearningCandidate mergeWith(LearningCandidate incoming) {
+        Objects.requireNonNull(incoming, "incoming");
         int mergedEvidence = evidenceCount + incoming.evidenceCount;
         double mergedScore = ((score * evidenceCount) + (incoming.score * incoming.evidenceCount))
                 / Math.max(1, mergedEvidence);
@@
                 lastSeenAt.isAfter(incoming.lastSeenAt) ? lastSeenAt : incoming.lastSeenAt,
                 List.copyOf(mergedSources),
-                hmac
+                null
         );
     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public LearningCandidate mergeWith(LearningCandidate incoming) {
int mergedEvidence = evidenceCount + incoming.evidenceCount;
double mergedScore = ((score * evidenceCount) + (incoming.score * incoming.evidenceCount))
/ Math.max(1, mergedEvidence);
var mergedSources = new LinkedHashSet<>(sourceRefs);
mergedSources.addAll(incoming.sourceRefs);
while (mergedSources.size() > MAX_SOURCE_REFS) {
mergedSources.remove(mergedSources.iterator().next());
}
return new LearningCandidate(
id,
category,
kind,
state,
content,
toolTag,
tags,
mergedScore,
mergedEvidence,
successCount + incoming.successCount,
failureCount + incoming.failureCount,
firstSeenAt.isBefore(incoming.firstSeenAt) ? firstSeenAt : incoming.firstSeenAt,
lastSeenAt.isAfter(incoming.lastSeenAt) ? lastSeenAt : incoming.lastSeenAt,
List.copyOf(mergedSources),
hmac
);
public LearningCandidate mergeWith(LearningCandidate incoming) {
Objects.requireNonNull(incoming, "incoming");
int mergedEvidence = evidenceCount + incoming.evidenceCount;
double mergedScore = ((score * evidenceCount) + (incoming.score * incoming.evidenceCount))
/ Math.max(1, mergedEvidence);
var mergedSources = new LinkedHashSet<>(sourceRefs);
mergedSources.addAll(incoming.sourceRefs);
while (mergedSources.size() > MAX_SOURCE_REFS) {
mergedSources.remove(mergedSources.iterator().next());
}
return new LearningCandidate(
id,
category,
kind,
state,
content,
toolTag,
tags,
mergedScore,
mergedEvidence,
successCount + incoming.successCount,
failureCount + incoming.failureCount,
firstSeenAt.isBefore(incoming.firstSeenAt) ? firstSeenAt : incoming.firstSeenAt,
lastSeenAt.isAfter(incoming.lastSeenAt) ? lastSeenAt : incoming.lastSeenAt,
List.copyOf(mergedSources),
null
);
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@aceclaw-memory/src/main/java/dev/aceclaw/memory/LearningCandidate.java`
around lines 61 - 86, Add a null-guard at the start of
LearningCandidate.mergeWith(LearningCandidate incoming) (e.g.,
Objects.requireNonNull(incoming) or an early return) to avoid accessing incoming
when it is null, and ensure the returned merged LearningCandidate clears the
stale signature by passing null for the hmac field (since
score/evidence/success/failure/firstSeenAt/lastSeenAt/sourceRefs affect
signablePayload()). Reference mergeWith, signablePayload, and hmac when making
these changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Continuous Learning 5/14] Implement Candidate Merge and Deduplication

1 participant