Skip to content

forge_v2_pipeline.sh stage 4: llama-cli hangs in interactive mode without -st flag #164

Description

@joelteply

Surfaced 2026-04-07 (recurring). Stage 4 (Layer 7 deployment-runtime gate) of `scripts/forge_v2_pipeline.sh` invokes llama-cli without the `-st` (single-turn) flag. llama-cli's default behavior is to drop into interactive chat mode after generating the response, which causes the pipeline script to hang waiting on stdin until killed.

This has now hit twice tonight:

  1. Manual v1.5 q5_K_S Layer 7 gate run (worked around manually)
  2. v2-7B forge automated pipeline run (worked around manually, killed and re-run)

Discipline note: This was a fallback at the script level — the known failure mode was being worked around manually each time it recurred instead of being fixed at the source. Per the no-fallbacks discipline (CLAUDE.md), recurring bugs get fixed at the source on first hit, not third.

Fix

```diff

  • "$LLAMA_CPP_BIN/llama-cli" -m "$GGUF_Q5KS" \
  • -p "def fibonacci(n):" \\
    
  • -n 50 --temp 0 -no-cnv 2>&1 | tee "$LLAMA_LOG"
    
  • "$LLAMA_CPP_BIN/llama-cli" -m "$GGUF_Q5KS" \
  • -p "def fibonacci(n):" \\
    
  • -n 50 --temp 0 -ngl 99 -st 2>&1 | tee "$LLAMA_LOG"
    

```

Two changes:

  1. `-no-cnv` (deprecated, not recognized by current llama.cpp builds) → `-st` (single-turn, exits after generation)
  2. Add `-ngl 99` so the model is offloaded to GPU; without it, llama-cli sits at 100% CPU mmap'd from disk and the gate takes minutes per token

Also tightening the post-gate check: instead of just looking for "wrong shape" / "failed to load", also assert the output contains a recognizable code keyword. If llama.cpp loads silently but produces empty output, the gate should still halt.

Status

Fixed in commit (this PR / upcoming push). Closing as soon as the next forge run validates the fix end-to-end.

References

  • sentinel-ai#161 (the v2 forge PR where this was found)
  • VALIDATED-TENSOR-SURGERY paper §4.1.3 (the failure-as-success pattern; this is an instance of the same pattern at the script layer instead of the model layer)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions