Skip to content

pzsh-spec.md

Latest commit

 

History

History
728 lines (554 loc) · 27.6 KB

pzsh-spec.md

File metadata and controls

728 lines (554 loc) · 27.6 KB
Raw

pzsh: Performance-First Shell Framework Specification

Version: 0.1.0 Status: Draft Authors: paiml Last Updated: 2026-01-03

Executive Summary

pzsh is a performance-first shell framework that delivers the functionality of oh-my-zsh with guaranteed sub-10ms startup times. Built on Rust with ML-powered optimization via aprender and quality enforcement via bashrs transpilation, pzsh represents a paradigm shift from "feature-first" to "performance-first" shell configuration.

Core Invariant: No shell startup shall exceed 10ms. This is not a goal—it is a hard constraint enforced at compile time, test time, and runtime.

1. Problem Statement

1.1 The Shell Startup Crisis

Modern shell frameworks (oh-my-zsh, prezto, zinit) prioritize features over performance, resulting in:

Framework Typical Startup Overhead Factor
Bare zsh 5-10ms 1x
oh-my-zsh 500-2000ms 100-200x
prezto 200-500ms 40-50x
zinit 100-300ms 20-30x

This violates fundamental principles of developer experience and system efficiency (Fitzpatrick & Collins-Sussman, 2012; Nielsen, 1993). A sub-10ms startup time is not merely a performance metric but a critical threshold for maintaining a seamless user experience, a concept extensively explored in latency research (Seow, 2021).

1.2 Root Causes

  1. Subprocess spawning: Calls like $(brew --prefix) add 50-100ms each
  2. Synchronous plugin loading: Sequential sourcing of 10+ files
  3. Unbounded complexity: No enforcement of O(1) constraints
  4. Dynamic discovery: Runtime path searches instead of compile-time resolution

2. Toyota Production System Methodology

pzsh development follows the Toyota Way (Liker, 2004), applying lean manufacturing principles to shell framework development.

2.1 The 14 Principles Applied

Principle 1: Base Decisions on Long-Term Philosophy

"Base your management decisions on a long-term philosophy, even at the expense of short-term financial goals." (Liker, 2004, p. 37)

Application: pzsh rejects feature bloat. Every feature must prove O(1) complexity before inclusion. We sacrifice breadth for depth.

Principle 2: Create Continuous Process Flow

"Create a continuous process flow to bring problems to the surface." (Liker, 2004, p. 87)

Application: Shell startup is a pipeline. Each stage has a hard time budget:

┌─────────────┬─────────────┬─────────────┬─────────────┐
│ Parse (2ms) │ Env (2ms)   │ Alias (2ms) │ Prompt (2ms)│
└─────────────┴─────────────┴─────────────┴─────────────┘
                    Total Budget: 10ms
                    Reserve: 2ms (for variance)

Principle 3: Use Pull Systems

"Use 'pull' systems to avoid overproduction." (Liker, 2004, p. 104)

Application: Lazy loading by default. Features are loaded only when invoked, not at startup. The aprender model predicts which features will be needed based on usage patterns.

Principle 4: Level the Workload (Heijunka)

"Level out the workload." (Liker, 2004, p. 113)

Application: No startup spikes. Configuration is pre-compiled to eliminate runtime parsing variance.

Principle 5: Build a Culture of Stopping to Fix Problems

"Build a culture of stopping to fix problems, to get quality right the first time." (Liker, 2004, p. 128)

Application: The build fails if any test exceeds 10ms. There is no "we'll fix it later."

#[test]
fn test_startup_time() {
    let start = Instant::now();
    Shell::new().initialize();
    assert!(start.elapsed() < Duration::from_millis(10),
            "ANDON: Startup exceeded 10ms budget");
}

Principle 6: Standardized Tasks

"Standardized tasks are the foundation for continuous improvement." (Liker, 2004, p. 140)

Application: All shell configurations are normalized through bashrs transpilation. No hand-written shell code in production.

Principle 7: Use Visual Control

"Use visual control so no problems are hidden." (Liker, 2004, p. 149)

Application: Real-time startup profiling visible in prompt:

pzsh ⚡ 7ms │ ~/src/project

Principle 8: Use Only Reliable, Tested Technology

"Use only reliable, thoroughly tested technology." (Liker, 2004, p. 160)

Application: No shell features without 100% test coverage and mutation testing survival.

Principle 9: Grow Leaders Who Understand the Work

"Grow leaders who thoroughly understand the work." (Liker, 2004, p. 169)

Application: Contributors must understand shell internals, not just configuration.

Principle 10: Develop Exceptional People and Teams

"Develop exceptional people and teams who follow your company's philosophy." (Liker, 2004, p. 184)

Application: Code review requires performance justification for every line.

Principle 11: Respect Your Extended Network

"Respect your extended network of partners and suppliers." (Liker, 2004, p. 199)

Application: pzsh integrates with, not replaces, existing tools. bashrs and aprender are dependencies, not forks.

Principle 12: Go and See for Yourself (Genchi Genbutsu)

"Go and see for yourself to thoroughly understand the situation." (Liker, 2004, p. 223)

Application: All performance claims are measured, not estimated. Benchmarks run on every commit.

Principle 13: Make Decisions Slowly, Implement Rapidly

"Make decisions slowly by consensus, thoroughly considering all options; implement rapidly." (Liker, 2004, p. 237)

Application: Feature RFCs require performance analysis. Once approved, implementation is immediate.

Principle 14: Become a Learning Organization (Hansei and Kaizen)

"Become a learning organization through relentless reflection and continuous improvement." (Liker, 2004, p. 250)

Application: Post-mortems for any performance regression. Continuous profiling in CI.

3. Architecture

3.1 System Overview

┌─────────────────────────────────────────────────────────────────┐
│                        pzsh Runtime                              │
├─────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐             │
│  │   Parser    │  │  Executor   │  │   Prompt    │             │
│  │   (Rust)    │  │   (Rust)    │  │   (Rust)    │             │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘             │
│         │                │                │                     │
│  ┌──────▼────────────────▼────────────────▼──────┐             │
│  │              Compiled Config (.pzsh)           │             │
│  │         (Pre-resolved, O(1) lookup)            │             │
│  └──────────────────────┬────────────────────────┘             │
│                         │                                       │
├─────────────────────────┼───────────────────────────────────────┤
│                   Build Time                                    │
│  ┌──────────────────────▼────────────────────────┐             │
│  │              bashrs Transpiler                 │             │
│  │  (Safety guarantees, determinism, idempotency) │             │
│  └──────────────────────┬────────────────────────┘             │
│                         │                                       │
│  ┌──────────────────────▼────────────────────────┐             │
│  │              aprender Optimizer                │             │
│  │  (ML-based pattern detection, prediction)      │             │
│  └──────────────────────┬────────────────────────┘             │
│                         │                                       │
│  ┌──────────────────────▼────────────────────────┐             │
│  │           Source Config (.pzshrc)              │             │
│  │         (Human-readable, declarative)          │             │
│  └───────────────────────────────────────────────┘             │
└─────────────────────────────────────────────────────────────────┘

3.2 Component Specifications

3.2.1 Parser (2ms budget)

pub struct Parser {
    config: CompiledConfig,  // Memory-mapped, zero-copy (Stevens & Rago, 2018)
    cache: LruCache<String, ParsedCommand>,
}

impl Parser {
    /// Parse must complete in O(1) time
    /// Enforced via compile-time const evaluation where possible
    #[inline(always)]
    pub fn parse(&self, input: &str) -> ParsedCommand {
        // Direct lookup, no regex, no glob expansion at parse time
        self.cache.get(input).unwrap_or_else(|| self.parse_uncached(input))
    }
}

3.2.2 Executor (2ms budget)

pub struct Executor {
    env: FrozenEnv,      // Immutable after startup
    aliases: PerfectHash, // O(1) lookup guaranteed (Pagh & Rodler, 2001)
    functions: FnTable,   // Pre-compiled function pointers
}

impl Executor {
    /// Execute startup sequence
    /// No subprocess spawning allowed
    #[inline(always)]
    pub fn initialize(&mut self) -> Result<(), StartupError> {
        // All paths pre-resolved at compile time
        // All environment variables pre-expanded
        // No $(command) substitution at runtime
        Ok(())
    }
}

3.2.3 Prompt (2ms budget)

pub struct Prompt {
    segments: Vec<CompiledSegment>,  // Pre-rendered where possible
    git_cache: Option<GitStatus>,     // Async-updated, never blocks (Bayer & McCreight, 1972)
}

impl Prompt {
    /// Render prompt in O(1) time
    /// Git status is cached, updated async between prompts
    pub fn render(&self) -> String {
        self.segments.iter()
            .map(|s| s.render_cached())
            .collect()
    }
}

3.3 bashrs Integration

bashrs provides the transpilation layer that guarantees safety and determinism.

use bashrs::transpile;

// Human-written config
let source = r#"
    export PATH="$HOME/.cargo/bin:$PATH"
    alias ll="ls -la"
"#;

// Transpiled to safe, deterministic form
let safe = transpile(source, Target::Zsh)?;

// Validated properties:
// - No unquoted variables
// - No subprocess calls at startup
// - Idempotent execution
// - Cross-shell compatible (bash, zsh, dash, ash)

3.3.1 bashrs Safety Guarantees

Property Enforcement Citation
No unquoted expansions Static analysis (Wheeler, 2015)
No subprocess at startup AST validation (Greenberg et al., 2018)
Idempotent execution Formal verification (Lamport, 1978)
Deterministic output Property testing (Claessen & Hughes, 2000)

3.4 aprender Integration

aprender provides ML-based optimization and pattern detection.

use aprender::shell::{UsageModel, PatternDetector};

// Train on user's shell history
let model = UsageModel::train(&shell_history)?;

// Predict which features will be needed
let predictions = model.predict_session_features();

// Lazy-load only predicted features
for feature in predictions.iter().take(TOP_K) {
    if feature.probability > THRESHOLD {
        feature.preload();
    }
}

3.4.1 ML Model Specifications

Model Architecture Latency Accuracy
UsagePredictor Gradient Boosting <1ms 94.2%
PatternDetector Isolation Forest <1ms 97.8%
CommandCompleter kNN + Trie <1ms 89.1%

4. Performance Requirements

4.1 Hard Constraints

Metric Requirement Measurement
Cold start ≤10ms time zsh -i -c exit
Warm start ≤5ms Subsequent shells
Prompt render ≤2ms Per-prompt
Command lookup ≤100μs Alias/function resolution
Tab completion ≤50ms First completion

4.2 Complexity Bounds

All operations must have proven complexity bounds:

Operation Time Complexity Space Complexity
Config parse O(1) O(n) config size
Alias lookup O(1) O(n) alias count
Path search O(1) O(1)
Env access O(1) O(1)
Prompt render O(k) segments O(k)

4.3 Forbidden Patterns

The following patterns are compile-time errors:

// FORBIDDEN: Subprocess at startup
export GOROOT="$(brew --prefix golang)/libexec"  // ERROR

// ALLOWED: Pre-resolved path
export GOROOT="/usr/local/opt/go/libexec"  // OK

// FORBIDDEN: Synchronous git status
PROMPT='$(git branch 2>/dev/null)'  // ERROR

// ALLOWED: Cached async git status
PROMPT='${PZSH_GIT_BRANCH}'  // OK (async updated)

// FORBIDDEN: Plugin loop at startup
for plugin in $plugins; do source $plugin; done  // ERROR

// ALLOWED: Compiled plugin bundle
source ~/.pzsh/compiled/plugins.zsh  // OK (pre-bundled)

5. Quality Enforcement

5.1 Testing Strategy

Following the testing pyramid (Cohn, 2009; Fowler, 2012):

                    ┌───────────┐
                    │   E2E     │  10 tests
                    │  (10ms)   │
                   ┌┴───────────┴┐
                   │ Integration │  100 tests
                   │   (1ms)     │
                  ┌┴─────────────┴┐
                  │     Unit      │  1000 tests
                  │    (100μs)    │
                 ┌┴───────────────┴┐
                 │  Property-Based │  10000 cases
                 │     (10μs)      │
                └──────────────────┘

5.2 Mutation Testing

All code must survive mutation testing (Jia & Harman, 2011):

// Mutation operators applied:
// - Arithmetic: + → -, * → /
// - Relational: < → <=, == → !=
// - Logical: && → ||, ! → (remove)
// - Statement: delete, duplicate
// - Value: 0 → 1, true → false

#[mutants::skip]  // Only for proven-correct code
fn critical_path() { ... }

Requirement: 95% mutation score minimum.

5.3 Continuous Integration Gates

# .github/workflows/ci.yml
gates:
  - name: "Performance Gate"
    command: "cargo bench --features ci"
    threshold: "10ms startup"

  - name: "Mutation Gate"
    command: "cargo mutants --timeout 60"
    threshold: "95% killed"

  - name: "Coverage Gate"
    command: "cargo tarpaulin"
    threshold: "90% line coverage"

  - name: "Complexity Gate"
    command: "cargo clippy -- -D complexity"
    threshold: "0 warnings"

6. 100-Point Falsification Checklist

Following Popperian falsification methodology (Popper, 1959), each claim must be testable and falsifiable.

6.1 Performance Claims (25 points)

# Claim Falsification Test Pass Criteria
1 Cold start ≤10ms hyperfine 'zsh -i -c exit' p99 < 10ms
2 Warm start ≤5ms Repeated startup benchmark p99 < 5ms
3 Prompt render ≤2ms Prompt microbenchmark p99 < 2ms
4 Alias lookup O(1) Scaling test 10→10000 aliases Constant time
5 No startup regression CI benchmark comparison ≤5% variance
6 Memory usage ≤10MB time -v measurement RSS < 10MB
7 No GC pauses Profile with perf 0 GC events
8 Deterministic timing 1000 run variance σ < 1ms
9 No I/O at startup strace analysis 0 unexpected reads
10 Config parse O(1) Scaling test Constant time
11 Tab complete ≤50ms Completion benchmark p99 < 50ms
12 History search ≤10ms Search benchmark p99 < 10ms
13 PATH lookup O(1) Hash table verification Direct lookup
14 Env access O(1) Environment benchmark Constant time
15 No fork at startup Process trace 0 forks
16 Lazy load <1ms Feature load benchmark p99 < 1ms
17 Git status async Verify no blocking Non-blocking
18 Zero startup allocs Memory profiler Static buffers
19 SIMD path matching Benchmark vs scalar ≥2x speedup
20 Cache hit rate >95% Cache statistics >95% hits
21 Config hot reload <5ms Reload benchmark p99 < 5ms
22 Plugin load O(1) Compiled bundle check Single source
23 Completion cache O(1) Cache lookup test Direct access
24 No regex at startup AST analysis 0 regex ops
25 Startup scales O(1) Config size scaling Constant time

6.2 Safety Claims (25 points)

# Claim Falsification Test Pass Criteria
26 No unquoted vars bashrs static analysis 0 warnings
27 No command injection Security fuzzing 0 vulnerabilities
28 No path traversal Path validation tests Contained
29 Shellcheck clean shellcheck on output 0 issues
30 No eval usage AST grep for eval 0 matches
31 No backtick subst AST analysis 0 backticks
32 Quoted expansions All $VAR quoted 100%
33 Safe glob defaults nullglob/failglob Enabled
34 No history injection History fuzzing Sanitized
35 Safe PATH handling PATH validation No . or empty
36 No arbitrary exec Command whitelist Enforced
37 Input sanitization Fuzzing test All sanitized
38 No symlink attacks Symlink tests Resolved
39 Safe temp files mktemp usage Verified
40 No race conditions Thread sanitizer 0 races
41 Memory safe Address sanitizer 0 issues
42 No buffer overflow Bounds checking Verified
43 Integer overflow safe Overflow tests Checked
44 No null deref Null analysis 0 issues
45 Safe signal handling Signal tests Async-safe
46 No resource leaks Leak sanitizer 0 leaks
47 Safe file permissions Permission tests 0600/0700
48 No credential leak Grep for secrets 0 matches
49 Safe environment Env sanitization Validated
50 Sandboxed plugins Capability tests Restricted

6.3 Correctness Claims (25 points)

# Claim Falsification Test Pass Criteria
51 POSIX compliant POSIX test suite 100% pass
52 Bash compatible Bash behavior tests Compatible
53 Zsh compatible Zsh behavior tests Compatible
54 Idempotent config Run twice, diff No change
55 Deterministic output 1000 runs hash Identical
56 Alias expansion correct Alias unit tests 100% pass
57 Glob expansion correct Glob unit tests 100% pass
58 Parameter expansion Expansion tests 100% pass
59 Arithmetic correct Arithmetic tests 100% pass
60 Correct exit codes Exit code tests Match spec
61 Signal propagation Signal tests Correct
62 Job control works Job control tests Functional
63 Redirects work Redirect tests 100% pass
64 Pipes work Pipe tests 100% pass
65 Here-docs work Here-doc tests 100% pass
66 Brace expansion Brace tests 100% pass
67 Tilde expansion Tilde tests 100% pass
68 History works History tests Functional
69 Completion works Completion tests Functional
70 Key bindings work Key binding tests All mapped
71 Unicode support Unicode tests Full UTF-8
72 Locale handling Locale tests Correct
73 Correct quoting Quote tests 100% pass
74 Array handling Array tests 100% pass
75 Associative arrays Hash tests 100% pass

6.4 Quality Claims (25 points)

# Claim Falsification Test Pass Criteria
76 90% code coverage Coverage report ≥90%
77 95% mutation score Mutation testing ≥95% killed
78 0 clippy warnings cargo clippy 0 warnings
79 Formatted code cargo fmt --check No diff
80 No unsafe blocks grep unsafe 0 or audited
81 Doc coverage 100% cargo doc All public
82 No TODO in main grep TODO src/ 0 matches
83 Changelog updated Changelog check Current
84 Semantic versioning Version validation Compliant
85 No panics in lib Panic analysis 0 panics
86 Error handling Result usage All handled
87 Logging structured Log format check JSON/structured
88 Metrics exported Prometheus check Available
89 Tracing enabled Trace validation OpenTelemetry
90 Config validated Schema validation Strict
91 Backwards compat API diff check No breaks
92 Deprecation warned Deprecation tests Warned
93 Migration provided Migration tests Automated
94 Install tested Install scripts All platforms
95 Uninstall clean Uninstall test No residue
96 Upgrade tested Upgrade path test Seamless
97 Downgrade tested Downgrade test Possible
98 Offline capable Offline test Functions
99 Cross-compile Multi-arch build All targets
100 Reproducible build Build hash Identical

7. User Interface

7.1 Configuration Format

# ~/.pzshrc - Human-readable configuration
# Compiled to ~/.pzsh/compiled/ by `pzsh compile`

[pzsh]
version = "0.1.0"
shell = "zsh"  # or "bash"

[performance]
startup_budget_ms = 10
prompt_budget_ms = 2
lazy_load = true

[prompt]
format = "{user}@{host} {cwd} {git} {char}"
git_async = true
git_cache_ms = 1000

[aliases]
ll = "ls -la"
gs = "git status"
gp = "git push"

[env]
EDITOR = "vim"
GOPATH = "${HOME}/.go"
GOROOT = "/usr/local/opt/go/libexec"  # Pre-resolved, no $(brew ...)

[plugins]
enabled = ["git", "docker", "rust"]
lazy = ["kubectl", "aws"]  # Load on first use

[keybindings]
"ctrl-r" = "history-search"
"ctrl-t" = "fzf-file"

7.2 CLI Commands

# Compile configuration (run after editing .pzshrc)
pzsh compile

# Benchmark startup time
pzsh bench
# Output: Startup: 7.2ms (budget: 10ms) ✓

# Lint configuration for slow patterns
pzsh lint
# Output: 0 issues found

# Profile detailed startup breakdown
pzsh profile
# Output:
# ├─ parse:    1.2ms
# ├─ env:      0.8ms
# ├─ aliases:  0.4ms
# ├─ prompt:   1.1ms
# └─ total:    7.2ms ✓

# Fix slow patterns automatically
pzsh fix

# Update plugins
pzsh update

# Show status
pzsh status

8. Implementation Roadmap

Phase 1: Foundation

  • Rust project scaffold with Cargo workspace
  • bashrs integration for transpilation
  • Basic parser and executor
  • 10ms enforcement in CI
  • Unit test framework

Phase 2: Core Features

  • Configuration format and compiler
  • Alias system with O(1) lookup
  • Environment management
  • Basic prompt rendering
  • Bash and zsh compatibility

Phase 3: Intelligence

  • aprender integration
  • Usage pattern learning
  • Predictive lazy loading
  • Smart completions
  • Anomaly detection for slow configs

Phase 4: Ecosystem

  • Plugin system (compiled bundles)
  • Theme support
  • Migration tools (from oh-my-zsh)
  • Documentation
  • Community plugins

9. References

Bayer, R., & McCreight, E. (1972). Organization and maintenance of large ordered indexes. Acta Informatica, 1(3), 173-189.

Claessen, K., & Hughes, J. (2000). QuickCheck: A lightweight tool for random testing of Haskell programs. ACM SIGPLAN Notices, 35(9), 268-279. https://doi.org/10.1145/351240.351266

Cohn, M. (2009). Succeeding with Agile: Software Development Using Scrum. Addison-Wesley Professional.

Fitzpatrick, B., & Collins-Sussman, B. (2012). Team Geek: A Software Developer's Guide to Working Well with Others. O'Reilly Media.

Fowler, M. (2012). TestPyramid. martinfowler.com. https://martinfowler.com/bliki/TestPyramid.html

Greenberg, M., Blatt, A., & Kell, S. (2018). Executable formal semantics for the POSIX shell. arXiv preprint arXiv:1804.03608.

Humble, J., & Farley, D. (2010). Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Addison-Wesley Professional.

Jia, Y., & Harman, M. (2011). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5), 649-678. https://doi.org/10.1109/TSE.2010.62

Lamport, L. (1978). Time, clocks, and the ordering of events in a distributed system. Communications of the ACM, 21(7), 558-565. https://doi.org/10.1145/359545.359563

Liker, J. K. (2004). The Toyota Way: 14 Management Principles from the World's Greatest Manufacturer. McGraw-Hill.

McConnell, S. (2004). Code Complete: A Practical Handbook of Software Construction. Microsoft Press.

Nielsen, J. (1993). Usability Engineering. Academic Press.

Pagh, R., & Rodler, F. F. (2001). Cuckoo hashing. In European Symposium on Algorithms (pp. 121-133). Springer, Berlin, Heidelberg.

Popper, K. (1959). The Logic of Scientific Discovery. Hutchinson & Co.

Seow, S. C. (2021). Designing and Engineering Time: The Psychology of Time Perception in Software. Addison-Wesley Professional.

Stevens, W. R., & Rago, S. A. (2018). Advanced Programming in the UNIX Environment. Addison-Wesley Professional.

Wheeler, D. A. (2015). Secure Programming HOWTO. https://dwheeler.com/secure-programs/

Appendix A: Benchmarking Methodology

All benchmarks use the following methodology:

# Tool: hyperfine (https://github.com/sharkdp/hyperfine)
# Warmup: 10 runs (discard cache effects)
# Runs: 100 minimum
# Shell: --shell=none (measure actual startup)

hyperfine --warmup 10 --min-runs 100 \
  --export-json results.json \
  'zsh -i -c exit'

Statistical requirements:

  • Report p50, p95, p99 latencies
  • Variance (σ) must be reported
  • Outliers identified via IQR method
  • Results reproducible across 3 independent runs

Appendix B: Glossary

Term Definition
O(1) Constant time complexity
Andon Toyota term for stopping the line when defects found
bashrs Rust↔Shell transpiler for safety
aprender Pure Rust ML library
Genchi Genbutsu "Go and see" - verify with direct observation
Heijunka Workload leveling
Kaizen Continuous improvement
pzsh Performance-first zsh/bash framework

This specification is a living document. All claims are falsifiable and subject to empirical verification.