add schema validation and markdown linting

[JeremyDev87]

Add JSON Schema Validation and Markdown Linting for AI Rules

📋 Summary

Adds comprehensive validation infrastructure for AI rules including JSON schema validation for agent definitions and Markdown linting for rule files. This ensures consistent structure and quality of AI rule files through automated validation.

Closes #46

🎯 Problem

Manual Validation Issues

AI rules were validated manually with no automated checks:

1. No JSON Schema Validation

   • Agent definition files could have invalid structure
   • Missing required fields not caught early
   • Inconsistent field names and types
   • No IDE autocomplete or validation

2. No Markdown Linting

   • Inconsistent formatting across rule files
   • No enforcement of markdown standards
   • Hard to maintain consistent documentation style

3. Limited Validation Script

   • Only checked directory structure and file existence
   • No content validation
   • No schema or format checking

4. No CI/CD Integration

   • Invalid rules could be merged
   • No automated quality checks
   • Manual review required for all changes

Business Impact

• Quality Issues: Invalid agent definitions could cause runtime errors
• Maintenance Burden: Manual validation is time-consuming
• Inconsistency: No enforced standards for rule files
• Developer Experience: No IDE support for agent definitions
• CI/CD Gaps: Quality issues discovered late in the process

✨ Solution

1. JSON Schema for Agent Definitions (agent.schema.json, 258 lines)

New File: Comprehensive JSON Schema Draft 7 schema

Features:

Required Fields Validation

• name - Agent display name (string, minLength: 1)
• description - Brief description (string, minLength: 10)
• role - Role definition (object, requires title)
• context_files - Context file paths (array, pattern: ^\.ai-rules/.*)

Agent Type Support

• Developer Agents: activation field
• Reviewer Agents: activation field
• Specialist Agents: modes field (planning, implementation, evaluation)
• Other Agents: Flexible structure

Comprehensive Property Definitions

• activation - Activation configuration
• modes - Mode-specific configuration
• workflow - Workflow configuration
• communication - Communication preferences
• evaluation_framework - Evaluation framework
• code_quality_checklist - Code quality items
• And 20+ other optional fields

Schema Highlights:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Codingbuddy Agent Definition",
  "required": ["name", "description", "role", "context_files"],
  "properties": {
    "name": {
      "type": "string",
      "minLength": 1,
      "examples": ["Frontend Developer", "Code Reviewer"]
    },
    "context_files": {
      "type": "array",
      "items": {
        "type": "string",
        "pattern": "^\\.ai-rules/.*"
      },
      "minItems": 1
    }
  }
}

Benefits:

• ✅ IDE autocomplete and validation
• ✅ Early error detection
• ✅ Consistent structure enforcement
• ✅ Documentation through schema

2. Schema Documentation (schemas/README.md, 66 lines)

New File: Complete schema usage guide

Sections:

Available Schemas

• Agent schema overview
• Required fields documentation
• Agent type classification

Usage Instructions

• VS Code integration setup
• CLI validation commands
• Manual validation with ajv

Schema Development

• Modification workflow
• Testing procedures
• Breaking change handling

Key Features:

• Clear usage examples
• VS Code integration guide
• Development guidelines

3. Markdown Linting Configuration (.markdownlint.json, 28 lines)

New File: Markdown linting rules

Configuration Highlights:

{
  "default": true,
  "MD003": { "style": "atx" },
  "MD004": { "style": "dash" },
  "MD007": { "indent": 2 },
  "MD013": false,  // Line length (disabled for code blocks)
  "MD022": false,  // Headings (disabled for flexibility)
  "MD046": { "style": "fenced" },
  "MD048": { "style": "backtick" }
}

Rules:

• ATX heading style (# Heading)
• Dash list style (- Item)
• 2-space indentation
• Fenced code blocks
• Backtick code spans
• Disabled rules for flexibility (line length, heading spacing)

Benefits:

• ✅ Consistent markdown formatting
• ✅ Enforced style standards
• ✅ Better readability
• ✅ Automated formatting checks

4. Enhanced Validation Script (validate-rules.sh, +191 lines)

Major Enhancements:

Command-Line Options

• --schema-only - Run only JSON schema validation
• --markdown-only - Run only Markdown linting
• --skip-schema - Skip schema validation
• --skip-markdown - Skip markdown linting
• -h, --help - Show help message

Three-Phase Validation

Phase 1: Directory Structure (if not schema/markdown-only)

• Check .ai-rules/ directory exists
• Verify subdirectories (rules, agents, adapters, schemas)
• Check required rule files exist and are non-empty
• Validate agent JSON syntax
• Check adapter files
• Verify main README

Phase 2: JSON Schema Validation (if not markdown-only)

• Validate all agent files against schema
• Use ajv-cli for validation
• Report invalid files with details
• Count validation errors

Phase 3: Markdown Linting (if not schema-only)

• Lint all markdown files in .ai-rules/
• Use markdownlint-cli2
• Report linting errors
• Count linting errors

Improved Output

• Color-coded messages (green/yellow/red)
• Section headers for clarity
• Error counting and summary
• Clear success/failure messages

Before:

# Only structure validation
./scripts/validate-rules.sh

After:

# Full validation (structure + schema + markdown)
./scripts/validate-rules.sh

# Schema only
./scripts/validate-rules.sh --schema-only

# Markdown only
./scripts/validate-rules.sh --markdown-only

5. CI/CD Integration (.github/workflows/dev.yml, +25 lines)

New Job: rules-validation

Features:

• Runs after dependency installation
• Validates JSON schema using ajv-cli
• Lints markdown files using markdownlint-cli2
• Fails CI if validation errors found
• Clear output messages

Workflow Integration:

install-dependencies
    ↓
rules-validation (NEW)
    ↓
(other jobs continue)

Benefits:

• ✅ Automated quality checks
• ✅ Prevents invalid rules from merging
• ✅ Consistent validation across all PRs
• ✅ Early error detection

6. Yarn Scripts (package.json, +3 scripts)

New Scripts:

{
  "validate:rules": "cd .. && ./scripts/validate-rules.sh",
  "validate:rules:schema": "cd .. && ./scripts/validate-rules.sh --schema-only",
  "validate:rules:markdown": "cd .. && ./scripts/validate-rules.sh --markdown-only"
}

Usage:

# Full validation
yarn validate:rules

# Schema only
yarn validate:rules:schema

# Markdown only
yarn validate:rules:markdown

Benefits:

• ✅ Easy to run validation
• ✅ Consistent command interface
• ✅ Integrated with development workflow

7. Documentation Updates

CONTRIBUTING.md (+3 lines)

• Added validate:rules command to testing section
• Instructions for validating rules before PR

development.md (+21 lines)

• New "Rules Validation" section
• Command reference
• Validation scope explanation

Key Additions:

### Rules Validation

Validate `.ai-rules/` files before committing:

```bash
# Full validation (structure + schema + markdown)
yarn validate:rules

# Schema validation only
yarn validate:rules:schema

# Markdown linting only
yarn validate:rules:markdown

## 📁 Files Changed

| File | Changes |
|------|---------|
| `.ai-rules/schemas/agent.schema.json` | New JSON schema (+258 lines) |
| `.ai-rules/schemas/README.md` | New schema documentation (+66 lines) |
| `.markdownlint.json` | New markdown linting config (+28 lines) |
| `scripts/validate-rules.sh` | Enhanced validation script (+191 lines, -88 lines) |
| `.github/workflows/dev.yml` | Added rules-validation job (+25 lines) |
| `mcp-server/package.json` | Added validation scripts (+3 lines) |
| `CONTRIBUTING.md` | Added validation instructions (+3 lines) |
| `docs/development.md` | Added validation section (+21 lines) |

**Total:** 8 files changed, +597 insertions, -88 deletions

## 🧪 Testing

### Schema Validation

- ✅ All existing agent files pass schema validation
- ✅ Required fields properly enforced
- ✅ Pattern validation works for `context_files`
- ✅ Optional fields don't break validation

### Markdown Linting

- ✅ All markdown files pass linting
- ✅ Consistent formatting enforced
- ✅ Disabled rules work correctly

### Script Functionality

- ✅ All command-line options work
- ✅ Error reporting is clear
- ✅ Exit codes are correct
- ✅ Color output works in CI

### CI/CD Integration

- ✅ Validation job runs successfully
- ✅ Fails on validation errors
- ✅ Clear error messages in CI logs

## 🎯 Benefits

### 1. **Early Error Detection**
Schema validation catches errors before runtime.

### 2. **IDE Support**
VS Code autocomplete and validation for agent files.

### 3. **Consistent Quality**
Automated checks ensure consistent structure and formatting.

### 4. **Developer Experience**
Clear error messages and easy-to-use commands.

### 5. **CI/CD Integration**
Automated validation prevents invalid rules from merging.

### 6. **Maintainability**
Schema serves as documentation and validation.

### 7. **Flexibility**
Command-line options allow selective validation.

## 📖 Usage Examples

### VS Code Integration

Add to `.vscode/settings.json`:
```json
{
  "json.schemas": [
    {
      "fileMatch": [".ai-rules/agents/*.json"],
      "url": "./.ai-rules/schemas/agent.schema.json"
    }
  ]
}

CLI Validation

# Full validation
yarn validate:rules

# Schema only
yarn validate:rules:schema

# Markdown only
yarn validate:rules:markdown

Manual Validation

# Using ajv directly
npx ajv validate -s .ai-rules/schemas/agent.schema.json -d ".ai-rules/agents/*.json"

🔗 Related Documentation

• Schemas README
• Development Guide
• Contributing Guidelines

📝 Design Decisions

Why JSON Schema Draft 7?

• Wide Support: Well-supported standard
• Tool Compatibility: Works with ajv, VS Code, and other tools
• Flexibility: Supports complex validation needs
• Documentation: Schema serves as documentation

Why Markdownlint?

• Standard Tool: Widely used markdown linter
• Configurable: Flexible rule configuration
• CI/CD Ready: Easy to integrate
• VS Code Support: Good editor integration

Why Three-Phase Validation?

• Modularity: Can run phases independently
• Performance: Skip unnecessary checks
• Debugging: Easier to identify issues
• Flexibility: Selective validation options

Why Command-Line Options?

• Selective Validation: Run only needed checks
• Debugging: Isolate validation issues
• Performance: Faster when checking specific aspects
• CI/CD: Different checks for different scenarios

Why CI/CD Integration?

• Automated Quality: No manual checks needed
• Early Detection: Catch errors before merge
• Consistency: Same validation for all PRs
• Documentation: Shows validation is important

✅ Acceptance Criteria

• JSON schema created for agent definitions
• Schema validates all existing agent files
• Markdown linting configuration added
• Validation script enhanced with schema and linting
• Command-line options added to script
• CI/CD job added for validation
• Yarn scripts added for easy validation
• Documentation updated with validation instructions
• VS Code integration documented

🚀 Impact

Validation Coverage

• Before: Structure validation only
• After: Structure + Schema + Markdown validation

Error Detection

• Before: Runtime errors or manual review
• After: Early detection via schema validation

Developer Experience

• Before: No IDE support
• After: Autocomplete and validation in VS Code

CI/CD Quality

• Before: Manual review required
• After: Automated validation prevents invalid rules

💡 Future Enhancements

Potential Improvements

1. Additional Schemas: Schema for rule files, config files
2. Custom Rules: Project-specific validation rules
3. Auto-Fix: Automatically fix linting issues
4. Pre-commit Hook: Run validation before commit
5. Schema Versioning: Handle schema evolution

📊 Before/After Comparison

Before

• ❌ No JSON schema validation
• ❌ No markdown linting
• ⚠️ Basic structure validation only
• ❌ No CI/CD integration
• ❌ No IDE support
• ❌ Manual quality checks

After

• ✅ Comprehensive JSON schema validation
• ✅ Markdown linting with configurable rules
• ✅ Three-phase validation (structure + schema + markdown)
• ✅ CI/CD integration with automated checks
• ✅ VS Code autocomplete and validation
• ✅ Automated quality enforcement

🎓 Lessons Learned

Best Practices

1. Schema First: Define schema before validation
2. Incremental Validation: Add validation gradually
3. Clear Errors: Provide helpful error messages
4. Flexible Tools: Support selective validation
5. Documentation: Document schema and usage

Common Patterns

• ✅ JSON Schema: Standard for JSON validation
• ✅ Markdownlint: Standard for markdown linting
• ✅ Command Options: Enable selective validation
• ✅ CI/CD Integration: Automate quality checks
• ✅ IDE Support: Improve developer experience