Skip to content

CLAUDE.md

Latest commit

 

History

History
431 lines (361 loc) · 13.6 KB

CLAUDE.md

File metadata and controls

431 lines (361 loc) · 13.6 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Korean Subtitle Extractor MVP

Project Context

You are building a micro-SaaS MVP that extracts hardcoded Korean subtitles from YouTube videos using OCR and optionally translates them to English. The application must be production-ready with minimal UI, robust error handling, and comprehensive testing.

Points to Remember

  • To know about the project, always start with the docs folder.
  • Never start implementation/code changes when asked for suggestions, questions, explanations.
  • Never add claude or Anthropic related texts anywhere in the project, even in commit messages
  • Never install libraries globally unless absolutely necessary. Always check if the library/package is installed before trying to install it
  • Always make changes with caution and if its core logic, keep it revertable preferrably as a separate commit

Development Commands

Project Setup

# Backend setup
cd backend
python -m venv venv
source venv/bin/activate  # Windows: venv/Scripts/activate
pip install -r requirements.txt

# Frontend setup  
cd frontend
npm install

# Environment setup
cp backend/.env.example backend/.env
cp frontend/.env.example frontend/.env

Development

# Start backend server
cd backend
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Start frontend dev server
cd frontend
npm run dev

# Run both concurrently (if available)
npm run dev:all

Testing

# Backend tests
cd backend
pytest tests/ --cov=app --cov-report=html
pytest tests/unit/  # Unit tests only
pytest tests/integration/  # Integration tests only

# Frontend tests
cd frontend  
npm run test
npm run test:watch
npm run test:coverage

# E2E tests
npm run test:e2e

Code Quality

# Backend linting and formatting
cd backend
pylint app/
black app/
isort app/

# Frontend linting and type checking
cd frontend
npm run lint
npm run lint:fix  
npm run type-check

Build and Deploy

# Frontend build
cd frontend
npm run build
npm run preview  # Preview production build

# Backend requirements
cd backend
pip freeze > requirements.txt

# Docker build (future)
docker build -t korean-subtitle-extractor .

Architecture Overview

Project Structure

korean-subtitle-extractor/
├── frontend/          # React TypeScript + Vite
│   ├── src/
│   │   ├── components/    # Shadcn UI components
│   │   ├── hooks/         # useWebSocket, useProcess
│   │   ├── services/      # API client, processing
│   │   ├── types/         # TypeScript definitions
│   │   └── pages/         # Home, Metrics
│   └── tests/
├── backend/           # Python FastAPI
│   ├── app/
│   │   ├── api/v1/        # API endpoints
│   │   ├── services/      # OCR, translation, SRT generation
│   │   ├── models/        # SQLAlchemy models
│   │   ├── utils/         # Text processing, validation
│   │   └── main.py
│   └── tests/
└── shared/            # Common types/constants

Processing Pipeline

  1. Video Download: yt-dlp extracts video info and validates duration (≤20min)
  2. Frame Extraction: OpenCV extracts 1 frame per second with timestamps
  3. OCR Processing: Google Cloud Vision API batch processes frames (10 frames/request)
  4. Text Deduplication: Removes duplicate text using Levenshtein distance (90% threshold)
  5. Translation: Google Cloud Translate API translates Korean to English (optional)
  6. SRT Generation: Creates properly formatted SRT files for both languages

Key Technologies

  • Frontend: React 18 + TypeScript (strict) + Vite + Shadcn/ui + React Query
  • Backend: FastAPI + SQLAlchemy + Pydantic + Redis (caching)
  • External APIs: Google Cloud Vision (OCR) + Google Cloud Translate
  • Database: SQLite (MVP) → PostgreSQL (production)

Critical Requirements

  1. 1 frame per second extraction rate for accuracy
  2. 20-minute maximum video duration for MVP
  3. No authentication - free service for validation
  4. Both Korean and English SRT output with translation toggle
  5. Minimal UI using Shadcn components - no fancy animations
  6. Complete test suite for backend
  7. Strict TypeScript for frontend

Development Patterns

Git Commit Standards

Commit at main checkpoint implementations, or for every subphase/phase implementation

# Use conventional commits
feat: add YouTube URL validation
fix: resolve OCR deduplication bug
test: add unit tests for SRT generation
docs: update API documentation
refactor: optimize frame extraction logic

Python Code Standards

# Always use type hints and async/await
from typing import List, Optional, Dict
from pydantic import BaseModel, Field

async def process_video(url: str) -> ProcessingResult:
    """Process video and extract subtitles."""
    pass

# Use Pydantic for validation with regex
class ProcessRequest(BaseModel):
    youtube_url: str = Field(..., regex=r'^https?://(www\.)?(youtube\.com/watch\?v=|youtu\.be/)[\w-]+')
    translate_to_english: bool = False

# Constants in uppercase
MAX_VIDEO_DURATION = 1200  # 20 minutes
FRAME_EXTRACTION_FPS = 1
OCR_BATCH_SIZE = 10

FastAPI Patterns

# Use background tasks for long operations
@app.post("/api/v1/process")
async def process_video(
    request: ProcessRequest,
    background_tasks: BackgroundTasks
):
    job_id = str(uuid.uuid4())
    background_tasks.add_task(process_video_task, job_id, request)
    return {"job_id": job_id, "status": "processing"}

# Use dependency injection for database
async def get_db() -> AsyncGenerator[AsyncSession, None]:
    async with async_session() as session:
        yield session

Google Cloud API Integration

# Batch OCR processing for efficiency (10 frames per request)
async def process_ocr_batch(cropped_frames: List[bytes]) -> List[OCRResult]:
    batch_requests = [
        vision.AnnotateImageRequest(
            image=vision.Image(content=frame_data),
            features=[vision.Feature(type_=vision.Feature.Type.TEXT_DETECTION)],
            image_context=vision.ImageContext(language_hints=["ko"])
        ) for frame_data in cropped_frames
    ]
    responses = client.batch_annotate_images(requests=batch_requests)
    return parse_ocr_responses(responses)

# CRITICAL: Bulk translation for better context preservation
async def translate_batch_bulk(texts: List[str]) -> List[str]:
    """Translate all Korean texts together for better context and meaning"""
    if not texts:
        return []
    
    # Join texts with delimiter for bulk translation
    combined_text = " ||DELIMITER|| ".join(texts)
    
    translated_combined = await translate_client.translate(
        combined_text,
        source_language="ko",
        target_language="en"
    )
    
    # Split back and map to original order
    translated_texts = translated_combined.split(" ||DELIMITER|| ")
    return translated_texts[:len(texts)]  # Ensure same length

Critical Data Model - Frame->Timestamp->OCR->Translation Relationships

from dataclasses import dataclass
from typing import Optional

@dataclass
class FrameOCRMapping:
    """
    CRITICAL: Core data model maintaining frame -> timestamp -> OCR -> translation relationships.
    This ensures we never lose the connection between original frame timing and processed text.
    """
    frame_number: int                    # For ordering and tracking
    timestamp_seconds: float            # Original frame timestamp (1fps = 1 second intervals)
    frame_data: bytes                   # Original frame data
    cropped_frame_data: bytes          # Cropped frame (remove channel logos)
    korean_text: str = ""              # OCR result (empty if no text detected)
    english_text: str = ""             # Translation result (empty if not translated)
    confidence_score: float = 0.0      # OCR confidence
    bounding_box: Optional[dict] = None # OCR bounding box
    is_empty: bool = True              # Track frames with no text
    cropped_frame_hash: str = ""       # For duplicate frame detection

def crop_frame_sides(frame_data: bytes, crop_ratio: float = 0.1) -> bytes:
    """CRITICAL: Remove YouTube channel logos from frame sides before OCR"""
    image = cv2.imdecode(np.frombuffer(frame_data, np.uint8), cv2.IMREAD_COLOR)
    height, width = image.shape[:2]
    
    # Remove crop_ratio from left and right sides
    crop_width = int(width * crop_ratio)
    cropped_image = image[:, crop_width:width-crop_width]
    
    # Enhance for better OCR
    gray = cv2.cvtColor(cropped_image, cv2.COLOR_BGR2GRAY)
    enhanced = cv2.addWeighted(gray, 1.2, gray, 0, 10)
    
    _, buffer = cv2.imencode('.jpg', enhanced)
    return buffer.tobytes()

def deduplicate_consecutive_mappings(mappings: List[FrameOCRMapping], threshold: float = 0.9) -> List[FrameOCRMapping]:
    """Remove duplicate text while preserving frame->timestamp relationships"""
    if not mappings:
        return []
    
    # Only process non-empty mappings for deduplication
    non_empty = [m for m in mappings if not m.is_empty]
    if not non_empty:
        return []
    
    unique = [non_empty[0]]
    for current in non_empty[1:]:
        last = unique[-1]
        similarity = SequenceMatcher(None, last.korean_text, current.korean_text).ratio()
        
        if similarity < threshold:
            unique.append(current)  # Different subtitle
        # Same subtitle - skip but preserve timing info for SRT generation
    
    return unique

Frontend Development Patterns

TypeScript Configuration (Strict Mode Required)

// tsconfig.json - Use strict mode with path aliases
{
  "compilerOptions": {
    "strict": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "paths": { "@/*": ["./src/*"] }
  }
}

Key Type Definitions

// types/processing.types.ts - Never use 'any', always define proper types
export interface ProcessingJob {
  id: string;
  youtube_url: string;
  status: ProcessingStatus;
  progress: number;
  error?: ProcessingError;
}

export enum ProcessingStatus {
  PENDING = 'pending',
  PROCESSING_OCR = 'processing_ocr',
  TRANSLATING = 'translating',
  COMPLETED = 'completed',
  FAILED = 'failed'
}

export interface ProcessingError {
  code: string;
  message: string;
}

React Component Patterns

// Use functional components with proper typing, memo, useCallback
import { FC, useState, useCallback, memo } from 'react';

interface URLProcessorProps {
  onSubmit: (url: string, translate: boolean) => Promise<void>;
  isProcessing: boolean;
}

export const URLProcessor: FC<URLProcessorProps> = memo(({ onSubmit, isProcessing }) => {
  const [url, setUrl] = useState<string>('');
  const [error, setError] = useState<string | null>(null);

  const validateYouTubeURL = useCallback((url: string): boolean => {
    const regex = /^(https?:\/\/)?(www\.)?(youtube\.com\/watch\?v=|youtu\.be\/)[\w-]+/;
    return regex.test(url);
  }, []);

  // Component implementation...
});

API Client Pattern

// services/api.ts - Use axios with proper error handling
class APIClient {
  private client = axios.create({
    baseURL: import.meta.env.VITE_API_URL,
    timeout: 30000,
  });

  async processVideo(url: string, translate: boolean): Promise<ProcessingJob> {
    const response = await this.client.post('/api/v1/process', {
      youtube_url: url,
      translate_to_english: translate,
    });
    return response.data;
  }
}

WebSocket for Real-Time Progress

// hooks/useWebSocket.ts - Handle reconnection and cleanup
export const useWebSocket = (jobId: string | null, onMessage: (msg: WebSocketMessage) => void) => {
  const ws = useRef<WebSocket | null>(null);
  
  const connect = useCallback(() => {
    if (!jobId) return;
    ws.current = new WebSocket(`${import.meta.env.VITE_WS_URL}/ws/progress/${jobId}`);
    ws.current.onmessage = (event) => onMessage(JSON.parse(event.data));
  }, [jobId, onMessage]);

  useEffect(() => {
    connect();
    return () => ws.current?.close();
  }, [connect]);
};

Environment Variables

# Backend (.env)
GOOGLE_APPLICATION_CREDENTIALS=path/to/credentials.json
GOOGLE_CLOUD_PROJECT=your-project-id
DATABASE_URL=sqlite:///./app.db
REDIS_URL=redis://localhost:6379
MAX_VIDEO_DURATION=1200
CORS_ORIGINS=["http://localhost:5173"]
GA4_MEASUREMENT_ID=G-XXXXXXXXXX
CLARITY_PROJECT_ID=your-clarity-id
FRAME_CROP_RATIO=0.1

# Frontend (.env)
VITE_API_URL=http://localhost:8000
VITE_WS_URL=ws://localhost:8000
VITE_GA4_MEASUREMENT_ID=G-XXXXXXXXXX
VITE_CLARITY_PROJECT_ID=your-clarity-id

Critical Pitfalls to Avoid

  1. Don't skip frame cropping - MUST remove YouTube channel logos before OCR
  2. Don't lose frame->timestamp->OCR->translation relationships - Use FrameOCRMapping throughout
  3. Don't skip OCR order preservation - Handle empty OCR results while maintaining frame order
  4. Don't translate texts individually - Use bulk translation for better context
  5. Don't process videos synchronously - Use FastAPI background tasks
  6. Don't use 'any' type in TypeScript - Always define proper types
  7. Don't make unbatched API calls - Batch OCR requests (10 frames/call)
  8. Don't forget to track analytics events - GA4 and Clarity for user behavior
  9. Don't skip WebSocket cleanup - Prevent memory leaks
  10. Don't store large files in database - Use filesystem for SRT files

Key API Endpoints

  • POST /api/v1/process - Start processing job
  • GET /api/v1/status/{job_id} - Get job status
  • GET /api/v1/download/{job_id}/{language} - Download SRT files
  • WebSocket /ws/progress/{job_id} - Real-time progress updates