Web Media Annotator

A powerful, web-based tool for annotating video and image media with frame-accurate precision. Built with a modular architecture that separates core annotation logic from the UI, enabling easy integration into any web application.

🎯 Project Goal

The ultimate goal is to create a professional-grade, browser-based media annotation suite capable of:

• Frame-Accurate Annotation: Drawing on specific video frames with vector precision.
• Temporal Annotations: defining start and end times for annotations (e.g., "follow this object for 5 seconds").
• Rich Media Support: Handling video playback, audio scrubbing, and potentially multi-track syncing.
• Export Flexibility: Exporting data in standard formats (JSON) or "burning in" annotations for review.
• Extensibility: Allowing developers to add custom tools and rendering logic via plugins.

✅ Implemented Features

🎨 Annotation Tools

• Vector Shapes: Draw Squares, Circles, Arrows, and Freehand lines.
• Text Support: Add text labels directly onto frames.
• Selection & Editing:
  • Multi-Select: Hold Ctrl/Cmd + Click to select multiple items.
  • Multi-Drag: Move entire selection groups at once.
• Undo/Redo: Robust history support for all actions (synced in collaborative sessions).
• Color Picking: EyeDropper tool to sample colors from the video.
• Delete Shortcuts: Quickly remove selection with Delete or Backspace.

🎥 Playback & Navigation

• Frame-Accurate Scrubbing: Navigate video frame-by-frame.
• Pan & Zoom:
  • Smooth Interaction: Pan (Hand tool) and Zoom (Scroll wheel) with inertia.
  • Fit-to-Content: Canvas automatically aligns to the video content, ignoring letterboxing/black bars.
  • Pixel-Perfect Sync: Annotations stay locked to video details at any zoom level using CSS transforms.
• Ghosting (Onion Skinning):
  • View previous (red) and future (green) frames overlaid on the current frame.
  • Configurable range (1-10 frames) and opacity.
• 🔊 Audio Scrubbing:
  • Real-time audio feedback while scrubbing the timeline, essential for lip-sync and cueing.
  • Works with both native video and force-downloaded media.

🚀 Performance & Media Support

• Force Download Strategy:
  • Option (preload="force-download") to pre-fetch the entire media file as a Blob.
  • Delivers instant seeking and zero-buffering playback even on slow connections.
• Client-Side Transcoding (FFmpeg WASM):
  • Play Anything: Drag & drop .mkv, .mov (ProRes), .avi, or .heic files.
  • The engine automatically transcodes them locally in the browser to MP4/JPG for frame-accurate annotation.
  • No server upload required – 100% private.
• Smart Caching:
  • efficiently manages large media files in memory for smooth performance.

⌨️ Shortcuts & Interaction

• Mouse: Middle-click to Pan (temporary), Scroll to Zoom.
• Keyboard:
  • Tools: S, P, A, C, Q, T, E.
  • Navigation: Space (Play/Pause), Arrows (Frame step), Ctrl+Arrows (Jump to annotation).
  • View: R (Reset Zoom/Pan).
  • History: Ctrl+Z (Undo), Ctrl+Y (Redo).
  • Edit: Ctrl+C (Copy), Ctrl+V (Paste), Del/Backspace (Delete).
• UI:
  • Unified Sidebar: Properties and information accessible in a slide-out drawer.
  • Shortcuts Modal: Instant help reference accessible from the toolbar.

💾 Import & Export

• Frame Export: Download current frame (Clean or Burned In) as PNG.
• Bulk Export (ZIP): Download all annotated frames as a ZIP archive (Clean or Burned In) with progress tracking.
• Data Portability: Download/Import annotation data as standard JSON.
• State Management: Centralized store with complete state persistence and SNAPSHOT history.

🤝 Real-Time Collaboration

• P2P Sync: Host or Join sessions via WebRTC (PeerJS).
• Shared State: Real-time synchronization of annotations, onion skin settings, and playback state via Yjs.
• Dynamic Updates: Collaborative undo/redo that respects remote changes.

🚧 Roadmap (Pending)

🧊 Simple 3D Viewer (Long Term)

• Goal: Support basic 3D model annotation (OBJ/GLB).
• Utility: Annotating 3D assets for game development or product visualization.

🎵 Audio Visuals

• Goal: Visualize the audio waveform in the timeline.

🛠 Plugins & API

• Goal: Finalize the Plugin API for custom tool creation.

🏗 Architecture

The project is structured as a Monorepo:

• packages/core: The headless library containing the state management (Store), rendering engine (Renderer), and tool logic (PanTool, FreehandTool, etc.). It has no UI dependencies other than a container element.
• apps/demo: A reference implementation using purely Vanilla TypeScript/CSS to demonstrate how to build a UI around the core library.

🌟 Versatile Usage Modes

This project is designed to be used in three distinct ways depending on your needs:

1. Static Web App (Quick Reviews)

   • Use as a standalone tool running completely in the browser.
   • Upload local video/image files which are processed in-memory (no server upload required).
   • Ideal for quick, secure reviews of daily work.

2. CDN Integration (Static Files)

   • Load the library via a script tag from a CDN (or locally hosted static files).
   • Easily embed an annotation widget into legacy apps or static sites using vanilla HTML/JS/CSS.

3. NPM Module (Build Integration)

   • Import @web-media-annotator/core into complex React/Vue/Angular applications.
   • Leverage the full TypeScript API for deep integration and custom UI development.

🚀 Getting Started

Prerequisites

• Node.js (v16+)

Installation

npm install @web-media-annotator/react @web-media-annotator/embed

Usage

1. Static Web App (Quick Reviews)

Zero Setup. No Server Required. Perfect for quickly reviewing local files. Just open the HTML file in your browser.

• Demo: quick-review.html
  • Drag & Drop interface for local video/image files.
  • Runs completely in the browser (using ObjectURLs).
• Static Test: static-test.html
  • Simple hardcoded example using a public video URL.

2. React Component

Best for modern React applications.

import { Annotator } from '@web-media-annotator/react';
import '@web-media-annotator/react/dist/style.css'; // If applicable

function App() {
  return (
    <div style={{ width: '1000px', height: '600px' }}>
      <Annotator 
        src="video.mp4" 
        fps={24} 
        preload="force-download" // Optional: fetches entire file for instant seeking
      />
    </div>
  );
}

3. Web Component (Drop-in)

Works with Vue, Svelte, Angular, or any HTML page.

Using npm:

import '@web-media-annotator/embed';

Using HTML:

<web-media-annotator 
    src="video.mp4" 
    fps="24"
    width="100%"
    height="100%"
    preload="force-download"
></web-media-annotator>

4. Direct CDN / UMD Usage

For legacy sites or static usage without a bundler.

<!-- Load the UMD script -->
<script src="https://unpkg.com/@web-media-annotator/embed/dist/web-media-annotator.umd.js"></script>

<!-- Use the custom element -->
<web-media-annotator src="video.mp4"></web-media-annotator>

Keyboard Shortcuts

Key	Action
Space	Play / Pause
← / →	Prev / Next Frame
P	Pencil Tool
E	Eraser
S	Select Tool
G	Toggle Ghosting (Onion Skin)
H	Toggle Hold (3 Frames)
+ / -	Adjust Stroke Size
Scroll	Zoom Canvas
Mid-Click	Pan Canvas

Development

1. Install Dependencies

   npm install

2. Run Demo

   npm run dev:demo

3. Build Packages

   npm run build

   Open your browser to the URL shown (usually http://localhost:5173 or similar).

🤝 Contributing

Contributions are welcome! Please focus on the packages/core for functional improvements and apps/demo for UI enhancements.

📄 License

This project is licensed under the Mozilla Public License 2.0 (MPL-2.0).

Why MPL-2.0? We chose this license specifically to support the Animation and VFX industry pipeline requirements:

• Commercial Safe: Studios can use this tool in commercial pipelines to create content without risking their proprietary IP or pipeline code.
• Pipeline Friendly: Unlike GPL, using this tool in a larger proprietary pipeline does not force you to open-source your entire pipeline.
• Open Contribution: We strongly believe in giving back. Any modifications to the source code files of this project must be open-sourced, ensuring the tool evolves for everyone's benefit.