Inspect mechanical assemblies with your hands. Analyze them with AI.
No mouse. No expensive software. Just your webcam and Gemini 3.
A browser-based, zero-install CAD inspection tool that combines real-time hand gesture control with Gemini 3 multimodal AI — turning any mechanical assembly into an interactive, AI-powered learning and inspection experience.
🔗 Try it live: gemini-3d-viewer.preview.emergentagent.com/poc
Go to https://gemini-3d-viewer.preview.emergentagent.com/poc in Chrome (recommended).
- Click "Enable Camera" on the permission modal
- Click "Allow" when the browser asks for camera access
- Wait for "Loading AI..." to complete
- Look at the top toolbar — you'll see model options: Gear, Piston, Engine, Assembly
- Click the "Engine" button, then select "V8 Engine" from the dropdown
- Wait for the V8 model to load — it will appear as a detailed multi-colored engine
Show your hand in the webcam preview (bottom-right circle) and use these gestures:
| Gesture | Action | How To |
|---|---|---|
| 🖐️ Open Palm | Rotate view | Hold open hand, move it around |
| 🤏 Pinch | Zoom in/out | Pinch thumb + index together (zoom in) or apart (zoom out) |
| ✊ Fist | Pan / Drag | Make a fist, move it to pan the viewport |
| ☝️ Point | Select part | Point index finger at a part — laser pointer appears |
💡 Tip: You can also use the +/− buttons in the bottom-left corner for zoom, and mouse controls (drag to rotate, scroll to zoom, right-drag to pan) as fallback.
- Click "✦ Ask Gemini" in the top toolbar to open the AI panel
- Click "✦ Tour" for an AI-guided engine walkthrough
- Click "✦ Identify" then point at a part to get AI identification
- Inside the Gemini panel, try the preset buttons:
- Analyze Assembly — full structural analysis
- Failure Points — potential weak spots
- Design Improvements — engineering suggestions
- How It Works — mechanical explanation
- ✦ AI Explode — AI recommends optimal explode level
- ✦ Generate Report — full engineering inspection report
Professional mechanical CAD inspection requires expensive software ($5,000+/year for SolidWorks, CATIA), specialized hardware, and significant training. Engineering students, freelance reviewers, and remote teams are locked out of proper design review workflows.
MechLab XR brings two breakthrough paradigms together in a single browser tab:
| Traditional CAD | MechLab XR | |
|---|---|---|
| Input | Mouse + keyboard | ✋ Hand gestures via webcam |
| Cost | $5,000+/year | Free, browser-based |
| Installation | Heavy desktop software | Zero install — open a URL |
| Intelligence | None — manual inspection | 🤖 Gemini 3 AI analysis |
| Learning Curve | Weeks of training | Intuitive hand gestures |
Six natural hand gestures replace traditional mouse/keyboard input, tracked in real-time via webcam using Google MediaPipe.
- Open Palm → Orbit / Rotate the 3D view
- Pinch In → Zoom in
- Pinch Out → Zoom out
- Fist → Pan / Drag the viewport
- Point → Select part + laser pointer
- Spread → Explode the assembly
Additional: live webcam preview with hand landmarks, gesture confidence indicator, adjustable sensitivity slider, +/− zoom buttons, and full mouse/keyboard fallback.
Five AI-powered features where Gemini 3 sees the live 3D viewport and provides contextual engineering analysis.
Gemini 3 receives a live screenshot of the 3D viewport and provides full structural analysis — identifying components, their arrangement, and mechanical relationships using visual reasoning.
Point at or click any part → Gemini 3 returns:
- Part name (e.g., Crankshaft, Oil Filter, Camshaft)
- Function within the assembly
- Likely material (e.g., Forged steel alloy, Cast aluminum)
- Stress risk level (🟢 Low / 🟡 Medium / 🔴 High)
From a single screenshot, Gemini 3 generates a 5-step interactive walkthrough of how the engine operates — intake, compression, combustion, power, exhaust.
One-click inspection reports with verdict (✅ PASS /
Gemini 3 analyzes the assembly visually and recommends the optimal explode level for inspection, then applies it with one click.
| Tool | Description |
|---|---|
| Exploded View | Animated explosion with parts separating along mechanical axes |
| Connector Lines | Dashed cyan lines showing part relationships |
| Wireframe Mode | Toggle wireframe overlay |
| Engineering Grid | Blueprint-style reference grid with fading edges |
| Contact Shadows | Soft shadow under model for depth |
| Idle Float | Gentle hovering animation when not exploded |
| Multi-Model | Gear, Piston, Engine, V8 Engine + custom .glb upload |
| +/− Zoom Buttons | On-screen zoom controls |
| Gemini 3 Capability | How MechLab XR Uses It |
|---|---|
| Multimodal Vision | Live 3D viewport screenshots analyzed in real-time |
| Structured JSON Output | Responses parsed into tooltips, reports, tours, and smart controls |
| Step-by-Step Reasoning | Mechanical process analysis broken into educational steps |
| Context Understanding | Engineering-specific prompts with model metadata |
| Image + Text Input | Screenshot + question + assembly context per request |
Every AI feature processes live screenshots — change the camera angle, explode the assembly, or load a different model, and Gemini 3 adapts accordingly.
┌───────────────────────────────────────────────────────────┐
│ Browser │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ MediaPipe │ │ Three.js │ │ Gemini 3 │ │
│ │ Hand Track │─▶│ 3D Engine │─▶│ Vision AI │ │
│ │ (Webcam) │ │ (WebGL) │ │ (API) │ │
│ └──────┬───────┘ └──────┬────────┘ └───────┬────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ React + Next.js UI │ │
│ │ Gesture HUD · Toolbar · AI Panel · Guided Tour │ │
│ └─────────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ Backend (Python) │
│ /api/ai/analyze │
│ Gemini 3 via LiteLLM│
└──────────────────────┘
| Layer | Technology | Purpose |
|---|---|---|
| Frontend | React + Next.js | UI framework and routing |
| 3D Engine | Three.js | WebGL rendering with SMAA, Bloom, ACES tone mapping |
| Gestures | MediaPipe Hands | Real-time hand tracking via webcam |
| AI | Google Gemini 3 API | Multimodal vision + structured reasoning |
| AI Gateway | LiteLLM | Model routing and API management |
| Styling | Tailwind CSS + Framer Motion | Dark glassmorphism theme |
| 3D Models | glTF 2.0 (.glb) | Multi-part mechanical assemblies |
| Platform | Emergent.sh | Full-stack development & deployment |
MechLab-XR/
├── frontend/
│ ├── src/
│ │ ├── components/
│ │ │ ├── Viewer3D.js # 3D engine, lighting, materials, explode
│ │ │ ├── GeminiPanel.js # AI chat, reports, smart explode
│ │ │ ├── GestureHUD.js # Gesture display overlay
│ │ │ ├── WebcamPreview.js # Webcam feed with hand landmarks
│ │ │ ├── Toolbar.js # Top toolbar with model selector
│ │ │ ├── ExplodePanel.js # Explode view controls
│ │ │ ├── SectionPanel.js # Section cut plane controls
│ │ │ └── POCViewer.js # Main viewer orchestrator
│ │ ├── hooks/
│ │ │ └── useHandTracking.js # MediaPipe hand tracking hook
│ │ └── utils/
│ │ └── gestureRecognition.js # Gesture detection & camera mapping
│ └── public/ # Static assets and 3D models (.glb)
├── backend/
│ └── server.py # Python API with Gemini 3 endpoint
└── README.md
- Node.js 18+ · Python 3.9+ · Webcam · Chrome browser
git clone https://github.com/SukinShetty/MechLab-XR.git
cd MechLab-XR
# Backend
cd backend && pip install -r requirements.txt && python server.py
# Frontend (new terminal)
cd frontend && npm install && npm run devGEMINI_API_KEY=your_gemini_3_api_key"Porsche Design Studio meets SolidWorks 2030"
- Dark theme with radial gradient spotlight background
- Cyan accent (#00d4ff) across all UI elements
- Glassmorphism panels with backdrop-blur
- Studio lighting with HDRI environment reflections
- Multi-material engine rendering (steel, bronze, blue-steel)
- Minimal, contextual UI — viewport stays clean
| Criteria | Weight | How MechLab XR Delivers |
|---|---|---|
| Technical Execution | 40% | Three.js + MediaPipe + Gemini 3 multimodal, 6 gestures, structured AI |
| Innovation / Wow Factor | 30% | First gesture-controlled CAD viewer with AI vision |
| Potential Impact | 20% | Democratizes CAD inspection for students & remote teams |
| Presentation | 10% | Live demo, video, comprehensive docs |
Developed using Emergent.sh — an AI-powered full-stack development platform for rapid prototyping and deployment.
gemini-3 · three.js · react · next.js · mediapipe · tailwind-css · webgl · emergent.sh · litellm · framer-motion
- Google Gemini 3 — Multimodal AI engine
- MediaPipe — Hand tracking
- Three.js — 3D rendering
- Emergent.sh — Development & deployment
- Sketchfab — 3D model assets (CC Attribution)
MIT License
Sukin Shetty · AI Architect