Jarvis AI Assistant

this was a really big project, which only 70% works but was made to play around with the capabilities of cursor and other AI tools.

A comprehensive AI assistant built with the Agno framework, featuring multiple specialized agents, advanced memory systems, and seamless integration with Google services. Designed to be your personal AI companion for research, communication, content creation, and task orchestration.

🌟 Features

🤖 Multi-Agent System

• Orchestrator Agent: Coordinates multiple agents and manages workflows
• Research Agent: Web search, fact-checking, and analysis capabilities
• Communication Agent: Email composition, meeting scheduling, and messaging
• Creator Agent: Document creation, presentations, and creative content
• Data Analyst Agent: Data analysis, visualization, and insights
• Financial Analyst Agent: Financial analysis and market research
• Media Analyst Agent: Social media and content analysis
• Social Manager Agent: Social media management and scheduling
• Developer Agent: Code assistance and development tasks
• Custom Agents: Fully customizable agents with personalized prompts and tools

🧠 Advanced Memory System

• Short-term Memory: Session context and temporary information
• Long-term Memory: Persistent user memories and preferences
• Semantic Search: Vector-based memory retrieval using embeddings
• Memory Types: User preferences, conversations, tasks, and factual information
• Automatic Memory Management: Intelligent storage and retrieval

📁 File Processing & Embeddings

• Multi-format Support: PDF, Word, Excel, text, JSON, and image files
• Semantic Search: Find relevant information across all your documents
• Vector Storage: ChromaDB for efficient similarity search
• Content Extraction: Intelligent text extraction from various file types

🔗 Google Services Integration

• Gmail: Read, compose, and send emails with AI assistance
• Google Calendar: View and create events, schedule meetings
• Google Drive: Access, search, and manage your files
• OAuth2 Authentication: Secure access to your Google account

🛠️ Extensible Tool System

• MCP Integration: Full Model Context Protocol support
• Web Search: DuckDuckGo and web scraping capabilities
• Custom Tools: Easy-to-add custom functionality
• Tool Orchestration: Combine multiple tools for complex tasks

🌐 Modern Web Interface

• React Frontend: Modern, responsive user interface
• Real-time Chat: WebSocket-powered streaming conversations
• Agent Management: Create, configure, and manage agents
• File Management: Upload, process, and search documents
• Memory Dashboard: View and manage your AI memories

🚀 FastAPI Backend

• High-performance API: Async FastAPI with WebSocket support
• RESTful Endpoints: Clean and intuitive API design
• Comprehensive Documentation: Auto-generated API docs
• Authentication: JWT-based security
• Background Tasks: Efficient file processing and embeddings

🚀 Quick Start

Prerequisites

• Python 3.10 or higher
• Node.js 18+ (for frontend)
• PostgreSQL 14+ (optional, SQLite fallback available)
• Google Cloud Console account (for Google services)
• Gemini API key

Installation

1. Clone and Setup

git clone <repository-url>
cd bronny
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

2. Install Backend Dependencies

pip install -r requirements.txt

3. Install Frontend Dependencies

cd frontend
npm install
cd ..

4. Configure Environment

# Copy example environment file
cp config/env.example config/.env

# Edit the generated config file
nano config/.env

# Add your API keys:
GEMINI_API_KEY=your_gemini_api_key_here
DATABASE_URL=postgresql+psycopg://user:pass@localhost:5432/bronny

5. Setup Google Services (Optional)

• Go to Google Cloud Console
• Create a project and enable Gmail, Calendar, Drive APIs
• Create OAuth2 credentials
• Download credentials.json to config/google_credentials.json

6. Start the Backend

python api/main.py

7. Start the Frontend (in another terminal)

cd frontend
npm run dev

Visit http://localhost:3000 for the web interface and http://localhost:8000/docs for the API documentation.

📖 Usage Examples

Web Interface

1. Chat with Agents: Select an agent and start a conversation
2. Upload Files: Drag and drop documents for processing and search
3. Manage Memories: View and organize your AI memories
4. Agent Configuration: Create and customize agents
5. Google Integration: Connect and use Google services

API Usage

Chat with Agents

import requests

response = requests.post("http://localhost:8000/api/chat", 
    headers={"Authorization": "Bearer your-token"},
    json={
        "message": "Research the latest developments in AI and create a summary",
        "agent_id": "orchestrator"
    }
)
print(response.json())

Memory Management

# Store a memory
requests.post("http://localhost:8000/api/memory/user123",
    headers={"Authorization": "Bearer your-token"},
    json={
        "content": "User prefers morning meetings",
        "memory_type": "user_preference",
        "importance": 8,
        "tags": ["meetings", "schedule"]
    }
)

# Search memories
response = requests.get("http://localhost:8000/api/memory/user123/search?query=meetings")
memories = response.json()["memories"]

File Processing

# Upload and process a file
with open("document.pdf", "rb") as f:
    response = requests.post("http://localhost:8000/api/files/upload",
        headers={"Authorization": "Bearer your-token"},
        files={"file": f}
    )

# Search through processed files
response = requests.get("http://localhost:8000/api/files/user123/search?query=machine learning")
results = response.json()["results"]

Google Services

# Get Gmail messages
response = requests.get("http://localhost:8000/api/google/gmail?query=is:unread",
    headers={"Authorization": "Bearer your-token"}
)
messages = response.json()["messages"]

# Send an email
requests.post("http://localhost:8000/api/google/gmail/send",
    headers={"Authorization": "Bearer your-token"},
    json={
        "to": "recipient@example.com",
        "subject": "AI-Generated Email",
        "body": "This email was composed with AI assistance!"
    }
)

Workflow Orchestration

# Execute a multi-step workflow
workflow = {
    "steps": [
        {
            "id": "research",
            "agent": "researcher",
            "task": "Research AI trends in 2024",
            "dependencies": []
        },
        {
            "id": "document",
            "agent": "creator",
            "task": "Create a comprehensive report based on the research",
            "dependencies": ["research"]
        }
    ]
}

response = requests.post("http://localhost:8000/api/workflows",
    headers={"Authorization": "Bearer your-token"},
    json=workflow
)
results = response.json()["results"]

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Frontend (React/TypeScript)                 │
│              Tailwind CSS + Vite + WebSocket Client            │
└─────────────────────────────────────────────────────────────────┘
                                │
                                │ HTTP/WebSocket
                                │
┌─────────────────────────────────────────────────────────────────┐
│                    FastAPI Backend                             │
│              Async Python + WebSocket Server                   │
└─────────────────────────────────────────────────────────────────┘
                                │
                                │
┌─────────────────────────────────────────────────────────────────┐
│                    Agno Framework                              │
│                                                                 │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐ │
│  │     Agents      │  │     Memory      │  │     Tools       │ │
│  │   (Specialized) │  │   Management    │  │  Integration    │ │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘ │
│                                                                 │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐ │
│  │  Orchestrator   │  │ File Processing │  │ Google Services │ │
│  │     Agent       │  │   & Embedding   │  │   Integration   │ │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
                                │
                                │
┌─────────────────────────────────────────────────────────────────┐
│                      Data Layer                                │
│                                                                 │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐ │
│  │   PostgreSQL    │  │    ChromaDB     │  │  Google APIs    │ │
│  │   (Memories &   │  │   (Embeddings   │  │  (Gmail, Drive, │ │
│  │   Sessions)     │  │   & Documents)  │  │   Calendar)     │ │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

📚 Documentation

• Architecture Overview: Detailed system design and component architecture
• API Documentation: Complete API reference and examples
• Frontend Documentation: React components and user interface guide
• Agent System: Agent ecosystem and customization guide
• Setup Guide: Detailed installation and configuration
• Deployment Guide: Production deployment instructions

🔧 Configuration

Environment Variables

# Core API Keys
GEMINI_API_KEY=your_gemini_api_key
OPENAI_API_KEY=your_openai_api_key  # Optional

# Database
DATABASE_URL=postgresql+psycopg://user:pass@localhost:5432/bronny

# Google Services
GOOGLE_CREDENTIALS_PATH=./config/google_credentials.json
GOOGLE_CLIENT_ID=your_client_id
GOOGLE_CLIENT_SECRET=your_client_secret

# Application Settings
DEBUG=true
HOST=0.0.0.0
PORT=8000

# Memory Configuration
ENABLE_USER_MEMORIES=true
ENABLE_SESSION_SUMMARIES=true
ENABLE_AGENTIC_MEMORY=true

# File Processing
MAX_FILE_SIZE=52428800  # 50MB
UPLOAD_DIRECTORY=./data/uploads
CHROMA_PERSIST_DIRECTORY=./data/chroma

Agent Customization

from agents.agent_factory import AgentFactory

# Create a custom agent
custom_agent = AgentFactory.create_agent(
    agent_type="research",
    agent_id="my_researcher",
    name="My Custom Researcher",
    description="Specialized research agent for my domain",
    model_id="gemini-2.5-flash",
    prompt_template="Your custom prompt here...",
    tools=[CustomTool()],  # Add custom tools
    enable_memory=True
)

🧪 Testing

# Run backend tests
python -m pytest tests/

# Test specific components
python -m pytest tests/test_agents.py
python -m pytest tests/test_memory.py
python -m pytest tests/test_tools.py

# Run frontend tests
cd frontend
npm test

# Integration tests
python -m pytest tests/test_integration.py

🚢 Deployment

Docker Deployment

# Build and run with Docker Compose
docker-compose up --build

# Or build manually
docker build -t bronny-ai .
docker run -p 8000:8000 bronny-ai

Production Deployment

1. Set up PostgreSQL database
2. Configure environment variables
3. Set up reverse proxy (Nginx/Traefik)
4. Enable HTTPS with SSL certificates
5. Set up monitoring and logging

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Development Setup

# Install development dependencies
pip install -r requirements.txt
cd frontend && npm install

# Set up pre-commit hooks
pre-commit install

# Run code formatting
black .
isort .
cd frontend && npm run lint

📋 Roadmap

Phase 1 (Current) ✅

• Core agent system with Agno integration
• Memory management system
• File processing and embeddings
• Google services integration
• RESTful API with FastAPI
• React frontend with real-time chat

Phase 2 (Next 1-2 months) 🔄

• Enhanced workflow orchestration
• Voice input/output integration
• Advanced analytics dashboard
• Mobile responsive improvements
• Plugin system foundation

Phase 3 (3-6 months) 📋

• Mobile application (React Native)
• Browser extension
• Advanced agent collaboration
• Multi-language support
• Enterprise features

Phase 4 (6+ months) 🚀

• Advanced AI capabilities
• Multi-tenant architecture
• Advanced security features
• Performance optimizations
• Global deployment options

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Agno Framework: For the powerful multi-agent system
• FastAPI: For the excellent web framework
• ChromaDB: For vector database capabilities
• Google APIs: For seamless service integration
• React: For the modern frontend framework

📞 Support

• Documentation: Check the docs/ directory
• Issues: Open an issue on GitHub
• Discussions: Use GitHub Discussions for questions

---

Built with ❤️ using the Agno framework, FastAPI, React, and modern Python/JavaScript technologies.