Merge pull request #47 from modelscope/feat/cli

[Docs] Update readme

Author: wangxingjun778

README.md

@@ -13,6 +13,7 @@
 [![ripgrep-all](https://img.shields.io/badge/ripgrep--all-Search-E67E22?style=flat-square&logo=rust&logoColor=white)](https://github.com/phiresky/ripgrep-all)
 [![OpenAI](https://img.shields.io/badge/OpenAI-API-412991?style=flat-square&logo=openai&logoColor=white)](https://github.com/openai/openai-python)
 [![Kreuzberg](https://img.shields.io/badge/Kreuzberg-Text_Extraction-4CAF50?style=flat-square)](https://github.com/kreuzberg-dev/kreuzberg)
+[![MCP](https://img.shields.io/badge/MCP-Python_SDK-8B5CF6?style=flat-square&logo=python&logoColor=white)](https://github.com/modelcontextprotocol/python-sdk)
 
 
 [**Quick Start**](#-quick-start) · [**Key Features**](#-key-features) · [**Web UI**](#-web-ui) · [**How it Works**](#-how-it-works) · [**FAQ**](#-faq)
@@ -138,6 +139,12 @@ It serves as a unified intelligent hub for AI agents, delivering deep insights a
 
 ## 🎉 News
 
+* 🚀 **Feb 5, 2026**: Release **v0.0.2** — MCP Support, CLI Commands & Knowledge Persistence!
+  - **MCP Integration**: Full [Model Context Protocol](https://modelcontextprotocol.io) support, works seamlessly with Claude Desktop and Cursor IDE.
+  - **CLI Commands**: New `sirchmunk` CLI with `init`, `config`, `serve`, and `search` commands.
+  - **KnowledgeCluster Persistence**: DuckDB-powered storage with Parquet export for efficient knowledge management.
+  - **Knowledge Reuse**: Semantic similarity-based cluster retrieval for faster searches via embedding vectors.
+
 * 🎉🎉 Jan 22, 2026: Introducing **Sirchmunk**: Initial Release v0.0.1 Now Available!
 
 
@@ -183,9 +190,9 @@ llm = OpenAIChat(
 
 async def main():
 
-    agent_search = AgenticSearch(llm=llm)
+    searcher = AgenticSearch(llm=llm)
 
-    result: str = await agent_search.search(
+    result: str = await searcher.search(
         query="How does transformer attention work?",
         search_paths=["/path/to/documents"],
     )
@@ -205,24 +212,34 @@ asyncio.run(main())
 
 Sirchmunk provides a powerful CLI for server management and search operations.
 
+#### Installation
+
+```bash
+pip install "sirchmunk[web]"
+
+# or install via UV
+uv pip install "sirchmunk[web]"
+```
+
+
 #### Initialize
 
 ```bash
-# Initialize Sirchmunk with default settings
+# Initialize Sirchmunk with default settings (Default work path: `~/.sirchmunk/`)
 sirchmunk init
 
-# Initialize with custom work path
+# Alternatively, initialize with custom work path
 sirchmunk init --work-path /path/to/workspace
 ```
 
 #### Configure
 
 ```bash
-# Generate configuration file (~/.sirchmunk/.env)
-sirchmunk config --generate
-
 # Show current configuration
 sirchmunk config
+
+# Regenerate configuration file if needed (Default config file: ~/.sirchmunk/.env)
+sirchmunk config --generate
 ```
 
 #### Start API Server
@@ -269,6 +286,36 @@ sirchmunk search "query" --api --api-url http://localhost:8584
 
 ---
 
+## 🔌 MCP Server
+
+Sirchmunk provides a [Model Context Protocol (MCP)](https://modelcontextprotocol.io) server that exposes its intelligent search capabilities as MCP tools. This enables seamless integration with AI assistants like **Claude Desktop** and **Cursor IDE**.
+
+### Quick Start
+
+```bash
+# Install MCP package
+pip install sirchmunk-mcp
+
+# Initialize and configure
+sirchmunk-mcp init
+sirchmunk-mcp config --generate
+
+# Edit ~/.sirchmunk/.mcp_env with your LLM API key
+
+# Test with MCP Inspector
+npx @modelcontextprotocol/inspector sirchmunk-mcp serve
+```
+
+### Features
+
+- **Multi-Mode Search**: DEEP mode for comprehensive analysis, FILENAME_ONLY for fast file discovery
+- **Knowledge Cluster Management**: Automatic extraction, storage, and reuse of knowledge
+- **Standard MCP Protocol**: Works with stdio and Streamable HTTP transports
+
+📖 **For detailed documentation, see [Sirchmunk MCP README](src/sirchmunk_mcp/README.md)**.
+
+---
+
 ## 🖥️ Web UI
 
 The web UI is built for fast, transparent workflows: chat, knowledge analytics, and system monitoring in one place.
@@ -386,7 +433,7 @@ Any OpenAI-compatible API endpoint, including (but not limited too):
 Simply specify the path in your search query:
 
 ```python
-result = await search.search(
+result = await searcher.search(
     query="Your question",
     search_paths=["/path/to/folder", "/path/to/file.pdf"]
 )
@@ -413,7 +460,7 @@ You can query them using DuckDB or the `KnowledgeManager` API.
 
 1. **Web Dashboard**: Visit the Monitor page for real-time statistics
 2. **API**: `GET /api/v1/monitor/llm` returns usage metrics
-3. **Code**: Access `search.llm_usages` after search completion
+3. **Code**: Access `searcher.llm_usages` after search completion
 
 </details>
 

README_zh.md

@@ -13,6 +13,7 @@
 [![ripgrep-all](https://img.shields.io/badge/ripgrep--all-Search-E67E22?style=flat-square&logo=rust&logoColor=white)](https://github.com/phiresky/ripgrep-all)
 [![OpenAI](https://img.shields.io/badge/OpenAI-API-412991?style=flat-square&logo=openai&logoColor=white)](https://github.com/openai/openai-python)
 [![Kreuzberg](https://img.shields.io/badge/Kreuzberg-Text_Extraction-4CAF50?style=flat-square)](https://github.com/kreuzberg-dev/kreuzberg)
+[![MCP](https://img.shields.io/badge/MCP-Python_SDK-8B5CF6?style=flat-square&logo=python&logoColor=white)](https://github.com/modelcontextprotocol/python-sdk)
 
 
 [**快速开始**](#-快速开始) · [**核心特性**](#-核心特性) · [**Web UI**](#-web-ui) · [**工作原理**](#-工作原理) · [**FAQ**](#-faq)
@@ -135,6 +136,12 @@
 
 ## 🎉 News
 
+* 🚀 **2026.2.5**: 发布 **v0.0.2** — MCP 支持、CLI 命令行 & 知识持久化！
+  - **MCP 集成**：完整支持 [Model Context Protocol](https://modelcontextprotocol.io)，与 Claude Desktop 和 Cursor IDE 无缝协作。
+  - **CLI 命令行**：全新 `sirchmunk` 命令行工具，支持 `init`、`config`、`serve` 和 `search` 命令。
+  - **KnowledgeCluster 持久化**：基于 DuckDB 存储，支持 Parquet 导出，高效管理知识聚类。
+  - **知识复用**：基于语义相似度的知识聚类检索，通过 embedding 向量加速搜索。
+
 * 🎉🎉 2026.1.22: **Sirchmunk** 初始版本 v0.0.1 现已发布！
 
 ---
@@ -202,24 +209,35 @@ asyncio.run(main())
 
 Sirchmunk 提供强大的 CLI，用于服务器管理和搜索操作。
 
+
+#### 安装
+
+```bash
+pip install "sirchmunk[web]"
+
+# 或使用UV安装
+uv pip install "sirchmunk[web]"
+```
+
+
 #### 初始化
 
 ```bash
-# 使用默认设置初始化 Sirchmunk
+# 使用默认设置初始化 Sirchmunk，默认工作路径为 `~/.sirchmunk/`
 sirchmunk init
 
-# 使用自定义工作路径初始化
+# 或者，也可以使用自定义工作路径初始化
 sirchmunk init --work-path /path/to/workspace
 ```
 
 #### 配置
 
 ```bash
-# 生成配置文件 (~/.sirchmunk/.env)
-sirchmunk config --generate
-
 # 显示当前配置
 sirchmunk config
+
+# 如需要，可重新生成配置文件 (默认配置文件路径，~/.sirchmunk/.env)
+sirchmunk config --generate
 ```
 
 #### 启动 API 服务器
@@ -266,6 +284,36 @@ sirchmunk search "查询" --api --api-url http://localhost:8584
 
 ---
 
+## 🔌 MCP 服务器
+
+Sirchmunk 提供 [Model Context Protocol (MCP)](https://modelcontextprotocol.io) 服务器，将其智能搜索能力作为 MCP 工具暴露。可与 **Claude Desktop** 和 **Cursor IDE** 等 AI 助手无缝集成。
+
+### 快速开始
+
+```bash
+# 安装 MCP 包
+pip install sirchmunk-mcp
+
+# 初始化和配置
+sirchmunk-mcp init
+sirchmunk-mcp config --generate
+
+# 编辑 ~/.sirchmunk/.mcp_env 配置 LLM API Key
+
+# 使用 MCP Inspector 测试
+npx @modelcontextprotocol/inspector sirchmunk-mcp serve
+```
+
+### 特性
+
+- **多模式搜索**：DEEP 模式进行全面分析，FILENAME_ONLY 模式快速发现文件
+- **知识聚类管理**：自动提取、存储和复用知识
+- **标准 MCP 协议**：支持 stdio 和 Streamable HTTP 传输
+
+📖 **详细文档请参阅 [Sirchmunk MCP README](src/sirchmunk_mcp/README.md)**。
+
+---
+
 ## 🖥️ Web UI
 
 Web UI 专为快速、透明的工作流设计：对话、知识分析、系统监控一体化。

requirements/core.txt

@@ -14,3 +14,4 @@ rapidfuzz
 duckdb
 kreuzberg>=4.0.0rc1
 sentence-transformers
+modelscope

src/sirchmunk/api/components/history_storage.py

@@ -33,12 +33,12 @@ def __init__(self, work_path: Optional[str] = None):
         Args:
             work_path: Base work path. If None, uses SIRCHMUNK_WORK_PATH env variable
         """
-        # Get work path from env if not provided
+        # Get work path from env if not provided, and expand ~ in path
         if work_path is None:
             work_path = os.getenv("SIRCHMUNK_WORK_PATH", DEFAULT_SIRCHMUNK_WORK_PATH)
 
-        # Create history storage path
-        self.history_path = Path(work_path) / ".cache" / "history"
+        # Create history storage path (expand ~ and resolve to absolute path)
+        self.history_path = Path(work_path).expanduser().resolve() / ".cache" / "history"
         self.history_path.mkdir(parents=True, exist_ok=True)
 
         # Initialize DuckDB

src/sirchmunk/api/components/monitor_tracker.py

@@ -353,7 +353,7 @@ def get_storage_info(self) -> Dict[str, Any]:
             Storage information
         """
         try:
-            work_path = Path(os.getenv("SIRCHMUNK_WORK_PATH", DEFAULT_SIRCHMUNK_WORK_PATH))
+            work_path = Path(os.getenv("SIRCHMUNK_WORK_PATH", DEFAULT_SIRCHMUNK_WORK_PATH)).expanduser().resolve()
             cache_path = work_path / ".cache"
 
             storage_info = {

src/sirchmunk/api/components/settings_storage.py

@@ -32,12 +32,12 @@ def __init__(self, work_path: Optional[str] = None):
         Args:
             work_path: Base work path. If None, uses SIRCHMUNK_WORK_PATH env variable
         """
-        # Get work path from env if not provided
+        # Get work path from env if not provided, and expand ~ in path
         if work_path is None:
             work_path = os.getenv("SIRCHMUNK_WORK_PATH", DEFAULT_SIRCHMUNK_WORK_PATH)
 
-        # Create settings storage path
-        self.settings_path = Path(work_path) / ".cache" / "settings"
+        # Create settings storage path (expand ~ and resolve to absolute path)
+        self.settings_path = Path(work_path).expanduser().resolve() / ".cache" / "settings"
         self.settings_path.mkdir(parents=True, exist_ok=True)
 
         # Initialize DuckDB

src/sirchmunk/cli/cli.py

@@ -184,6 +184,22 @@ def cmd_init(args: argparse.Namespace) -> int:
         llm_base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")
         print(f"  • LLM_BASE_URL: {llm_base_url}")
 
+        # Pre-download embedding model
+        print()
+        print("Downloading embedding model...")
+        print("  (This may take a few minutes on first run)")
+        try:
+            from sirchmunk.utils.embedding_util import EmbeddingUtil
+            
+            model_cache_dir = str(work_path / ".cache" / "models")
+            model_dir = EmbeddingUtil.preload_model(
+                cache_dir=model_cache_dir,
+            )
+            print(f"  ✓ Embedding model downloaded: {model_dir}")
+        except Exception as e:
+            print(f"  ✗ Failed to download embedding model: {e}")
+            print("    Model will be downloaded on first search.")
+        
         print()
         print("=" * 60)
         print("✅ Initialization complete!")
@@ -393,19 +409,24 @@ async def _search_local(
     """
     from sirchmunk.search import AgenticSearch
     from sirchmunk.llm.openai_chat import OpenAIChat
-    from sirchmunk.utils.constants import LLM_BASE_URL, LLM_API_KEY, LLM_MODEL_NAME
+    
+    # Read LLM config from environment at runtime (after .env is loaded)
+    # Don't use constants module values as they are loaded at import time
+    llm_base_url = os.getenv("LLM_BASE_URL", "https://api.openai.com/v1")
+    llm_api_key = os.getenv("LLM_API_KEY", "")
+    llm_model_name = os.getenv("LLM_MODEL_NAME", "gpt-5.2")
 
     # Validate API key
-    if not LLM_API_KEY:
+    if not llm_api_key:
         print("❌ LLM_API_KEY is not set.")
         print("   Configure it in ~/.sirchmunk/.env or set the environment variable.")
         return 1
 
     # Create LLM client
     llm = OpenAIChat(
-        base_url=LLM_BASE_URL,
-        api_key=LLM_API_KEY,
-        model=LLM_MODEL_NAME,
+        base_url=llm_base_url,
+        api_key=llm_api_key,
+        model=llm_model_name,
     )
 
     # Create search instance

src/sirchmunk/learnings/knowledge_base.py

@@ -51,7 +51,9 @@ def __init__(
         self.llm = llm
         self.metadata_map = metadata_map
         self.work_path: Path = (
-            DEFAULT_SIRCHMUNK_WORK_PATH if work_path is None else Path(work_path).resolve()
+            Path(DEFAULT_SIRCHMUNK_WORK_PATH).expanduser().resolve() 
+            if work_path is None 
+            else Path(work_path).expanduser().resolve()
         )
         self.metadata_path: Path = (
             self.work_path / StorageStructure.CACHE_DIR / StorageStructure.METADATA_DIR

src/sirchmunk/retrieve/text_retriever.py

@@ -29,7 +29,7 @@ class GrepRetriever(BaseRetriever):
     def __init__(self, work_path: Union[str, Path] = None, **kwargs):
         super().__init__()
 
-        self.work_path: Path = Path(work_path or DEFAULT_SIRCHMUNK_WORK_PATH)
+        self.work_path: Path = Path(work_path or DEFAULT_SIRCHMUNK_WORK_PATH).expanduser().resolve()
         self.rga_cache: Path = (
             self.work_path / StorageStructure.CACHE_DIR / StorageStructure.GREP_DIR
         )

src/sirchmunk/scan/file_scanner.py

@@ -58,9 +58,9 @@ def __init__(
             corpus_path = [corpus_path]
         self.corpus_paths: List[Path] = [Path(p).resolve() for p in corpus_path]
 
-        # Set work and metadata paths
+        # Set work and metadata paths (expand ~ and resolve to absolute path)
         self.work_path: Path = (
-            Path.cwd() if work_path is None else Path(work_path).resolve()
+            Path.cwd() if work_path is None else Path(work_path).expanduser().resolve()
         )
         self.metadata_path: Path = (
             self.work_path / StorageStructure.CACHE_DIR / StorageStructure.METADATA_DIR