feat: add UDF deploy script and writer skill for IDMP by huskar-t · Pull Request #11 · taosdata/agent-skills

huskar-t · 2026-06-03T03:27:16Z

Description

feat: add UDF deploy script and writer skill for IDMP

Issue(s)

resolve: https://project.feishu.cn/taosdata_td/job/detail/7006895703

Checklist

Please check the items in the checklist if applicable.

Is the user manual updated?
Are the test cases passed and automated?
Is there no significant decrease in test coverage?

gemini-code-assist

Code Review

This pull request introduces a suite of automation skills and helper scripts for the TDengine IDMP ecosystem, including tools for blog generation, data simulation, asset tree exporting, telemetry, TDGPT model deployment, and UDF compilation and deployment. The review feedback highlights critical security improvements for deploy_udf.py to prevent command and SQL injection vulnerabilities by validating UDF names and safely escaping shell arguments with shlex. Other recommendations include implementing pagination in export_idmp_tree.py to support large asset trees, enhancing REST API error handling, and cleaning up unused imports and regex patterns across several scripts.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-06-03T03:29:51Z

+    func_keyword = "AGGREGATE FUNCTION" if args.type == "aggregate" else "FUNCTION"
+    create_sql = f"CREATE {func_keyword} {udf_name} AS '{target_path}' OUTPUTTYPE {args.output_type}"
+    if args.type == "aggregate" and args.bufsize:
+        create_sql += f" BUFSIZE {args.bufsize}"


Use the safely quoted paths in the remote copy command to prevent command injection.

Suggested change

create_sql += f" BUFSIZE {args.bufsize}"

remote_cmd = f"mkdir -p /var/lib/taos/udf && cp /tmp/{safe_src_filename} {safe_target_path}"

gemini-code-assist · 2026-06-03T03:29:51Z

+        log_error("Unsupported file extension. Only .c, .cpp, and .py are supported.")
+        sys.exit(1)
+
+    udf_name = args.name if args.name else src_name


Validate the udf_name using a regular expression to ensure it only contains alphanumeric characters and underscores. This prevents potential SQL injection in the DROP FUNCTION / CREATE FUNCTION statements and command injection in remote execution paths.

Suggested change

udf_name = args.name if args.name else src_name

udf_name = args.name if args.name else src_name

if not re.match(r'^[a-zA-Z_][a-zA-Z0-9_]*$', udf_name):

log_error(f"Invalid UDF name: '{udf_name}'. Only alphanumeric characters and underscores are allowed, and it must start with a letter or underscore.")

sys.exit(1)

gemini-code-assist · 2026-06-03T03:29:51Z

+
+        if ext in [".c", ".cpp"]:
+            log_info("Compiling UDF on remote server...")
+            compiler = "g++" if ext == ".cpp" else "gcc"


Define safely quoted versions of target_path and src_filename using shlex.quote to prevent command injection vulnerabilities when executing commands on the remote server via SSH.

elif mode == "remote_ssh": if not args.ssh_host: log_error("SSH host must be provided for remote_ssh mode.") sys.exit(1) safe_target_path = shlex.quote(target_path) safe_src_filename = shlex.quote(src_filename)

gemini-code-assist · 2026-06-03T03:29:51Z

+                log_error(f"Remote compilation failed: {e}")
+                sys.exit(1)
+        else:
+            log_info("Deploying Python UDF on remote server...")


Use the safely quoted safe_src_filename for the remote scp destination path to prevent shell metacharacter expansion issues.

Suggested change

log_info("Deploying Python UDF on remote server...")

run_command(["scp"] + port_arg + [args.src, f"{ssh_dest}:/tmp/{safe_src_filename}"])

gemini-code-assist · 2026-06-03T03:29:51Z

+            except Exception as e:
+                log_error(f"Failed to deploy on remote server: {e}")
+                sys.exit(1)
+


Use the safely quoted paths in the remote compilation command to prevent command injection.

Suggested change

remote_cmd = f"mkdir -p /var/lib/taos/udf && {compiler} -shared -fPIC -O2 -o {safe_target_path} /tmp/{safe_src_filename}"

gemini-code-assist · 2026-06-03T03:29:51Z

+ALLOWED_FUN_FUNCS = re.compile(r'\b(?!sin|cos|random)\b[a-zA-Z_][a-zA-Z0-9_]*\s*\(')
+ALLOWED_FUN_PATTERN = re.compile(r'^[\d\s\+\-\*\/\.\(\)xX]*(?:(?:sin|cos|random)\s*\([^)]*\)[\d\s\+\-\*\/\.]*)*$')


Remove the unused regular expression patterns ALLOWED_FUN_FUNCS and ALLOWED_FUN_PATTERN to clean up the codebase.

Suggested change

ALLOWED_FUN_FUNCS = re.compile(r'\b(?!sin|cos|random)\b[a-zA-Z_][a-zA-Z0-9_]*\s*$')

ALLOWED_FUN_PATTERN = re.compile(r'^[\d\s\+\-\*\/\.\($xX]*(?:(?:sin|cos|random)\s*$[^)]*$[\d\s\+\-\*\/\.]*)*$')

# Unused patterns removed

gemini-code-assist · 2026-06-03T03:29:51Z

+import sys
+import os
+import argparse
+import re


Remove the unused import re to keep the code clean.

Suggested change

import re

# import re removed as it is unused

gemini-code-assist · 2026-06-03T03:29:51Z

+#!/usr/bin/env python3
+import json
+import requests
+import re


Remove the unused import re to keep the code clean.

Suggested change

import re

# import re removed as it is unused

gemini-code-assist · 2026-06-03T03:29:51Z

@@ -0,0 +1,254 @@
+#!/usr/bin/env python3
+import json


Remove the unused import json to keep the code clean.

Suggested change

import json

# import json removed as it is unused

gemini-code-assist · 2026-06-03T03:29:51Z

@@ -0,0 +1,126 @@
+import os
+import numpy as np
+import pandas as pd


Remove the unused import pandas in the template to keep it clean and lightweight.

Suggested change

import pandas as pd

# import pandas removed as it is unused in this template

Copilot

Pull request overview

This PR introduces a set of new “skills” under skills/ to support IDMP demo orchestration and TDengine UDF authoring/deployment, and adds a shared telemetry sender to standardize skill-invocation reporting.

Changes:

Added udf-writer (documentation + templates) and udf-deploy (deployment guide + deploy_udf.py automation script).
Added multiple IDMP workflow skills (easy-use orchestrator, sample data generator, structure exporter, blog generator) plus supporting scripts/templates.
Introduced skill-telemetry (Python + PowerShell) as a centralized telemetry sender.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 19 comments.

Show a summary per file

File	Description
skills/udf-writer/SKILL.md	New skill spec for UDF authoring; includes embedded telemetry block.
skills/udf-writer/references/udf_scalar_template.md	Added scalar UDF C template/reference.
skills/udf-writer/references/udf_aggregate_template.md	Added aggregate UDF (UDAF) C template/reference.
skills/udf-writer/README.md	New README for udf-writer (noted version alignment issue).
skills/udf-deploy/SKILL.md	New skill spec for UDF compilation/deployment; includes embedded telemetry + troubleshooting notes.
skills/udf-deploy/scripts/deploy_udf.py	New Python script to compile/deploy/register UDFs (local/docker/ssh).
skills/tdgpt-model-writer/SKILL.md	New skill spec for TDgpt model/service code generation; includes embedded telemetry.
skills/tdgpt-model-writer/assets/pytorch_lstm_template.py	Added PyTorch LSTM service template (noted safety/edge-case issues).
skills/tdgpt-model-writer/assets/data_align_template.py	Added pandas-based data alignment template.
skills/tdgpt-model-deploy/SKILL.md	New skill spec for TDgpt model deployment; includes embedded telemetry.
skills/skill-telemetry/SKILL.md	New centralized telemetry skill definition.
skills/skill-telemetry/scripts/telemetry.py	Telemetry sender implementation (Python).
skills/skill-telemetry/scripts/telemetry.ps1	Telemetry sender implementation (PowerShell).
skills/skill-telemetry/README.md	Telemetry sender documentation.
skills/idmp-structure-exporter/SKILL.md	New skill spec for exporting IDMP asset tree structure.
skills/idmp-structure-exporter/scripts/export_idmp_tree.py	Export script for IDMP asset tree (noted pagination correctness issue).
skills/idmp-structure-exporter/README.md	Structure-exporter README.
skills/idmp-sample-data-generator/templates/uomclass_template.json	Added UOM class request-body template.
skills/idmp-sample-data-generator/templates/uom_template.json	Added UOM request-body template (endpoint text alignment issue).
skills/idmp-sample-data-generator/templates/idmp_sample_data_v1.json	Added full sample-data JSON template/spec.
skills/idmp-sample-data-generator/SKILL.md	New skill spec for sample data generation/loading (endpoint text alignment issue).
skills/idmp-sample-data-generator/scripts/validate_sample_data.py	Validator script for sample_data.json (noted dead-code constants).
skills/idmp-sample-data-generator/scripts/upload_sample.py	Upload/load/cleanup automation for IDMP samples.
skills/idmp-sample-data-generator/scripts/uom_check.py	UOM existence check + auto-fix/auto-create helper.
skills/idmp-sample-data-generator/scripts/duplicate_check.py	Detect and auto-fix naming conflicts in IDMP.
skills/idmp-sample-data-generator/README.md	Sample-data-generator README.
skills/idmp-easyuse/SKILL.md	New top-level orchestrator skill spec (noted `state.json` key inconsistency).
skills/idmp-easyuse/scripts/prepare_project_dir.py	Script to create project directory structure.
skills/idmp-easyuse/scripts/get_login_token.py	Script to obtain IDMP login token.
skills/idmp-easyuse/references/research_specification.md	Added research document specification.
skills/idmp-easyuse/references/idmp_panel_types.md	Added panel type reference doc.
skills/idmp-easyuse/references/idmp_mcp_config.md	Added MCP server configuration reference.
skills/idmp-easyuse/references/idmp_analysis_abilities.md	Added analysis/trigger capability reference.
skills/idmp-easyuse/README.md	EasyUse README.
skills/idmp-blog-generator/SKILL.md	New blog generator skill spec.
skills/idmp-blog-generator/references/编写原则.md	Detailed blog writing guidelines.
skills/idmp-blog-generator/README.md	Blog-generator README.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    src_filename = os.path.basename(args.src)
+    src_name, ext = os.path.splitext(src_filename)
+    ext = ext.lower()
+
+    if ext not in [".c", ".cpp", ".py"]:
+        log_error("Unsupported file extension. Only .c, .cpp, and .py are supported.")
+        sys.exit(1)
+
+    udf_name = args.name if args.name else src_name
+


+def execute_sql_cli(container_name, user, password, sql):
+    """Execute SQL using taos CLI inside Docker container."""
+    cmd = ["docker", "exec", "-i", container_name, "taos", "-u", user, "-p", password, "-s", sql]
+    stdout, stderr = run_command(cmd)
+    if "Query OK" in stdout or "affected" in stdout or not stderr:
+        return {"code": 0, "desc": stdout}
+    else:
+        raise RuntimeError(f"taos CLI execution failed: {stderr or stdout}")


+## Telemetry (MUST)
+
+> **隐私声明**：以下统计功能仅用于 Skill 调用量分析和质量改进，不收集任何用户输入内容、
+> 文件内容或敏感信息。所有数据仅在公司内部网络传输和存储。
+
+### Agent Execution Rule
+
+**当此 Skill 被激活时，Agent MUST 在开始执行 Skill 逻辑之前，先运行以下 telemetry 命令。**
+这是强制要求，不可跳过。失败不阻塞后续流程，但必须尝试执行。


+| `agent` | Agent caller or process name | 调用方 Agent 标识或进程名，如：copilot,claude,codex,warp,gemini 等 |
+| `os` | `uname -s` / 硬编码 | 操作系统：linux, darwin, windows |
+| `distro` | `/etc/os-release` / `sw_vers` / `Win32_OperatingSystem` | 发行版，如 Ubuntu 24.04, macOS 15.3 |
+| `local_ip` | `hostname -I` | Agent 所在机器 the 本地 IP |


+- Author: limingjun
+- Team: 售前
+- Version: 0.1.0


+1. **创建缺失单位**：
+   针对每一行“待处理”条目，调用 API：
+   - **URL**: `POST /api/v1/uomclasses/{uomClass id}/uom`
+   - **Body**: 参考 `templates/uom_template.json` 构造和说明，编写 uom 的 json 文件。
+   - **数据来源**：从脚本输出表格中提取 `uomClass id`、`uom` 和 `基础单位id`。


+
+### 3. 返回值填充与内存管理规范
+
+- **痛点**：若在 UDF 循环中直接对输出指针进行操作，可能会导致 TDengine UDF 守护进程（`udfd`）无法记录写入的行数，抛出类似 `scala result num of rows 0 not equal to input rows 1` 的报错。


+        if self.input_is_empty() or len(self.list) < self.lookback_window:
+            raise ValueError(f"Input sequence length ({len(self.list)}) is less than lookback window ({self.lookback_window})")
+


+        # 构建输出返回格式
+        # 预测值的数量取决于用户参数 self.rows
+        # 本模板演示单步预测后自回归扩展 self.rows 个预测点，或直接重复映射
+        pred_list = [pred_val_real] * self.rows
+
+        # 生成对应的预测结果时间轴（使用基类的 start_ts 与 time_step）
+        ts_list = [self.start_ts + i * self.time_step for i in range(self.rows)]


+---
+name: skill-telemetry
+description: "Client-side telemetry reporter for Skill invocations. Sends usage events to telemetry-server when any Skill is activated. Keywords: telemetry, usage report, skill tracking"
+metadata:
+  author: Linhe Huo
+  version: 0.2.0
+  owner_team: ai
+---


zitsen

✅ Approved — all new skill files, clean additions, no existing code modified.

feat: add UDF deploy script and writer skill for IDMP

5b80912

Copilot AI review requested due to automatic review settings June 3, 2026 03:27

Copilot started reviewing on behalf of huskar-t June 3, 2026 03:27 View session

gemini-code-assist Bot reviewed Jun 3, 2026

View reviewed changes

Copilot AI reviewed Jun 3, 2026

View reviewed changes

zitsen approved these changes Jun 3, 2026

View reviewed changes

zitsen merged commit 0b67242 into main Jun 3, 2026
1 check passed

zitsen deleted the chore/xftan/add-skills branch June 3, 2026 03:48

	create_sql += f" BUFSIZE {args.bufsize}"
	remote_cmd = f"mkdir -p /var/lib/taos/udf && cp /tmp/{safe_src_filename} {safe_target_path}"

	log_info("Deploying Python UDF on remote server...")
	run_command(["scp"] + port_arg + [args.src, f"{ssh_dest}:/tmp/{safe_src_filename}"])

		ALLOWED_FUN_FUNCS = re.compile(r'\b(?!sin\|cos\|random)\b[a-zA-Z_][a-zA-Z0-9_]\s\(')
		ALLOWED_FUN_PATTERN = re.compile(r'^[\d\s\+\-\\/\.\(\)xX](?:(?:sin\|cos\|random)\s\([^)]\)[\d\s\+\-\\/\.])*$')

	ALLOWED_FUN_FUNCS = re.compile(r'\b(?!sin\|cos\|random)\b[a-zA-Z_][a-zA-Z0-9_]\s\(')
	ALLOWED_FUN_PATTERN = re.compile(r'^[\d\s\+\-\\/\.\(\)xX](?:(?:sin\|cos\|random)\s\([^)]\)[\d\s\+\-\\/\.])*$')
	# Unused patterns removed

	import pandas as pd
	# import pandas removed as it is unused in this template


		### 3. 返回值填充与内存管理规范

		- 痛点：若在 UDF 循环中直接对输出指针进行操作，可能会导致 TDengine UDF 守护进程（`udfd`）无法记录写入的行数，抛出类似 `scala result num of rows 0 not equal to input rows 1` 的报错。

		if self.input_is_empty() or len(self.list) < self.lookback_window:
		raise ValueError(f"Input sequence length ({len(self.list)}) is less than lookback window ({self.lookback_window})")

Uh oh!

Conversation

huskar-t commented Jun 3, 2026

Description

Issue(s)

Checklist

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

zitsen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants