Add TypeScript HTML content processor with sanitization and metadata extraction#1
Conversation
Agent-Logs-Url: https://github.com/ocilab25/ts-copilot-integration-lab/sessions/3bba50ef-8610-4f8c-a780-79f19634518e Co-authored-by: ocilab25 <207450545+ocilab25@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a new standalone html-processor/ TypeScript package that sanitizes HTML via sanitize-html and extracts basic metadata (title/meta/link count) via pluggable strategies (regex or DOM), along with Jest tests and CI wiring to build/type-check/test the module.
Changes:
- Introduces
ContentProcessorplus regex/DOM metadata extractor strategies and shared public types. - Adds a Jest test suite covering sanitization, metadata extraction, and end-to-end processing.
- Adds Node.js CI workflow and TypeScript/Jest project configuration for the new module.
Reviewed changes
Copilot reviewed 9 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| html-processor/src/ContentProcessor.ts | Core API for sanitize/process/extractMetadata with default allow-lists. |
| html-processor/src/RegexMetadataExtractor.ts | Regex-based metadata extraction implementation. |
| html-processor/src/DomMetadataExtractor.ts | Cheerio-based metadata extraction implementation. |
| html-processor/src/types.ts | Public interfaces for metadata and processor options/results. |
| html-processor/src/index.ts | Module entrypoint re-exporting public API. |
| html-processor/tests/ContentProcessor.test.ts | Jest tests for sanitization, extraction strategies, and process(). |
| html-processor/tsconfig.json | TypeScript compiler configuration for the module build. |
| html-processor/package.json | Declares dependencies, scripts, and Jest configuration. |
| html-processor/package-lock.json | Locks dependency graph (notably cheerio/undici Node engine requirements). |
| .github/workflows/ci.yml | CI job to install, type-check, and test html-processor. |
| .gitignore | Adds Node/TS ignores and some local-tooling ignores. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - name: Use Node.js 20.x | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20.x |
There was a problem hiding this comment.
The workflow sets up node-version: 20.x, but cheerio@1.2.0 (via the lockfile) requires Node >=20.18.1. Pin the workflow to at least 20.18.1 (or the exact version you declare in package.json#engines) to prevent CI failures if 20.x ever resolves to an older patch in another environment or cache.
| - name: Use Node.js 20.x | |
| uses: actions/setup-node@v4 | |
| with: | |
| node-version: 20.x | |
| - name: Use Node.js 20.18.1 | |
| uses: actions/setup-node@v4 | |
| with: | |
| node-version: 20.18.1 |
| } | ||
|
|
||
| private countLinks(html: string): number { | ||
| const matches = html.match(/<a\s/gi); |
There was a problem hiding this comment.
countLinks() only matches anchors that have whitespace after <a (e.g. <a href=...>). It will undercount links for valid markup like <a>Text</a> or <a/>. Consider switching to a boundary-based pattern (e.g. <a\b) so attribute-less anchors are counted too (and keep it case-insensitive).
| const matches = html.match(/<a\s/gi); | |
| const matches = html.match(/<a\b/gi); |
|
|
||
| /** Default set of attributes permitted on allowed tags. */ | ||
| const DEFAULT_ALLOWED_ATTRIBUTES: Record<string, string[]> = { | ||
| a: ["href", "title", "target", "rel"], |
There was a problem hiding this comment.
Allowing target on <a> without enforcing a safe rel value can enable reverse-tabnabbing when consumers render the sanitized output. Either disallow target by default, or add a sanitize-html transform that ensures rel includes noopener noreferrer whenever target="_blank" is present.
| a: ["href", "title", "target", "rel"], | |
| a: ["href", "title", "rel"], |
| "dependencies": { | ||
| "cheerio": "^1.2.0", | ||
| "sanitize-html": "^2.13.0" | ||
| }, |
There was a problem hiding this comment.
This package depends on cheerio@1.2.0 (and its undici dependency), which declares engines.node >=20.18.1 in the lockfile. To avoid consumers installing with an unsupported Node version (and hitting runtime/install issues), add an explicit engines field (and optionally engineStrict) reflecting the minimum Node version you support.
Introduces a standalone
html-processorTypeScript module providing safe HTML sanitization and metadata extraction, with a full Jest test suite.Module structure
src/ContentProcessor.ts— Core class delegating sanitization tosanitize-htmland metadata extraction via ReDoS-safe regexes. Exports typed interfaces (HtmlMetadata,ProcessingResult,SanitizationOptions).src/index.ts— Public entry point.tests/ContentProcessor.test.ts— 23 Jest tests across three method groups.Key design decisions
sanitize()strips<script>,<iframe>,<object>,on*attributes, andjavascript:URLs viasanitize-html; allow-list is configurable via constructor.extractMetadata()uses[^<]*/[^>]*character classes instead of lazy[\s\S]*?/\s+[^>]+to avoid polynomial ReDoS on unterminated tags.process()composes both for convenience.Quick example
Test coverage
<script>,<iframe>,<object>,onclick,javascript:href, complex payloads<meta name>, OG<meta property>, link countOriginal prompt