Skip to content

Commit e3ba07a

Browse files
authored
feat(parsers): add Dart public API parser via package:analyzer (alternative to #35) (#41)
* feat(parsers): add Dart public API parser via package:analyzer Alternative to #35: extract the Dart/Flutter public API surface with a small package:analyzer tool instead of the dartdoc_json CLI plus a Node normalizer. The tool parses lib/**.dart syntactically (no pub get, no network), emits the existing ParseResult shape, and is invoked via a parse-dart npm script so the reusable workflow's generic parse step is unchanged. It handles extension types and enhanced enums by construction, and honors .sdk-parse-ignore for parity with the TypeScript and Swift parsers. Removes the need for dartdoc_json activation, per-package flutter pub get, the jq merge, and the Node normalizer. * ci: run dart_symbol_extractor format, analyze and test in capability CI * ci: install Dart SDK via direct download instead of marketplace action The supabase/sdk Actions policy only permits GitHub-owned and verified-creator actions, so dart-lang/setup-dart caused a workflow startup failure. Install the SDK with a plain download instead, which also keeps the reusable workflow independent of each calling repo's allowed-actions policy. * fix(dart-parser): correct class-type-alias kind and directory ignores Self-review found three issues: - A class-type alias (class C = A with B;) was emitted with kind variable instead of class, because ClassTypeAlias is a subtype of TypeAlias. - Directory ignore patterns (build/, examples/) excluded only the directory entry itself, not the files nested beneath it, so contents leaked into the surface. - A single unreadable or unparseable lib file aborted the whole extraction; it is now skipped with a stderr warning. Adds regression tests for each. * refactor: use dart-lang/setup-dart action and drop SymbolKind extension Install the Dart SDK with the dart-lang/setup-dart action instead of a manual download. Replace the SymbolKindJson extension with SymbolKind.name, special casing only classKind since 'class' is a reserved word. * Fix alignment * docs: remove design spec doc * refactor(dart-parser): collapse container dispatch with pattern matching Replace the if-else type ladder in _visitTopLevel with a switch on Dart 3 patterns: the five class-like containers share one case, and a small _emit helper centralizes the private-name guard for simple symbols. Output is unchanged. * refactor: move dart_symbol_extractor up to scripts/ Relocate the package from scripts/capability-matrix/dart_symbol_extractor to scripts/dart_symbol_extractor as a sibling of the Node validator. Update the parse-dart npm script path, both workflows' working directories, the validate-capabilities paths filter, and the CLAUDE.md reference. * ci: invoke dart extractor directly, drop parse-dart npm script The reusable workflow now runs 'dart run bin/extract.dart' from the extractor package for the dart language, instead of dispatching through an npm script. Removes the parse-dart entry from package.json.
1 parent 8c50361 commit e3ba07a

20 files changed

Lines changed: 1011 additions & 8 deletions

.github/workflows/validate-capabilities.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ on:
88
- "capabilities/**"
99
- "schema/**"
1010
- "scripts/capability-matrix/**"
11+
- "scripts/dart_symbol_extractor/**"
1112
- ".github/workflows/validate-capabilities.yml"
1213
schedule:
1314
- cron: "0 6 * * *" # nightly drift sweep
@@ -30,6 +31,21 @@ jobs:
3031
- run: npm run typecheck
3132
- run: npm run validate
3233

34+
dart-parser:
35+
name: Dart parser (format, analyze, test)
36+
runs-on: ubuntu-latest
37+
steps:
38+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
39+
- uses: dart-lang/setup-dart@65eb853c7ba17dde3be364c3d2858773e7144260 # v1.7.2
40+
- working-directory: scripts/dart_symbol_extractor
41+
run: dart pub get
42+
- working-directory: scripts/dart_symbol_extractor
43+
run: dart format --output=none --set-exit-if-changed .
44+
- working-directory: scripts/dart_symbol_extractor
45+
run: dart analyze
46+
- working-directory: scripts/dart_symbol_extractor
47+
run: dart test
48+
3349
references:
3450
name: Tier 2 (references)
3551
runs-on: ubuntu-latest

.github/workflows/validate-sdk-compliance.yml

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ on:
88
type: string
99
default: sdk-compliance.yaml
1010
language:
11-
description: SDK language for public API check — must match a parse-<language> script (e.g. swift, javascript)
11+
description: SDK language for public API check — must match a parse-<language> script (e.g. swift, javascript, dart)
1212
type: string
1313
required: true
1414
sdk-ref:
@@ -73,26 +73,45 @@ jobs:
7373
run: npm ci
7474
working-directory: _sdk-spec/scripts/capability-matrix
7575

76+
# The Dart parser is a small package:analyzer tool run directly with
77+
# `dart run`. It parses syntactically, so the target packages need no
78+
# `pub get` — only the Dart SDK and the tool's own dependencies.
79+
- name: Setup Dart SDK
80+
if: inputs.language == 'dart'
81+
uses: dart-lang/setup-dart@65eb853c7ba17dde3be364c3d2858773e7144260 # v1.7.2
82+
83+
- name: Get Dart extractor dependencies
84+
if: inputs.language == 'dart'
85+
run: dart pub get
86+
working-directory: _sdk-spec/scripts/dart_symbol_extractor
87+
7688
- name: Resolve parse command
89+
if: inputs.language != 'dart'
7790
id: resolve
7891
run: |
7992
case "${{ inputs.language }}" in
8093
swift) echo "cmd=parse-swift" >> "$GITHUB_OUTPUT" ;;
8194
javascript) echo "cmd=parse-ts" >> "$GITHUB_OUTPUT" ;;
82-
*) echo "::error::Unsupported language '${{ inputs.language }}'. Supported values: swift, javascript"; exit 1 ;;
95+
*) echo "::error::Unsupported language '${{ inputs.language }}'. Supported values: swift, javascript, dart"; exit 1 ;;
8396
esac
8497
85-
- name: Parse PR branch
98+
- name: Parse PR and base branch
99+
if: inputs.language != 'dart'
86100
run: |
87101
npm run --silent ${{ steps.resolve.outputs.cmd }} -- "$GITHUB_WORKSPACE/_sdk-pr" \
88102
> "$GITHUB_WORKSPACE/pr-symbols.json"
103+
npm run --silent ${{ steps.resolve.outputs.cmd }} -- "$GITHUB_WORKSPACE/_sdk-base" \
104+
> "$GITHUB_WORKSPACE/base-symbols.json"
89105
working-directory: _sdk-spec/scripts/capability-matrix
90106

91-
- name: Parse base branch
107+
- name: Parse PR and base branch (Dart)
108+
if: inputs.language == 'dart'
92109
run: |
93-
npm run --silent ${{ steps.resolve.outputs.cmd }} -- "$GITHUB_WORKSPACE/_sdk-base" \
110+
dart run bin/extract.dart "$GITHUB_WORKSPACE/_sdk-pr" \
111+
> "$GITHUB_WORKSPACE/pr-symbols.json"
112+
dart run bin/extract.dart "$GITHUB_WORKSPACE/_sdk-base" \
94113
> "$GITHUB_WORKSPACE/base-symbols.json"
95-
working-directory: _sdk-spec/scripts/capability-matrix
114+
working-directory: _sdk-spec/scripts/dart_symbol_extractor
96115

97116
- name: Check new symbols against capability matrix
98117
run: |

CLAUDE.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,14 +63,15 @@ capabilities/*.yaml → validate (AJV schema) → aggregate (GitHub API fetc
6363
- `swift-parser.ts` — Line-by-line Swift scanner; extracts public/open symbols from classes, structs, actors, enums, extensions
6464
- `parse-ts.ts` — CLI wrapper for `ts-parser.ts`; takes an SDK root path and emits `ParseResult` JSON
6565
- `parse-swift.ts` — CLI wrapper for `swift-parser.ts`; same contract as `parse-ts.ts`
66+
- `scripts/dart_symbol_extractor/` (sibling Dart package) — Small `package:analyzer` tool that walks `lib/**.dart` syntactically and emits the same `ParseResult` JSON; run directly with `dart run bin/extract.dart <sdk-root>`. Parses without `pub get`; supports extension types and enhanced enums
6667
- `parse-ignore.ts` — Loads `.sdk-parse-ignore` (gitignore syntax) to exclude paths from symbol parsing
6768
- `api-check.ts` — Diff logic: `checkNewSymbols(base, pr, compliance)` returns symbols added in PR not in the compliance file
6869
- `check-api-symbols.ts` — CLI; compares two `ParseResult` files against `sdk-compliance.yaml`, exits 1 with a clear error on uncovered symbols
6970

7071
### CI Workflows
7172

7273
- `validate-capabilities.yml` — Runs on push to main, PRs, and nightly; Tier 1: schema/tests/typecheck/structural; Tier 2 (PRs + nightly): reference checks against GitHub
73-
- `validate-sdk-compliance.yml`**Reusable workflow** called by SDK repos; validates `sdk-compliance.yaml` and blocks PRs that add public symbols not registered in the compliance file (requires `language` input: `swift` or `javascript`)
74+
- `validate-sdk-compliance.yml`**Reusable workflow** called by SDK repos; validates `sdk-compliance.yaml` and blocks PRs that add public symbols not registered in the compliance file (requires `language` input: `swift`, `javascript`, or `dart`)
7475
- `aggregate-capabilities.yml` — Hourly cron that fetches all SDK compliance data and rebuilds the site
7576
- `deploy-pages.yml` — Deploys to GitHub Pages on main push
7677

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ jobs:
7070
validate:
7171
uses: supabase/sdk/.github/workflows/validate-sdk-compliance.yml@main
7272
with:
73-
language: swift # one of: swift, javascript
73+
language: swift # one of: swift, javascript, dart
7474
```
7575
7676
This checks out the canonical feature list from this repo and runs two checks on every PR:
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.dart_tool/
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
include: package:lints/recommended.yaml
2+
3+
analyzer:
4+
exclude:
5+
- test/fixtures/**
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
import 'dart:convert';
2+
import 'dart:io';
3+
4+
import 'package:dart_symbol_extractor/dart_symbol_extractor.dart';
5+
6+
void main(List<String> arguments) {
7+
if (arguments.isEmpty) {
8+
stderr.writeln('Usage: extract <path-to-sdk-root>');
9+
exit(1);
10+
}
11+
12+
try {
13+
final symbols = parseDartProject(arguments.first);
14+
final result = {'symbols': symbols.map((s) => s.toJson()).toList()};
15+
stdout.writeln(const JsonEncoder.withIndent(' ').convert(result));
16+
} catch (error) {
17+
stderr.writeln('Error: $error');
18+
exit(1);
19+
}
20+
}
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
export 'src/extractor.dart';
2+
export 'src/ignore_matcher.dart';
3+
export 'src/parsed_symbol.dart';
Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
import 'dart:io';
2+
3+
import 'package:analyzer/dart/analysis/utilities.dart';
4+
import 'package:analyzer/dart/ast/ast.dart';
5+
import 'package:path/path.dart' as p;
6+
7+
import 'ignore_matcher.dart';
8+
import 'parsed_symbol.dart';
9+
10+
/// Source file suffixes for generated Dart that is never authored public API.
11+
const _generatedSuffixes = ['.g.dart', '.freezed.dart', '.gr.dart'];
12+
13+
/// Extracts public API symbols from a single Dart source string.
14+
///
15+
/// Parsing is purely syntactic (no element resolution), so it needs neither
16+
/// `pub get` nor a resolvable package graph. Dart privacy is name-based, so a
17+
/// declaration is public when its name does not start with `_`. This matches
18+
/// the altitude of the TypeScript and Swift parsers, which collect declarations
19+
/// per file without following exports.
20+
List<ParsedSymbol> extractFromSource(String source, String relPath) {
21+
final symbols = <ParsedSymbol>[];
22+
final unit = parseString(
23+
content: source,
24+
path: relPath,
25+
throwIfDiagnostics: false,
26+
).unit;
27+
28+
for (final declaration in unit.declarations) {
29+
_visitTopLevel(declaration, relPath, symbols);
30+
}
31+
return symbols;
32+
}
33+
34+
void _visitTopLevel(
35+
CompilationUnitMember declaration,
36+
String relPath,
37+
List<ParsedSymbol> out,
38+
) {
39+
switch (declaration) {
40+
// Class-like containers expose their name and members identically. Unnamed
41+
// extensions (name == null) fall through, as they have no qualifiable
42+
// surface.
43+
case ClassDeclaration(:final name, :final members):
44+
case MixinDeclaration(:final name, :final members):
45+
case EnumDeclaration(:final name, :final members):
46+
case ExtensionTypeDeclaration(:final name, :final members):
47+
case ExtensionDeclaration(name: final name?, :final members):
48+
_emitContainer(name.lexeme, members, relPath, out);
49+
50+
case FunctionDeclaration(:final name, :final isGetter, :final isSetter)
51+
when !isGetter && !isSetter:
52+
_emit(name.lexeme, SymbolKind.function, relPath, out);
53+
54+
// `class C = A with M;` is a class, not a typedef, so it precedes TypeAlias.
55+
case ClassTypeAlias(:final name):
56+
_emit(name.lexeme, SymbolKind.classKind, relPath, out);
57+
58+
case TypeAlias(:final name):
59+
_emit(name.lexeme, SymbolKind.variable, relPath, out);
60+
61+
case TopLevelVariableDeclaration(:final variables):
62+
for (final variable in variables.variables) {
63+
_emit(variable.name.lexeme, SymbolKind.variable, relPath, out);
64+
}
65+
}
66+
}
67+
68+
void _emit(
69+
String name, SymbolKind kind, String relPath, List<ParsedSymbol> out) {
70+
if (_isPrivate(name)) return;
71+
out.add(ParsedSymbol(name: name, kind: kind, file: relPath));
72+
}
73+
74+
void _emitContainer(
75+
String containerName,
76+
List<ClassMember> members,
77+
String relPath,
78+
List<ParsedSymbol> out,
79+
) {
80+
if (_isPrivate(containerName)) return;
81+
out.add(
82+
ParsedSymbol(
83+
name: containerName,
84+
kind: SymbolKind.classKind,
85+
file: relPath,
86+
),
87+
);
88+
89+
for (final member in members) {
90+
if (member is MethodDeclaration) {
91+
final name = member.name.lexeme;
92+
if (_isPrivate(name)) continue;
93+
final kind = (member.isGetter || member.isSetter)
94+
? SymbolKind.property
95+
: SymbolKind.method;
96+
out.add(
97+
ParsedSymbol(name: '$containerName.$name', kind: kind, file: relPath),
98+
);
99+
} else if (member is FieldDeclaration) {
100+
for (final field in member.fields.variables) {
101+
final name = field.name.lexeme;
102+
if (_isPrivate(name)) continue;
103+
out.add(
104+
ParsedSymbol(
105+
name: '$containerName.$name',
106+
kind: SymbolKind.property,
107+
file: relPath,
108+
),
109+
);
110+
}
111+
} else if (member is ConstructorDeclaration) {
112+
// The unnamed constructor reuses the class name, matching the
113+
// `ClassName.ClassName` form the other parsers emit.
114+
final ctorName = member.name?.lexeme ?? containerName;
115+
if (_isPrivate(ctorName)) continue;
116+
out.add(
117+
ParsedSymbol(
118+
name: '$containerName.$ctorName',
119+
kind: SymbolKind.method,
120+
file: relPath,
121+
),
122+
);
123+
}
124+
}
125+
}
126+
127+
bool _isPrivate(String name) => name.startsWith('_');
128+
129+
/// Discovers every package under [projectRoot] (a directory containing a
130+
/// `pubspec.yaml` with a `lib/` directory) and extracts the public API surface
131+
/// of its `lib/**.dart` files. Paths are reported relative to [projectRoot].
132+
List<ParsedSymbol> parseDartProject(String projectRoot) {
133+
final root = p.normalize(p.absolute(projectRoot));
134+
final ignore = IgnoreMatcher.load(root);
135+
final symbols = <ParsedSymbol>[];
136+
137+
for (final packageDir in _findPackageDirs(root, ignore)) {
138+
final libDir = Directory(p.join(packageDir, 'lib'));
139+
if (!libDir.existsSync()) continue;
140+
141+
for (final entity in libDir.listSync(recursive: true, followLinks: false)) {
142+
if (entity is! File || !entity.path.endsWith('.dart')) continue;
143+
if (_generatedSuffixes.any(entity.path.endsWith)) continue;
144+
145+
final relPath = p.relative(entity.path, from: root);
146+
if (ignore.ignores(p.split(relPath).join('/'))) continue;
147+
148+
try {
149+
symbols.addAll(extractFromSource(entity.readAsStringSync(), relPath));
150+
} catch (error) {
151+
// A single unreadable or unparseable file must not fail the whole
152+
// check; skip it and surface a warning on stderr.
153+
stderr.writeln('warning: skipped $relPath: $error');
154+
}
155+
}
156+
}
157+
158+
symbols.sort((a, b) => a.name.compareTo(b.name));
159+
return symbols;
160+
}
161+
162+
Iterable<String> _findPackageDirs(String root, IgnoreMatcher ignore) sync* {
163+
final rootDir = Directory(root);
164+
if (!rootDir.existsSync()) return;
165+
166+
for (final entity in rootDir.listSync(recursive: true, followLinks: false)) {
167+
if (entity is! File || p.basename(entity.path) != 'pubspec.yaml') continue;
168+
169+
final dir = p.dirname(entity.path);
170+
final relDir = p.relative(dir, from: root);
171+
if (relDir != '.') {
172+
final segments = p.split(relDir);
173+
if (segments.any((s) => s.startsWith('.') || s == 'build')) continue;
174+
if (ignore.ignores(segments.join('/'), isDirectory: true)) continue;
175+
}
176+
yield dir;
177+
}
178+
}

0 commit comments

Comments
 (0)