Validate v4 using concerto-conformance tests

[ekarademir]

Using concerto-conformance repository, validate concerto-v4 and report any bugs surfaced.

[Ridhimakale]

Hi, I’m Ridhima

I’ve started working on validating Concerto v4 using the concerto-conformance test suite and wanted to share my current findings.

I ran the available test scripts locally:

npm run test:semantic runs successfully and all 69 scenarios and 184 steps pass.

npm run test:instances executes but reports 0 scenarios / 0 steps.

From what I can see, instance validation tests under the validate/ directory are present (models and step definitions), but they don’t appear to be wired to any Cucumber scenarios at the moment, so instance-level conformance is not currently being exercised for v4.

Please let me know if this behavior is expected for v4, or if instance validation scenarios should be added/updated. I’d be happy to help further based on the intended direction.

[Shubh-Raj]

Using concerto-conformance repository, validate concerto-v4 and report any bugs surfaced.

Ran the conformance tests against Concerto v4.0.0-alpha.2:

69 scenarios, 184 steps - All Passed

Tested: concepts, enums, imports, maps, namespaces, scalars

No bugs surfaced from the current test suite. May need to add new tests for v4-specific features.

[rockaxorb13]

After my PR which migrates concerto-core and concerto-util from Javascript to Typescript got merged in the v4 branch,

I successfully wired and validated V4 instance tests.

Here are my findings and modifications:

1. The validate/features/validate.feature file was empty, which is why Cucumber found "0 scenarios."
2. The current NPM release (4.0.0-alpha.2) is outdated and does not contain the recent TypeScript migration fixes merged into main.

The fix:
I linked my local concerto-core (built from the latest v4.0.0 branch) to concerto-conformance and populated the feature file with scenarios matching validateSteps.js.

Result:

2 scenarios (2 passed)
4 steps (4 passed)

I will open a PR to populate the feature files permanently so these tests run correctly in CI.

[rockaxorb13]

@ekarademir As asked in the last tech workgroup meeting, I linked the new concerto-core (local) to the concerto-conformance repository as the npm version of concerto-core v4 is outdated, being last updated in October 2025

'4.0.0-alpha.1': '2025-10-20T20:09:14.252Z',
'4.0.0-alpha.2': '2025-10-20T20:17:46.056Z',

Meaning, the NPM release is missing the TypeScript migration code that was recently merged. Additionally I linked my local build of concerto-core, util, vocabulary and cto to concerto-conformance to test the latest v4 code.

I got

69 scenarios (69 passed)
184 steps (184 passed)
0m00.805s (executing steps: 0m00.728s)

Implying that the latest v4 branch is rock solid and is passing all the scenarios for conformance.

I'll take up the validation in playground and come up with an update soon.