XML Deserialisation - Substitution group namespace lost inside VIRTUAL wrapper types

[davidalk]

Summary

XML substitution-group members lose namespace-based disambiguation when the substituted attribute sits inside a type marked xmlRepresentation: VIRTUAL.

The problem

When a substitution-group attribute is a direct property of a bean, SubstitutedMethodProperty.getActualType() reads the element's QName from the live FromXmlParser/StAX reader and resolves the correct type from the SubstitutionMap by fully-qualified name. This is correct XSD semantics: substitution-group members are always global element declarations, so they are always namespace-qualified in instance documents, and (localName, namespaceURI) identifies the member unambiguously.

However, when the substituted attribute sits inside a VIRTUAL type, Jackson treats the wrapper as an unwrapped property: its content is buffered into a generic TokenBuffer and replayed through a plain JsonParser. TokenBuffer has no concept of XML namespaces, so at replay time the instanceof FromXmlParser check in getActualType() fails and resolution silently degrades to deserializeWithCandidates, which:

1. tries every type registered under the element's local name — the namespace is ignored entirely;
2. keeps whichever candidate populates the most fields (RosettaModelObjectSizeEstimator::getNumberOfFields);
3. collects candidates in a TreeSet ordered by field count, so candidates with equal counts are silently deduplicated and the winner of a tie is decided by candidate iteration order;
4. as a side effect, deserialises the same content once per candidate.

Any model where two substitution-group members share a local name across namespaces gets the wrong runtime type whenever the "wrong" candidate absorbs the content into more fields — with no warning or error.

Real-world impact

This was root-caused from a production model in which a proprietary schema extends a public standard schema, and both declare a global element with the same local name as members of the same substitution group, distinguished only by namespace — a standard XSD extension pattern. In the non-flattened form of that model (the substitution group sits inside a VIRTUAL model-group wrapper), an element whose namespace unambiguously identifies the base-schema member per the XSD (verified with Xerces validation) deserialises as the extension type instead. Flattening the model moves the attribute onto the live-parser namespace-lookup path and produces the schema-correct type — i.e. the same document deserialises to different types depending on whether the model is flattened.

Proposed solutions

Option A — keep the live XML parser for VIRTUAL attributes (avoid TokenBuffer replay)

Implement VIRTUAL/unwrapped handling with a custom deserialisation path that reads the wrapper's content directly from the original FromXmlParser instead of Jackson's buffer-and-replay unwrapped mechanism.

• Pros: fixes the root cause; namespace information is available for all nested decisions (not just substitution groups); removes the per-candidate re-deserialisation cost.
• Cons: the most invasive option — Jackson's unwrapped-property buffering is deep in BeanDeserializer; reordering/interleaving of virtual and real properties is exactly what the buffering exists to handle, so a custom path must re-solve that (possibly building on the content-model machinery from XML disambiguation #614). Highest regression risk.

Option B — preserve QNames across the buffering

Capture namespace information before the content enters the TokenBuffer and make it available at replay time, e.g. (a) encode the fully-qualified name into buffered field names ({urn:my.schema}camel) and decode in SubstitutedMethodProperty, or (b) stash a positional field-name→QName map as a DeserializationContext attribute.

• Pros: targeted; keeps Jackson's unwrapped mechanism intact; moderate blast radius.
• Cons: (a) leaks encoded names to every consumer of the buffer and must be decoded everywhere field names are compared; (b) is fragile to nesting/repeated elements and requires hooking the point where Jackson copies tokens into the buffer, which is not currently rune-controlled. Both are workarounds for TokenBuffer being namespace-blind rather than a principled fix.

Option C — make the fallback safe(r) rather than fixing namespace loss

Keep deserializeWithCandidates but: log a WARN whenever resolution falls back to content-based guessing between >1 candidates; replace the TreeSet with an explicit max-selection with a deterministic, documented tie-break (or fail fast on ties); optionally prefer content-model compatibility (from #614) over raw populated-field count.

• Pros: small, low-risk, immediately shippable; makes today's silent wrong answers at least visible and deterministic.
• Cons: does not fix the bug — the namespace is still ignored, and the FiML/FpML case still resolves to the wrong type. Mitigation only; should be done in addition to A or B, not instead.

Recommendation

Option A (or B if A proves too disruptive), plus the Option C hardening regardless — the warning and deterministic tie-break are worth having even once namespaces are preserved, since the fallback still runs for genuinely namespace-less input.

Until fixed, the only consumer-side workaround is to flatten models so substitution-group attributes are direct properties rather than nested inside VIRTUAL wrappers.