Feature request: build a Model without resolving external references

[mromaszewicz]

Hi Guys,

I'm the author of an OpenAPI spec to Go code generator (https://github.com/oapi-codegen/oapi-codegen), and I'm working on V3.1 and V3.2 support by switching to your wonderful library.

One place where I have trouble is with handling external references. Let's say that I have a spec that looks like this:

components:
  schemas:
    ObjectA:
      properties:
        name:
          type: string
        object_b:
          $ref: ../packageB/spec.yaml#/components/schemas/ObjectB

As a code generator, I don't care about the contents of that external spec. All I know is that there is an ObjectB defined somewhere else, and I just need to import a package and reference it, I don't need to fetch and parse its spec.

This is a use case in oapi-codegen which comes up a bunch. Someone might have hand implemented something that they're referring to via spec and the spec may not exist, or may no longer be available.

Right now, my libopenapi port sets up the BasePath and relies on the specs being available, but this will be a regression in our behavior.

I've considered other workarounds; rewriting the spec before parsing to use a stub schema, or not calling BuildV3Model() at all, which bypasses your parsing and validation, but this is a big loss for me.

In an ideal world, I would love to see some kind of flag which says to skip resolving external schemas, and in the resulting Document, I'd have some kind of node which has Ref=something, but a flag indicating that it's external, or simply a nil Schema?

How would you do this? I'm happy to send a PR if you can suggest a way compatible with your design.

[daveshanley]

Hello!

It should not be hard; it means you won't get the graph, but you will get the model. It just means skipping a few bits of how the model is composed using the rolodex.

The index and the rolodex are complex, It's a lot to grok (AI does not understand it without extensive docs), however - It would not be much work for me to throw this in.

Would a SkipResolvingExternalRefs flag in the DocumentConfiguration suffice?

[daveshanley]

Curiously: Did you try this

https://github.com/pb33f/libopenapi/blob/main/datamodel/document_config.go#L82

// AllowFileReferences will allow the index to locate relative file references. This is disabled by default.
	//
	// This behavior is now driven by the inclusion of a BasePath. If a BasePath is set, then the
	// rolodex will look for relative file references. If no BasePath is set, then the rolodex will not look for
	// relative file references.
	//
	// This value when set, will force the creation of a local file system even when the BasePath has not been set.
	// it will suck in and index everything from the current working directory, down... so be warned
	// FileFilter should be used to limit the scope of the rolodex.
	AllowFileReferences bool

	// AllowRemoteReferences will allow the index to lookup remote references. This is disabled by default.
	//
	// This behavior is now driven by the inclusion of a BaseURL. If a BaseURL is set, then the
	// rolodex will look for remote references. If no BaseURL is set, then the rolodex will not look for
	// remote references. This value has no effect as of version 0.13.0 and will be removed in a future release.
	//
	// This value when set, will force the creation of a remote file system even when the BaseURL has not been set.
	// it will suck in every http link it finds, and recurse through all references located in each document.
	AllowRemoteReferences bool

	// AvoidIndexBuild will avoid building the index. This is disabled by default, only use if you are sure you don't need it.
	// This is useful for developers building out models that should be indexed later on.
	AvoidIndexBuild bool

[mromaszewicz]

My buddy Claude has summarized our experiments to get this working, and here are the findings:

We tried the suggested approach of not setting BaseURL/AllowRemoteReferences combined with SkipCircularReferenceCheck = true:

What works:

• Remote references are NOT fetched ✓
• Model is non-nil ✓
• idx.GetAllReferences() discovers all external refs (including nested) ✓
• Top-level $ref schemas: IsReference() and GetReference() work ✓
• Local schemas without external refs remain fully accessible ✓

The problem:
For schemas that contain external refs in their properties, Schema() returns nil. This means we can't iterate properties to generate struct fields without falling back to parsing raw YAML via GoLow().GetValueNode().

Why we need SkipResolvingExternalRefs:
We want libopenapi to parse the schema structure normally but leave external $ref nodes unresolved (preserving IsReference()=true and GetReference() on those nodes). This would let code generators like oapi-codegen handle external refs via import mappings without needing to reimplement schema parsing at the YAML level.

[daveshanley]

Mmmmkay. Thanks, Claude!

I'll fire up my Claude and see how it feels about other Claude's analysis. :D

[mromaszewicz]

Thank you for doing this! It'll allow for much cleaner code on my end.

[daveshanley]

Thank you for doing this! It'll allow for much cleaner code on my end.

No worries, the purpose of this library is to be a toolkit, twistable into any shape you want, extensible as much as you need.

[daveshanley]

Alright, in v0.33.4 you will find a new configuration option in the DocumentConfiguration called SkipExternalRefResolution

This will do exactly what you're looking for.

Here is some sample code for your agent to review on how to use it.

https://github.com/pb33f/libopenapi/blob/main/document_examples_test.go#L648

Enjoy.

[mromaszewicz]

Please have a look at this PR. I made a small change to your test which demonstrates the problem I am still seeing after your fix.

#524