Feature request: Adapt by-type-arbitraries to Java Records and Sealed Interfaces

[io7m]

Hello!

One way I use jqwik is to generate instances of messages in message protocols. A message protocol is declared as a sealed interface, and the sealed interface is implemented by other sealed interfaces and record types:

sealed interface Message {}
sealed interface Command implements Message {}
sealed interface Response implements Message {}
record CommandHello(String text) implements Command {}
record CommandPing() implements Command {}
record ResponseHello(String text) implements Response {}
record ResponsePing() implements Response {}

... and so on. The tests then generate an arbitrary message, serialize it, and deserialize it, and check it against the original message as a round-trip test of the serializers.

For this kind of setup, it's a lot of work to manually write arbitraries for each message type. If we assume that all record components are either value types, records, or collection types (including Optional), and that all interfaces are sealed, it feels like it would be relatively easy to generate arbitrary instances reflectively. I have done similar work generating JSON schemas for sealed "sum and product" hierarchies like this:

https://github.com/io7m-com/sumjack

I wonder if there's some kind of Arbitraries.createForSimpleRecord() method that could do this, or some way that I could register types for generation in this manner.

Doing it by hand results in having to write a lot of arbitraries: https://github.com/io7m-com/cardant/tree/develop/com.io7m.cardant.tests.arbitraries/src/main/java/com/io7m/cardant/tests/arbitraries

[jlink]

Have you checked how well your use case is already handled by https://jqwik.net/docs/current/user-guide.html#generation-from-a-types-interface ?
In theory it should mostly do what you want, but newer features like sealed types and records may get into its way.

[io7m]

Wow, I'm sorry, I completely missed this. I went through the documentation looking for something like this and managed not to see it. 😊

I'll try it out today and see how it copes with the sets of classes I have.

[io7m]

There are a couple of issues, one of which you hinted might happen.

1. Sealed interfaces don't work. If I write a test such as this:

  @Property
  public void testMessages(
    final @ForAll @UseType MAMessageType message)
  {
    final JsonMapper mapper = MAdminMappers.mapper();
    assertEquals(
      message,
      mapper.readValue(mapper.writeValueAsBytes(message), MAMessageType.class)
    );
  }

I'll get an exception:

timestamp = 2026-03-04T10:52:52.416838647, MAdminMappersTest:testMessages = 
  net.jqwik.api.JqwikException:
    No usable generator executables (constructors or factory methods) have been provided for type [interface org.thepalaceproject.mimace.adminproto.MAMessageType].

I would expect jqwik to iterate over the result of MAMessageType.class.getPermittedSubclasses() (recursively) to locate classes that it can instantiate.

2. Records don't seem to work. If I have a class like this:

public record MABookGet(
  @JsonProperty(value = "ID", required = true)
  UUID id)
  implements MACommandType<MABookGetResponse>
{
  /**
   * A command to retrieve a book.
   *
   * @param id The book ID
   */

  public MABookGet
  {
    Objects.requireNonNull(id, "ID");
  }
}

And a test like this:

  @Property
  public void testMessagesBookGet(
    final @ForAll @UseType MABookGet message)
  {
    final JsonMapper mapper = MAdminMappers.mapper();
    assertEquals(
      message,
      mapper.readValue(mapper.writeValueAsBytes(message), MABookGet.class)
    );
  }

I'll get this response:

timestamp = 2026-03-04T10:55:14.311238952, MAdminMappersTest:testMessagesBookGet = 
  net.jqwik.api.TooManyFilterMissesException:
    net.jqwik.engine.properties.arbitraries.randomized.IgnoreExceptionGenerator@248deced missed more than 10000 times.

In the debugger, I see that in IgnoreExceptionGenerator, an java.lang.reflect.InvocationTargetException is being raised when (presumably) trying to construct the values to pass to the MABookGet constructor:

java.lang.IllegalArgumentException: Invalid UUID string: 뫩櫖㵶딄賋

[jlink]

The problems you detected are more or less what I expected - with the exception of UUID maybe.
You could probably work around them by providing the creator methods for sealed interfaces and UUID explicitly.

A real fix, however, requires an upgrade to a base Java version which has sealed interfaces and records, so that jqwik could use the related reflection mechanisms for these constructs.

In a normal world, I'd already be in the midst of upgrading to JUnit 6 and thus newer Java versions. This would enable to fix this problem as well. In today's world, with all the "AI" stealing most of Open Source's code and attention, my motivation for doing any further OS development at all is basically zero. This has been aggrevated by the fact that I've been trying to get at least some sponsoring for years - and failed.

What I will do is change the title of this issue to reflect the missing feature. And hope for some change in motiviation and/or sponsoring...

Sorry for the downer.

[io7m]

I understand, and feel similarly.

[jlink]

If you have some time to spare, you could start with https://github.com/jqwik-team/jqwik/blob/main/engine/src/main/java/net/jqwik/engine/properties/arbitraries/DefaultTypeArbitrary.java and derive your own implementation for Records and Sealed Interfaces.

[io7m]

I'll give it a shot this weekend!

[io7m]

Is it possible to run tests in an IDE?

I'm able to run tests from the command-line with ./gradlew tests and so on, but attempting to run any test in IDEA results in:

It seems to be a general failure to build. Presumably the project isn't being imported properly, but I have no idea why.

[jlink]

I'm running the tests in IntelliJ all the time. Maybe you're using a fixed Gradle version in IntelliJ that doesn't fit? What does the build log say?

[io7m]

Unfortunately, the only build output isn't exactly helpful:

11:09:52: Executing ':buildSrc:classes :testing:classes :testing:testClasses :test-modular-api:classes :test-modular-api:testClasses :base:classes :documentation:classes :time:classes :time:testClasses :engine:classes :web:classes :kotlin:classes :api:classes :buildSrc:testClasses :base:testClasses :documentation:testClasses :api:testClasses :engine:testClasses :web:testClasses :kotlin:testClasses'…

> Task :buildSrc:extractPluginRequests UP-TO-DATE
> Task :buildSrc:generatePluginAdapters UP-TO-DATE
> Task :buildSrc:compileJava
> Task :buildSrc:compileGroovy NO-SOURCE
> Task :buildSrc:compileGroovyPlugins UP-TO-DATE
> Task :buildSrc:pluginDescriptors UP-TO-DATE
> Task :buildSrc:processResources UP-TO-DATE
> Task :buildSrc:classes
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :api:makeModuleInfoDir

> Task :api:compileJava FAILED
9 actionable tasks: 3 executed, 6 up-to-date
error: module not found: org.opentest4j
error: module not found: org.junit.platform.commons
2 errors

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':api:compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
> Run with --info option to get more log output.
> Run with --scan to get full insights.

BUILD FAILED in 649ms

Looking at the IDE settings, it's set to Wrapper, so it should be using the same Gradle installation that ./gradlew would be using, and ./gradlew clean assemble test works fine... 🤔

[jlink]

Just checked out the latest code and tried it in IntelliJ.

Gradle settings:
Build and run using: Gradle
Run tests using: IntelliJ IDEA

I'm using Java 17 locally.

I can run the test suite in those modules that have tests:

• engine
• kotlin
• time
• web

Maybe some difference in our import procedures. How do you import?

[io7m]

Apologies for the delay in getting back. Unfortunately, I've admitted defeat with this. Sadly, this has been my experience with Gradle and IDEA to date: "It sometimes works on my machine". I've tried every way I can think of to import the project into IDEA... Multiple clones, deleting the cache, etc. Something fundamentally isn't happy.

However, I did determine that I don't actually need to modify jqwik to get what I was looking for. I register these two arbitrary providers:

import net.jqwik.api.Arbitraries;
import net.jqwik.api.Arbitrary;
import net.jqwik.api.providers.ArbitraryProvider;
import net.jqwik.api.providers.TypeUsage;

import java.util.ArrayList;
import java.util.Set;

public final class RqArbSealedInterface
  implements ArbitraryProvider
{
  public RqArbSealedInterface()
  {

  }

  @Override
  public boolean canProvideFor(
    final TypeUsage targetType)
  {
    final Class<?> rawType = targetType.getRawType();
    return rawType.isSealed() && rawType.isInterface();
  }

  @Override
  public Set<Arbitrary<?>> provideFor(
    final TypeUsage targetType,
    final SubtypeProvider subtypeProvider)
  {
    final var rawType =
      targetType.getRawType();

    final var subclasses =
      rawType.getPermittedSubclasses();
    final var arbitraries =
      new ArrayList<Arbitrary<?>>(subclasses.length);

    for (int index = 0; index < subclasses.length; ++index) {
      arbitraries.add(Arbitraries.defaultFor(subclasses[index]));
    }

    return Set.of(Arbitraries.oneOf(arbitraries));
  }
}

import com.fasterxml.classmate.ResolvedType;
import com.fasterxml.classmate.TypeResolver;
import net.jqwik.api.Arbitraries;
import net.jqwik.api.Arbitrary;
import net.jqwik.api.providers.ArbitraryProvider;
import net.jqwik.api.providers.TypeUsage;

import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.util.Set;

public final class RqArbRecord
  implements ArbitraryProvider
{
  private static final TypeResolver TYPE_RESOLVER =
    new TypeResolver();

  public RqArbRecord()
  {

  }

  @Override
  public int priority()
  {
    return -100;
  }

  private static TypeUsage typeUsageOfResolved(
    final ResolvedType t)
  {
    final var params =
      t.getTypeParameters()
        .stream()
        .map(RqArbRecord::typeUsageOfResolved)
        .toList();

    final var paramsArray =
      params.toArray(new TypeUsage[0]);

    return TypeUsage.of(
      t.getErasedType(),
      paramsArray
    );
  }

  @Override
  public boolean canProvideFor(
    final TypeUsage targetType)
  {
    final Class<?> rawType = targetType.getRawType();
    return rawType.isRecord();
  }

  @Override
  public Set<Arbitrary<?>> provideFor(
    final TypeUsage targetType,
    final SubtypeProvider subtypeProvider)
  {
    final var rawType =
      targetType.getRawType();
    final var components =
      rawType.getRecordComponents();

    final var argumentArbitraries =
      new Arbitrary<?>[components.length];
    final var argumentClasses =
      new Class<?>[components.length];

    for (int index = 0; index < components.length; ++index) {
      final var component =
        components[index];

      final ResolvedType resolvedType =
        TYPE_RESOLVER.resolve(component.getGenericType());
      final TypeUsage typeUsage =
        typeUsageOfResolved(resolvedType);

      argumentArbitraries[index] = Arbitraries.defaultFor(typeUsage);
      argumentClasses[index] = component.getType();
    }

    final Constructor<?> constructor;
    try {
      constructor = rawType.getConstructor(argumentClasses);
    } catch (final NoSuchMethodException e) {
      throw new IllegalStateException(e);
    }

    return Set.of(
      Arbitraries.create(() -> {
        try {
          final var arguments = new Object[components.length];
          for (int index = 0; index < components.length; ++index) {
            arguments[index] = argumentArbitraries[index].sample();
          }
          return constructor.newInstance(arguments);
        } catch (final InstantiationException | IllegalAccessException |
                       InvocationTargetException e) {
          throw new IllegalStateException(e);
        }
      })
    );
  }
}

I'm fairly sure the RqArbRecord is doing various things that arbitraries are not supposed to do, but it does seem to work with every "sum and product" class hierarchy I've thrown at it so far. I use classmate from the Jackson authors to fully apply generic types.

Where it can fall down is where a record class has some kind of precondition on the constructor directly (such as "this string isn't allowed to be empty"). The best way I've found to deal with this is to extract something like that into its own record (like NonEmptyString) and just register a custom arbitrary for that record type.

[jlink]

You are right as for the things arbitraries should never do ;-)
The problem is argumentArbitraries[index].sample() which leads to all kinds of unwanted side-effects and misbehaviours.
The idiomatic and safe way would be to use combinators for your purpose. Something like:

var listOfArgumentArbitraries = argumentArbitraries.toList();
return Combinators.combine(listOfArgumentArbitraries)
   .as(arguments -> constructor.newInstance(arguments));

I left out some details, but I hope you get the main idea.

As for constraints that are enforced in the constructor: Exceptions can be ignored during sample creation using .ignoreException(..). Therefore, if the constraints don't filter out most of the random instances this should just work fine.