Unsure why my std::variant is serialising but not deserialising...?

[nickpelling-ramonspace]

Can anyone please help? I'm trying to use Glaze to serialise and deserialise a std::variant (VariantStore in the fragment below). The serialisation works fine, but calling read_json() to try to deserialise from "world" gives error_code 34, i.e. no_matching_variant_type (I believe). I get the same error_code if I set the value to (int) 12345678, so it doesn't seem to be to do with std::string usage.

All my other Glaze calls work fine: I'm using v5.0.0. Any ideas? Apologies if it's something really obvious! Thanks!

EDIT: I suspect that this works under recent gcc (e.g. 13.3.0), but fails on my embedded device's compiler (gcc 12.3.0). If this is correct, is there a glz::custom workaround? This is with -std=c++23

#include <variant>
#include <vector>
#include <string>
#include <fmt/format.h>
#include "glaze/glaze.hpp"

#define _TRACE fmt::print

using VariantStore = std::variant<
    bool,
    int,
    float,
    std::string,
    std::vector<int>,
    std::vector<float>,
    std::vector<std::string>
>;

bool TestVariantStore()
{
    // Test string variant
    VariantStore value = std::string{"world"};

    // Serialize VariantStore
    std::string json;
    auto ec1 = glz::write_json(value, json);
    if (ec1.ec != glz::error_code::none) {
        _TRACE("TestVariantStore: glz::write_json failed with error code {}\n",
               static_cast<uint32_t>(ec1.ec));
        return false;
    }

    // Sanity check
    if (json.empty()) {
        _TRACE("TestVariantStore: JSON output is empty\n");
        return false;
    }

    // Deserialize VariantStore
    std::string_view json_view = json;
    VariantStore value2{};
    auto ec2 = glz::read_json(value2, json_view);
    if (ec2.ec != glz::error_code::none) {
        _TRACE("TestVariantStore: glz::read_json failed with error code {}\n",
               static_cast<uint32_t>(ec2.ec));
        return false;
    }

    // Compare VariantStore values
    if (value != value2) {
        _TRACE("TestVariantStore: VariantStore value mismatch\n");
        return false;
    }

    return true;
}

[stephenberry]

Thanks for sharing this detailed report!

The issue is that Glaze v5.0.0 doesn't support auto-deduction for variants with multiple types of the same JSON category. Your variant has three array types:

std::vector<int>, std::vector<float>, std::vector<std::string>

All three serialize to JSON arrays, and in v5.0.0, Glaze couldn't distinguish between them during deserialization.

This was fixed in v5.6.1 with the addition of parsability-based variant support (#1916). With this feature, Glaze tries each variant alternative in order until one successfully parses.

Solution

Upgrade to Glaze v5.6.1 or later (v7.0.0 is current). Your code should work without any changes:

using VariantStore = std::variant<
    bool, int, float, std::string,
    std::vector<int>, std::vector<float>,
    std::vector<std::string>
>;

VariantStore v;
glz::read_json(v, "[1, 2, 3]");        // → std::vector<int>
glz::read_json(v, "[1.5, 2.5]");       // → std::vector<float>
glz::read_json(v, R"(["a", "b"])");    // → std::vector<std::string>

Workaround (if you must stay on v5.0.0)

If upgrading isn't possible, you can use tagged variants:

template <>
struct glz::meta<VariantStore> {
   static constexpr std::string_view tag = "type";
   static constexpr auto ids = std::array{
      "bool", "int", "float", "string",
      "int_array", "float_array", "string_array"
   };
};

Your JSON would then include an explicit type tag:

{"type":"int_array", ...}

GCC 12 Note

GCC 12 works with much of Glaze, but recent versions are targetting GCC 13 and up. But, your primary issue is resolved in newer versions of Glaze.

[nickpelling-ramonspace]

Hi Stephen, thanks for the very quick (and correct) reply. The constraint I have is that the embedded toolchain I'm working with is locked to gcc 12.3.0, which means that Glaze 5.0.0 is the (exactly) latest version I can use. So I have to work with what I've got.

Since posting the above, I've made some progress towards a solution, which also gets around the std::vector disambiguation issue you mentioned. Basically, manually unrolling the std::variant inside the read_json() specialisation seems to solve the core template problem:

namespace glz
{
inline error_ctx read_json(VariantStore &v, std::string_view json)
{
	error_ctx ec;

#define TRY_VARIANT_TYPE(TYPE)                                   \
	{                                                        \
		TYPE temp{};                                     \
		ec = glz::read_json(temp, json);                 \
		if (ec.ec == glz::error_code::none) {            \
			v = std::move(temp);                     \
			_TRACE("Succeeded parsing " #TYPE "\n"); \
			ec.ec = glz::error_code::none;           \
			return ec;                               \
		}                                                \
	}

	TRY_VARIANT_TYPE(bool)
	TRY_VARIANT_TYPE(int)
	TRY_VARIANT_TYPE(float)
	TRY_VARIANT_TYPE(std::string)
	TRY_VARIANT_TYPE(std::vector<int>)
	TRY_VARIANT_TYPE(std::vector<float>)
	TRY_VARIANT_TYPE(std::vector<std::string>)

	// No matching type found
	_TRACE("Failed to parse {}\n", json);
	ec.ec = glz::error_code::no_matching_variant_type;
	return ec;
}
} // namespace glz

My (slightly extended) test code now reports success, which is great!

[TRACE] TestVariantStore: Note that JSON #0 = true
[TRACE] Succeeded parsing bool
[TRACE] TestVariantStore: Note that JSON #1 = 42
[TRACE] Succeeded parsing int
[TRACE] TestVariantStore: Note that JSON #2 = 125.125
[TRACE] Succeeded parsing float
[TRACE] TestVariantStore: Note that JSON #3 = "hello"
[TRACE] Succeeded parsing std::string
[TRACE] TestVariantStore: Note that JSON #4 = [1,2,3]
[TRACE] Succeeded parsing std::vector<int>
[TRACE] TestVariantStore: Note that JSON #5 = [125.125,25.25,5.5]
[TRACE] Succeeded parsing std::vector<float>
[TRACE] TestVariantStore: Note that JSON #6 = ["one","two","three"]
[TRACE] Succeeded parsing std::vector<std::string>

The problem I'm now stuck on is that Glaze 5.0.0 doesn't seem to be passing std::optional<VariantStore> through to this. Is that a known issue?

Thanks again for your help!

[stephenberry]

If you put this custom parsing logic inside a glz::from specialization for your VariantStore type, when it should work within a std::optional. Internally Glaze uses to/from specializations for various types. If you fully specialize for VariantStore for this custom reading with a glz::from specialization it should work with the rest of Glaze.

You could also see if GCC 12 builds your code with a newer version of Glaze, but I can't guarantee that will work.