Loadflow Parameters .from_json is order-sensitive

[nicow-elia]

Describe the current behavior

When storing and loading powsybl loadflow parameters, the from_json crashes if the order of keys in the json has been altered. See:

from pypowsybl.loadflow import Parameters
params = Parameters()
params_json = params.to_json()
ordered_json = json.dumps(json.loads(params_json), sort_keys=True)
# Passes:
Parameters.from_json(params_json)
# Crashes:
Parameters.from_json(ordered_json)

Describe the expected behavior

from_json should be insensitive to the order of keys in the json file.

Describe the steps

No response

Environment

• pypowsybl 1.14.0

Relevant Log Output

No response

Extra Information

No response

[nicow-elia]

Not sure if related to #1153

[HugoKulesza]

Hello @nicow-elia , thank you for the issue ! I checked this to see where it is coming from.
It is not linked to the other issues recently opened concerning the loadflow parameters.

It is an issue coming from the underlying java powsybl-core package and not directly from pypowsybl wrapping.

The json deserialization for loadflow parameters (and actually many other java parameters objects that can be serialized) currently expects to have the "version" field on top to be able to use it to check for the rest of the fields' name/values, so everything can be orederd except this one field.

How blocking is this issue for you ? I can check if/when this can be fixed depending on this

[nicow-elia]

It is not blocking, but I think generally it should be fixed because the json specification explicitely states objects as unordered key/value collections and hence order should not be relied upon in general.

So probably fixing it at the source and implementing the json deserialization in an order-agnostic way is preferrable. Maybe you could introduce a toplevel object with { "version": "xyz", "body": {all-the-data} }

[HugoKulesza]

I did some work to fix in java all the JSON deserializations I could find that depended on the order of the keys : powsybl/powsybl-core#3886.
I hope to have it integrated in the next release train so that pypowsybl also has a fixed behaviour.

[HugoKulesza]

Hello @nicow-elia, just coming back for this subject. Some more discussion has happened around what I tried to do on powsybl-core side to fix this issue.
The conclusion was that fixing it would badly degrade deserialization performance for a lot of objects (ex: json IIDM network files, security analysis result, etc... that can be quite huge) since it would imply that either all the object has to be stored in memory while searching for the version or the file must be read entirely twice. The solution you proposed would have the same issue.

What has been concluded was that the documentation should be improved to clearly state this necessary order, and the error in case of a field not being in its place should be clearer (and not only NullPointerException as it is today).