how to parse logged .json files?

[zach-nervana]

The json files stored in the experiments directory are not valid json objects. Is there anything preventing us from using valid json? Is there a method available for loading this custom file type?

[zach-nervana]

Alternatively:

• this log could be stored in multiple files
• we could switch to yaml which can be generated incrementally and validly while only appending to the existing file. With json, this is not possible which is what gets us into this weirdness using that serialization type.

[gal-leibovich]

I don't see any reason not to move to YAML.
Usually, we would have just looked at the files manually, not using any tool, in order to compare a set of hyper-parameters between two or more experiments. This was quite inefficient, but didn't bother us too much as it wasn't something being done too often.

I like the idea of storing in multiple files, as it will allow storing hyper-parameters of different agents (e.g. in different hierarchies) and/or different constructs in different files and will thus allow for better readability. I think we just need to watch out, for not splitting to too many different files, as this has the potential to also become cumbersome.

[zach-nervana]

A few notes after looking into this:

• This log is actually generated all at once, so there is no need to use a format which allows incremental appends.
• generating a valid json will most likely involve creating custom json encoder as described here: https://hynek.me/articles/serialization/
• this issue would be nonexistent if we used gin: https://github.com/google/gin-config