NEC agent checkpoint creation issue

[Axel-CH]

Hi,
I've an issue with the NEC agent checkpoints. When I ativate the checkpoint option for exemple:

coach -p CartPole_NEC -r -s 120

After the specified time, when the agent trie to save the checkpoint, I have this error:

File rl_coach\agents\nec_agent.py", line 209, in save_checkpoint
TypeError: can't pickle annoy.Annoy objects

Aparently it's not possible to pickle annoy objects (annoy pickle issue)

Does anyone have the same issue?
Other question is the checkpoint system is the only way to save the model weight and bias to run it in production afterward?
Thank you for your help

[gal-leibovich]

This used to work with previous versions of Annoy. Probably was broken with one of their new releases, as this agent code hasn't changed for a while now.

Have you tried saving the index instead as the annoy pickle issue suggests?

[Axel-CH]

I tried:

1. To save DND and DND.dicts objects with jsonpickle module but i'm getting a memory error as this process is eating all my RAM.

2. To perform a json dump on DND and DND.dicts objects but i'm getting

TypeError: Object of type 'QDND' is not JSON serializable

3. To downgrade annoy to version 1.8.3 but getting error:

Failed building wheel for annoy (I'm on Windows...). Only recent version of annoy install themselves sucessfully.

Note that I managed to make Coach to work on Windows by fixing multiple issues with the os path system, and I don't think it's related to the current issue.

[gal-leibovich]

I have reproduced the issue on Linux with Annoy 1.15.0. Downgrading to 1.8.3, resolves the issue though (on Ubuntu).

It's cool to hear that you managed to get Coach running on Windows! It would be great if you could either submit a PR with the needed changes, or try to summarize them here - #190, for the benefit of the community.

[gal-leibovich]

Checkpoint restore for NEC also seems to be broken. Regardless of the save issue with the latest Annoy versions.