[rllib] h5 export format is not supported

[mcrowson]

What is the problem?

I'm calling tune.run, but no file is being output in the local_dir.

ray: 0.8.6
tensorflow: 2.2.0
Arch Linux

Reproduction

Tune it and ask for the h5

from ray import tune
from ray.rllib.agents.ppo import PPOTrainer
tune.run(PPOTrainer,
         config={"env": "CartPole-v0"},
         export_formats=['h5'],
         local_dir='cart_outputs',
         stop={"training_iteration": 1}
)

See outputs in local_dir

> tree cart_outputs
cart_outputs
└── PPO
    ├── experiment_state-2020-07-06_09-17-52.json
    └── PPO_CartPole-v0_0_2020-07-06_09-17-52g7nd9506
        ├── events.out.tfevents.1594041472.refindly-arch
        ├── params.json
        ├── params.pkl
        ├── progress.csv
        └── result.json

2 directories, 6 files

[amogkam]

Hi @mcrowson what file are you expecting to see? The contents that you see in local_dir look like correct behavior. By default Tune creates a new directory for each trial and logs results for Tensorboard, CSV, and JSON.

[mcrowson]

I expect to see an .h5 file. I'm trying to export the weights of the tuned model.

[richardliaw]

Yeah, looks like the main problem here is that h5 is not supported in rllib. Can you try using ExportFormats.MODEL?

[mcrowson]

I get the same output when using that approach

from ray import tune
from ray.rllib.agents.ppo import PPOTrainer
from ray.tune.trial import ExportFormat

tune.run(PPOTrainer,
         config={"env": "CartPole-v0"},
         export_formats=[ExportFormat.H5],
         local_dir='cart_outputs2',
         stop={"training_iteration": 1}
)

> tree cart_outputs2/
cart_outputs2/
└── PPO
    ├── experiment_state-2020-07-09_10-18-47.json
    └── PPO_CartPole-v0_0_2020-07-09_10-18-47xl56z7k8
        ├── events.out.tfevents.1594304327.refindly-arch
        ├── params.json
        ├── params.pkl
        ├── progress.csv
        └── result.json

2 directories, 6 files

[mcrowson]

However if I use the MODEL as you suggest it does give me the TF model, but no H5.

from ray import tune
from ray.rllib.agents.ppo import PPOTrainer
from ray.tune.trial import ExportFormat

tune.run(PPOTrainer,
         config={"env": "CartPole-v0"},
         export_formats=[ExportFormat.MODEL],
         local_dir='cart_outputs3',
         stop={"training_iteration": 1}
)

> tree cart_outputs3 
cart_outputs3
└── PPO
    ├── experiment_state-2020-07-09_10-20-49.json
    └── PPO_CartPole-v0_0_2020-07-09_10-20-49nzews0q6
        ├── events.out.tfevents.1594304449.refindly-arch
        ├── model
        │   ├── events.out.tfevents.1594304456.refindly-arch
        │   ├── saved_model.pb
        │   └── variables
        │       ├── variables.data-00000-of-00002
        │       ├── variables.data-00001-of-00002
        │       └── variables.index
        ├── params.json
        ├── params.pkl
        ├── progress.csv
        └── result.json

4 directories, 11 files

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.