The following applies to DDPG and TD3, and possibly other models. The following libraries were installed in a virtual environment:

numpy==1.16.4
stable-baselines==2.10.0
gym==0.14.0
tensorflow==1.14.0

Episode rewards do not seem to be updated in model.learn() before callback.on_step(). Depending on which callback.locals variable is used, this means that:

• episode rewards may n

For a lot of exercises, there are different variations (wide grip, narrow grip, reverse, etc. etc.). It would be nice if this could be modelled and presented to the user.

We would basically only need a new many-to-many table, linking all the exercises together, most of the work here would be to go through the DB and actually group them together. Perhaps some text similarity script could help

Per this comment in #12

There seem to be some vulnerabilities in our code that might fail easily. I suggest adding more unit tests for the following:

• Custom agents (there's only VPG and PPO on CartPole-v0 as of now. We should preferably add more to cover discrete-offpolicy, continuous-offpolicy and continuous-onpolicy)
• Evaluation for the Bandits and Classical agents
• Testing of convergence of agents as proposed i

Problem Description

Procgen Environments (https://github.com/openai/procgen) are new environments to test out the generalization ability of agents. It would be nice to include some of the games into the Open RL Benchmark (http://benchmark.cleanrl.dev/)

This is a good first issue for contributors. I think contributors can simply modify the network model slightly (https://github.com/vwxyzjn/c