gym-uds-api

This project presents a basic API for interacting with OpenAI Gym environments in languages other than Python through the gRPC framework and local Unix domain sockets.

The API comes with example C++ bindings supporting one-dimensional observation spaces of type Box and action spaces of type Discrete (suitable, for instance, for the CartPole-v0 environment).

Requisites

Example instructions are provided for macOS (tested on macOS Catalina 10.15.3).

0. Install Homebrew.

1. Install pkg-config:

    ~$ brew install pkg-config
   

2. Install protobuf:

    ~$ brew install protobuf
   

3. Install gRPC:

    ~$ brew install grpc
   

4. Install the grpcio_tools, NumPy and OpenAI Gym pip packages (for Python 3):

    ~$ pip install grpcio_tools numpy gym
   

Tested with:

~$ brew list --versions pkg-config protobuf grpc
grpc 1.27.3
pkg-config 0.29.2
protobuf 3.11.4

~$ pip list | egrep "grpcio|numpy|gym"
grpcio       1.27.2
grpcio-tools 1.27.2
gym          0.17.0
numpy        1.18.1

Installation

1. Clone this repository:

    ~$ git clone https://github.com/integeruser/gym-uds-api
   

2. cd to the gym-uds-api directory and generate the gRPC headers and sources for the Python server and client:

    gym-uds-api$ python -m grpc_tools.protoc -I=. --python_out=. --grpc_python_out=. ./gym-uds.proto
   

3. To build the C++ client, generate the gRPC headers and sources for C++ and move them to the binding-cpp directory:

    gym-uds-api$ protoc -I=. --cpp_out=. --grpc_out=. --plugin=protoc-gen-grpc=$(which grpc_cpp_plugin) ./gym-uds.proto
    gym-uds-api$ mv ./gym-uds.pb.h binding-cpp/include/
    gym-uds-api$ mv ./gym-uds.pb.cc binding-cpp/src/
    gym-uds-api$ mv ./gym-uds.grpc.pb.h binding-cpp/include/
    gym-uds-api$ mv ./gym-uds.grpc.pb.cc binding-cpp/src/
   

Usage

1. Start the Python server:

    gym-uds-api$ python ./gym-uds-server.py CartPole-v0
   

2. On a second terminal, execute the dummy Python client:

    gym-uds-api$ python ./gym-uds-test-client.py
    Ep. 1: 15.00
    Ep. 2: 12.00
    Ep. 3: 20.00
   

3. Alternatively, on a second terminal, cd to the binding-cpp directory, then build and execute the dummy C++ client:

    gym-uds-api/binding-cpp$ make
    gym-uds-api/binding-cpp$ bin/gym-uds-client
    Ep. 1: 19
    Ep. 2: 13
    Ep. 3: 10