Boost.Python - Passing boost::python::object as argument to python function?

Got this fantastic solution from the c++sig mailing list.

Implement a std::map<std::string, boost::python::object> in the C++ class, then overload __getattr__() and __setattr__() to read from and write to that std::map. Then just send it to the python with boost::python::ptr() as usual, no need to keep an object around on the C++ side or send one to the python. It works perfectly.

Edit: I also found I had to override the __setattr__() function in a special way as it was breaking things I added with add_property(). Those things worked fine when getting them, since python checks a class's attributes before calling __getattr__(), but there's no such check with __setattr__(). It just calls it directly. So I had to make some changes to turn this into a full solution. Here's the full implementation of the solution:

First create a global variable:

boost::python::object PyMyModule_global;

Create a class as follows (with whatever other information you want to add to it):

class MyClass
{
public:
   //Python checks the class attributes before it calls __getattr__ so we don't have to do anything special here.
   boost::python::object Py_GetAttr(std::string str)
   {
      if(dict.find(str) == dict.end())
      {
         PyErr_SetString(PyExc_AttributeError, JFormat::format("MyClass instance has no attribute '{0}'", str).c_str());
         throw boost::python::error_already_set();
      }
      return dict[str];
   }

   //However, with __setattr__, python doesn't do anything with the class attributes first, it just calls __setattr__.
   //Which means anything that's been defined as a class attribute won't be modified here - including things set with
   //add_property(), def_readwrite(), etc.
   void Py_SetAttr(std::string str, boost::python::object val)
   {
      try
      {
         //First we check to see if the class has an attribute by this name.
         boost::python::object obj = PyMyModule_global["MyClass"].attr(str.c_str());
         //If so, we call the old cached __setattr__ function.
         PyMyModule_global["MyClass"].attr("__setattr_old__")(ptr(this), str, val);
      }
      catch(boost::python::error_already_set &e)
      {
         //If it threw an exception, that means that there is no such attribute.
         //Put it on the persistent dict.
         PyErr_Clear();
         dict[str] = val;
      }
   }
private:
   std::map<std::string, boost::python::object> dict;
};

Then define the python module as follows, adding whatever other defs and properties you want:

BOOST_PYTHON_MODULE(MyModule)
{
   boost::python::class_<MyClass>("MyClass", boost::python::no_init)
      .def("__getattr__", &MyClass::Py_GetAttr)
      .def("__setattr_new__", &MyClass::Py_SetAttr);
}

Then initialize python:

void PyInit()
{
   //Initialize module
   PyImport_AppendInittab( "MyModule", &initMyModule );
   //Initialize Python
   Py_Initialize();

   //Grab __main__ and its globals
   boost::python::object main = boost::python::import("__main__");
   boost::python::object global = main.attr("__dict__");

   //Import the module and grab its globals
   boost::python::object PyMyModule = boost::python::import("MyModule");
   global["MyModule"] = PyMyModule;
   PyMyModule_global = PyMyModule.attr("__dict__");

   //Overload MyClass's setattr, so that it will work with already defined attributes while persisting new ones
   PyMyModule_global["MyClass"].attr("__setattr_old__") = PyMyModule_global["MyClass"].attr("__setattr__");
   PyMyModule_global["MyClass"].attr("__setattr__") = PyMyModule_global["MyClass"].attr("__setattr_new__");
}

Once you've done all of this, you'll be able to persist changes to the instance made in python over to the C++. Anything that's defined in C++ as an attribute will be handled properly, and anything that's not will be appended to dict instead of the class's __dict__.