In file binary segmentation (camvid).ipynb, block 5, there is:

# Lets look at data we have
dataset = Dataset(x_train_dir, y_train_dir, classes=['car', 'pedestrian'])
image, mask = dataset[5] # get some sample
visualize(
    image=image, 
    cars_mask=mask[..., 0].squeeze(),
    sky_mask=mask[..., 1].squeeze(),
    background_mask=mask[..., 2].squeeze(),
)

here, sky_mask

https://github.com/yangxue0827/R2CNN_FPN_Tensorflow/blob/37da5d7e002792b6c73a0aefaed865ca1911b2d3/libs/box_utils/coordinate_convert.py#L43

See here,

Developer should keep in mind that the returned rotatedRect can contain negative indices when data is close to the containing Mat element bo

What is the Map in pascal voc 07 dataset using this code? As i read your code, i find that you do not fix the BN parameters. The batchsize is small, following the paper FPN or Mask RCNN, you need to set the training and trainbale to False for the slim.batch_norm op. I am testing whether this is the problem for the bad generalization for mutiple objects.