How to merge 2 ROS topics into 1?

Question

I have 2 topics being published by an intel realsense camera. One topic is posting a depth image, and the other is posting a color image. Every color image has a corresponding depth image with the same timestamp in the header. I want to merge the color and depth topic into one topic that would publish the pairs of depth and color images. Is there a ROS function that does this based on timestamps?

Here are the subscribers I have created:

self.image_sub = rospy.Subscriber("image", Image, mask_detect, queue_size=1, buff_size=2**24)
depth_image_sub = rospy.Subscriber("depth_image", Image,
aquire_depth_image, queue_size=1000)

I want to be able to do something like this (psuedocode):

color_depth = rospy.Subscriber(["image", "depth_image"], callback_function, mergePolicy="EXACTTIME")

Is there a standard way of doing this in ROS, or a simple way of doing it?

Answer 1

If you want to fuse the images by time, you will have to write your own nodelet. But I believe the realsense camera already does this for you in the form of an RGB pointcloud.

That message is off by default. You can turn it on in the launch file by setting:

  <arg name="enable_pointcloud"   default="true"/>
  <arg name="pointcloud_texture_stream" default="RS2_STREAM_COLOR"/>
  <arg name="pointcloud_texture_index"  default="0"/>

Your correlated depth and RGB should then be published as a single message called:

depth/points

FYI this is the data type of the depth/points topic: http://docs.ros.org/api/sensor_msgs/html/msg/PointCloud2.html

Answer 2

Each topic has a unique type or ROS message it's associated with. To merge the two topics into one, you would have to take the two messages and make a new message from them. This would be to have a callback which subscribes to both, as you already do, and then publishes them at your desired merged topic and message type.

I can't ask this in a comment, but why do you want to merge the two?

The most common reason, especially for camera data, is that you want to make sure the callback is being triggered on a set of data taken at the exact same time. The standard way is with ROS message_filters:

image_sub = message_filters.Subscriber("image", Image)
depth_sub = message_filters.Subscriber("depth_image", Image)
sync = message_filters.TimeSynchronizer([image_sub, depth_sub], 1)
sync.registerCallback(mask_detect_callback)