ROS 2 Lyrical Luth · Lesson 49 of 86

RGB camera pipelines

Camera drivers, camera_info, rectification with image_proc and cv_bridge.

Estimated time · 60–90 min

Learning goals

What you will understand

Explain rgb camera pipelines in your own words
Use it in a small robotics example
Verify the result instead of assuming it worked

Understand the idea first

The mental model

A camera pipeline is a chain of nodes: a driver publishes raw images and camera_info; image_proc rectifies (removes lens distortion) and can convert colours; your node subscribes to the result. cv_bridge converts between ROS Image messages and OpenCV cv::Mat.

Pick the encoding explicitly (bgr8 for OpenCV colour, mono8 for grey). Calibrate real cameras with camera_calibration to obtain camera_info. Process frames at the rate you need, not the rate the camera delivers, and never copy images more than necessary.

Guided lesson

Work through these steps

1.
Source the ROS 2 Lyrical environment in a fresh terminal.
2.
Read the rgb camera pipelines example and identify what each line does.
3.
Build only the affected package, source the workspace overlay, and run the example.
4.
Inspect the result from a second terminal with the ros2 CLI or RViz2 and compare it with the expected result.

Worked example

Read it before you run it

#include "cv_bridge/cv_bridge.hpp"
// Converts between ROS images and OpenCV.
void on_image(const sensor_msgs::msg::Image::ConstSharedPtr& msg) {
// Callback for every frame.
  cv::Mat frame = cv_bridge::toCvShare(msg, "bgr8")->image;
// Wrap the message data as an OpenCV colour image without copying.
  cv::Mat gray;
// Output image.
  cv::cvtColor(frame, gray, cv::COLOR_BGR2GRAY);
// Convert to greyscale.
  RCLCPP_INFO_THROTTLE(get_logger(), *get_clock(), 1000, "frame %dx%d", gray.cols, gray.rows);
// Log the size once per second.
}
// End of the callback.

What the example is doing

1.The ros2 commands query the running graph; they do not prove that a node's internal behavior is correct.
2.Build from the workspace root, then source install/setup.bash so the shell can discover the new package.
3.Use a second sourced terminal to inspect nodes and interfaces while the example runs.

Watch it run · Sensor view

See what this code does

The terminal types this lesson's practice solution and runs it. The scene shows the kind of robot behaviour that code drives. It plays by itself and loops.

Practice code

Now type it yourself

Copy the starter into your ROS 2 package, replace every TODO, then build with colcon build and run it from a sourced terminal.

void on_image(const sensor_msgs::msg::Image::ConstSharedPtr& msg) {
  // TODO: wrap the message as a bgr8 cv::Mat with cv_bridge::toCvShare
  RCLCPP_INFO_THROTTLE(get_logger(), *get_clock(), 1000, "frame %dx%d", frame.cols, frame.rows);
}

Expected output

Frame size is logged once per second

Stuck? Show the full solution (try it yourself first)

void on_image(const sensor_msgs::msg::Image::ConstSharedPtr& msg) {
  cv::Mat frame = cv_bridge::toCvShare(msg, "bgr8")->image;
  RCLCPP_INFO_THROTTLE(get_logger(), *get_clock(), 1000, "frame %dx%d", frame.cols, frame.rows);
}

Expected result

How you know it worked

ROS_DISTRO is lyrical in every terminal used for the exercise.
The complete package from the exercise builds and the expected graph entity appears in a second terminal.

Practical exercise

Apply the lesson

Build a node that subscribes to /camera/image, converts each frame to greyscale with cv_bridge and republishes it, then check the rate with ros2 topic hz and view both images in rqt_image_view.

Before you continue

ROS_DISTRO reports lyrical in every terminal you used
The example builds and runs without errors
You checked the result with a ros2 command, RViz2 or a test, not just by reading the code

If something goes wrong

ros2 cannot find a package or executable.

Source /opt/ros/lyrical/setup.bash, build from the workspace root, then source install/setup.bash in that terminal.

Nodes run but cannot discover each other.

Confirm both terminals use Lyrical, the same ROS_DOMAIN_ID, and compatible network/discovery settings.