Ubuntu Linux for Robotics · Lesson 70 of 96

Depth processing

Depth images, encodings, depth_image_proc and turning depth into point clouds.

Estimated time · 45–75 min

Learning goals

What you will understand

Explain depth processing in your own words
Use it in a small workstation example
Verify the result instead of assuming it worked

Understand the idea first

The mental model

A depth camera publishes a depth image where each pixel is a distance, usually 16-bit millimetres (16UC1) or 32-bit float metres (32FC1). Check the encoding with ros2 topic echo --field encoding. With the camera intrinsics from camera_info you can convert depth to a point cloud: depth_image_proc's point_cloud_xyz does exactly this.

Depth is invalid on shiny, dark or very close surfaces (zeros or NaN) and noisy at long range. Filter invalid pixels, limit the range (for example 0.3–5 m), and run at a lower rate than the camera if the CPU is limited.

Guided lesson

Work through these steps

1.
Open a terminal and record the current state before changing anything.
2.
Read the depth processing commands and predict what each one does.
3.
Run the commands one at a time and compare each output with your prediction.
4.
Write the commands that worked in a setup note you can repeat later.

Worked example

Read it before you run it

sudo apt install ros-lyrical-depth-image-proc
# Install the depth processing nodes.
ros2 topic echo /camera/depth/image_rect_raw --once --field encoding
# Check whether depth is 16UC1 or 32FC1.
ros2 run depth_image_proc point_cloud_xyz_node --ros-args -r depth/image_rect:=/camera/depth/image_rect_raw -r depth/camera_info:=/camera/depth/camera_info
# Convert depth to a point cloud.
ros2 topic hz /points
# Check the point cloud rate.
rviz2
# View the PointCloud2.

What the example is doing

1.Lines without sudo only inspect or change files you own; sudo authorizes a system-wide change.
2.Read command output before moving to the next line. Stop if names, versions, disks, or paths differ from the lesson.
3.Comments beginning with # explain safety checks and are not executed by the shell.

Watch it run · Sensor view

See what this code does

The terminal types this lesson's practice solution and runs it. The scene shows the kind of robot behaviour that code drives. It plays by itself and loops.

Practice code

Now type it yourself

Open a terminal, run the starter line by line (or save it as practice.sh and run bash practice.sh), and replace every TODO with your own command.

# TODO: print only the encoding field of one depth image message

Expected output

encoding: 16UC1

Stuck? Show the full solution (try it yourself first)

ros2 topic echo /camera/depth/image_rect_raw --once --field encoding

Expected result

How you know it worked

Each command completes without an unexplained error.
Your notes contain the command, the important output, and what it proves.

Practical exercise

Apply the lesson

Convert a depth stream to a point cloud, view it in RViz2, and measure how many points are invalid (zero) when the camera looks at a shiny surface.

Before you continue

You recorded the starting state
Every command and output is understood
The change can be repeated from your notes

If something goes wrong

A command says permission denied.

Check ownership and permissions first. Do not add sudo unless the command changes a documented system location.

A command or package is not found.

Check spelling, Ubuntu version, enabled repositories, and whether the previous installation command succeeded.