ROS2_WS workspace (RMV && ROS2 Humble)

Overview

This workspace contains several packages for automated computer vision tasks for RMV. Key packages are:

• hik_camera — Hikvision MVS SDK camera node (requires MVS SDK);
• from_video — read local video file and publish /image_raw and /camera_info;
• armor_detector — detects armor plates and classifies numbers (uses ONNX model);
• vision_launch — top-level launch wrapper to start the full stack;
• auto_aim_interfaces — This is part of an open-source package, thanks to their authors :) , which inspires the interface-based design of most part of this workspace. Original package has more functionalities than the two formats in this workspace.

This project expects OpenCV >= 4.7 due to the need for ONNX DNN support. Ubuntu 22.04 ships OpenCV 4.5 via apt by default; if you compiled and installed OpenCV 4.12 (recommended for ONNX features) to /usr/local, follow the "Build and runtime steps" below to ensure the workspace uses OpenCV 4.12 and a matching cv_bridge.

Usage

Suppose you use zsh:

0. Don't forget to source your /opt/ros/humble/setup.zsh before everything when you open a new terminal window! (Ignore this if you already has modified your ~/.zshrc or something has the same function. :) )
1. Run build.zsh at the root directory of this workspace.
2. source launch_using_camera.zsh if you wish to use HikVision camera; source launch_using_video.zsh if you want to use the provided video.

Parameters and Macros

Parameters

Parameters' names are all based on their actual functions.

Macros

There are some C macros available at some of .cpp files' beginning. Toggling them can output logs accordingly.

Build and runtime steps (if needed)

1. Build cv_bridge in this workspace against your local OpenCV (4.12):

cd /home/bis/BiS_tmp.d/ros2_ws
export OpenCV_DIR=/usr/local/lib/cmake/opencv4
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
colcon build --packages-select cv_bridge --cmake-args -DOpenCV_DIR=$OpenCV_DIR

2. (Optional but recommended) Create workspace-local OpenCV shared libs to avoid system library ordering issues. This repository includes a helper that copies the /usr/local OpenCV shared libs into third_party/opencv_libs/lib and provides a setup script to prepend that path to LD_LIBRARY_PATH:

cd /home/bis/BiS_tmp.d/ros2_ws
chmod +x scripts/copy_opencv_libs_to_ws.sh third_party/opencv_libs/setup_opencv_ws.sh
./scripts/copy_opencv_libs_to_ws.sh
source third_party/opencv_libs/setup_opencv_ws.sh

3. Rebuild packages that depend on OpenCV/cv_bridge (or rebuild the whole workspace):

export CMAKE_PREFIX_PATH=$(pwd)/install:$CMAKE_PREFIX_PATH
export LD_LIBRARY_PATH=$PWD/install/cv_bridge/lib:/usr/local/lib:$LD_LIBRARY_PATH
colcon build --packages-select armor_detector from_video

4. Launch the unified stack using the workspace helper (local installs):

source install/local_setup.zsh
# optionally source the workspace-local OpenCV libs
source third_party/opencv_libs/setup_opencv_ws.sh
zsh ./launch_all.zsh --local input_type:=video video_path:="/dev/null"

Exact environment commands to run before launching (copy/paste):

# prefer workspace cv_bridge and /usr/local OpenCV
export LD_LIBRARY_PATH=/home/bis/BiS_tmp.d/ros2_ws/install/cv_bridge/lib:/usr/local/lib:$LD_LIBRARY_PATH
source /home/bis/BiS_tmp.d/ros2_ws/install/local_setup.zsh
# optional: use workspace-local copy of OpenCV libs
source /home/bis/BiS_tmp.d/ros2_ws/third_party/opencv_libs/setup_opencv_ws.sh
zsh /home/bis/BiS_tmp.d/ros2_ws/launch_all.zsh --local input_type:=video video_path:="/dev/null"

Notes and troubleshooting

• If you see OpenCV ABI mismatches (linker warnings about libopencv_core.so.4.5d vs libopencv_core.so.412), use the instructions above to build cv_bridge and prefer the workspace cv_bridge at runtime by setting LD_LIBRARY_PATH (or using the copy script).
• The NumberClassifier attempts a lightweight forward-test at startup and will disable classification if the model is incompatible with the runtime OpenCV DNN backend. Check armor_detector logs for details.
• MVS SDK: By default hik_camera/CMakeLists.txt looks for an MVS directory in the repo, otherwise falls back to /opt/MVS. Place the SDK accordingly.
• QoS: The image and camera_info topics use best-effort, keep-last(1) (sensor-data QoS). If you see messages not delivered, check subscriber QoS compatibility.

Problems encountered and reflections

• In armor detector package, .cpp codes need to reference to each other. Thus, standalone header files are needed in order to pass compilation.

• These CMakeLists.txt codes works differently:

   install(
   	DIRECTORY MVS/lib/64/
   	DESTINATION lib
   )

   install(
   	DIRECTORY MVS/lib/64
   	DESTINATION lib
   )

  Use the former!