Fejemis Vision ROS2

fejemis_vision package

The fejemis_vision package is the computer vision subsystem used in the Fejemis robot platform. It is responsible for:

• Human detection
• Cable detection
• Depth-based segmentation
• Pose estimation
• Object tracking

The package is located in: src/fejemis/fejemis_vision

launch/: Contains ROS 2 launch files used to start the vision system.

• main-launch.py - launch full detection
• cable_test.launch.py, net_test.launch.py, people_test.launch.py - launches individual detection

pyth/:

• main.py - Main ROS 2 vision node
• cable_detection.py - Cable and linear object detection
• net_test.py - Used for testing and validating the net detection pipeline
• people_detection_pose_segmentation.py - Human detection using pose estimation and segmentation
• people_detection_depth_segmentation.py - Human detection using depth-based segmentation
• people_detection_deep_sort.py - Multi-person tracking using DeepSORT

Each detector has a test script to run for testing.These test uses test images to do offline testing of the detection models fromdifferent test images folder.

config/:

• Configuration files allow tuning the perception system without modifying source code.

deep_learning/:

• The deep_learning/ directory contains the neural network models and configuration files used by the vision system for object and human detection.

Setup:

• OpenCV should be installed using "pip install opencv-python"
• The various python packages used can be installed using the requiremtents.txt file by running "pip install requiremtents.txt"
• To run the scripts using roslaunch make sure to set scripts to be executable permissionns:

chmod +x main.py
chmod +x net_test.py
chmod +x people_test_basic.py
chmod +x cable_test.py 

• To run the main script:

ros2 launch fejemis_vision main.launch.py

• To run any of the test scripts:

ros2 launch fejemis_vision cable_test.launch.py
ros2 launch fejemis_vision people_test.launch.py
ros2 launch fejemis_vision net.launch.py

Progress

The main work done was to convert this package from ROS1 to compatible with ROS 2. Limited testing was done due to limited time, and more modifying needs to be done.

• Each detections worked on offline testing meaning testing with existing images.

• The result testing each detector from camera stream was mixed. People detection and net detection had the most working detection and displaying the markers in rviz. Cable detectors seems random where it some time detected cables and ofte drawed cables on random places and some time

This can look into:

• The detection parameters was not fully tested and twearked, which can be done to improve the noise expecially for cable detectors.

• From people detection only the people_detection_pose_segmentation.py observed to work. There other was not invastigated due to limited time and can be further invastigate.

• In the original ROS jsk-visualization was used for markering in Rviz. However at the time of this project this package was not compatible in ROS2. Instead inbuilt marker array was used to draw the object in Rviz.

To access the old ROS 1 repository: https://github.com/MP-EL/fejemis_vision