Fejemis Vision ROS2 - Revision history

S253734 at 18:41, 28 May 2026

2026-05-28T18:41:12Z

← Older revision		Revision as of 20:41, 28 May 2026
Line 1:		Line 1:
	Back to [[Fejemis ~~2026~~]]		Back to [[Fejemis ROS2 Software]]

	== fejemis_vision package ==		== fejemis_vision package ==

S253734 at 18:40, 28 May 2026

2026-05-28T18:40:23Z

S253734: /* RViz Visualization */

2026-05-28T18:38:16Z

RViz Visualization

← Older revision		Revision as of 20:38, 28 May 2026
Line 136:		Line 136:
	=== RViz Visualization ===		=== RViz Visualization ===

	The original ROS 1 implementation used the jsk-visualization package for RViz markers.		The original ROS 1 implementation used the '''jsk-visualization''' package for RViz markers.

	At the time of this project, jsk-visualization was not fully compatible with ROS 2. Therefore, the system was modified to use built-in ROS 2 MarkerArray visualization messages instead.		At the time of this project, '''jsk-visualization''' was not fully compatible with ROS 2. Therefore, the system was modified to use built-in ROS 2 '''MarkerArray''' visualization messages instead.

	This allowed detected objects and positions to still be visualized in RViz.		This allowed detected objects and positions to still be visualized in RViz.

S253734: /* Areas for Future Improvement */

2026-05-28T18:37:15Z

Areas for Future Improvement

← Older revision		Revision as of 20:37, 28 May 2026
Line 130:		Line 130:

	* Detection parameters and thresholds were not fully tuned. Additional tuning may significantly reduce detection noise, especially for cable detection		* Detection parameters and thresholds were not fully tuned. Additional tuning may significantly reduce detection noise, especially for cable detection
	* Only people_detection_pose_segmentation.py was verified to work reliably during live testing. The remaining human detection methods were not fully investigated due to time limitations		* Only ''people_detection_pose_segmentation.py'' was verified to work reliably during live testing. The remaining human detection methods were not fully investigated due to time limitations
	* Additional filtering and depth processing could improve robustness		* Additional filtering and depth processing could improve robustness
	* Performance optimization may still be required for Raspberry Pi deployment		* Performance optimization may still be required for Raspberry Pi deployment

S253734: /* Current Results */

2026-05-28T18:34:19Z

Current Results

← Older revision		Revision as of 20:34, 28 May 2026
Line 125:		Line 125:
	* People detection and net detection showed the most stable and reliable performance		* People detection and net detection showed the most stable and reliable performance
	* RViz marker visualization worked correctly for several detection pipelines		* RViz marker visualization worked correctly for several detection pipelines
	* Cable detection was significantly more unstable and noisy. The cables were sometimes correctly detected ~~and~~ false detections and random line generation frequently occurred		* Cable detection was significantly more unstable and noisy. The cables were sometimes correctly detected, however false detections and random line generation frequently occurred

	=== Areas for Future Improvement ===		=== Areas for Future Improvement ===

S253734: /* launch/ */

2026-05-28T18:30:33Z

launch/

← Older revision		Revision as of 20:30, 28 May 2026
Line 22:		Line 22:

	Main launch files:		Main launch files:
	* ''~~main-launch~~.py'' - Launches the complete perception system		* ''main_launch.py'' - Launches the complete perception system

	Testing launch files:		Testing launch files:

S253734: /* fejemis_vision package */

2026-05-28T18:29:23Z

fejemis_vision package

← Older revision		Revision as of 20:29, 28 May 2026
Line 25:		Line 25:

	Testing launch files:		Testing launch files:
	* cable_test.launch.py - Launches the cable detection pipeline		* ''cable_test.launch.py'' - Launches the cable detection pipeline
	* net_test.launch.py - Launches the net detection pipeline		* ''net_test.launch.py'' - Launches the net detection pipeline
	* people_test.launch.py - Launches the people detection pipeline		* ''people_test.launch.py'' - Launches the people detection pipeline

	These launch files initialize the required ROS 2 nodes, camera subscriptions, and RViz visualization.		These launch files initialize the required ROS 2 nodes, camera subscriptions, and RViz visualization.
Line 35:		Line 35:

	Main scripts:		Main scripts:
	* main.py - Main ROS 2 vision node coordinating the perception system		* ''main.py'' - Main ROS 2 vision node coordinating the perception system

	Detection scripts:		Detection scripts:
	* cable_detection.py - Detects cables and linear structures in the environment		* ''cable_detection.py'' - Detects cables and linear structures in the environment
	* net_detection.py - Detects net or mesh-like structures using computer vision methods		* ''net_detection.py'' - Detects net or mesh-like structures using computer vision methods
	* people_detection_pose_segmentation.py - Human detection using pose estimation and segmentation		* ''people_detection_pose_segmentation.py'' - Human detection using pose estimation and segmentation
	* people_detection_depth_segmentation.py - Human detection using depth-based segmentation		* ''people_detection_depth_segmentation.py'' - Human detection using depth-based segmentation
	* people_detection_simple_segmentation.py - Simpler segmentation-based human detection approach		* ''people_detection_simple_segmentation.py'' - Simpler segmentation-based human detection approach

	Tracking:		Tracking:
	* people_detection_deep_sort.py - Multi-person tracking using the DeepSORT tracking algorithm		* ''people_detection_deep_sort.py'' - Multi-person tracking using the DeepSORT tracking algorithm

	Testing scripts:		Testing scripts:
	* cable_test.py		* ''cable_test.py''
	* net_test.py		* ''net_test.py''
	* people_test_basic.py		* ''people_test_basic.py''

	These scripts are used for testing and validating the detection pipelines independently.		These scripts are used for testing and validating the detection pipelines independently.
Line 57:		Line 57:
	The package contains several directories with offline test images used during development and debugging:		The package contains several directories with offline test images used during development and debugging:

	* cable_test_images/		* ''cable_test_images/''
	* net_test_images/		* ''net_test_images/''
	* people_test_images/		* ''people_test_images/''
	* other_imgs/		* ''other_imgs/''

	These images are used to evaluate the detection models without requiring a live camera stream.		These images are used to evaluate the detection models without requiring a live camera stream.
Line 68:		Line 68:

	Files include:		Files include:
	* yolov4-tiny.cfg - YOLOv4-tiny network configuration		* ''yolov4-tiny.cfg'' - YOLOv4-tiny network configuration
	* yolov4-tiny.weights - Pretrained YOLOv4-tiny weights		* ''yolov4-tiny.weights'' - Pretrained YOLOv4-tiny weights
	* classes.txt - Object class labels used by the detector		* ''classes.txt'' - Object class labels used by the detector
	* mars-small128.pb - Feature extraction model used by DeepSORT tracking		* ''mars-small128.pb'' - Feature extraction model used by DeepSORT tracking

	The package uses YOLOv4-tiny to achieve lightweight real-time object detection suitable for embedded systems such as Raspberry Pi devices.		The package uses YOLOv4-tiny to achieve lightweight real-time object detection suitable for embedded systems such as Raspberry Pi devices.

S253734 at 18:24, 28 May 2026

2026-05-28T18:24:57Z

Show changes

S253734: Created page with " == fejemis_vision package == The fejemis_vision package is the computer vision subsystem used in the Fejemis robot platform. It is responsible for: * Human detection * Cable detection * Depth-based segmentation * Pose estimation * Object tracking The package is located in: ''src/fejemis/fejemis_vision'' launch/: Contains ROS 2 launch files used to start the vision system. * ''main-launch.py'' - launch full detection * ''cable_test.launch.py'', ''net_test.launch.p..."

2026-05-28T18:06:03Z

Created page with " == fejemis_vision package == The fejemis_vision package is the computer vision subsystem used in the Fejemis robot platform. It is responsible for: * Human detection * Cable detection * Depth-based segmentation * Pose estimation * Object tracking The package is located in: ''src/fejemis/fejemis_vision'' launch/: Contains ROS 2 launch files used to start the vision system. * ''main-launch.py'' - launch full detection * ''cable_test.launch.py'', ''net_test.launch.p..."

New page

== fejemis_vision package ==

The fejemis_vision package is the computer vision subsystem used in the Fejemis robot platform. It is responsible for:

* Human detection
* Cable detection
* Depth-based segmentation
* Pose estimation
* Object tracking

The package is located in: ''src/fejemis/fejemis_vision''

launch/:
Contains ROS 2 launch files used to start the vision system.

* ''main-launch.py'' - launch full detection
* ''cable_test.launch.py'', ''net_test.launch.py'', ''people_test.launch.py'' - launches individual detection

pyth/:
* main.py - Main ROS 2 vision node
* cable_detection.py - Cable and linear object detection
* net_test.py - Used for testing and validating the net detection pipeline
* people_detection_pose_segmentation.py - Human detection using pose estimation and segmentation
* people_detection_depth_segmentation.py - Human detection using depth-based segmentation
* people_detection_deep_sort.py - Multi-person tracking using DeepSORT

Each detector has a test script to run for testing.These test uses test images to do offline testing of the detection models fromdifferent test images folder.

config/:
* Configuration files allow tuning the perception system without modifying source code.

deep_learning/:
* The deep_learning/ directory contains the neural network models and configuration files used by the vision system for object and human detection.

Setup:

* OpenCV should be installed using "pip install opencv-python"
* The various python packages used can be installed using the requiremtents.txt file by running "pip install requiremtents.txt"
* To run the scripts using roslaunch make sure to set scripts to be executable permissionns:

chmod +x main.py
chmod +x net_test.py
chmod +x people_test_basic.py
chmod +x cable_test.py

* To run the main script:

ros2 launch fejemis_vision main.launch.py

* To run any of the test scripts:

ros2 launch fejemis_vision cable_test.launch.py
ros2 launch fejemis_vision people_test.launch.py
ros2 launch fejemis_vision net.launch.py

== Progress ==

The main work done was to convert this package from ROS1 to compatible with ROS 2. Limited testing was done due to limited time, and more modifying needs to be done.

* Each detections worked on offline testing meaning testing with existing images.

* The result testing each detector from camera stream was mixed. People detection and net detection had the most working detection and displaying the markers in rviz. Cable detectors seems random where it some time detected cables and ofte drawed cables on random places and some time

This can look into:

* The detection parameters was not fully tested and twearked, which can be done to improve the noise expecially for cable detectors.

* From people detection only the people_detection_pose_segmentation.py observed to work. There other was not invastigated due to limited time and can be further invastigate.

* In the original ROS jsk-visualization was used for markering in Rviz. However at the time of this project this package was not compatible in ROS2. Instead inbuilt marker array was used to draw the object in Rviz.

To access the old ROS 1 repository: https://github.com/MP-EL/fejemis_vision

← Older revision		Revision as of 20:40, 28 May 2026
Line 22:		Line 22:

	Main launch files:		Main launch files:
	* ''main_launch.py'' - Launches the complete perception system		* ''main_launch.py'' - Launches the complete perception system.

	Testing launch files:		Testing launch files:
	* ''cable_test.launch.py'' - Launches the cable detection pipeline		* ''cable_test.launch.py'' - Launches the cable detection pipeline.
	* ''net_test.launch.py'' - Launches the net detection pipeline		* ''net_test.launch.py'' - Launches the net detection pipeline.
	* ''people_test.launch.py'' - Launches the people detection pipeline		* ''people_test.launch.py'' - Launches the people detection pipeline.

	These launch files initialize the required ROS 2 nodes, camera subscriptions, and RViz visualization.		These launch files initialize the required ROS 2 nodes, camera subscriptions, and RViz visualization.
Line 38:		Line 38:

	Detection scripts:		Detection scripts:
	* ''cable_detection.py'' - Detects cables and linear structures in the environment		* ''cable_detection.py'' - Detects cables and linear structures in the environment.
	* ''net_detection.py'' - Detects net or mesh-like structures using computer vision methods		* ''net_detection.py'' - Detects net or mesh-like structures using computer vision methods.
	* ''people_detection_pose_segmentation.py'' - Human detection using pose estimation and segmentation		* ''people_detection_pose_segmentation.py'' - Human detection using pose estimation and segmentation.
	* ''people_detection_depth_segmentation.py'' - Human detection using depth-based segmentation		* ''people_detection_depth_segmentation.py'' - Human detection using depth-based segmentation.
	* ''people_detection_simple_segmentation.py'' - Simpler segmentation-based human detection approach		* ''people_detection_simple_segmentation.py'' - Simpler segmentation-based human detection approach.

	Tracking:		Tracking:
	* ''people_detection_deep_sort.py'' - Multi-person tracking using the DeepSORT tracking algorithm		* ''people_detection_deep_sort.py'' - Multi-person tracking using the DeepSORT tracking algorithm.

	Testing scripts:		Testing scripts:
Line 68:		Line 68:

	Files include:		Files include:
	* ''yolov4-tiny.cfg'' - YOLOv4-tiny network configuration		* ''yolov4-tiny.cfg'' - YOLOv4-tiny network configuration.
	* ''yolov4-tiny.weights'' - Pretrained YOLOv4-tiny weights		* ''yolov4-tiny.weights'' - Pretrained YOLOv4-tiny weights.
	* ''classes.txt'' - Object class labels used by the detector		* ''classes.txt'' - Object class labels used by the detector.
	* ''mars-small128.pb'' - Feature extraction model used by DeepSORT tracking		* ''mars-small128.pb'' - Feature extraction model used by DeepSORT tracking.

	The package uses YOLOv4-tiny to achieve lightweight real-time object detection suitable for embedded systems such as Raspberry Pi devices.		The package uses YOLOv4-tiny to achieve lightweight real-time object detection suitable for embedded systems such as Raspberry Pi devices.
Line 121:		Line 121:
	=== Current Results ===		=== Current Results ===

	* All detection pipelines were successfully tested offline using stored test images		* All detection pipelines were successfully tested offline using stored test images.
	* Live camera stream testing produced mixed results		* Live camera stream testing produced mixed results.
	* People detection and net detection showed the most stable and reliable performance		* People detection and net detection showed the most stable and reliable performance.
	* RViz marker visualization worked correctly for several detection pipelines		* RViz marker visualization worked correctly for several detection pipelines.
	* Cable detection was significantly more unstable and noisy. The cables were sometimes correctly detected, however false detections and random line generation frequently occurred		* Cable detection was significantly more unstable and noisy. The cables were sometimes correctly detected, however false detections and random line generation frequently occurred.

	=== Areas for Future Improvement ===		=== Areas for Future Improvement ===

	* Detection parameters and thresholds were not fully tuned. Additional tuning may significantly reduce detection noise, especially for cable detection		* Detection parameters and thresholds were not fully tuned. Additional tuning may significantly reduce detection noise, especially for cable detection.
	* Only ''people_detection_pose_segmentation.py'' was verified to work reliably during live testing. The remaining human detection methods were not fully investigated due to time limitations		* Only ''people_detection_pose_segmentation.py'' was verified to work reliably during live testing. The remaining human detection methods were not fully investigated due to time limitations.
	* Additional filtering and depth processing could improve robustness		* Additional filtering and depth processing could improve robustness.
	* Performance optimization may still be required for Raspberry Pi deployment		* Performance optimization may still be required for Raspberry Pi deployment.

	=== RViz Visualization ===		=== RViz Visualization ===