Back to [[Fejemis 2026]]

== fejemis_vision package ==

The fejemis_vision package is the computer vision subsystem used in the Fejemis robot platform. It is responsible for environmental perception and object detection using RGB and depth camera data.

Main functionalities include:

* Human detection
* Cable detection
* Net detection
* Depth-based segmentation
* Pose estimation
* Object tracking

The package is located in: ''src/fejemis/fejemis_vision''

=== Directory Structure ===

====launch/====
Contains ROS 2 launch files used to start the vision system and individual detection pipelines.

Main launch files:
* ''main-launch.py'' - Launches the complete perception system

Testing launch files:
* ''cable_test.launch.py'' - Launches the cable detection pipeline
* ''net_test.launch.py'' - Launches the net detection pipeline
* ''people_test.launch.py'' - Launches the people detection pipeline

These launch files initialize the required ROS 2 nodes, camera subscriptions, and RViz visualization.

==== pyth/ ====
Contains the main Python source code for the vision and detection system.

Main scripts:
* ''main.py'' - Main ROS 2 vision node coordinating the perception system

Detection scripts:
* ''cable_detection.py'' - Detects cables and linear structures in the environment
* ''net_detection.py'' - Detects net or mesh-like structures using computer vision methods
* ''people_detection_pose_segmentation.py'' - Human detection using pose estimation and segmentation
* ''people_detection_depth_segmentation.py'' - Human detection using depth-based segmentation
* ''people_detection_simple_segmentation.py'' - Simpler segmentation-based human detection approach

Tracking:
* ''people_detection_deep_sort.py'' - Multi-person tracking using the DeepSORT tracking algorithm

Testing scripts:
* ''cable_test.py''
* ''net_test.py''
* ''people_test_basic.py''

These scripts are used for testing and validating the detection pipelines independently.

==== Test Image Directories ====
The package contains several directories with offline test images used during development and debugging:

* ''cable_test_images/''
* ''net_test_images/''
* ''people_test_images/''
* ''other_imgs/''

These images are used to evaluate the detection models without requiring a live camera stream.

==== deep_learning/ ====
Contains neural network models and configuration files used by the detection system.

Files include:
* ''yolov4-tiny.cfg'' - YOLOv4-tiny network configuration
* ''yolov4-tiny.weights'' - Pretrained YOLOv4-tiny weights
* ''classes.txt'' - Object class labels used by the detector
* ''mars-small128.pb'' - Feature extraction model used by DeepSORT tracking

The package uses YOLOv4-tiny to achieve lightweight real-time object detection suitable for embedded systems such as Raspberry Pi devices.

==== config/ ====
Contains configuration files used to tune the perception system.

Typical configuration includes:

* Detection thresholds
* Camera parameters
* Segmentation parameters
* ROS topic settings

These configuration files allow tuning the system without modifying the source code.

== Setup ==

=== Install Dependencies ===

Install OpenCV:
pip install opencv-python

Install the required Python packages:
pip install -r requirements.txt

=== Make Python Scripts Executable ===
Before launching the ROS 2 nodes, ensure the Python scripts have executable permissions:
chmod +x main.py
chmod +x net_test.py
chmod +x people_test_basic.py
chmod +x cable_test.py

=== Launch files ===
To launch the full perception pipeline:
ros2 launch fejemis_vision main.launch.py

To run individual test pipelines:
ros2 launch fejemis_vision cable_test.launch.py
ros2 launch fejemis_vision people_test.launch.py
ros2 launch fejemis_vision net.launch.py

== Progress and Current State ==

A major part of the work involved converting the original package from ROS 1 to ROS 2 compatibility.

Due to limited project time, only partial testing and validation were completed. Additional development and tuning are still required.

=== Current Results ===

* All detection pipelines were successfully tested offline using stored test images
* Live camera stream testing produced mixed results
* People detection and net detection showed the most stable and reliable performance
* RViz marker visualization worked correctly for several detection pipelines
* Cable detection was significantly more unstable and noisy. The cables were sometimes correctly detected and false detections and random line generation frequently occurred

=== Areas for Future Improvement ===

* Detection parameters and thresholds were not fully tuned. Additional tuning may significantly reduce detection noise, especially for cable detection
* Only people_detection_pose_segmentation.py was verified to work reliably during live testing. The remaining human detection methods were not fully investigated due to time limitations
* Additional filtering and depth processing could improve robustness
* Performance optimization may still be required for Raspberry Pi deployment

=== RViz Visualization ===

The original ROS 1 implementation used the jsk-visualization package for RViz markers.

At the time of this project, jsk-visualization was not fully compatible with ROS 2. Therefore, the system was modified to use built-in ROS 2 MarkerArray visualization messages instead.

This allowed detected objects and positions to still be visualized in RViz.

== Original ROS 1 Repository ==

The original ROS 1 implementation can be found here: [https://github.com/MP-EL/fejemis_vision MP-EL/fejemis_vision]

2026-05-27T12:20:30Z

S253734: /* Setup for External Computers */

= Raspberry Pi =

The robot uses two Raspberry Pi 5 units, referred to as the "Main" and "Aux" Raspberry Pi.

The "Main Raspi" is responsible for core robot functionality and launches the odometry, lifecycle management, and bridge-related ROS nodes.

The "Aux Raspi" handles the majority of the remaining ROS launch packages, including the computationally intensive perception and support nodes.

== Hardware Specifications ==

=== Main Raspberry Pi ===

* Raspberry Pi 5
* 16 GB RAM
* 60 GB storage (SD card)

=== Aux Raspberry Pi ===

* Raspberry Pi 5
* 8 GB RAM
* 512 GB external SSD storage

== System Architecture ==

During testing, it was observed that the Main Raspi could not reliably handle all ROS launch packages simultaneously. To improve stability and performance, most workloads were therefore moved to the Aux Raspberry Pi.

One likely reason for this difference is the storage medium. The Main Raspberry Pi uses an SD card, while the Aux Raspberry Pi uses an external SSD, which provides significantly faster read/write performance and better handling of high I/O workloads generated by ROS 2 nodes, logging, and vision processing.

This split architecture allows the robot to distribute computational load more effectively and improves overall system responsiveness and reliability.

== Ethernet Network ==

=== Router ===
The router is used to connect the Raspberry Pi units and external devices on the robot network. It allows communication between the onboard computers and provides Ethernet access for external computers during development, debugging, and monitoring.

The router has a total of five Ethernet ports:

* Two ports are currently occupied by the Main and Aux Raspberry Pi units
* The remaining ports can be used to connect external computers directly to the robot network

Router power supply:

* 5V / 0.6A

=== Network Configuration ===
Network/CIDR:

* 192.168.1.0/24

Subnet mask:

* 255.255.255.0

=== Ethernet IP Addresses ===

* Main Raspberry Pi: 192.168.1.1
* Aux Raspberry Pi: 192.168.1.2

=== Setup for External Computers ===

To connect a computer to the robot network through Ethernet, configure the network interface manually.

The following settings must be configured:

* Subnet mask: 255.255.255.0
* Assign an available static IP address in the range 192.168.1.x

Note:

* 192.168.1.1 and 192.168.1.2 are already assigned to the Raspberry Pi units and must not be reused
* Example valid addresses:
** 192.168.1.10
** 192.168.1.20
** 192.168.1.100

After configuration, the computer should be able to communicate with both Raspberry Pi units over Ethernet.

=== Linear actuator ===

The actuator is a ...

=== Lidar ===

The lidar is a ...

=== Realsense D455 ===

The camera is a ...

=== Teensy Configuration ===

[[File:teensy-configuration.png | 600px]]

Figure: The main hardware blocks. Two Teensy processors are the interface to the hardware. The drive processor controls the drive motors, safety and battery system. The front processor controls the brush and the front wheel to lift the brush. A main PC integrates the functionality with additional sensors to allow autonomous operation.

===== Teensy firmware =====

[[Fejemis Teensy]] software is build using standard Arduino library configurations.
The interface to the main PC is organized as text-lines.

===== Electrical =====

[[Fejemis electrical]] wiring etc.

=== Battery control ===

The [[Fejemis battery control]] is a 24V system (2x3cell LiPo 5Ah) with power on-off and measurement electronics. There is on-board chargers for all batteries.

=== Brush unit ===

The [[Fejemis brush unit]] is a commercial brush that comes with its own battery (12V), charger and motor control. The unit is slightly modified to to allow measurement and control from the software.