Camera calibration
Back to Robobot
Back to Robobot software description
Camera calibration using openCV
Test pattern
The camera calibration test pattern is found here: https://github.com/opencv/opencv/blob/3.4/doc/pattern.png; print it.
Take test images
The images must also have the test pattern near the corners of the images.
Tape the test pattern on a solid frame, as a curved pattern will ruin the calibration.
Place the pattern in front of the robot.
Check position using motion (Video stremaer).
View the stream using a browser on port 8081, like http://10.197.216.165:8081/, but with the robot IP, move the pattern (or the robot) for an appropriate view:
local@jack:~ $ cd svn/robobot/raubase/build/ local@jack:~/svn/robobot/raubase/build $ motion [0:motion] [NTC] [ALL] conf_load: Processing thread 0 - config file /etc/motion/motion.conf [0:motion] [NTC] [ALL] motion_startup: Logging to file (/home/local/.motion/motion.log) ^C
Stop the video streamer (using ctrl-C) and let the raubase take the image.
local@jack:~/svn/robobot/raubase/build $ ./raubase -i # UService:: created directory log_20240123_094405.808/ # UService::setup: Ignoring robot hardware (Regbot and GPIO) # SpyVision:: disabled in robot.ini # Video device 0: width=1280, height=720, format=MJPG, FPS=25 # Camera is running (to stabilize illumination) # UCam::run: read frame 0/11 # ready to save # found '%' in ini[camera][imageName] # saved image to img/img_raw_20240123_094407.424.jpg # saved image to img/img_rec_20240123_094407.424.jpg # --------- terminating ----------- # UCam::run: camera released # UCam:: logfile closed # UService:: configuration saved to robot.ini
The raubase saves the image in an img subdirectory, and the raw image name starts with img_raw_2024....
Repeat this for 6-10 images with the pattern all around the camera view, something like:
The run the calibration
local@jack:~/svn/robobot/raubase/build $ ./raubase -m # UService:: created directory log_20240123_105137.289/ # UService::setup: Ignoring robot hardware (Regbot and GPIO) # SpyVision:: disabled in robot.ini # Video device 0: width=1280, height=720, format=MJPG, FPS=25 # Camera is running (to stabilize illumination) # ready to calibrate (stopping camera) # UCam::run: camera released # 0 succes img/img_raw_20240123_101039.126.jpg # 1 succes img/img_raw_20240123_103400.651.jpg # 2 succes img/img_raw_20240123_103617.847.jpg # 3 succes img/img_raw_20240123_103723.998.jpg # 4 succes img/img_raw_20240123_103833.044.jpg # 5 succes img/img_raw_20240123_103925.440.jpg # 6 succes img/img_raw_20240123_104011.273.jpg # 7 succes img/img_raw_20240123_105019.243.jpg # Camera matrix 0: 1050.6 0.0 712.6 # Camera matrix 1: 0.0 1046.7 430.6 # Camera matrix 2: 0.0 0.0 1.0 # Distortion k1: -0.397866 # Distortion k2: 0.173319 # Distortion p1: 9.58965e-05 # Distortion p2: -0.000660305 # Distortion k3: -0.0385043 # Average pixel error is 0.26 # Image 0 error 0.29 pixels # Image 1 error 0.20 pixels # Image 2 error 0.34 pixels # Image 3 error 0.32 pixels # Image 4 error 0.26 pixels # Image 5 error 0.16 pixels # Image 6 error 0.19 pixels # Image 7 error 0.23 pixels # --------- terminating ----------- # UCam:: logfile closed # UService:: configuration saved to robot.ini local@jack:~/svn/robobot/raubase/build $
The code for this is in the scam.cpp file in a function called bool UCam::calibrate().
In this case, the average pixel error is calculated to 0.26 pixels.
Camera materix
The result is a camera matrix (CM):
# Camera matrix 0: 1050.6 0.0 712.6 # Camera matrix 1: 0.0 1046.7 430.6 # Camera matrix 2: 0.0 0.0 1.0
The camera matrix gives the pixel position in the image for a given P = (x,y,z) position in the real world in meters (here, x is left, y is down, and z is the distance along the camera centre axis; the camera itself (the focal point) is at (0,0,0) in this coordinate system. i.e.:
Px = CM * P or px/w = 1050.6 * x + 712.6 py/w = 1046.7 * y + 430.6 w = z
Where px and py are the pixel position in the image; w is the scaling at this distance (z) - from meters to pixels; 1050.6 and 1046.7 are the focal lengths (f) in pixels (should be the same value for most cameras), 712.6 and 430.6 are the pixel positions in the centre of the lens. If the camera chip is mounted correctly, this is also the centre of the image.
More details here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html.
Lens distortion
The lens distortion factors are found to be
# Distortion k1: -0.397866 # Distortion k2: 0.173319 # Distortion p1: 9.58965e-05 # Distortion p2: -0.000660305 # Distortion k3: -0.0385043
How to intrepid these can be found here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html
These factors are used to generate a rectified image (saved with the image name img_rec_2024... when saving an image), and a rectified image is available from the scam module function cv::Mat getFrame(), and used like:
cv::Mat frame = cam.getFrame();
The raw distorted frame is also available with:
frame = cam.getFrameRaw();
Configuration file (robot.ini)
The result of the calibration is saved in the configuration file in the camera section:
[camera] device = 0 width = 1280 height = 720 fps = 25 matrix = 1050.6 0.0 712.6 0.0 1046.7 430.6 0.0 0.0 1.0 distortion = -0.397866 0.173319 9.58965e-05 -0.000660305 -0.0385043
