Raspberry Camera API: Difference between revisions

From Rsewiki
No edit summary
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
Back to [[Robobot]]
Back to [[Robobot]]


===Raspicam===
== Camera use ==


To be used from C++ direct or through openCV.
Raspberry pi operating system version "bullseye" uses a new camera library.


A rather short raspberry pi camera API is available from https://www.uco.es/investiga/grupos/ava/node/40.
The info is taken from https://www.raspberrypi.com/documentation/accessories/camera.html#libcamera-and-libcamera-apps?ref=https://githubhelp.com


And can be downloaded from https://sourceforge.net/projects/raspicam/files/,
=== LibCamera ===


or get the file from sourceforge with no GUI, with:
If not already installed, install with


  cd ~/Downloads
  sudo apt install libcamera-apps
wget --no-check-certificate -O raspicam-0.1.9.zip https://downloads.sourceforge.net/project/raspicam/raspicam-0.1.9.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fraspicam%2F%3Fsource%3Dtyp_redirect&ts=1486483484&use_mirror=netix


Once fetched and available on the raspberry, then
It should not be needed to install libcamera-dev
Unpack and install:
unzip raspicam-0.1.9.zip
cd raspicam-0.1.9
mkdir build
cd build
cmake ..
make -j3
sudo make install
sudo ldconfig


The library is installed in /usr/local, and therefore to make cmake find it the path needs to be added to startup configuration, in file ~/.bashrc
==== Camera streaming ====


cd
The libcamera library can do streaming out of the box with the command:
nano ~/.bashrc
add at the end:
export CMAKE_PREFIX_PATH=/usr/local/lib
source ~/.bashrc  # to implement export


Test with openCV
  libcamera-vid -n -t 0 --inline --listen --rotation 0 --framerate 30 --width 1920 --height 1080 -o tcp://0.0.0.0:8888
  raspicam_cv_test
That takes 100 images and saves 3.


=== Camera streamer ===
It will listen to port 8888 for TCP connections on the localhost.


To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
From another PC the stream can be displayed by VLC, (or the same parameter in the VLC "media" -> "open network stream")
sudo apt-get install git
vlc tcp/h264://192.168.2.14:8888


Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.
if 192.168.2.14 is the IP of the Raspberry pi.
The streaming will terminate when the VLC stops the connection.


git clone https://github.com/jacksonliam/mjpg-streamer.git
A start-script is placed in the "local" home folder.


After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.
./stream_cam.sh


cd mjpg-streamer/mjpg-streamer-experimental/
with examples of different solutions.
make -j4    # optional CMAKE_BUILD_TYPE=Debug
sudo make install


Make a script to start the streamer
== camera from OpenCV ==


cd
The raspberry cameras seem to prefer 10 bit/pixel in Bayer RGGB format - other formats are not consistent for both new and old camera.
nano start-stream.sh


copy this into the file (uncomment only one line - the others are examples for other resolutions)
=== Format check ===


#!/bin/bash
Install package v4l-utils (also installed by default)
#mjpg_streamer -i "input_raspicam.so -y 1600 -x 1200 -fps 5 -rot 90" -o output_http.so
mjpg_streamer -i "input_raspicam.so -y 972 -x 1296 -fps 3 -rot=0" -o output_http.so
#mjpg_streamer -i "input_raspicam.so -y 600 -x 800 -fps 3 -rot=0" -o output_http.so
#mjpg_streamer -i "input_raspicam.so -y 640 -x 480 -fps 5" -o output_http.so


Change the file to be executable
sudo apt install v4l-utils


chmod +x start-stream.sh
The supported image formats from both cameras are:


=== Start streaming at boot ===
v4l2-ctl -d/dev/video0 --list-formats


Start of applications can be controlled by the /etc/rc.local script:
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture
        [0]: 'BA81' (8-bit Bayer BGBG/GRGR)
        [1]: 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
        [2]: 'BG10' (10-bit Bayer BGBG/GRGR)


nano /etc/rc.local
For the used format 'BG10' the frame sizes supported is found


add the following lines before 'exit 0' (close to the end)
v4l2-ctl -d /dev/video0 --list-framesizes BG10


  # start camera streaming
  ioctl: VIDIOC_ENUM_FRAMESIZES
su - local /home/local/start-stream.sh &
        Size: Discrete 2592x1944
        Size: Discrete 1920x1080
        Size: Discrete 1296x972
        Size: Discrete 640x480


Copy and 'shift-ctrl-v' will do


Comment the last line (su - local /home/local/start-stream.sh &) if camera streaming should not start.


to stop streaming use (if started)
But formats can be converted to other formats, visible as other video devices e.g /dev/video12.


  pkill mjpg_streamer
  v4l2-ctl -d/dev/video12 --list-formats


ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar
        [0]: 'YUYV' (YUYV 4:2:2)
        [1]: 'YVYU' (YVYU 4:2:2)
        [2]: 'VYUY' (VYUY 4:2:2)
        [3]: 'UYVY' (UYVY 4:2:2)
        [4]: 'YU12' (Planar YUV 4:2:0)
        [5]: 'YV12' (Planar YVU 4:2:0)
        [6]: 'RGB3' (24-bit RGB 8-8-8)
        [7]: 'BGR3' (24-bit BGR 8-8-8)
        [8]: 'BGR4' (32-bit BGRA/X 8-8-8-8)
        [9]: 'RGBP' (16-bit RGB 5-6-5)
        [10]: 'NV12' (Y/CbCr 4:2:0)
        [11]: 'NV21' (Y/CrCb 4:2:0)


=== Watch camera stream ===
Format BA81 did not give a proper result for both cameras.
 
I didn't succeed in using the converted formats, maybe because they are based on the BA81 format.
When the robot is on a net connection - cable or wifi - you can watch the stream from a browser
Format 'BG10' works for both cameras and is easily converted to RGB format using OpenCV (https://docs.opencv.org/3.4/db/d64/tutorial_js_colorspaces.html).
 
for streaming:
HTTP://jasmin.local:8080/?action=stream
or for a snapshot
HTTP://jasmin.local:8080/?action=snapshot
where jasmin should be replaced with the robot name, or jasmin.local can be replaced with the IP address.

Latest revision as of 14:49, 4 January 2022

Back to Robobot

Camera use

Raspberry pi operating system version "bullseye" uses a new camera library.

The info is taken from https://www.raspberrypi.com/documentation/accessories/camera.html#libcamera-and-libcamera-apps?ref=https://githubhelp.com

LibCamera

If not already installed, install with 

sudo apt install libcamera-apps

It should not be needed to install libcamera-dev

Camera streaming

The libcamera library can do streaming out of the box with the command: 

libcamera-vid -n -t 0 --inline --listen --rotation 0 --framerate 30 --width 1920 --height 1080 -o tcp://0.0.0.0:8888

It will listen to port 8888 for TCP connections on the localhost.

From another PC the stream can be displayed by VLC, (or the same parameter in the VLC "media" -> "open network stream") 

vlc tcp/h264://192.168.2.14:8888

if 192.168.2.14 is the IP of the Raspberry pi. The streaming will terminate when the VLC stops the connection.

A start-script is placed in the "local" home folder. 

./stream_cam.sh

with examples of different solutions.

camera from OpenCV

The raspberry cameras seem to prefer 10 bit/pixel in Bayer RGGB format - other formats are not consistent for both new and old camera.

Format check

Install package v4l-utils (also installed by default) 

sudo apt install v4l-utils

The supported image formats from both cameras are: 

v4l2-ctl -d/dev/video0 --list-formats

ioctl: VIDIOC_ENUM_FMT
       Type: Video Capture
       [0]: 'BA81' (8-bit Bayer BGBG/GRGR)
       [1]: 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
       [2]: 'BG10' (10-bit Bayer BGBG/GRGR)

For the used format 'BG10' the frame sizes supported is found 

v4l2-ctl -d /dev/video0 --list-framesizes BG10

ioctl: VIDIOC_ENUM_FRAMESIZES
       Size: Discrete 2592x1944
       Size: Discrete 1920x1080
       Size: Discrete 1296x972
       Size: Discrete 640x480


But formats can be converted to other formats, visible as other video devices e.g /dev/video12. 

v4l2-ctl -d/dev/video12 --list-formats

ioctl: VIDIOC_ENUM_FMT
       Type: Video Capture Multiplanar
       [0]: 'YUYV' (YUYV 4:2:2)
       [1]: 'YVYU' (YVYU 4:2:2)
       [2]: 'VYUY' (VYUY 4:2:2)
       [3]: 'UYVY' (UYVY 4:2:2)
       [4]: 'YU12' (Planar YUV 4:2:0)
       [5]: 'YV12' (Planar YVU 4:2:0)
       [6]: 'RGB3' (24-bit RGB 8-8-8)
       [7]: 'BGR3' (24-bit BGR 8-8-8)
       [8]: 'BGR4' (32-bit BGRA/X 8-8-8-8)
       [9]: 'RGBP' (16-bit RGB 5-6-5)
       [10]: 'NV12' (Y/CbCr 4:2:0)
       [11]: 'NV21' (Y/CrCb 4:2:0)

Format BA81 did not give a proper result for both cameras. I didn't succeed in using the converted formats, maybe because they are based on the BA81 format. Format 'BG10' works for both cameras and is easily converted to RGB format using OpenCV (https://docs.opencv.org/3.4/db/d64/tutorial_js_colorspaces.html).

Retrieved from "https://rsewiki.electro.dtu.dk/index.php?title=Raspberry_Camera_API&oldid=5735"