Vision based Line Follower

By Jesper H. Christensen

Hardware needed

For installing and utilizing the line follower application for RegBot robots the following things are needed:

• A fully functional RegBot programmed with latest version of the robot software. Consult this section to ensure that.
• A clean Raspberry Pi (model 2 and above is preferred). In case the Pi is clean with an OS newly installed simply skip the install OS section in Software Installation.
• A Raspberry Pi camera modul.
• A mount for Raspberry Pi + camera module. Consult JCA for this.
• A few female/female jumper wires for connecting the Raspberry Pi with the RegBot.
• A WiFi dongle e.g. EDIMAX EW-7811Un (Raspberry Pi model 3 includes WiFi).

Also it would be good to have access to a "HDMI input"-screen in order to locally make the initial connection to WiFi and setup SSH for remote access.

Software installation

Instructions on how to install the needed software on a Raspberry Pi.

Raspberry Pi

Installing the OS

Use this link to download and install the newest Raspian OS on a SD card. Detailed instructions on how to write an image to a SD card is found within the link as well.

When the OS has been successfully installed insert the SD into the Raspberry Pi - connect a screen via the Pi's HDMI output and a keyboard + mouse set.

Setting up WiFi

Plug in the WiFi dongle and open up a terminal window.

In terminal open the wpa-supplicant configuration file:

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Go to the bottom of the file and enter the information about the WiFi:

network={ 
    ssid="Your_SSID" 
    psk="Your_wifi_password" 
} 

Save the file by pressing Ctrl+X then Y, then finally press Enter.

Restart the Pi by typing sudo reboot and it should automatically connect to WiFi when finished restarting.

Setting up SSH

Make sure SSH is enabled. In terminal type:

sudo raspi-config

Scroll down to the "ssh" option and enable it.

Change the password while at it to e.g. filippa corresponding to the RegBots name - default password is raspberry.

Get the Pi's IP address by typing sudo ifconfig.

Next to the wlan0 entry the inet address is present which is the IP address of the Raspberry Pi, and remote access to the device is available by typing:

On Mac and Linux:

terminal: ssh pi@ip_addr

On Windows:

putty: choose SSH --> pi@ip_addr

By now no screen, mouse or keyboard is needed connected for utilizing the Raspberry Pi.

Assign .local domain to Raspberry Pi

The Raspberry Pi's IP address will frequently change and this will result in spending a lot of time looking up IP addresses of devices. Instead one possibility is to assign a static IP to the device or a .local address to the device, which is the option explained here.

For Linux and Apple computers this should work right out of the box while Windows-users have to download Apple's Bonjour client (note that if Windows-users have iTunes installed then Bonjour is most likely installed as well, as it comes with iTunes). The Bonjour client can be downloaded here.

While still in terminal on Raspberry Pi install Avahi:

sudo apt-get install avahi-daemon

After the installation is done remote access is available by typing:

ssh pi@hostname.local

Default hostname is raspberry. Change the hostname to RegBotXX corresponding to the RegBots number by following these instructions.

Mount remote folder

Working on applications that needs to compile and run locally on the Raspberry Pi it is very handy to mount the working folder locally on a PC or laptop. Doing this will make it possible to develop the code in a preferred IDE, and drag and drop files to/from the device.

On Mac and Linux go to terminal and type:

sshfs pi@hostname.local:/remote/folder ~/local/folder

Remember to create the local folder before mounting to it.

Network Bootstrapper

In order to remotely SSH into the device, the Pi and computer needs to be on the same network. In case the Pi does not know any of the available WiFi networks it will stay unconnected and SSH is not possible.

To overcome this a network bootstrapper must be implemented. Basically all it does is create a AD-HOC network if the Pi does not know or cannot connect to any of the available networks. Connecting the computer to this AD-HOC network opens up SSH access and will allow communication. Use this to put in information of any of the available WiFi networks, or continue as normal on the AD-HOC.

To do this start by installing and configure DHCP Server:

sudo apt-get update
sudo apt-get install isc-dhcp-server

The service will start automatically right after installation and will give an error like:

"Not configured to listen on any interface" 

Configure some defaults for the DHCP server:

sudo nano /etc/default/isc-dhcp-server

Only change needed:

# On what interfaces should the DHCP server (dhcpd) serve DHCP requests?
# Separate multiple interfaces with spaces, e.g. "eth0 eth1". 
INTERFACES="wlan0"

Save the file by pressing Ctrl+X then Y, then finally press Enter.

Now edit the config file for the DHCP server:

sudo nano /etc/dhcp/dhcpd.conf

Edit it to look similar to this:

DHCPDARGS=wlan0; #args for the dhcpd daemon -> limit DHCP to the wlan0 interface
default-lease-time 600;
max-lease-time 7200;

option subnet-mask 255.255.255.0;
option broadcast-address 10.0.0.255;
option domain-name "RegBotxxNet";
option routers 10.0.0.1; #default gateway

subnet 10.0.0.0 netmask 255.255.255.0 {
    range 10.0.0.2 10.0.0.20; #IP range to offer
}
 
#static IP-assignment
host myLaptop {
    hardware ethernet 11:aa:22:bb:33:cc;
    fixed-address 10.0.0.100;
}

Save the file by pressing Ctrl+X then Y, then finally press Enter.

Now update the interfaces config file:

sudo nano /etc/network/interfaces

Should look similar to this:

# start interfaces upon start of the system
auto lo wlan0

# register loopback interface
iface lo inet loopback

# use dhcp and allow interface to be started when kernel detects a hotplug event
allow-hotplug eth0
iface eth0 inet dhcp

# use manual ip configuration for wlan0 interface and allow hotplug as well
allow-hotplug wlan0
iface wlan0 inet manual

Save the file by pressing Ctrl+X then Y, then finally press Enter.

Next up is to edit the rc.local-file. This script is running every time the Raspberry Pi boots:

sudo nano /etc/rc.local

This scripts will be using bash as the interpreter, so the first line of the file needs to be changed to:

#!/bin/bash

Now go to the bottom of the script, but before the exit 0 line, and add:

# RPi Network Conf Bootstrapper

createAdHocNetwork(){
    echo "Creating ad-hoc network"
    ifconfig wlan0 down
    iwconfig wlan0 mode ad-hoc
    iwconfig wlan0 essid RegBotxxNet      #SSID
    ifconfig wlan0 10.0.0.200 netmask 255.255.255.0 up
    /usr/sbin/dhcpd wlan0
    echo "Ad-hoc network created"
}

ssids=( 'MyWlan' 'MyOtherWlan' )
connected=false
for ssid in "${ssids[@]}"
do
    if iwlist wlan0 scan | grep $ssid > /dev/null
    then
        echo "First WiFi in range has SSID:" $ssid
        echo "Starting supplicant for WPA/WPA2"
        wpa_supplicant -B -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.conf > /dev/null 2>&1
        echo "Obtaining IP from DHCP"
        if dhclient -1 wlan0
        then
            echo "Connected to WiFi"
            connected=true
            break
        else
            echo "DHCP server did not respond with an IP lease (DHCPOFFER)"
            wpa_cli terminate
            break
        fi
    else
        echo "Not in range, WiFi with SSID:" $ssid
    fi
done

if ! $connected; then
    createAdHocNetwork
fi

Save the file by pressing Ctrl+X then Y, then finally press Enter.

As the DHCP only needs to be started when created a AD-HOC network type:

sudo update-rc.d -f isc-dhcp-server remove

Finally reboot the system and assuming everything succeeded without any errors the Raspberry Pi will connect to any known network, and if no known networks is available the device will automatically create a AD-HOC network.

sudo reboot

OpenCV 3

This guide will give instructions on how to get OpenCV 3 installed and working on the Raspberry Pi.

If not already SSH into the device.

Start by update and upgrade the Pi:

sudo apt-get update
sudo apt-get upgrade
sudo rpi-update 

sudo reboot 

Install dependencies

When rebooting is complete SSH into the device again and install some developer tools:

sudo apt-get install build-essential git cmake pkg-config

Image I/O packages:

sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev 

Video I/O packages:

sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev

GTK development library to compile the highgui sub-module of OpenCV:

sudo apt-get install libgtk2.0-dev 

Dependencies:

sudo apt-get install libatlas-base-dev gfortran 

Python header files (in case further development or prototyping should take place in python):

sudo apt-get install python2.7-dev python3-dev

OpenCV source code

Get OpenCV 3.0.0 from github and extract in root of Pi:

cd ~
wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.0.0.zip
unzip opencv.zip

Get the opencv_contrib repo as well:

wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.0.0.zip
unzip opencv_contrib.zip

Compile and install OpenCV

cd ~/opencv-3.0.0/
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
  -D CMAKE_INSTALL_PREFIX=/usr/local \
  -D INSTALL_C_EXAMPLES=ON \
  -D INSTALL_PYTHON_EXAMPLES=ON \
  -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.0.0/modules \
  -D BUILD_EXAMPLES=ON ..

Now that the build is all setup, next thing is to compile OpenCV:

make -j4

-j4 stands for the number of cores to use and even when using full 4 cores this can take a very long time.

Assuming no errors occurred when compiling, install it to the system:

sudo make install
sudo ldconfig

Now OpenCV 3.0.0 should be installed on the Raspberry Pi system and ready to use!

Line Follower application

Here are instructions on how to get the Line Follower application up and running on the Raspberry Pi.

Raspberry Pi Camera Module API

First off the RaspiCam API needs to be downloaded.

Get raspicam-0.1.3.zip from sourceforge:

cd ~
curl -L -k http://sourceforge.net/projects/raspicam/files/raspicam-0.1.3.zip/download > raspicam-0.1.3.zip
unzip raspicam-0.1.3.zip
cd raspicam-0.1.3
mkdir build
cd build
cmake ..

Finally compile, install and update the ldconfig

make
sudo make install
sudo ldconfig

Line Follower source code

Grab the latest version of the Line Follower source code and compile:

cd ~
git clone git://github.com/JesperChristensen89/LineFollower.git
cd LineFollower/build/
cmake ..
make

Hopefully no errors should have occured, and the application was created successfully.

Autorun line follower application

In order for the application to run automatically every time the system reboots a mix of tmux and Crontab is used. This will make it possible to SSH into the running instance of the application.

Get tmux:

sudo apt-get tmux

Open up Crontab to make the application run at every boot:

crontab -e

Insert the line:

@reboot tmux new-session -d -s line; tmux new-window -d -n 'linefollow' -t line:1 'sudo /home/pi/LineFollower/build/line_detection'

To save the file press Ctrl+X then Y then finally press Enter.

Reboot the system:

sudo reboot

RegBot integration

In order for Raspberry Pi to run and communicate with the RegBot it needs to be connected via the jumper-wires.

The Raspberry Pi pins is located as the figure shows:

An overview of the RegBot's PCB is:

Use Pin 2 and Pin 6 to supply power:

• Connect Pin 2 to RegBot Sharp IR pin 2.
• Connect Pin 6 to RegBot Sharp IR pin 1.

Use Pin 8, Pin 10 and Pin 14 for serial communication back and forth to the RegBot:

• Connect Pin 8 to RegBot UART pin 4.
• Connect Pin 10 to RegBot UART pin 5.
• Connect Pin 14 to RegBot UART pin 1.

Turn on the system, and the Raspberry Pi should now be connected through the RegBot and all communications to the RegBot is now wireless through the Raspberry Pi's Line Follower application via SSH.

However when uploading to RegBot a wired USB connection to the Teensy board is still needed.