Rsewiki - User contributions [en]

Python interface

2020-12-09T10:52:17Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By editing the object by the name, the actual user interface now can also be tweaked by editing the Python code, but this usually requires a more in-depth experience and knowledge of how Python user-interface code works.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program. Its objects don't have any functionality, because the actions have not been programmed yet for them.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined. We sort of define/initialize the functions that are going to be used in the program.

The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration just by reading and sending byte-type network messages from/to the IP adress port.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

$ pip install sockets

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client function connected with the button action when "clicked".
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type, and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP address using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented as well, for example, to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')

'''Note''' '' *To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section right above this section.

It is possible to get a continuous stream of data of the selected parameter by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream. When the subscription has been made, the Pi will provide a continuous stream of replies for the subscribed parameter.

For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field.

'''Note''' ''*If the manual speed is not specified, the remote control is not possible.''

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip for designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

''' How to set up mjpeg streamer on Raspberry-Pi '''

Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img) # Putting image stream on the label
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img) # Putting snapped image on the frame
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image) # Saving the image under the filename

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video recording thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recording status label visible, which indicates that the recording is happening. And the

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting the video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-09T10:41:44Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By editing the object by the name, the actual user interface now can also be tweaked by editing the Python code, but this usually requires a more in-depth experience and knowledge of how Python user-interface code works.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program. Its objects don't have any functionality, because the actions have not been programmed yet for them.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined. We sort of define/initialize the functions that are going to be used in the program.

The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration just by reading and sending byte-type network messages from/to the IP adress port.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

$ pip install sockets

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client function connected with the button action when "clicked".
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type, and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP address using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented as well, for example, to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')

'''Note''' '' *To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section right above this section.

It is possible to get a continuous stream of data of the selected parameter by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream. When the subscription has been made, the Pi will provide a continuous stream of replies for the subscribed parameter.

For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field.

'''Note''' ''*If the manual speed is not specified, the remote control is not possible.''

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip for designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

''' How to set up mjpeg streamer on Raspberry-Pi '''

Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-09T10:41:32Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By editing the object by the name, the actual user interface now can also be tweaked by editing the Python code, but this usually requires a more in-depth experience and knowledge of how Python user-interface code works.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program. Its objects don't have any functionality, because the actions have not been programmed yet for them.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined. We sort of define/initialize the functions that are going to be used in the program.

The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration just by reading and sending byte-type network messages from/to the IP adress port.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

$ pip install sockets

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client function connected with the button action when "clicked".
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type, and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP address using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented as well, for example, to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')

'''Note''' '' *To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section right above this section.

It is possible to get a continuous stream of data of the selected parameter by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream. When the subscription has been made, the Pi will provide a continuous stream of replies for the subscribed parameter.

For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field.

'''Note''' ''*If the manual speed is not specified, the remote control is not possible.''

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip for designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

''' How to set up mjpeg streamer on Raspberry-Pi '''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-09T10:38:04Z

S192320: /* Manual Robot Control from the GUI */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By editing the object by the name, the actual user interface now can also be tweaked by editing the Python code, but this usually requires a more in-depth experience and knowledge of how Python user-interface code works.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program. Its objects don't have any functionality, because the actions have not been programmed yet for them.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined. We sort of define/initialize the functions that are going to be used in the program.

The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration just by reading and sending byte-type network messages from/to the IP adress port.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

$ pip install sockets

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client function connected with the button action when "clicked".
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type, and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP address using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented as well, for example, to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')

'''Note''' '' *To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section right above this section.

It is possible to get a continuous stream of data of the selected parameter by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream. When the subscription has been made, the Pi will provide a continuous stream of replies for the subscribed parameter.

For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field.

'''Note''' ''*If the manual speed is not specified, the remote control is not possible.''

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-09T10:28:48Z

S192320: /* Receiving, Decoding and Translating received messages */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By editing the object by the name, the actual user interface now can also be tweaked by editing the Python code, but this usually requires a more in-depth experience and knowledge of how Python user-interface code works.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program. Its objects don't have any functionality, because the actions have not been programmed yet for them.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined. We sort of define/initialize the functions that are going to be used in the program.

The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration just by reading and sending byte-type network messages from/to the IP adress port.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

$ pip install sockets

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client function connected with the button action when "clicked".
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type, and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP address using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented as well, for example, to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')

'''Note''' '' *To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section right above this section.

It is possible to get a continuous stream of data of the selected parameter by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream. When the subscription has been made, the Pi will provide a continuous stream of replies for the subscribed parameter.

For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-09T10:19:27Z

S192320: /* Programming GUI */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By editing the object by the name, the actual user interface now can also be tweaked by editing the Python code, but this usually requires a more in-depth experience and knowledge of how Python user-interface code works.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program. Its objects don't have any functionality, because the actions have not been programmed yet for them.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined. We sort of define/initialize the functions that are going to be used in the program.

The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration just by reading and sending byte-type network messages from/to the IP adress port.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

$ pip install sockets

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client function connected with the button action when "clicked".
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type, and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP address using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented as well, for example, to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')

'''Note''' '' *To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-09T10:18:40Z

S192320: /* Programming GUI */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By editing the object by the name, the actual user interface now can also be tweaked by editing the Python code, but this usually requires a more in-depth experience and knowledge of how Python user-interface code works.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program. Its objects don't have any functionality, because the actions have not been programmed yet for them.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined. We sort of define/initialize the functions that are going to be used in the program.

The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration just by reading and sending byte-type network messages from/to the IP adress port.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

$ pip install sockets

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client function connected with the button action when "clicked".
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type, and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP address using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented as well, for example, to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')

'''Note'''''' *To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-09T10:04:38Z

S192320: /* Qt GUI interface design */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By editing the object by the name, the actual user interface now can also be tweaked by editing the Python code, but this usually requires a more in-depth experience and knowledge of how Python user-interface code works.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program. Its objects don't have any functionality, because the actions have not been programmed yet for them.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined. We sort of define/initialize the functions that are going to be used in the program.

The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-09T09:59:30Z

S192320: /* Qt GUI interface design */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface. The framework provides the developer with the most common user-interface objects, however, it might be lacking some of the objects that other frameworks support, like basic shapes.

This user interface was designed using Python 3.7.9 and Qt version 5.

'''Note'''''*To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

When your user-interface is finished and is saved as .ui file from The Qt designer, the file is ready to be translated to C++ or Python interface code.
The .ui file will be converted to .py file with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

'''Note''''' *This file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.''

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Source Code

2020-12-06T17:45:22Z

S192320: /* User Interface Code */

==User Interface Code==

==Main Application Code==

==Linesensor==

==IR distance sensors==

==Servo==

==Video Threads==

Code

2020-12-06T17:45:10Z

S192320: /* The User Interface */

Code

2020-12-06T17:44:39Z

S192320: Created page with "==The User Interface== # -*- coding: utf-8 -*- # Form implementation generated from reading ui file 'Regbot_V1_9.ui' # # Created by: PyQt5 UI code generator 5...."

==The User Interface==

# -*- coding: utf-8 -*-

# Form implementation generated from reading ui file 'Regbot_V1_9.ui'
#
# Created by: PyQt5 UI code generator 5.13.2
#
# WARNING! All changes made in this file will be lost!

from PyQt5 import QtCore, QtGui, QtWidgets

class Ui_MainWindow(object):
def setupUi(self, MainWindow):
MainWindow.setObjectName("MainWindow")
MainWindow.resize(1122, 555)
self.centralwidget = QtWidgets.QWidget(MainWindow)
self.centralwidget.setObjectName("centralwidget")
self.SidebarMenu = QtWidgets.QToolBox(self.centralwidget)
self.SidebarMenu.setGeometry(QtCore.QRect(0, 0, 331, 721))
self.SidebarMenu.setAutoFillBackground(True)
self.SidebarMenu.setObjectName("SidebarMenu")
self.SidebarMenuPage1 = QtWidgets.QWidget()
self.SidebarMenuPage1.setGeometry(QtCore.QRect(0, 0, 331, 640))
self.SidebarMenuPage1.setObjectName("SidebarMenuPage1")
self.line = QtWidgets.QFrame(self.SidebarMenuPage1)
self.line.setGeometry(QtCore.QRect(10, 220, 311, 21))
self.line.setFrameShape(QtWidgets.QFrame.HLine)
self.line.setFrameShadow(QtWidgets.QFrame.Sunken)
self.line.setObjectName("line")
self.IP_Connect_frame = QtWidgets.QFrame(self.SidebarMenuPage1)
self.IP_Connect_frame.setGeometry(QtCore.QRect(0, 0, 331, 111))
self.IP_Connect_frame.setAutoFillBackground(True)
self.IP_Connect_frame.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.IP_Connect_frame.setFrameShadow(QtWidgets.QFrame.Raised)
self.IP_Connect_frame.setObjectName("IP_Connect_frame")
self.label_2 = QtWidgets.QLabel(self.IP_Connect_frame)
self.label_2.setGeometry(QtCore.QRect(30, 90, 91, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_2.setFont(font)
self.label_2.setObjectName("label_2")
self.Disconnect_Cmd = QtWidgets.QPushButton(self.IP_Connect_frame)
self.Disconnect_Cmd.setGeometry(QtCore.QRect(180, 60, 121, 23))
self.Disconnect_Cmd.setObjectName("Disconnect_Cmd")
self.connect_Cmd = QtWidgets.QPushButton(self.IP_Connect_frame)
self.connect_Cmd.setGeometry(QtCore.QRect(30, 60, 131, 23))
self.connect_Cmd.setObjectName("connect_Cmd")
self.label = QtWidgets.QLabel(self.IP_Connect_frame)
self.label.setGeometry(QtCore.QRect(120, 30, 61, 21))
self.label.setObjectName("label")
self.IP_Adress_field = QtWidgets.QLineEdit(self.IP_Connect_frame)
self.IP_Adress_field.setGeometry(QtCore.QRect(170, 30, 131, 20))
self.IP_Adress_field.setObjectName("IP_Adress_field")
self.IP_NetworkStatus_label = QtWidgets.QLabel(self.IP_Connect_frame)
self.IP_NetworkStatus_label.setGeometry(QtCore.QRect(140, 90, 141, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.IP_NetworkStatus_label.setFont(font)
self.IP_NetworkStatus_label.setObjectName("IP_NetworkStatus_label")
self.label_7 = QtWidgets.QLabel(self.IP_Connect_frame)
self.label_7.setGeometry(QtCore.QRect(10, 10, 131, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_7.setFont(font)
self.label_7.setObjectName("label_7")
self.SidebarMenu.addItem(self.SidebarMenuPage1, "")
self.SidebarMenuPage2 = QtWidgets.QWidget()
self.SidebarMenuPage2.setGeometry(QtCore.QRect(0, 0, 331, 640))
self.SidebarMenuPage2.setObjectName("SidebarMenuPage2")
self.TextCmd_field = QtWidgets.QLineEdit(self.SidebarMenuPage2)
self.TextCmd_field.setGeometry(QtCore.QRect(10, 580, 311, 20))
self.TextCmd_field.setObjectName("TextCmd_field")
self.send_Cmd = QtWidgets.QPushButton(self.SidebarMenuPage2)
self.send_Cmd.setGeometry(QtCore.QRect(240, 610, 81, 23))
self.send_Cmd.setObjectName("send_Cmd")
self.TX_checkBox = QtWidgets.QCheckBox(self.SidebarMenuPage2)
self.TX_checkBox.setGeometry(QtCore.QRect(10, 610, 41, 17))
self.TX_checkBox.setObjectName("TX_checkBox")
self.RX_checkBox = QtWidgets.QCheckBox(self.SidebarMenuPage2)
self.RX_checkBox.setGeometry(QtCore.QRect(50, 610, 41, 17))
self.RX_checkBox.setObjectName("RX_checkBox")
self.ConsoleHelp_Cmd = QtWidgets.QPushButton(self.SidebarMenuPage2)
self.ConsoleHelp_Cmd.setGeometry(QtCore.QRect(150, 610, 81, 23))
self.ConsoleHelp_Cmd.setObjectName("ConsoleHelp_Cmd")
self.CommandConsole_Window = QtWidgets.QPlainTextEdit(self.SidebarMenuPage2)
self.CommandConsole_Window.setGeometry(QtCore.QRect(10, 0, 311, 571))
self.CommandConsole_Window.setAcceptDrops(False)
self.CommandConsole_Window.setToolTip("")
self.CommandConsole_Window.setHorizontalScrollBarPolicy(QtCore.Qt.ScrollBarAsNeeded)
self.CommandConsole_Window.setLineWrapMode(QtWidgets.QPlainTextEdit.NoWrap)
self.CommandConsole_Window.setObjectName("CommandConsole_Window")
self.ConsoleClear_Cmd = QtWidgets.QPushButton(self.SidebarMenuPage2)
self.ConsoleClear_Cmd.setGeometry(QtCore.QRect(90, 610, 51, 23))
self.ConsoleClear_Cmd.setObjectName("ConsoleClear_Cmd")
self.SidebarMenu.addItem(self.SidebarMenuPage2, "")
self.SidebarMenuPage3 = QtWidgets.QWidget()
self.SidebarMenuPage3.setGeometry(QtCore.QRect(0, 0, 331, 640))
self.SidebarMenuPage3.setObjectName("SidebarMenuPage3")
self.main_start = QtWidgets.QPushButton(self.SidebarMenuPage3)
self.main_start.setGeometry(QtCore.QRect(170, 60, 151, 23))
self.main_start.setObjectName("main_start")
self.main_remote_ctrl = QtWidgets.QCheckBox(self.SidebarMenuPage3)
self.main_remote_ctrl.setGeometry(QtCore.QRect(10, 30, 141, 17))
self.main_remote_ctrl.setTristate(True)
self.main_remote_ctrl.setObjectName("main_remote_ctrl")
self.main_stop = QtWidgets.QPushButton(self.SidebarMenuPage3)
self.main_stop.setGeometry(QtCore.QRect(10, 60, 151, 23))
self.main_stop.setObjectName("main_stop")
self.main_mission_state = QtWidgets.QCheckBox(self.SidebarMenuPage3)
self.main_mission_state.setGeometry(QtCore.QRect(10, 10, 101, 17))
self.main_mission_state.setTristate(True)
self.main_mission_state.setObjectName("main_mission_state")
self.SidebarMenu.addItem(self.SidebarMenuPage3, "")
self.tabWidget = QtWidgets.QTabWidget(self.centralwidget)
self.tabWidget.setGeometry(QtCore.QRect(330, 0, 961, 721))
self.tabWidget.setAutoFillBackground(True)
self.tabWidget.setObjectName("tabWidget")
self.tab = QtWidgets.QWidget()
self.tab.setObjectName("tab")
self.MotorGUI_ctrl_frame = QtWidgets.QFrame(self.tab)
self.MotorGUI_ctrl_frame.setGeometry(QtCore.QRect(0, 0, 321, 171))
self.MotorGUI_ctrl_frame.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.MotorGUI_ctrl_frame.setFrameShadow(QtWidgets.QFrame.Raised)
self.MotorGUI_ctrl_frame.setObjectName("MotorGUI_ctrl_frame")
self.label_5 = QtWidgets.QLabel(self.MotorGUI_ctrl_frame)
self.label_5.setGeometry(QtCore.QRect(140, 100, 71, 21))
self.label_5.setObjectName("label_5")
self.enableKeyboardRC_checkBox = QtWidgets.QCheckBox(self.MotorGUI_ctrl_frame)
self.enableKeyboardRC_checkBox.setGeometry(QtCore.QRect(150, 40, 141, 17))
self.enableKeyboardRC_checkBox.setObjectName("enableKeyboardRC_checkBox")
self.guiRC_Reverse_Cmd = QtWidgets.QPushButton(self.MotorGUI_ctrl_frame)
self.guiRC_Reverse_Cmd.setGeometry(QtCore.QRect(50, 110, 31, 31))
self.guiRC_Reverse_Cmd.setObjectName("guiRC_Reverse_Cmd")
self.guiRC_Forward_Cmd = QtWidgets.QPushButton(self.MotorGUI_ctrl_frame)
self.guiRC_Forward_Cmd.setGeometry(QtCore.QRect(50, 50, 31, 31))
self.guiRC_Forward_Cmd.setObjectName("guiRC_Forward_Cmd")
self.manualGuiSpeed_Cmd = QtWidgets.QLineEdit(self.MotorGUI_ctrl_frame)
self.manualGuiSpeed_Cmd.setGeometry(QtCore.QRect(220, 100, 51, 20))
self.manualGuiSpeed_Cmd.setObjectName("manualGuiSpeed_Cmd")
self.enableGuiRC_checkBox = QtWidgets.QCheckBox(self.MotorGUI_ctrl_frame)
self.enableGuiRC_checkBox.setGeometry(QtCore.QRect(150, 60, 111, 17))
self.enableGuiRC_checkBox.setObjectName("enableGuiRC_checkBox")
self.guiRC_Right_Cmd = QtWidgets.QPushButton(self.MotorGUI_ctrl_frame)
self.guiRC_Right_Cmd.setGeometry(QtCore.QRect(80, 80, 31, 31))
self.guiRC_Right_Cmd.setObjectName("guiRC_Right_Cmd")
self.guiRC_Left_Cmd = QtWidgets.QPushButton(self.MotorGUI_ctrl_frame)
self.guiRC_Left_Cmd.setGeometry(QtCore.QRect(20, 80, 31, 31))
self.guiRC_Left_Cmd.setObjectName("guiRC_Left_Cmd")
self.label_8 = QtWidgets.QLabel(self.MotorGUI_ctrl_frame)
self.label_8.setGeometry(QtCore.QRect(10, 10, 151, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_8.setFont(font)
self.label_8.setObjectName("label_8")
self.frame = QtWidgets.QFrame(self.tab)
self.frame.setGeometry(QtCore.QRect(0, 170, 321, 321))
self.frame.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame.setObjectName("frame")
self.label_9 = QtWidgets.QLabel(self.frame)
self.label_9.setGeometry(QtCore.QRect(10, 10, 201, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_9.setFont(font)
self.label_9.setObjectName("label_9")
self.label_21 = QtWidgets.QLabel(self.frame)
self.label_21.setGeometry(QtCore.QRect(10, 149, 91, 21))
self.label_21.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_21.setObjectName("label_21")
self.Robot_WheelRadius_Right = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_WheelRadius_Right.setGeometry(QtCore.QRect(109, 230, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_WheelRadius_Right.sizePolicy().hasHeightForWidth())
self.Robot_WheelRadius_Right.setSizePolicy(sizePolicy)
self.Robot_WheelRadius_Right.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_WheelRadius_Right.setToolTip("<html><head/><body>Wheel radius</body></html>")
self.Robot_WheelRadius_Right.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelRadius_Right.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelRadius_Right.setReadOnly(False)
self.Robot_WheelRadius_Right.setDecimals(4)
self.Robot_WheelRadius_Right.setMinimum(0.001)
self.Robot_WheelRadius_Right.setMaximum(0.99)
self.Robot_WheelRadius_Right.setSingleStep(0.001)
self.Robot_WheelRadius_Right.setProperty("value", 0.03)
self.Robot_WheelRadius_Right.setObjectName("Robot_WheelRadius_Right")
self.Robot_WheelRadius_Left = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_WheelRadius_Left.setGeometry(QtCore.QRect(109, 190, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_WheelRadius_Left.sizePolicy().hasHeightForWidth())
self.Robot_WheelRadius_Left.setSizePolicy(sizePolicy)
self.Robot_WheelRadius_Left.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_WheelRadius_Left.setToolTip("<html><head/><body>Wheel radius</body></html>")
self.Robot_WheelRadius_Left.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelRadius_Left.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelRadius_Left.setReadOnly(False)
self.Robot_WheelRadius_Left.setDecimals(4)
self.Robot_WheelRadius_Left.setMinimum(0.001)
self.Robot_WheelRadius_Left.setMaximum(0.99)
self.Robot_WheelRadius_Left.setSingleStep(0.001)
self.Robot_WheelRadius_Left.setProperty("value", 0.03)
self.Robot_WheelRadius_Left.setObjectName("Robot_WheelRadius_Left")
self.label_22 = QtWidgets.QLabel(self.frame)
self.label_22.setGeometry(QtCore.QRect(10, 118, 91, 23))
self.label_22.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_22.setObjectName("label_22")
self.Robot_GearRatio = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_GearRatio.setGeometry(QtCore.QRect(110, 120, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_GearRatio.sizePolicy().hasHeightForWidth())
self.Robot_GearRatio.setSizePolicy(sizePolicy)
self.Robot_GearRatio.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_GearRatio.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_GearRatio.setReadOnly(False)
self.Robot_GearRatio.setDecimals(3)
self.Robot_GearRatio.setMaximum(999.99)
self.Robot_GearRatio.setSingleStep(0.001)
self.Robot_GearRatio.setProperty("value", 9.68)
self.Robot_GearRatio.setObjectName("Robot_GearRatio")
self.label_24 = QtWidgets.QLabel(self.frame)
self.label_24.setGeometry(QtCore.QRect(10, 169, 121, 20))
self.label_24.setAlignment(QtCore.Qt.AlignCenter)
self.label_24.setObjectName("label_24")
self.label_127 = QtWidgets.QLabel(self.frame)
self.label_127.setGeometry(QtCore.QRect(50, 30, 51, 20))
self.label_127.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_127.setObjectName("label_127")
self.Robot_WheelBase = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_WheelBase.setGeometry(QtCore.QRect(110, 90, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_WheelBase.sizePolicy().hasHeightForWidth())
self.Robot_WheelBase.setSizePolicy(sizePolicy)
self.Robot_WheelBase.setToolTip("<html><head/><body>Distance between drivng wheels (0.155m)</body></html>")
self.Robot_WheelBase.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelBase.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelBase.setReadOnly(False)
self.Robot_WheelBase.setDecimals(4)
self.Robot_WheelBase.setMaximum(5.0)
self.Robot_WheelBase.setSingleStep(0.01)
self.Robot_WheelBase.setProperty("value", 0.155)
self.Robot_WheelBase.setObjectName("Robot_WheelBase")
self.label_128 = QtWidgets.QLabel(self.frame)
self.label_128.setGeometry(QtCore.QRect(50, 60, 51, 20))
self.label_128.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_128.setObjectName("label_128")
self.label_25 = QtWidgets.QLabel(self.frame)
self.label_25.setGeometry(QtCore.QRect(20, 90, 81, 21))
self.label_25.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_25.setObjectName("label_25")
self.RobotID = QtWidgets.QDoubleSpinBox(self.frame)
self.RobotID.setEnabled(True)
self.RobotID.setGeometry(QtCore.QRect(110, 31, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.RobotID.sizePolicy().hasHeightForWidth())
self.RobotID.setSizePolicy(sizePolicy)
self.RobotID.setToolTip("<html><head/><body>This is the ID of the robot [1..9999].To change the ID of robot: change ID, click "save", click "save on robot".Change ID to -1 to revert to factory settings. And then change ID back to right value</body></html>")
self.RobotID.setStatusTip("")
self.RobotID.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.RobotID.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.RobotID.setReadOnly(False)
self.RobotID.setSpecialValueText("")
self.RobotID.setSuffix("")
self.RobotID.setDecimals(0)
self.RobotID.setMinimum(-2.0)
self.RobotID.setMaximum(10000.0)
self.RobotID.setObjectName("RobotID")
self.RobotHW_type = QtWidgets.QDoubleSpinBox(self.frame)
self.RobotHW_type.setEnabled(True)
self.RobotHW_type.setGeometry(QtCore.QRect(110, 60, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.RobotHW_type.sizePolicy().hasHeightForWidth())
self.RobotHW_type.setSizePolicy(sizePolicy)
self.RobotHW_type.setToolTip("<html><head/><body>The hardware version1 = old version with no wifi nor line sensor plugs on main board and round power supply plug on main board.2 = more plugs and satellite PCB for power management and wifi.3 = power management and on-off switch on-board (big motor controller board)4 = same as 3, but with small motor controller board.5 = same as 2, but updated with new sattelite boards (power and wifi) etc.6 = hardware version 4 (motor controller integrated on board)To change: click edit change HW setting, click "save", click "save to flash".(ID must be > 0)</body></html>")
self.RobotHW_type.setStatusTip("")
self.RobotHW_type.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.RobotHW_type.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.RobotHW_type.setReadOnly(False)
self.RobotHW_type.setSpecialValueText("")
self.RobotHW_type.setSuffix("")
self.RobotHW_type.setDecimals(0)
self.RobotHW_type.setMinimum(-2.0)
self.RobotHW_type.setMaximum(10.0)
self.RobotHW_type.setObjectName("RobotHW_type")
self.Robot_PulsePerRev = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_PulsePerRev.setGeometry(QtCore.QRect(110, 150, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_PulsePerRev.sizePolicy().hasHeightForWidth())
self.Robot_PulsePerRev.setSizePolicy(sizePolicy)
self.Robot_PulsePerRev.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PulsePerRev.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PulsePerRev.setReadOnly(False)
self.Robot_PulsePerRev.setDecimals(0)
self.Robot_PulsePerRev.setMinimum(1.0)
self.Robot_PulsePerRev.setMaximum(1000.0)
self.Robot_PulsePerRev.setProperty("value", 48.0)
self.Robot_PulsePerRev.setObjectName("Robot_PulsePerRev")
self.label_26 = QtWidgets.QLabel(self.frame)
self.label_26.setGeometry(QtCore.QRect(10, 210, 121, 20))
self.label_26.setAlignment(QtCore.Qt.AlignCenter)
self.label_26.setObjectName("label_26")
self.Robot_RevEncoder_Right = QtWidgets.QCheckBox(self.frame)
self.Robot_RevEncoder_Right.setGeometry(QtCore.QRect(190, 90, 131, 21))
self.Robot_RevEncoder_Right.setObjectName("Robot_RevEncoder_Right")
self.Robot_RevMotor = QtWidgets.QCheckBox(self.frame)
self.Robot_RevMotor.setGeometry(QtCore.QRect(190, 33, 92, 16))
self.Robot_RevMotor.setObjectName("Robot_RevMotor")
self.Robot_RevEncoder_Left = QtWidgets.QCheckBox(self.frame)
self.Robot_RevEncoder_Left.setGeometry(QtCore.QRect(190, 60, 131, 21))
self.Robot_RevEncoder_Left.setObjectName("Robot_RevEncoder_Left")
self.Robot_BatteryIdleVolts = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_BatteryIdleVolts.setGeometry(QtCore.QRect(110, 290, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_BatteryIdleVolts.sizePolicy().hasHeightForWidth())
self.Robot_BatteryIdleVolts.setSizePolicy(sizePolicy)
self.Robot_BatteryIdleVolts.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_BatteryIdleVolts.setToolTip("<html><head/><body>Go idle when battery voltage is below this level.For a 3 cell LiPo battery it should go in idle mode at 3 x 3.2V = 9.6VAt 10V about 10% capacity is left before battery destruction, and in idle mode the robot uses about 30mA, so battey will selfdestruct after another about 3 hours.So remember to switch off and to recharge.</body></html>")
self.Robot_BatteryIdleVolts.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_BatteryIdleVolts.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_BatteryIdleVolts.setReadOnly(False)
self.Robot_BatteryIdleVolts.setDecimals(1)
self.Robot_BatteryIdleVolts.setMinimum(9.0)
self.Robot_BatteryIdleVolts.setMaximum(17.0)
self.Robot_BatteryIdleVolts.setSingleStep(0.1)
self.Robot_BatteryIdleVolts.setProperty("value", 9.9)
self.Robot_BatteryIdleVolts.setObjectName("Robot_BatteryIdleVolts")
self.label_222 = QtWidgets.QLabel(self.frame)
self.label_222.setGeometry(QtCore.QRect(20, 290, 81, 20))
self.label_222.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_222.setObjectName("label_222")
self.save_id_on_robot = QtWidgets.QPushButton(self.frame)
self.save_id_on_robot.setEnabled(False)
self.save_id_on_robot.setGeometry(QtCore.QRect(210, 230, 92, 23))
self.save_id_on_robot.setToolTip("<html><head/><body>Save ID and robot specific settings (this frame) on robot. NB! you need to press "save on robot" in configuration block to save into EE-prom (flash)</body></html>")
self.save_id_on_robot.setCheckable(False)
self.save_id_on_robot.setChecked(False)
self.save_id_on_robot.setObjectName("save_id_on_robot")
self.label_129 = QtWidgets.QLabel(self.frame)
self.label_129.setGeometry(QtCore.QRect(10, 265, 91, 21))
self.label_129.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_129.setObjectName("label_129")
self.Robot_BalanceOffset = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_BalanceOffset.setGeometry(QtCore.QRect(110, 263, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_BalanceOffset.sizePolicy().hasHeightForWidth())
self.Robot_BalanceOffset.setSizePolicy(sizePolicy)
self.Robot_BalanceOffset.setToolTip("<html><head/><body>Balance offset - where center of gravity (COG) is just above support point (wheels).Angle is in radians.Positive is tilt forward.</body></html>")
self.Robot_BalanceOffset.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_BalanceOffset.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_BalanceOffset.setReadOnly(False)
self.Robot_BalanceOffset.setDecimals(4)
self.Robot_BalanceOffset.setMinimum(-1.0)
self.Robot_BalanceOffset.setMaximum(1.0)
self.Robot_BalanceOffset.setSingleStep(0.001)
self.Robot_BalanceOffset.setObjectName("Robot_BalanceOffset")
self.Robot_edit = QtWidgets.QPushButton(self.frame)
self.Robot_edit.setEnabled(True)
self.Robot_edit.setGeometry(QtCore.QRect(210, 259, 92, 23))
self.Robot_edit.setToolTip("<html><head/><body>Switch to edit mode - rather than listen to robot messages - for these fields.</body></html>")
self.Robot_edit.setCheckable(False)
self.Robot_edit.setChecked(False)
self.Robot_edit.setObjectName("Robot_edit")
self.Robot_OnBattery = QtWidgets.QCheckBox(self.frame)
self.Robot_OnBattery.setGeometry(QtCore.QRect(190, 120, 111, 21))
self.Robot_OnBattery.setObjectName("Robot_OnBattery")
self.robot_cancel = QtWidgets.QPushButton(self.frame)
self.robot_cancel.setEnabled(False)
self.robot_cancel.setGeometry(QtCore.QRect(210, 288, 92, 23))
self.robot_cancel.setToolTip("<html><head/><body>Revert to values on robot</body></html>")
self.robot_cancel.setCheckable(False)
self.robot_cancel.setChecked(False)
self.robot_cancel.setObjectName("robot_cancel")
self.tabWidget_2 = QtWidgets.QTabWidget(self.tab)
self.tabWidget_2.setGeometry(QtCore.QRect(0, 490, 321, 211))
self.tabWidget_2.setObjectName("tabWidget_2")
self.tab_6 = QtWidgets.QWidget()
self.tab_6.setObjectName("tab_6")
self.groupBox = QtWidgets.QGroupBox(self.tab_6)
self.groupBox.setGeometry(QtCore.QRect(0, 0, 311, 181))
self.groupBox.setTitle("")
self.groupBox.setObjectName("groupBox")
self.Robot_CurrentLeft_Value = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_CurrentLeft_Value.setGeometry(QtCore.QRect(110, 100, 71, 21))
self.Robot_CurrentLeft_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_CurrentLeft_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_CurrentLeft_Value.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_CurrentLeft_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_CurrentLeft_Value.setReadOnly(True)
self.Robot_CurrentLeft_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_CurrentLeft_Value.setDecimals(2)
self.Robot_CurrentLeft_Value.setMinimum(-100000.0)
self.Robot_CurrentLeft_Value.setMaximum(100000.0)
self.Robot_CurrentLeft_Value.setObjectName("Robot_CurrentLeft_Value")
self.Robot_Distance = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_Distance.setGeometry(QtCore.QRect(110, 130, 71, 21))
self.Robot_Distance.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_Distance.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_Distance.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_Distance.setReadOnly(True)
self.Robot_Distance.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_Distance.setDecimals(3)
self.Robot_Distance.setMinimum(-999999.0)
self.Robot_Distance.setMaximum(999999.99)
self.Robot_Distance.setObjectName("Robot_Distance")
self.label_36 = QtWidgets.QLabel(self.groupBox)
self.label_36.setGeometry(QtCore.QRect(186, 24, 71, 16))
self.label_36.setAlignment(QtCore.Qt.AlignCenter)
self.label_36.setObjectName("label_36")
self.label_37 = QtWidgets.QLabel(self.groupBox)
self.label_37.setGeometry(QtCore.QRect(106, 24, 71, 16))
self.label_37.setAlignment(QtCore.Qt.AlignCenter)
self.label_37.setObjectName("label_37")
self.label_42 = QtWidgets.QLabel(self.groupBox)
self.label_42.setGeometry(QtCore.QRect(11, 100, 91, 16))
self.label_42.setMinimumSize(QtCore.QSize(0, 0))
self.label_42.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_42.setObjectName("label_42")
self.Robot_EncLeft_Value = QtWidgets.QSpinBox(self.groupBox)
self.Robot_EncLeft_Value.setGeometry(QtCore.QRect(110, 40, 71, 21))
self.Robot_EncLeft_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_EncLeft_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_EncLeft_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_EncLeft_Value.setReadOnly(True)
self.Robot_EncLeft_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_EncLeft_Value.setMinimum(-2147483647)
self.Robot_EncLeft_Value.setMaximum(2147483647)
self.Robot_EncLeft_Value.setObjectName("Robot_EncLeft_Value")
self.Robot_CurrentRight_Value = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_CurrentRight_Value.setGeometry(QtCore.QRect(190, 100, 71, 21))
self.Robot_CurrentRight_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_CurrentRight_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_CurrentRight_Value.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_CurrentRight_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_CurrentRight_Value.setReadOnly(True)
self.Robot_CurrentRight_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_CurrentRight_Value.setDecimals(2)
self.Robot_CurrentRight_Value.setMinimum(-100000.0)
self.Robot_CurrentRight_Value.setMaximum(100000.0)
self.Robot_CurrentRight_Value.setObjectName("Robot_CurrentRight_Value")
self.label_28 = QtWidgets.QLabel(self.groupBox)
self.label_28.setGeometry(QtCore.QRect(11, 40, 91, 16))
self.label_28.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_28.setObjectName("label_28")
self.label_38 = QtWidgets.QLabel(self.groupBox)
self.label_38.setGeometry(QtCore.QRect(9, 5, 121, 21))
font = QtGui.QFont()
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_38.setFont(font)
self.label_38.setObjectName("label_38")
self.label_32 = QtWidgets.QLabel(self.groupBox)
self.label_32.setGeometry(QtCore.QRect(10, 130, 91, 16))
self.label_32.setMinimumSize(QtCore.QSize(0, 0))
self.label_32.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_32.setObjectName("label_32")
self.label_29 = QtWidgets.QLabel(self.groupBox)
self.label_29.setGeometry(QtCore.QRect(1, 70, 101, 16))
self.label_29.setMinimumSize(QtCore.QSize(0, 0))
self.label_29.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_29.setObjectName("label_29")
self.Robot_EncRight_Value = QtWidgets.QSpinBox(self.groupBox)
self.Robot_EncRight_Value.setGeometry(QtCore.QRect(190, 40, 71, 21))
self.Robot_EncRight_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_EncRight_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_EncRight_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_EncRight_Value.setReadOnly(True)
self.Robot_EncRight_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_EncRight_Value.setMinimum(-2147483647)
self.Robot_EncRight_Value.setMaximum(2147483647)
self.Robot_EncRight_Value.setObjectName("Robot_EncRight_Value")
self.Robot_WheelRightSpeed_Value = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_WheelRightSpeed_Value.setGeometry(QtCore.QRect(190, 70, 71, 21))
self.Robot_WheelRightSpeed_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_WheelRightSpeed_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_WheelRightSpeed_Value.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelRightSpeed_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelRightSpeed_Value.setReadOnly(True)
self.Robot_WheelRightSpeed_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_WheelRightSpeed_Value.setDecimals(3)
self.Robot_WheelRightSpeed_Value.setMinimum(-100000.0)
self.Robot_WheelRightSpeed_Value.setMaximum(100000.0)
self.Robot_WheelRightSpeed_Value.setObjectName("Robot_WheelRightSpeed_Value")
self.Robot_WheelLeftSpeed_Value = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_WheelLeftSpeed_Value.setGeometry(QtCore.QRect(110, 70, 71, 21))
self.Robot_WheelLeftSpeed_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_WheelLeftSpeed_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_WheelLeftSpeed_Value.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelLeftSpeed_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelLeftSpeed_Value.setReadOnly(True)
self.Robot_WheelLeftSpeed_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_WheelLeftSpeed_Value.setDecimals(3)
self.Robot_WheelLeftSpeed_Value.setMinimum(-100000.0)
self.Robot_WheelLeftSpeed_Value.setMaximum(100000.0)
self.Robot_WheelLeftSpeed_Value.setObjectName("Robot_WheelLeftSpeed_Value")
self.tabWidget_2.addTab(self.tab_6, "")
self.tab_7 = QtWidgets.QWidget()
self.tab_7.setObjectName("tab_7")
self.groupBox_3 = QtWidgets.QGroupBox(self.tab_7)
self.groupBox_3.setGeometry(QtCore.QRect(0, 0, 311, 181))
self.groupBox_3.setTitle("")
self.groupBox_3.setObjectName("groupBox_3")
self.Robot_TiltDegrees = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_TiltDegrees.setGeometry(QtCore.QRect(190, 131, 71, 21))
self.Robot_TiltDegrees.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_TiltDegrees.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_TiltDegrees.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_TiltDegrees.setReadOnly(True)
self.Robot_TiltDegrees.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_TiltDegrees.setDecimals(3)
self.Robot_TiltDegrees.setMinimum(-999999.0)
self.Robot_TiltDegrees.setMaximum(999999.99)
self.Robot_TiltDegrees.setObjectName("Robot_TiltDegrees")
self.label_76 = QtWidgets.QLabel(self.groupBox_3)
self.label_76.setGeometry(QtCore.QRect(190, 80, 71, 21))
self.label_76.setAlignment(QtCore.Qt.AlignCenter)
self.label_76.setObjectName("label_76")
self.label_45 = QtWidgets.QLabel(self.groupBox_3)
self.label_45.setGeometry(QtCore.QRect(10, 0, 271, 31))
font = QtGui.QFont()
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_45.setFont(font)
self.label_45.setObjectName("label_45")
self.Robot_PoseX = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_PoseX.setGeometry(QtCore.QRect(100, 40, 81, 21))
self.Robot_PoseX.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_PoseX.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_PoseX.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PoseX.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PoseX.setReadOnly(True)
self.Robot_PoseX.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_PoseX.setDecimals(3)
self.Robot_PoseX.setMinimum(-999999.0)
self.Robot_PoseX.setMaximum(999999.99)
self.Robot_PoseX.setObjectName("Robot_PoseX")
self.Robot_PoseY = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_PoseY.setGeometry(QtCore.QRect(100, 71, 81, 21))
self.Robot_PoseY.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_PoseY.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PoseY.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PoseY.setReadOnly(True)
self.Robot_PoseY.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_PoseY.setDecimals(3)
self.Robot_PoseY.setMinimum(-999999.0)
self.Robot_PoseY.setMaximum(999999.99)
self.Robot_PoseY.setObjectName("Robot_PoseY")
self.Robot_PoseReset = QtWidgets.QPushButton(self.groupBox_3)
self.Robot_PoseReset.setGeometry(QtCore.QRect(190, 40, 71, 21))
self.Robot_PoseReset.setToolTip("Reset robot pose and distance to 0.0")
self.Robot_PoseReset.setObjectName("Robot_PoseReset")
self.label_44 = QtWidgets.QLabel(self.groupBox_3)
self.label_44.setGeometry(QtCore.QRect(20, 100, 71, 21))
self.label_44.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_44.setObjectName("label_44")
self.label_34 = QtWidgets.QLabel(self.groupBox_3)
self.label_34.setGeometry(QtCore.QRect(10, 40, 81, 21))
self.label_34.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_34.setObjectName("label_34")
self.label_35 = QtWidgets.QLabel(self.groupBox_3)
self.label_35.setGeometry(QtCore.QRect(20, 71, 71, 20))
self.label_35.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_35.setObjectName("label_35")
self.Robot_Tilt = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_Tilt.setGeometry(QtCore.QRect(100, 131, 81, 21))
self.Robot_Tilt.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_Tilt.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_Tilt.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_Tilt.setReadOnly(True)
self.Robot_Tilt.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_Tilt.setDecimals(3)
self.Robot_Tilt.setMinimum(-999999.0)
self.Robot_Tilt.setMaximum(999999.99)
self.Robot_Tilt.setObjectName("Robot_Tilt")
self.Robot_PoseTheta = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_PoseTheta.setGeometry(QtCore.QRect(100, 100, 81, 21))
self.Robot_PoseTheta.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_PoseTheta.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PoseTheta.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PoseTheta.setReadOnly(True)
self.Robot_PoseTheta.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_PoseTheta.setDecimals(3)
self.Robot_PoseTheta.setMinimum(-999999.0)
self.Robot_PoseTheta.setMaximum(999999.99)
self.Robot_PoseTheta.setObjectName("Robot_PoseTheta")
self.Robot_PoseDegrees = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_PoseDegrees.setGeometry(QtCore.QRect(190, 100, 71, 21))
self.Robot_PoseDegrees.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_PoseDegrees.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PoseDegrees.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PoseDegrees.setReadOnly(True)
self.Robot_PoseDegrees.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_PoseDegrees.setDecimals(3)
self.Robot_PoseDegrees.setMinimum(-999999.0)
self.Robot_PoseDegrees.setMaximum(999999.99)
self.Robot_PoseDegrees.setObjectName("Robot_PoseDegrees")
self.label_125 = QtWidgets.QLabel(self.groupBox_3)
self.label_125.setGeometry(QtCore.QRect(10, 130, 81, 21))
self.label_125.setMinimumSize(QtCore.QSize(0, 0))
self.label_125.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_125.setObjectName("label_125")
self.tabWidget_2.addTab(self.tab_7, "")
self.tabWidget.addTab(self.tab, "")
self.tab_2 = QtWidgets.QWidget()
self.tab_2.setObjectName("tab_2")
self.frame_2 = QtWidgets.QFrame(self.tab_2)
self.frame_2.setGeometry(QtCore.QRect(0, 0, 711, 421))
self.frame_2.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_2.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_2.setObjectName("frame_2")
self.label_10 = QtWidgets.QLabel(self.frame_2)
self.label_10.setGeometry(QtCore.QRect(10, 10, 101, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_10.setFont(font)
self.label_10.setObjectName("label_10")
self.enableLineSensor_checkBox = QtWidgets.QCheckBox(self.frame_2)
self.enableLineSensor_checkBox.setGeometry(QtCore.QRect(10, 30, 111, 17))
self.enableLineSensor_checkBox.setObjectName("enableLineSensor_checkBox")
self.frame_3 = QtWidgets.QFrame(self.frame_2)
self.frame_3.setGeometry(QtCore.QRect(10, 50, 421, 231))
self.frame_3.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_3.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_3.setObjectName("frame_3")
self.line_bar_1 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_1.setGeometry(QtCore.QRect(10, 20, 16, 181))
self.line_bar_1.setMaximum(1000)
self.line_bar_1.setProperty("value", 0)
self.line_bar_1.setOrientation(QtCore.Qt.Vertical)
self.line_bar_1.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_1.setObjectName("line_bar_1")
self.line_bar_4 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_4.setGeometry(QtCore.QRect(70, 20, 16, 181))
self.line_bar_4.setMaximum(1000)
self.line_bar_4.setProperty("value", 0)
self.line_bar_4.setOrientation(QtCore.Qt.Vertical)
self.line_bar_4.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_4.setObjectName("line_bar_4")
self.line_bar_6 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_6.setGeometry(QtCore.QRect(110, 20, 16, 181))
self.line_bar_6.setMaximum(1000)
self.line_bar_6.setProperty("value", 0)
self.line_bar_6.setOrientation(QtCore.Qt.Vertical)
self.line_bar_6.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_6.setObjectName("line_bar_6")
self.line_bar_5 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_5.setGeometry(QtCore.QRect(90, 20, 16, 181))
self.line_bar_5.setMaximum(1000)
self.line_bar_5.setProperty("value", 0)
self.line_bar_5.setOrientation(QtCore.Qt.Vertical)
self.line_bar_5.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_5.setObjectName("line_bar_5")
self.line_bar_3 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_3.setGeometry(QtCore.QRect(50, 20, 16, 181))
self.line_bar_3.setMaximum(1000)
self.line_bar_3.setProperty("value", 0)
self.line_bar_3.setOrientation(QtCore.Qt.Vertical)
self.line_bar_3.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_3.setObjectName("line_bar_3")
self.line_bar_7 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_7.setGeometry(QtCore.QRect(130, 20, 16, 181))
self.line_bar_7.setMaximum(1000)
self.line_bar_7.setProperty("value", 0)
self.line_bar_7.setOrientation(QtCore.Qt.Vertical)
self.line_bar_7.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_7.setObjectName("line_bar_7")
self.line_bar_2 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_2.setGeometry(QtCore.QRect(30, 20, 16, 181))
self.line_bar_2.setMaximum(1000)
self.line_bar_2.setProperty("value", 0)
self.line_bar_2.setOrientation(QtCore.Qt.Vertical)
self.line_bar_2.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_2.setObjectName("line_bar_2")
self.line_bar_8 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_8.setGeometry(QtCore.QRect(150, 20, 16, 181))
self.line_bar_8.setMaximum(1000)
self.line_bar_8.setProperty("value", 0)
self.line_bar_8.setOrientation(QtCore.Qt.Vertical)
self.line_bar_8.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_8.setObjectName("line_bar_8")
self.label_11 = QtWidgets.QLabel(self.frame_3)
self.label_11.setGeometry(QtCore.QRect(10, 0, 151, 21))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_11.setFont(font)
self.label_11.setObjectName("label_11")
self.label_12 = QtWidgets.QLabel(self.frame_3)
self.label_12.setGeometry(QtCore.QRect(10, 200, 16, 21))
self.label_12.setLayoutDirection(QtCore.Qt.RightToLeft)
self.label_12.setObjectName("label_12")
self.label_13 = QtWidgets.QLabel(self.frame_3)
self.label_13.setGeometry(QtCore.QRect(30, 200, 16, 21))
self.label_13.setObjectName("label_13")
self.label_14 = QtWidgets.QLabel(self.frame_3)
self.label_14.setGeometry(QtCore.QRect(110, 200, 16, 21))
self.label_14.setObjectName("label_14")
self.label_15 = QtWidgets.QLabel(self.frame_3)
self.label_15.setGeometry(QtCore.QRect(50, 200, 16, 21))
self.label_15.setObjectName("label_15")
self.label_16 = QtWidgets.QLabel(self.frame_3)
self.label_16.setGeometry(QtCore.QRect(70, 200, 16, 21))
self.label_16.setObjectName("label_16")
self.label_17 = QtWidgets.QLabel(self.frame_3)
self.label_17.setGeometry(QtCore.QRect(90, 200, 20, 21))
self.label_17.setObjectName("label_17")
self.label_18 = QtWidgets.QLabel(self.frame_3)
self.label_18.setGeometry(QtCore.QRect(130, 200, 16, 21))
self.label_18.setObjectName("label_18")
self.label_19 = QtWidgets.QLabel(self.frame_3)
self.label_19.setGeometry(QtCore.QRect(150, 200, 16, 21))
self.label_19.setObjectName("label_19")
self.ls_right_side = QtWidgets.QDoubleSpinBox(self.frame_3)
self.ls_right_side.setGeometry(QtCore.QRect(300, 170, 61, 21))
self.ls_right_side.setToolTip("<html><head/><body>Position of right edge (cm)</body></html>")
self.ls_right_side.setStyleSheet("background-color: rgb(230,230,230);")
self.ls_right_side.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ls_right_side.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_right_side.setReadOnly(True)
self.ls_right_side.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.ls_right_side.setSpecialValueText("")
self.ls_right_side.setDecimals(3)
self.ls_right_side.setMinimum(-100.0)
self.ls_right_side.setMaximum(100.0)
self.ls_right_side.setObjectName("ls_right_side")
self.ls_left_side = QtWidgets.QDoubleSpinBox(self.frame_3)
self.ls_left_side.setGeometry(QtCore.QRect(230, 170, 61, 21))
self.ls_left_side.setToolTip("<html><head/><body>Position of left edge (cm)</body></html>")
self.ls_left_side.setStyleSheet("background-color: rgb(230,230,230);")
self.ls_left_side.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ls_left_side.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_left_side.setReadOnly(True)
self.ls_left_side.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.ls_left_side.setSpecialValueText("")
self.ls_left_side.setDecimals(3)
self.ls_left_side.setMinimum(-100.0)
self.ls_left_side.setMaximum(100.0)
self.ls_left_side.setObjectName("ls_left_side")
self.label_118 = QtWidgets.QLabel(self.frame_3)
self.label_118.setGeometry(QtCore.QRect(230, 40, 61, 18))
self.label_118.setAlignment(QtCore.Qt.AlignCenter)
self.label_118.setObjectName("label_118")
self.ls_left_edge = QtWidgets.QProgressBar(self.frame_3)
self.ls_left_edge.setEnabled(True)
self.ls_left_edge.setGeometry(QtCore.QRect(240, 60, 41, 101))
self.ls_left_edge.setMinimum(-3)
self.ls_left_edge.setMaximum(103)
self.ls_left_edge.setProperty("value", 24)
self.ls_left_edge.setTextVisible(False)
self.ls_left_edge.setOrientation(QtCore.Qt.Vertical)
self.ls_left_edge.setInvertedAppearance(False)
self.ls_left_edge.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.ls_left_edge.setObjectName("ls_left_edge")
self.ls_right_edge = QtWidgets.QProgressBar(self.frame_3)
self.ls_right_edge.setGeometry(QtCore.QRect(310, 60, 41, 101))
self.ls_right_edge.setMinimum(-3)
self.ls_right_edge.setMaximum(103)
self.ls_right_edge.setProperty("value", 24)
self.ls_right_edge.setTextVisible(False)
self.ls_right_edge.setOrientation(QtCore.Qt.Vertical)
self.ls_right_edge.setInvertedAppearance(False)
self.ls_right_edge.setObjectName("ls_right_edge")
self.label_121 = QtWidgets.QLabel(self.frame_3)
self.label_121.setGeometry(QtCore.QRect(300, 40, 61, 18))
self.label_121.setAlignment(QtCore.Qt.AlignCenter)
self.label_121.setObjectName("label_121")
self.label_20 = QtWidgets.QLabel(self.frame_3)
self.label_20.setGeometry(QtCore.QRect(180, 0, 101, 21))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_20.setFont(font)
self.label_20.setObjectName("label_20")
self.label_27 = QtWidgets.QLabel(self.frame_3)
self.label_27.setGeometry(QtCore.QRect(290, 20, 48, 20))
self.label_27.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_27.setObjectName("label_27")
self.label_171 = QtWidgets.QLabel(self.frame_3)
self.label_171.setGeometry(QtCore.QRect(176, 20, 48, 20))
self.label_171.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_171.setObjectName("label_171")
self.ls_crossing_cnt = QtWidgets.QLineEdit(self.frame_3)
self.ls_crossing_cnt.setGeometry(QtCore.QRect(344, 20, 54, 20))
self.ls_crossing_cnt.setStatusTip("")
self.ls_crossing_cnt.setStyleSheet("background-color: rgb(230,230,230);")
self.ls_crossing_cnt.setObjectName("ls_crossing_cnt")
self.ls_line_valid_cnt = QtWidgets.QLineEdit(self.frame_3)
self.ls_line_valid_cnt.setGeometry(QtCore.QRect(230, 20, 54, 20))
self.ls_line_valid_cnt.setStyleSheet("background-color: rgb(230,230,230);")
self.ls_line_valid_cnt.setObjectName("ls_line_valid_cnt")
self.groupBox_5 = QtWidgets.QGroupBox(self.frame_2)
self.groupBox_5.setGeometry(QtCore.QRect(430, 10, 251, 361))
self.groupBox_5.setObjectName("groupBox_5")
self.CalibrateWhite_Cmd = QtWidgets.QPushButton(self.groupBox_5)
self.CalibrateWhite_Cmd.setGeometry(QtCore.QRect(20, 310, 101, 31))
self.CalibrateWhite_Cmd.setObjectName("CalibrateWhite_Cmd")
self.label_217 = QtWidgets.QLabel(self.groupBox_5)
self.label_217.setGeometry(QtCore.QRect(20, 75, 21, 16))
self.label_217.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_217.setObjectName("label_217")
self.label_218 = QtWidgets.QLabel(self.groupBox_5)
self.label_218.setGeometry(QtCore.QRect(20, 117, 21, 16))
self.label_218.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_218.setObjectName("label_218")
self.label_219 = QtWidgets.QLabel(self.groupBox_5)
self.label_219.setGeometry(QtCore.QRect(20, 159, 21, 16))
self.label_219.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_219.setObjectName("label_219")
self.label_221 = QtWidgets.QLabel(self.groupBox_5)
self.label_221.setGeometry(QtCore.QRect(20, 96, 21, 16))
self.label_221.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_221.setObjectName("label_221")
self.label_223 = QtWidgets.QLabel(self.groupBox_5)
self.label_223.setGeometry(QtCore.QRect(20, 138, 21, 16))
self.label_223.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_223.setObjectName("label_223")
self.label_224 = QtWidgets.QLabel(self.groupBox_5)
self.label_224.setGeometry(QtCore.QRect(20, 33, 21, 16))
self.label_224.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_224.setObjectName("label_224")
self.label_225 = QtWidgets.QLabel(self.groupBox_5)
self.label_225.setGeometry(QtCore.QRect(20, 180, 21, 16))
self.label_225.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_225.setObjectName("label_225")
self.label_226 = QtWidgets.QLabel(self.groupBox_5)
self.label_226.setGeometry(QtCore.QRect(20, 54, 21, 16))
self.label_226.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_226.setObjectName("label_226")
self.ls_max_white_7 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_7.setGeometry(QtCore.QRect(50, 159, 50, 15))
self.ls_max_white_7.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_7.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_7.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_7.setObjectName("ls_max_white_7")
self.ls_max_white_8 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_8.setGeometry(QtCore.QRect(50, 180, 50, 15))
self.ls_max_white_8.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_8.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_8.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_8.setObjectName("ls_max_white_8")
self.ls_max_white_1 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_1.setGeometry(QtCore.QRect(50, 33, 50, 15))
self.ls_max_white_1.setMinimumSize(QtCore.QSize(0, 0))
self.ls_max_white_1.setAutoFillBackground(False)
self.ls_max_white_1.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_1.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_1.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_1.setObjectName("ls_max_white_1")
self.ls_max_white_3 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_3.setGeometry(QtCore.QRect(50, 75, 50, 15))
self.ls_max_white_3.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_3.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_3.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_3.setObjectName("ls_max_white_3")
self.ls_max_white_5 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_5.setGeometry(QtCore.QRect(50, 117, 50, 15))
self.ls_max_white_5.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_5.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_5.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_5.setObjectName("ls_max_white_5")
self.ls_max_white_2 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_2.setGeometry(QtCore.QRect(50, 54, 50, 15))
self.ls_max_white_2.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_2.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_2.setObjectName("ls_max_white_2")
self.ls_max_white_4 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_4.setGeometry(QtCore.QRect(50, 96, 50, 15))
self.ls_max_white_4.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_4.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_4.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_4.setObjectName("ls_max_white_4")
self.ls_max_white_6 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_6.setGeometry(QtCore.QRect(50, 138, 50, 15))
self.ls_max_white_6.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_6.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_6.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_6.setObjectName("ls_max_white_6")
self.ls_max_black_5 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_5.setGeometry(QtCore.QRect(150, 117, 50, 15))
self.ls_max_black_5.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_5.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_5.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_5.setObjectName("ls_max_black_5")
self.ls_max_black_4 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_4.setGeometry(QtCore.QRect(150, 96, 50, 15))
self.ls_max_black_4.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_4.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_4.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_4.setObjectName("ls_max_black_4")
self.ls_max_black_8 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_8.setGeometry(QtCore.QRect(150, 180, 50, 15))
self.ls_max_black_8.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_8.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_8.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_8.setObjectName("ls_max_black_8")
self.ls_max_black_1 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_1.setGeometry(QtCore.QRect(150, 33, 50, 15))
self.ls_max_black_1.setMinimumSize(QtCore.QSize(0, 0))
self.ls_max_black_1.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_1.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_1.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_1.setObjectName("ls_max_black_1")
self.ls_max_black_2 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_2.setGeometry(QtCore.QRect(150, 54, 50, 15))
self.ls_max_black_2.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_2.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_2.setObjectName("ls_max_black_2")
self.ls_max_black_3 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_3.setGeometry(QtCore.QRect(150, 75, 50, 15))
self.ls_max_black_3.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_3.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_3.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_3.setObjectName("ls_max_black_3")
self.ls_max_black_6 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_6.setGeometry(QtCore.QRect(150, 138, 50, 15))
self.ls_max_black_6.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_6.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_6.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_6.setObjectName("ls_max_black_6")
self.ls_max_black_7 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_7.setGeometry(QtCore.QRect(150, 159, 50, 15))
self.ls_max_black_7.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_7.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_7.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_7.setObjectName("ls_max_black_7")
self.label_227 = QtWidgets.QLabel(self.groupBox_5)
self.label_227.setGeometry(QtCore.QRect(50, 14, 51, 16))
self.label_227.setAlignment(QtCore.Qt.AlignCenter)
self.label_227.setObjectName("label_227")
self.CalibrateBlack_Cmd = QtWidgets.QPushButton(self.groupBox_5)
self.CalibrateBlack_Cmd.setGeometry(QtCore.QRect(130, 310, 101, 31))
self.CalibrateBlack_Cmd.setObjectName("CalibrateBlack_Cmd")
self.label_228 = QtWidgets.QLabel(self.groupBox_5)
self.label_228.setGeometry(QtCore.QRect(150, 15, 51, 16))
self.label_228.setAlignment(QtCore.Qt.AlignCenter)
self.label_228.setObjectName("label_228")
self.ls_swap_left_right = QtWidgets.QCheckBox(self.groupBox_5)
self.ls_swap_left_right.setGeometry(QtCore.QRect(140, 220, 81, 20))
self.ls_swap_left_right.setObjectName("ls_swap_left_right")
self.ls_show_normalized = QtWidgets.QCheckBox(self.groupBox_5)
self.ls_show_normalized.setGeometry(QtCore.QRect(30, 220, 101, 17))
self.ls_show_normalized.setObjectName("ls_show_normalized")
self.label_23 = QtWidgets.QLabel(self.groupBox_5)
self.label_23.setGeometry(QtCore.QRect(130, 240, 75, 21))
self.label_23.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_23.setObjectName("label_23")
self.line_disp_max_value = QtWidgets.QSpinBox(self.groupBox_5)
self.line_disp_max_value.setGeometry(QtCore.QRect(30, 260, 75, 20))
self.line_disp_max_value.setToolTip("<html><head/><body>Maximum value used as 100% in bar display.</body></html>")
self.line_disp_max_value.setStatusTip("")
self.line_disp_max_value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.line_disp_max_value.setMaximum(4250)
self.line_disp_max_value.setSingleStep(256)
self.line_disp_max_value.setProperty("value", 4096)
self.line_disp_max_value.setObjectName("line_disp_max_value")
self.label_156 = QtWidgets.QLabel(self.groupBox_5)
self.label_156.setGeometry(QtCore.QRect(34, 240, 61, 20))
self.label_156.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_156.setObjectName("label_156")
self.ls_crossing_detect = QtWidgets.QDoubleSpinBox(self.groupBox_5)
self.ls_crossing_detect.setGeometry(QtCore.QRect(130, 260, 75, 20))
self.ls_crossing_detect.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_crossing_detect.setDecimals(1)
self.ls_crossing_detect.setMinimum(2.0)
self.ls_crossing_detect.setMaximum(8.0)
self.ls_crossing_detect.setSingleStep(0.1)
self.ls_crossing_detect.setProperty("value", 4.5)
self.ls_crossing_detect.setObjectName("ls_crossing_detect")
self.line_2 = QtWidgets.QFrame(self.groupBox_5)
self.line_2.setGeometry(QtCore.QRect(20, 200, 211, 21))
self.line_2.setFrameShape(QtWidgets.QFrame.HLine)
self.line_2.setFrameShadow(QtWidgets.QFrame.Sunken)
self.line_2.setObjectName("line_2")
self.line_3 = QtWidgets.QFrame(self.groupBox_5)
self.line_3.setGeometry(QtCore.QRect(20, 290, 211, 21))
self.line_3.setFrameShape(QtWidgets.QFrame.HLine)
self.line_3.setFrameShadow(QtWidgets.QFrame.Sunken)
self.line_3.setObjectName("line_3")
self.ls_power_high = QtWidgets.QCheckBox(self.frame_2)
self.ls_power_high.setGeometry(QtCore.QRect(230, 30, 91, 17))
self.ls_power_high.setObjectName("ls_power_high")
self.ls_line_white = QtWidgets.QCheckBox(self.frame_2)
self.ls_line_white.setGeometry(QtCore.QRect(140, 30, 81, 17))
self.ls_line_white.setObjectName("ls_line_white")
self.tabWidget.addTab(self.tab_2, "")
self.tab_8 = QtWidgets.QWidget()
self.tab_8.setObjectName("tab_8")
self.frame_37 = QtWidgets.QFrame(self.tab_8)
self.frame_37.setGeometry(QtCore.QRect(0, 0, 521, 291))
self.frame_37.setMinimumSize(QtCore.QSize(250, 0))
self.frame_37.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_37.setFrameShadow(QtWidgets.QFrame.Sunken)
self.frame_37.setObjectName("frame_37")
self.label_56 = QtWidgets.QLabel(self.frame_37)
self.label_56.setGeometry(QtCore.QRect(10, 10, 104, 16))
font = QtGui.QFont()
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_56.setFont(font)
self.label_56.setObjectName("label_56")
self.ir_bar_1 = QtWidgets.QProgressBar(self.frame_37)
self.ir_bar_1.setGeometry(QtCore.QRect(320, 200, 171, 41))
self.ir_bar_1.setMaximum(130000)
self.ir_bar_1.setProperty("value", 0)
self.ir_bar_1.setOrientation(QtCore.Qt.Horizontal)
self.ir_bar_1.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.ir_bar_1.setFormat("")
self.ir_bar_1.setObjectName("ir_bar_1")
self.ir_bar_2 = QtWidgets.QProgressBar(self.frame_37)
self.ir_bar_2.setGeometry(QtCore.QRect(280, 60, 41, 141))
self.ir_bar_2.setMaximum(130000)
self.ir_bar_2.setProperty("value", 0)
self.ir_bar_2.setOrientation(QtCore.Qt.Vertical)
self.ir_bar_2.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.ir_bar_2.setFormat("")
self.ir_bar_2.setObjectName("ir_bar_2")
self.ir_d1_meters = QtWidgets.QLineEdit(self.frame_37)
self.ir_d1_meters.setGeometry(QtCore.QRect(400, 170, 81, 20))
self.ir_d1_meters.setStyleSheet("background-color: rgb(230,230,230);")
self.ir_d1_meters.setAlignment(QtCore.Qt.AlignCenter)
self.ir_d1_meters.setObjectName("ir_d1_meters")
self.label_279 = QtWidgets.QLabel(self.frame_37)
self.label_279.setGeometry(QtCore.QRect(360, 170, 41, 20))
self.label_279.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_279.setObjectName("label_279")
self.label_281 = QtWidgets.QLabel(self.frame_37)
self.label_281.setGeometry(QtCore.QRect(330, 140, 71, 18))
self.label_281.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_281.setObjectName("label_281")
self.ir_d1_raw = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d1_raw.setGeometry(QtCore.QRect(400, 140, 81, 20))
self.ir_d1_raw.setToolTip("<html><head/><body>Raw measured value 2Y0A21F sensor from A/D converter</body></html>")
self.ir_d1_raw.setStyleSheet("background-color: rgb(230,230,230);")
self.ir_d1_raw.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d1_raw.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d1_raw.setReadOnly(True)
self.ir_d1_raw.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.ir_d1_raw.setSpecialValueText("")
self.ir_d1_raw.setDecimals(0)
self.ir_d1_raw.setMinimum(0.0)
self.ir_d1_raw.setMaximum(1000000.0)
self.ir_d1_raw.setObjectName("ir_d1_raw")
self.label_287 = QtWidgets.QLabel(self.frame_37)
self.label_287.setGeometry(QtCore.QRect(490, 170, 16, 16))
self.label_287.setObjectName("label_287")
self.ir_d2_meters = QtWidgets.QLineEdit(self.frame_37)
self.ir_d2_meters.setGeometry(QtCore.QRect(402, 90, 81, 20))
self.ir_d2_meters.setStyleSheet("background-color: rgb(230,230,230);")
self.ir_d2_meters.setAlignment(QtCore.Qt.AlignCenter)
self.ir_d2_meters.setObjectName("ir_d2_meters")
self.label_280 = QtWidgets.QLabel(self.frame_37)
self.label_280.setGeometry(QtCore.QRect(360, 90, 51, 20))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Preferred, QtWidgets.QSizePolicy.Maximum)
sizePolicy.setHorizontalStretch(0)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.label_280.sizePolicy().hasHeightForWidth())
self.label_280.setSizePolicy(sizePolicy)
self.label_280.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_280.setObjectName("label_280")
self.ir_d2_raw = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d2_raw.setGeometry(QtCore.QRect(400, 60, 81, 20))
self.ir_d2_raw.setToolTip("<html><head/><body>Raw measured value 2Y0A21F sensor from A/D converter</body></html>")
self.ir_d2_raw.setStyleSheet("background-color: rgb(230,230,230);")
self.ir_d2_raw.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d2_raw.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d2_raw.setReadOnly(True)
self.ir_d2_raw.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.ir_d2_raw.setSpecialValueText("")
self.ir_d2_raw.setDecimals(0)
self.ir_d2_raw.setMinimum(0.0)
self.ir_d2_raw.setMaximum(1000000.0)
self.ir_d2_raw.setObjectName("ir_d2_raw")
self.label_282 = QtWidgets.QLabel(self.frame_37)
self.label_282.setGeometry(QtCore.QRect(330, 60, 71, 18))
self.label_282.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_282.setObjectName("label_282")
self.label_288 = QtWidgets.QLabel(self.frame_37)
self.label_288.setGeometry(QtCore.QRect(490, 90, 16, 16))
self.label_288.setObjectName("label_288")
self.ir_d2_80cm = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d2_80cm.setGeometry(QtCore.QRect(170, 200, 85, 20))
self.ir_d2_80cm.setToolTip("<html><head/><body>Raw value at 50 cm from base of sensor (about 24000)</body></html>")
self.ir_d2_80cm.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d2_80cm.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d2_80cm.setReadOnly(False)
self.ir_d2_80cm.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.ir_d2_80cm.setSpecialValueText("")
self.ir_d2_80cm.setDecimals(0)
self.ir_d2_80cm.setMinimum(0.0)
self.ir_d2_80cm.setMaximum(128000.0)
self.ir_d2_80cm.setProperty("value", 24100.0)
self.ir_d2_80cm.setObjectName("ir_d2_80cm")
self.ir_d1_80cm = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d1_80cm.setGeometry(QtCore.QRect(170, 148, 85, 20))
self.ir_d1_80cm.setToolTip("<html><head/><body>Raw value at 50cm from base of sensor (about 24000)</body></html>")
self.ir_d1_80cm.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d1_80cm.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d1_80cm.setReadOnly(False)
self.ir_d1_80cm.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.ir_d1_80cm.setSpecialValueText("")
self.ir_d1_80cm.setDecimals(0)
self.ir_d1_80cm.setMinimum(0.0)
self.ir_d1_80cm.setMaximum(128000.0)
self.ir_d1_80cm.setProperty("value", 24100.0)
self.ir_d1_80cm.setObjectName("ir_d1_80cm")
self.ir_d2_20cm = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d2_20cm.setGeometry(QtCore.QRect(170, 174, 85, 20))
self.ir_d2_20cm.setToolTip("<html><head/><body>Raw value at 13 cm from base of sensor (about 80000)</body></html>")
self.ir_d2_20cm.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d2_20cm.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d2_20cm.setReadOnly(False)
self.ir_d2_20cm.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.ir_d2_20cm.setSpecialValueText("")
self.ir_d2_20cm.setDecimals(0)
self.ir_d2_20cm.setMinimum(0.0)
self.ir_d2_20cm.setMaximum(128000.0)
self.ir_d2_20cm.setProperty("value", 72300.0)
self.ir_d2_20cm.setObjectName("ir_d2_20cm")
self.label_284 = QtWidgets.QLabel(self.frame_37)
self.label_284.setGeometry(QtCore.QRect(32, 148, 132, 20))
self.label_284.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_284.setObjectName("label_284")
self.label_286 = QtWidgets.QLabel(self.frame_37)
self.label_286.setGeometry(QtCore.QRect(32, 174, 132, 20))
self.label_286.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_286.setObjectName("label_286")
self.label_285 = QtWidgets.QLabel(self.frame_37)
self.label_285.setGeometry(QtCore.QRect(32, 200, 132, 20))
self.label_285.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_285.setObjectName("label_285")
self.label_283 = QtWidgets.QLabel(self.frame_37)
self.label_283.setGeometry(QtCore.QRect(32, 122, 132, 20))
self.label_283.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_283.setObjectName("label_283")
self.ir_d1_20cm = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d1_20cm.setGeometry(QtCore.QRect(170, 122, 85, 20))
self.ir_d1_20cm.setToolTip("<html><head/><body>Raw value at 13 cm from base of sensor (about 80000)</body></html>")
self.ir_d1_20cm.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d1_20cm.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d1_20cm.setReadOnly(False)
self.ir_d1_20cm.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.ir_d1_20cm.setSpecialValueText("")
self.ir_d1_20cm.setDecimals(0)
self.ir_d1_20cm.setMinimum(0.0)
self.ir_d1_20cm.setMaximum(128000.0)
self.ir_d1_20cm.setProperty("value", 72000.0)
self.ir_d1_20cm.setObjectName("ir_d1_20cm")
self.checkBox_ir_use = QtWidgets.QCheckBox(self.frame_37)
self.checkBox_ir_use.setGeometry(QtCore.QRect(30, 70, 132, 17))
self.checkBox_ir_use.setObjectName("checkBox_ir_use")
self.checkBox_ir_installed = QtWidgets.QCheckBox(self.frame_37)
self.checkBox_ir_installed.setGeometry(QtCore.QRect(30, 44, 111, 17))
self.checkBox_ir_installed.setObjectName("checkBox_ir_installed")
self.label_277 = QtWidgets.QLabel(self.frame_37)
self.label_277.setGeometry(QtCore.QRect(30, 100, 132, 13))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_277.setFont(font)
self.label_277.setObjectName("label_277")
self.ir_apply = QtWidgets.QPushButton(self.frame_37)
self.ir_apply.setGeometry(QtCore.QRect(170, 230, 85, 23))
self.ir_apply.setObjectName("ir_apply")
self.label_278 = QtWidgets.QLabel(self.frame_37)
self.label_278.setGeometry(QtCore.QRect(310, 30, 171, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_278.setFont(font)
self.label_278.setObjectName("label_278")
self.ir_edit = QtWidgets.QPushButton(self.frame_37)
self.ir_edit.setGeometry(QtCore.QRect(30, 230, 132, 23))
self.ir_edit.setObjectName("ir_edit")
self.ir_cancel = QtWidgets.QPushButton(self.frame_37)
self.ir_cancel.setGeometry(QtCore.QRect(260, 230, 51, 23))
self.ir_cancel.setObjectName("ir_cancel")
self.tabWidget.addTab(self.tab_8, "")
self.tab_3 = QtWidgets.QWidget()
self.tab_3.setObjectName("tab_3")
self.frame_4 = QtWidgets.QFrame(self.tab_3)
self.frame_4.setGeometry(QtCore.QRect(0, 0, 711, 691))
self.frame_4.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_4.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_4.setObjectName("frame_4")
self.EnableCamera_checkBox = QtWidgets.QCheckBox(self.frame_4)
self.EnableCamera_checkBox.setGeometry(QtCore.QRect(20, 20, 121, 17))
self.EnableCamera_checkBox.setObjectName("EnableCamera_checkBox")
self.frame_5 = QtWidgets.QFrame(self.frame_4)
self.frame_5.setGeometry(QtCore.QRect(10, 430, 461, 211))
self.frame_5.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_5.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_5.setObjectName("frame_5")
self.Camera_Record_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_Record_Cmd.setGeometry(QtCore.QRect(240, 10, 91, 23))
self.Camera_Record_Cmd.setObjectName("Camera_Record_Cmd")
self.Camera_StopRecord_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_StopRecord_Cmd.setGeometry(QtCore.QRect(340, 10, 91, 23))
self.Camera_StopRecord_Cmd.setObjectName("Camera_StopRecord_Cmd")
self.Camera_Snapshot_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_Snapshot_Cmd.setGeometry(QtCore.QRect(10, 10, 111, 23))
self.Camera_Snapshot_Cmd.setObjectName("Camera_Snapshot_Cmd")
self.Camera_SnapFilename = QtWidgets.QLineEdit(self.frame_5)
self.Camera_SnapFilename.setGeometry(QtCore.QRect(240, 70, 191, 20))
self.Camera_SnapFilename.setObjectName("Camera_SnapFilename")
self.label_55 = QtWidgets.QLabel(self.frame_5)
self.label_55.setGeometry(QtCore.QRect(240, 50, 121, 20))
self.label_55.setObjectName("label_55")
self.Camera_SnapSave_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_SnapSave_Cmd.setGeometry(QtCore.QRect(340, 100, 91, 23))
self.Camera_SnapSave_Cmd.setObjectName("Camera_SnapSave_Cmd")
self.Snapshot_Label = QtWidgets.QLabel(self.frame_5)
self.Snapshot_Label.setGeometry(QtCore.QRect(10, 40, 221, 161))
self.Snapshot_Label.setLayoutDirection(QtCore.Qt.LeftToRight)
self.Snapshot_Label.setAutoFillBackground(True)
self.Snapshot_Label.setFrameShape(QtWidgets.QFrame.Box)
self.Snapshot_Label.setFrameShadow(QtWidgets.QFrame.Raised)
self.Snapshot_Label.setLineWidth(-2)
self.Snapshot_Label.setText("")
self.Snapshot_Label.setScaledContents(True)
self.Snapshot_Label.setIndent(0)
self.Snapshot_Label.setObjectName("Snapshot_Label")
self.VideoRecording_Label = QtWidgets.QLabel(self.frame_5)
self.VideoRecording_Label.setGeometry(QtCore.QRect(240, 30, 71, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.VideoRecording_Label.setFont(font)
self.VideoRecording_Label.setObjectName("VideoRecording_Label")
self.Camera_ClearSnap_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_ClearSnap_Cmd.setGeometry(QtCore.QRect(130, 10, 91, 23))
self.Camera_ClearSnap_Cmd.setObjectName("Camera_ClearSnap_Cmd")
self.frame_6 = QtWidgets.QFrame(self.frame_4)
self.frame_6.setGeometry(QtCore.QRect(10, 40, 561, 381))
self.frame_6.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_6.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_6.setObjectName("frame_6")
self.Stream_Label = QtWidgets.QLabel(self.frame_6)
self.Stream_Label.setGeometry(QtCore.QRect(10, 0, 541, 371))
self.Stream_Label.setText("")
self.Stream_Label.setScaledContents(True)
self.Stream_Label.setObjectName("Stream_Label")
self.tabWidget.addTab(self.tab_3, "")
self.tab_4 = QtWidgets.QWidget()
self.tab_4.setObjectName("tab_4")
self.frame_7 = QtWidgets.QFrame(self.tab_4)
self.frame_7.setGeometry(QtCore.QRect(0, 0, 541, 631))
self.frame_7.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_7.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_7.setObjectName("frame_7")
self.label_30 = QtWidgets.QLabel(self.frame_7)
self.label_30.setGeometry(QtCore.QRect(10, 0, 151, 21))
font = QtGui.QFont()
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_30.setFont(font)
self.label_30.setObjectName("label_30")
self.enableServo1_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo1_checkBox.setGeometry(QtCore.QRect(10, 30, 111, 17))
self.enableServo1_checkBox.setObjectName("enableServo1_checkBox")
self.enableServo2_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo2_checkBox.setGeometry(QtCore.QRect(10, 150, 111, 17))
self.enableServo2_checkBox.setObjectName("enableServo2_checkBox")
self.enableServo3_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo3_checkBox.setGeometry(QtCore.QRect(10, 270, 111, 17))
self.enableServo3_checkBox.setObjectName("enableServo3_checkBox")
self.Servo1_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo1_Pos_dial.setGeometry(QtCore.QRect(210, 50, 81, 81))
self.Servo1_Pos_dial.setMinimum(-920)
self.Servo1_Pos_dial.setMaximum(920)
self.Servo1_Pos_dial.setTracking(False)
self.Servo1_Pos_dial.setObjectName("Servo1_Pos_dial")
self.label_31 = QtWidgets.QLabel(self.frame_7)
self.label_31.setGeometry(QtCore.QRect(10, 50, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_31.setFont(font)
self.label_31.setObjectName("label_31")
self.servo1_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo1_current_pos.setGeometry(QtCore.QRect(100, 50, 91, 20))
self.servo1_current_pos.setStatusTip("")
self.servo1_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo1_current_pos.setObjectName("servo1_current_pos")
self.label_33 = QtWidgets.QLabel(self.frame_7)
self.label_33.setGeometry(QtCore.QRect(10, 80, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_33.setFont(font)
self.label_33.setObjectName("label_33")
self.manualServoPos1_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos1_Val.setGeometry(QtCore.QRect(100, 80, 91, 20))
self.manualServoPos1_Val.setObjectName("manualServoPos1_Val")
self.manualServoPos1_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos1_Cmd.setGeometry(QtCore.QRect(100, 110, 91, 23))
self.manualServoPos1_Cmd.setObjectName("manualServoPos1_Cmd")
self.label_39 = QtWidgets.QLabel(self.frame_7)
self.label_39.setGeometry(QtCore.QRect(220, 30, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_39.setFont(font)
self.label_39.setObjectName("label_39")
self.Servo2_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo2_Pos_dial.setGeometry(QtCore.QRect(210, 170, 81, 81))
self.Servo2_Pos_dial.setMinimum(-920)
self.Servo2_Pos_dial.setMaximum(920)
self.Servo2_Pos_dial.setTracking(False)
self.Servo2_Pos_dial.setObjectName("Servo2_Pos_dial")
self.label_40 = QtWidgets.QLabel(self.frame_7)
self.label_40.setGeometry(QtCore.QRect(220, 150, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_40.setFont(font)
self.label_40.setObjectName("label_40")
self.manualServoPos2_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos2_Val.setGeometry(QtCore.QRect(100, 200, 91, 20))
self.manualServoPos2_Val.setObjectName("manualServoPos2_Val")
self.servo2_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo2_current_pos.setGeometry(QtCore.QRect(100, 170, 91, 20))
self.servo2_current_pos.setStatusTip("")
self.servo2_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo2_current_pos.setObjectName("servo2_current_pos")
self.label_41 = QtWidgets.QLabel(self.frame_7)
self.label_41.setGeometry(QtCore.QRect(10, 200, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_41.setFont(font)
self.label_41.setObjectName("label_41")
self.manualServoPos2_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos2_Cmd.setGeometry(QtCore.QRect(100, 230, 91, 23))
self.manualServoPos2_Cmd.setObjectName("manualServoPos2_Cmd")
self.label_43 = QtWidgets.QLabel(self.frame_7)
self.label_43.setGeometry(QtCore.QRect(10, 170, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_43.setFont(font)
self.label_43.setObjectName("label_43")
self.Servo3_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo3_Pos_dial.setGeometry(QtCore.QRect(210, 290, 81, 81))
self.Servo3_Pos_dial.setMinimum(-920)
self.Servo3_Pos_dial.setMaximum(920)
self.Servo3_Pos_dial.setTracking(False)
self.Servo3_Pos_dial.setObjectName("Servo3_Pos_dial")
self.label_46 = QtWidgets.QLabel(self.frame_7)
self.label_46.setGeometry(QtCore.QRect(220, 270, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_46.setFont(font)
self.label_46.setObjectName("label_46")
self.manualServoPos3_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos3_Val.setGeometry(QtCore.QRect(100, 320, 91, 20))
self.manualServoPos3_Val.setObjectName("manualServoPos3_Val")
self.servo3_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo3_current_pos.setGeometry(QtCore.QRect(100, 290, 91, 20))
self.servo3_current_pos.setStatusTip("")
self.servo3_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo3_current_pos.setObjectName("servo3_current_pos")
self.label_47 = QtWidgets.QLabel(self.frame_7)
self.label_47.setGeometry(QtCore.QRect(10, 320, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_47.setFont(font)
self.label_47.setObjectName("label_47")
self.manualServoPos3_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos3_Cmd.setGeometry(QtCore.QRect(100, 350, 91, 23))
self.manualServoPos3_Cmd.setObjectName("manualServoPos3_Cmd")
self.label_48 = QtWidgets.QLabel(self.frame_7)
self.label_48.setGeometry(QtCore.QRect(10, 290, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_48.setFont(font)
self.label_48.setObjectName("label_48")
self.enableServo4_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo4_checkBox.setGeometry(QtCore.QRect(10, 390, 111, 17))
self.enableServo4_checkBox.setObjectName("enableServo4_checkBox")
self.Servo4_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo4_Pos_dial.setGeometry(QtCore.QRect(210, 410, 81, 81))
self.Servo4_Pos_dial.setMinimum(-920)
self.Servo4_Pos_dial.setMaximum(920)
self.Servo4_Pos_dial.setTracking(False)
self.Servo4_Pos_dial.setObjectName("Servo4_Pos_dial")
self.label_49 = QtWidgets.QLabel(self.frame_7)
self.label_49.setGeometry(QtCore.QRect(220, 390, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_49.setFont(font)
self.label_49.setObjectName("label_49")
self.manualServoPos4_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos4_Val.setGeometry(QtCore.QRect(100, 440, 91, 20))
self.manualServoPos4_Val.setObjectName("manualServoPos4_Val")
self.servo4_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo4_current_pos.setGeometry(QtCore.QRect(100, 410, 91, 20))
self.servo4_current_pos.setStatusTip("")
self.servo4_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo4_current_pos.setObjectName("servo4_current_pos")
self.label_50 = QtWidgets.QLabel(self.frame_7)
self.label_50.setGeometry(QtCore.QRect(10, 440, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_50.setFont(font)
self.label_50.setObjectName("label_50")
self.manualServoPos4_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos4_Cmd.setGeometry(QtCore.QRect(100, 470, 91, 23))
self.manualServoPos4_Cmd.setObjectName("manualServoPos4_Cmd")
self.label_51 = QtWidgets.QLabel(self.frame_7)
self.label_51.setGeometry(QtCore.QRect(10, 410, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_51.setFont(font)
self.label_51.setObjectName("label_51")
self.enableServo5_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo5_checkBox.setGeometry(QtCore.QRect(10, 510, 111, 17))
self.enableServo5_checkBox.setObjectName("enableServo5_checkBox")
self.Servo5_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo5_Pos_dial.setGeometry(QtCore.QRect(210, 530, 81, 81))
self.Servo5_Pos_dial.setMinimum(-920)
self.Servo5_Pos_dial.setMaximum(920)
self.Servo5_Pos_dial.setTracking(False)
self.Servo5_Pos_dial.setObjectName("Servo5_Pos_dial")
self.label_52 = QtWidgets.QLabel(self.frame_7)
self.label_52.setGeometry(QtCore.QRect(220, 510, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_52.setFont(font)
self.label_52.setObjectName("label_52")
self.manualServoPos5_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos5_Val.setGeometry(QtCore.QRect(100, 560, 91, 20))
self.manualServoPos5_Val.setObjectName("manualServoPos5_Val")
self.servo5_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo5_current_pos.setGeometry(QtCore.QRect(100, 530, 91, 20))
self.servo5_current_pos.setStatusTip("")
self.servo5_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo5_current_pos.setObjectName("servo5_current_pos")
self.label_53 = QtWidgets.QLabel(self.frame_7)
self.label_53.setGeometry(QtCore.QRect(10, 560, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_53.setFont(font)
self.label_53.setObjectName("label_53")
self.manualServoPos5_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos5_Cmd.setGeometry(QtCore.QRect(100, 590, 91, 23))
self.manualServoPos5_Cmd.setObjectName("manualServoPos5_Cmd")
self.label_54 = QtWidgets.QLabel(self.frame_7)
self.label_54.setGeometry(QtCore.QRect(10, 530, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_54.setFont(font)
self.label_54.setObjectName("label_54")
self.Servo1Steering_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.Servo1Steering_checkBox.setGeometry(QtCore.QRect(110, 30, 111, 17))
self.Servo1Steering_checkBox.setObjectName("Servo1Steering_checkBox")
self.val_servo1_offset = QtWidgets.QDoubleSpinBox(self.frame_7)
self.val_servo1_offset.setGeometry(QtCore.QRect(300, 50, 81, 18))
self.val_servo1_offset.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.val_servo1_offset.setReadOnly(False)
self.val_servo1_offset.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.val_servo1_offset.setSpecialValueText("")
self.val_servo1_offset.setDecimals(0)
self.val_servo1_offset.setMinimum(-200.0)
self.val_servo1_offset.setMaximum(200.0)
self.val_servo1_offset.setSingleStep(0.01)
self.val_servo1_offset.setProperty("value", 10.0)
self.val_servo1_offset.setObjectName("val_servo1_offset")
self.label_servo_6 = QtWidgets.QLabel(self.frame_7)
self.label_servo_6.setGeometry(QtCore.QRect(310, 70, 75, 13))
self.label_servo_6.setAlignment(QtCore.Qt.AlignCenter)
self.label_servo_6.setObjectName("label_servo_6")
self.val_servo1_scale = QtWidgets.QDoubleSpinBox(self.frame_7)
self.val_servo1_scale.setGeometry(QtCore.QRect(300, 130, 81, 18))
self.val_servo1_scale.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.val_servo1_scale.setReadOnly(False)
self.val_servo1_scale.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.val_servo1_scale.setSpecialValueText("")
self.val_servo1_scale.setDecimals(1)
self.val_servo1_scale.setMinimum(-180.0)
self.val_servo1_scale.setMaximum(180.0)
self.val_servo1_scale.setSingleStep(1.0)
self.val_servo1_scale.setProperty("value", 160.0)
self.val_servo1_scale.setObjectName("val_servo1_scale")
self.label_servo_9 = QtWidgets.QLabel(self.frame_7)
self.label_servo_9.setGeometry(QtCore.QRect(300, 110, 87, 13))
self.label_servo_9.setAlignment(QtCore.Qt.AlignCenter)
self.label_servo_9.setObjectName("label_servo_9")
self.label_servo_8 = QtWidgets.QLabel(self.frame_7)
self.label_servo_8.setGeometry(QtCore.QRect(303, 31, 81, 13))
self.label_servo_8.setAlignment(QtCore.Qt.AlignCenter)
self.label_servo_8.setObjectName("label_servo_8")
self.val_steer_distance = QtWidgets.QDoubleSpinBox(self.frame_7)
self.val_steer_distance.setGeometry(QtCore.QRect(300, 90, 81, 18))
self.val_steer_distance.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.val_steer_distance.setReadOnly(False)
self.val_steer_distance.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.val_steer_distance.setSpecialValueText("")
self.val_steer_distance.setDecimals(3)
self.val_steer_distance.setMinimum(-1.0)
self.val_steer_distance.setMaximum(1.0)
self.val_steer_distance.setSingleStep(0.01)
self.val_steer_distance.setProperty("value", 0.16)
self.val_steer_distance.setObjectName("val_steer_distance")
self.tabWidget.addTab(self.tab_4, "")
self.tab_5 = QtWidgets.QWidget()
self.tab_5.setObjectName("tab_5")
self.tabWidget.addTab(self.tab_5, "")
MainWindow.setCentralWidget(self.centralwidget)
self.menubar = QtWidgets.QMenuBar(MainWindow)
self.menubar.setGeometry(QtCore.QRect(0, 0, 1122, 21))
self.menubar.setObjectName("menubar")
self.menuFile = QtWidgets.QMenu(self.menubar)
self.menuFile.setObjectName("menuFile")
self.menuHelp = QtWidgets.QMenu(self.menubar)
self.menuHelp.setObjectName("menuHelp")
self.menuTools = QtWidgets.QMenu(self.menubar)
self.menuTools.setObjectName("menuTools")
MainWindow.setMenuBar(self.menubar)
self.statusbar = QtWidgets.QStatusBar(MainWindow)
self.statusbar.setObjectName("statusbar")
MainWindow.setStatusBar(self.statusbar)
self.menubar.addAction(self.menuFile.menuAction())
self.menubar.addAction(self.menuTools.menuAction())
self.menubar.addAction(self.menuHelp.menuAction())

self.retranslateUi(MainWindow)
self.tabWidget.setCurrentIndex(2)
self.tabWidget_2.setCurrentIndex(0)
QtCore.QMetaObject.connectSlotsByName(MainWindow)

def retranslateUi(self, MainWindow):
_translate = QtCore.QCoreApplication.translate
MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow"))
self.label_2.setText(_translate("MainWindow", "Network Status: "))
self.Disconnect_Cmd.setText(_translate("MainWindow", "Disconnect"))
self.connect_Cmd.setText(_translate("MainWindow", "Connect"))
self.label.setText(_translate("MainWindow", "IP adress:"))
self.IP_NetworkStatus_label.setText(_translate("MainWindow", "Status"))
self.label_7.setText(_translate("MainWindow", "INTERNET CONNECTION"))
self.SidebarMenu.setItemText(self.SidebarMenu.indexOf(self.SidebarMenuPage1), _translate("MainWindow", "Robot Connection"))
self.send_Cmd.setText(_translate("MainWindow", "Send"))
self.TX_checkBox.setText(_translate("MainWindow", "TX"))
self.RX_checkBox.setText(_translate("MainWindow", "RX"))
self.ConsoleHelp_Cmd.setText(_translate("MainWindow", "Help"))
self.ConsoleClear_Cmd.setText(_translate("MainWindow", "Clear"))
self.SidebarMenu.setItemText(self.SidebarMenu.indexOf(self.SidebarMenuPage2), _translate("MainWindow", "Debugger"))
self.main_start.setText(_translate("MainWindow", "Start"))
self.main_remote_ctrl.setToolTip(_translate("MainWindow", "<html><head/><body>Remote control is active,and mission paused.</body></html>"))
self.main_remote_ctrl.setText(_translate("MainWindow", "Enable Remote-Control"))
self.main_stop.setText(_translate("MainWindow", "Stop"))
self.main_mission_state.setToolTip(_translate("MainWindow", "<html><head/><body>Shows that a mission is active.Number is0: not active1: waiting 1 second to start2: running mission line6,7: listening to button (more button presses)8: stop mission</body></html>"))
self.main_mission_state.setText(_translate("MainWindow", "Enable Mission"))
self.SidebarMenu.setItemText(self.SidebarMenu.indexOf(self.SidebarMenuPage3), _translate("MainWindow", "Mission"))
self.label_5.setText(_translate("MainWindow", "Manual Speed:"))
self.enableKeyboardRC_checkBox.setToolTip(_translate("MainWindow", "<html><head/><body>Enable Remote Keyboard control</body></html>"))
self.enableKeyboardRC_checkBox.setText(_translate("MainWindow", "Enable keyboard control"))
self.guiRC_Reverse_Cmd.setText(_translate("MainWindow", "V"))
self.guiRC_Forward_Cmd.setText(_translate("MainWindow", "^"))
self.enableGuiRC_checkBox.setToolTip(_translate("MainWindow", "<html><head/><body>Enable Remote - GUI control </body></html>"))
self.enableGuiRC_checkBox.setText(_translate("MainWindow", "Enable GUI control"))
self.guiRC_Right_Cmd.setText(_translate("MainWindow", ">"))
self.guiRC_Left_Cmd.setText(_translate("MainWindow", "<"))
self.label_8.setText(_translate("MainWindow", "Motor Remote GUI control"))
self.label_9.setText(_translate("MainWindow", "Robot Motor hardware parameters"))
self.label_21.setToolTip(_translate("MainWindow", "<html><head/><body>Number of encoder pulses on each motor revolution (normally 48)</body></html>"))
self.label_21.setText(_translate("MainWindow", "Enc. pulse per rev:"))
self.label_22.setText(_translate("MainWindow", "Gear ratio [N]:"))
self.Robot_GearRatio.setToolTip(_translate("MainWindow", "<html><head/><body>Gear ratio from motor to wheel axle (9.68).</body></html>"))
self.label_24.setText(_translate("MainWindow", "Wheel radius (Left) [m]:"))
self.label_127.setText(_translate("MainWindow", "Robot ID:"))
self.label_128.setText(_translate("MainWindow", "Hardware:"))
self.label_25.setToolTip(_translate("MainWindow", "<html><head/><body>Wheel base is distance between wheels (mid-wheel to mid-wheel) for turn calculations.</body></html>"))
self.label_25.setText(_translate("MainWindow", "Wheel-base [m]:"))
self.Robot_PulsePerRev.setToolTip(_translate("MainWindow", "<html><head/><body>Number of encoder ticks per motor revolution (48)</body></html>"))
self.label_26.setText(_translate("MainWindow", "Wheel radius (Right) [m]:"))
self.Robot_RevEncoder_Right.setToolTip(_translate("MainWindow", "<html><head/><body>Checking this makes odometry go the other way, i.e. encoder pins swapped (right motor).This should NOT be checked on normal REGBOT.</body></html>"))
self.Robot_RevEncoder_Right.setText(_translate("MainWindow", "Reverse Right Encoder"))
self.Robot_RevMotor.setToolTip(_translate("MainWindow", "<html><head/><body>Checking this makes wheels go the other way when orderd forward.This should NOT be checked on normal REGBOT.</body></html>"))
self.Robot_RevMotor.setText(_translate("MainWindow", "Reverse Motor"))
self.Robot_RevEncoder_Left.setToolTip(_translate("MainWindow", "<html><head/><body>Checking this makes odometry go the other way, i.e. encoder pins swapped.This should NOT be checked on normal REGBOT.</body></html>"))
self.Robot_RevEncoder_Left.setText(_translate("MainWindow", "Reverse Left Encoder"))
self.label_222.setToolTip(_translate("MainWindow", "<html><head/><body>Battery is assumed to be of type LiPo 3cell. Any battery with a normal voltage between 9 and 15V should do.</body></html>"))
self.label_222.setText(_translate("MainWindow", "Go idle at [V]:"))
self.save_id_on_robot.setText(_translate("MainWindow", "Save"))
self.label_129.setText(_translate("MainWindow", "Tilt offset [Rad]:"))
self.Robot_edit.setText(_translate("MainWindow", "Edit"))
self.Robot_OnBattery.setText(_translate("MainWindow", "Robot On Battery"))
self.robot_cancel.setText(_translate("MainWindow", "Cancel"))
self.label_36.setText(_translate("MainWindow", "Right"))
self.label_37.setText(_translate("MainWindow", "Left"))
self.label_42.setText(_translate("MainWindow", "Current [A]:"))
self.label_28.setText(_translate("MainWindow", "Encoder:"))
self.label_38.setText(_translate("MainWindow", "Drive information"))
self.label_32.setText(_translate("MainWindow", "Distance [m]:"))
self.label_29.setText(_translate("MainWindow", "Wheel Speed [m/s]:"))
self.tabWidget_2.setTabText(self.tabWidget_2.indexOf(self.tab_6), _translate("MainWindow", "Drive Info"))
self.label_76.setText(_translate("MainWindow", "degrees"))
self.label_45.setText(_translate("MainWindow", "Pose (relative to starting/reset point)"))
self.Robot_PoseReset.setText(_translate("MainWindow", "reset"))
self.label_44.setText(_translate("MainWindow", "Heading [rad]:"))
self.label_34.setText(_translate("MainWindow", "X (forward) [m]:"))
self.label_35.setText(_translate("MainWindow", "Y (left) [m]:"))
self.label_125.setText(_translate("MainWindow", "Tilt [rad]:"))
self.tabWidget_2.setTabText(self.tabWidget_2.indexOf(self.tab_7), _translate("MainWindow", "Pose Info"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab), _translate("MainWindow", "Motor"))
self.label_10.setText(_translate("MainWindow", "Line Sensor"))
self.enableLineSensor_checkBox.setText(_translate("MainWindow", "Enable Line Sensor"))
self.line_bar_1.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_4.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_6.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_5.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_3.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_7.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_2.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_8.setFormat(_translate("MainWindow", "%p%"))
self.label_11.setText(_translate("MainWindow", "Sensor readings:"))
self.label_12.setText(_translate("MainWindow", "L1"))
self.label_13.setText(_translate("MainWindow", "L2"))
self.label_14.setText(_translate("MainWindow", "L6"))
self.label_15.setText(_translate("MainWindow", "L3"))
self.label_16.setText(_translate("MainWindow", "L4"))
self.label_17.setText(_translate("MainWindow", "L5"))
self.label_18.setText(_translate("MainWindow", "L7"))
self.label_19.setText(_translate("MainWindow", "L8"))
self.label_118.setText(_translate("MainWindow", "Left edge"))
self.ls_left_edge.setToolTip(_translate("MainWindow", "<html><head/><body>Left edge positionGray if an edge is not detected</body></html>"))
self.ls_right_edge.setToolTip(_translate("MainWindow", "<html><head/><body>Right edge positionEither no edge is valid or both are valid.</body></html>"))
self.label_121.setText(_translate("MainWindow", "Right edge"))
self.label_20.setText(_translate("MainWindow", "Edge Analysis:"))
self.label_27.setText(_translate("MainWindow", "Crossing"))
self.label_171.setText(_translate("MainWindow", "Line valid"))
self.ls_crossing_cnt.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.ls_line_valid_cnt.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 5.0 = no line, 5=line seen in 5ms. </body></html>"))
self.groupBox_5.setTitle(_translate("MainWindow", "Sensor Calibration"))
self.CalibrateWhite_Cmd.setText(_translate("MainWindow", "Calibrate White"))
self.label_217.setText(_translate("MainWindow", "L3"))
self.label_218.setText(_translate("MainWindow", "L5"))
self.label_219.setText(_translate("MainWindow", "L7"))
self.label_221.setText(_translate("MainWindow", "L4"))
self.label_223.setText(_translate("MainWindow", "L6"))
self.label_224.setText(_translate("MainWindow", "L1"))
self.label_225.setText(_translate("MainWindow", "L8"))
self.label_226.setText(_translate("MainWindow", "L2"))
self.ls_max_white_7.setText(_translate("MainWindow", "L1"))
self.ls_max_white_8.setText(_translate("MainWindow", "L1"))
self.ls_max_white_1.setText(_translate("MainWindow", "L1"))
self.ls_max_white_3.setText(_translate("MainWindow", "L1"))
self.ls_max_white_5.setText(_translate("MainWindow", "L1"))
self.ls_max_white_2.setText(_translate("MainWindow", "L1"))
self.ls_max_white_4.setText(_translate("MainWindow", "L1"))
self.ls_max_white_6.setText(_translate("MainWindow", "L1"))
self.ls_max_black_5.setText(_translate("MainWindow", "L1"))
self.ls_max_black_4.setText(_translate("MainWindow", "L1"))
self.ls_max_black_8.setText(_translate("MainWindow", "L1"))
self.ls_max_black_1.setText(_translate("MainWindow", "L1"))
self.ls_max_black_2.setText(_translate("MainWindow", "L1"))
self.ls_max_black_3.setText(_translate("MainWindow", "L1"))
self.ls_max_black_6.setText(_translate("MainWindow", "L1"))
self.ls_max_black_7.setText(_translate("MainWindow", "L1"))
self.label_227.setText(_translate("MainWindow", "White"))
self.CalibrateBlack_Cmd.setText(_translate("MainWindow", "Calibrate Black "))
self.label_228.setText(_translate("MainWindow", "Black"))
self.ls_swap_left_right.setToolTip(_translate("MainWindow", "<html><head/><body>Swap the detected left and right edge.Check this if LED\'s are (mostly) in the forward (X) direction.(Normally the LED end of the line sensor PCB is closer to driving wheel axis).</body></html>"))
self.ls_swap_left_right.setText(_translate("MainWindow", "Swap Sides"))
self.ls_show_normalized.setToolTip(_translate("MainWindow", "<html><head/><body>If line is not white (bright), then assumed black (dark)The normalized value should be shown as 75%.</body></html>"))
self.ls_show_normalized.setText(_translate("MainWindow", "Show normalized"))
self.label_23.setText(_translate("MainWindow", "Crossing det."))
self.label_156.setText(_translate("MainWindow", "Max RAW"))
self.ls_crossing_detect.setToolTip(_translate("MainWindow", "<html><head/><body>Distance between left and right edge in a 0 to 8 scale.Value depends distance between edges with an extra value if width increse fast. Therefore the width can temorarily be higher than 6 (edge values are from -3 to 3)this should make it easier to detect a crossing.For Regbot in balance (and normal tape (38mm)), then about 4.8 cm to trigger crossing line.For Robobot this could be as low as 3-4, as the linesensor is wider.</body></html>"))
self.ls_power_high.setToolTip(_translate("MainWindow", "<html><head/><body>If not in auto mode, should the line sensor LED power be high (ca. 30mA), else low (ca 10mA)</body></html>"))
self.ls_power_high.setText(_translate("MainWindow", "High power"))
self.ls_line_white.setToolTip(_translate("MainWindow", "<html><head/><body>If line is not white (bright), then assumed black (dark)</body></html>"))
self.ls_line_white.setText(_translate("MainWindow", "White line"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_2), _translate("MainWindow", "Edge Detection"))
self.label_56.setText(_translate("MainWindow", "Distance sensor"))
self.label_279.setText(_translate("MainWindow", "D1 right"))
self.label_281.setText(_translate("MainWindow", "D1 right RAW"))
self.label_287.setText(_translate("MainWindow", "m"))
self.label_280.setText(_translate("MainWindow", "D2 front"))
self.label_282.setText(_translate("MainWindow", "D2 front RAW"))
self.label_288.setText(_translate("MainWindow", "m"))
self.label_284.setText(_translate("MainWindow", "D1 right (50cm)"))
self.label_286.setText(_translate("MainWindow", "D2 front (13cm)"))
self.label_285.setText(_translate("MainWindow", "D2 front (50cm)"))
self.label_283.setText(_translate("MainWindow", "D1 right (13cm)"))
self.checkBox_ir_use.setToolTip(_translate("MainWindow", "<html><head/><body>Should be turned on at power on (will be turned on automatically in missions that use the sensor - if installed). If not installed, then distance is 10m</body></html>"))
self.checkBox_ir_use.setText(_translate("MainWindow", "Sensor Enable"))
self.checkBox_ir_installed.setToolTip(_translate("MainWindow", "<html><head/><body>Is the sensor installed (can not be turned on if not installed)</body></html>"))
self.checkBox_ir_installed.setText(_translate("MainWindow", "Sensor Installed"))
self.label_277.setText(_translate("MainWindow", "Calibrate values (RAW)"))
self.ir_apply.setText(_translate("MainWindow", "Apply"))
self.label_278.setText(_translate("MainWindow", "Distance values (measured)"))
self.ir_edit.setText(_translate("MainWindow", "Edit"))
self.ir_cancel.setText(_translate("MainWindow", "Cancel"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_8), _translate("MainWindow", "IR Distance"))
self.EnableCamera_checkBox.setToolTip(_translate("MainWindow", "<html><head/><body>To enable Camera stream, install mjpeg-streamer and run follwing commands: cd mjpg-streamer/mjpg-streamer-experimental/ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"</body></html>"))
self.EnableCamera_checkBox.setText(_translate("MainWindow", "Enable Camera"))
self.Camera_Record_Cmd.setText(_translate("MainWindow", "Record"))
self.Camera_StopRecord_Cmd.setText(_translate("MainWindow", "Stop Recording"))
self.Camera_Snapshot_Cmd.setText(_translate("MainWindow", "Take Snapshot"))
self.label_55.setText(_translate("MainWindow", "Snapshot Save Filename"))
self.Camera_SnapSave_Cmd.setText(_translate("MainWindow", "Save"))
self.VideoRecording_Label.setText(_translate("MainWindow", "<html><head/><body>RECORDING...</body></html>"))
self.Camera_ClearSnap_Cmd.setText(_translate("MainWindow", "Clear Snap"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_3), _translate("MainWindow", "Camera"))
self.label_30.setText(_translate("MainWindow", "Servo Controls"))
self.enableServo1_checkBox.setText(_translate("MainWindow", "Enable Servo 1"))
self.enableServo2_checkBox.setText(_translate("MainWindow", "Enable Servo 2"))
self.enableServo3_checkBox.setText(_translate("MainWindow", "Enable Servo 3"))
self.label_31.setText(_translate("MainWindow", "Current Position"))
self.servo1_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_33.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos1_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_39.setText(_translate("MainWindow", "Position Dial"))
self.label_40.setText(_translate("MainWindow", "Position Dial"))
self.servo2_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_41.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos2_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_43.setText(_translate("MainWindow", "Current Position"))
self.label_46.setText(_translate("MainWindow", "Position Dial"))
self.servo3_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_47.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos3_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_48.setText(_translate("MainWindow", "Current Position"))
self.enableServo4_checkBox.setText(_translate("MainWindow", "Enable Servo 4"))
self.label_49.setText(_translate("MainWindow", "Position Dial"))
self.servo4_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_50.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos4_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_51.setText(_translate("MainWindow", "Current Position"))
self.enableServo5_checkBox.setText(_translate("MainWindow", "Enable Servo 5"))
self.label_52.setText(_translate("MainWindow", "Position Dial"))
self.servo5_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_53.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos5_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_54.setText(_translate("MainWindow", "Current Position"))
self.Servo1Steering_checkBox.setText(_translate("MainWindow", "Enable Steering"))
self.val_servo1_offset.setToolTip(_translate("MainWindow", "<html><head/><body>Steering offset to align steering wheel with straight forward.Units is control units, i.e. about 50 is 5 degrees.Zero offset means forward is 1.5ms pulse (center for servo)</body></html>"))
self.label_servo_6.setText(_translate("MainWindow", "distance"))
self.val_servo1_scale.setToolTip(_translate("MainWindow", "<html><head/><body>Servo angle change from 1ms to 2ms PWM pulse to servo.(typically 160 deg for hitec7235 and 90 deg for "old" servos)</body></html>"))
self.label_servo_9.setText(_translate("MainWindow", "scale"))
self.label_servo_8.setText(_translate("MainWindow", "Offset"))
self.val_steer_distance.setToolTip(_translate("MainWindow", "<html><head/><body>Distance from drive axle to steering wheel.Positive is forward (front wheel). In meters.Typically 0.16 m for ROBOBOT.</body></html>"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_4), _translate("MainWindow", "Servo"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_5), _translate("MainWindow", "Mission"))
self.menuFile.setTitle(_translate("MainWindow", "File"))
self.menuHelp.setTitle(_translate("MainWindow", "Help"))
self.menuTools.setTitle(_translate("MainWindow", "Tools"))

if __name__ == "__main__":
import sys
app = QtWidgets.QApplication(sys.argv)
MainWindow = QtWidgets.QMainWindow()
ui = Ui_MainWindow()
ui.setupUi(MainWindow)
MainWindow.show()
sys.exit(app.exec_())

Source Code

2020-12-06T17:43:40Z

S192320: /* User Interface Code */

==User Interface Code==

[[Code]]

==Main Application Code==

==Linesensor==

==IR distance sensors==

==Servo==

==Video Threads==

Source Code

2020-12-06T17:42:20Z

S192320: /* User Interface Code */

==User Interface Code==

==Main Application Code==

==Linesensor==

==IR distance sensors==

==Servo==

==Video Threads==

User Interface Code

2020-12-06T17:41:09Z

S192320: Blanked the page

User Interface Code

2020-12-06T17:40:31Z

S192320:

# -*- coding: utf-8 -*-

# Form implementation generated from reading ui file 'Regbot_V1_9.ui'
#
# Created by: PyQt5 UI code generator 5.13.2
#
# WARNING! All changes made in this file will be lost!

from PyQt5 import QtCore, QtGui, QtWidgets

class Ui_MainWindow(object):
def setupUi(self, MainWindow):
MainWindow.setObjectName("MainWindow")
MainWindow.resize(1122, 555)
self.centralwidget = QtWidgets.QWidget(MainWindow)
self.centralwidget.setObjectName("centralwidget")
self.SidebarMenu = QtWidgets.QToolBox(self.centralwidget)
self.SidebarMenu.setGeometry(QtCore.QRect(0, 0, 331, 721))
self.SidebarMenu.setAutoFillBackground(True)
self.SidebarMenu.setObjectName("SidebarMenu")
self.SidebarMenuPage1 = QtWidgets.QWidget()
self.SidebarMenuPage1.setGeometry(QtCore.QRect(0, 0, 331, 640))
self.SidebarMenuPage1.setObjectName("SidebarMenuPage1")
self.line = QtWidgets.QFrame(self.SidebarMenuPage1)
self.line.setGeometry(QtCore.QRect(10, 220, 311, 21))
self.line.setFrameShape(QtWidgets.QFrame.HLine)
self.line.setFrameShadow(QtWidgets.QFrame.Sunken)
self.line.setObjectName("line")
self.IP_Connect_frame = QtWidgets.QFrame(self.SidebarMenuPage1)
self.IP_Connect_frame.setGeometry(QtCore.QRect(0, 0, 331, 111))
self.IP_Connect_frame.setAutoFillBackground(True)
self.IP_Connect_frame.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.IP_Connect_frame.setFrameShadow(QtWidgets.QFrame.Raised)
self.IP_Connect_frame.setObjectName("IP_Connect_frame")
self.label_2 = QtWidgets.QLabel(self.IP_Connect_frame)
self.label_2.setGeometry(QtCore.QRect(30, 90, 91, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_2.setFont(font)
self.label_2.setObjectName("label_2")
self.Disconnect_Cmd = QtWidgets.QPushButton(self.IP_Connect_frame)
self.Disconnect_Cmd.setGeometry(QtCore.QRect(180, 60, 121, 23))
self.Disconnect_Cmd.setObjectName("Disconnect_Cmd")
self.connect_Cmd = QtWidgets.QPushButton(self.IP_Connect_frame)
self.connect_Cmd.setGeometry(QtCore.QRect(30, 60, 131, 23))
self.connect_Cmd.setObjectName("connect_Cmd")
self.label = QtWidgets.QLabel(self.IP_Connect_frame)
self.label.setGeometry(QtCore.QRect(120, 30, 61, 21))
self.label.setObjectName("label")
self.IP_Adress_field = QtWidgets.QLineEdit(self.IP_Connect_frame)
self.IP_Adress_field.setGeometry(QtCore.QRect(170, 30, 131, 20))
self.IP_Adress_field.setObjectName("IP_Adress_field")
self.IP_NetworkStatus_label = QtWidgets.QLabel(self.IP_Connect_frame)
self.IP_NetworkStatus_label.setGeometry(QtCore.QRect(140, 90, 141, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.IP_NetworkStatus_label.setFont(font)
self.IP_NetworkStatus_label.setObjectName("IP_NetworkStatus_label")
self.label_7 = QtWidgets.QLabel(self.IP_Connect_frame)
self.label_7.setGeometry(QtCore.QRect(10, 10, 131, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_7.setFont(font)
self.label_7.setObjectName("label_7")
self.SidebarMenu.addItem(self.SidebarMenuPage1, "")
self.SidebarMenuPage2 = QtWidgets.QWidget()
self.SidebarMenuPage2.setGeometry(QtCore.QRect(0, 0, 331, 640))
self.SidebarMenuPage2.setObjectName("SidebarMenuPage2")
self.TextCmd_field = QtWidgets.QLineEdit(self.SidebarMenuPage2)
self.TextCmd_field.setGeometry(QtCore.QRect(10, 580, 311, 20))
self.TextCmd_field.setObjectName("TextCmd_field")
self.send_Cmd = QtWidgets.QPushButton(self.SidebarMenuPage2)
self.send_Cmd.setGeometry(QtCore.QRect(240, 610, 81, 23))
self.send_Cmd.setObjectName("send_Cmd")
self.TX_checkBox = QtWidgets.QCheckBox(self.SidebarMenuPage2)
self.TX_checkBox.setGeometry(QtCore.QRect(10, 610, 41, 17))
self.TX_checkBox.setObjectName("TX_checkBox")
self.RX_checkBox = QtWidgets.QCheckBox(self.SidebarMenuPage2)
self.RX_checkBox.setGeometry(QtCore.QRect(50, 610, 41, 17))
self.RX_checkBox.setObjectName("RX_checkBox")
self.ConsoleHelp_Cmd = QtWidgets.QPushButton(self.SidebarMenuPage2)
self.ConsoleHelp_Cmd.setGeometry(QtCore.QRect(150, 610, 81, 23))
self.ConsoleHelp_Cmd.setObjectName("ConsoleHelp_Cmd")
self.CommandConsole_Window = QtWidgets.QPlainTextEdit(self.SidebarMenuPage2)
self.CommandConsole_Window.setGeometry(QtCore.QRect(10, 0, 311, 571))
self.CommandConsole_Window.setAcceptDrops(False)
self.CommandConsole_Window.setToolTip("")
self.CommandConsole_Window.setHorizontalScrollBarPolicy(QtCore.Qt.ScrollBarAsNeeded)
self.CommandConsole_Window.setLineWrapMode(QtWidgets.QPlainTextEdit.NoWrap)
self.CommandConsole_Window.setObjectName("CommandConsole_Window")
self.ConsoleClear_Cmd = QtWidgets.QPushButton(self.SidebarMenuPage2)
self.ConsoleClear_Cmd.setGeometry(QtCore.QRect(90, 610, 51, 23))
self.ConsoleClear_Cmd.setObjectName("ConsoleClear_Cmd")
self.SidebarMenu.addItem(self.SidebarMenuPage2, "")
self.SidebarMenuPage3 = QtWidgets.QWidget()
self.SidebarMenuPage3.setGeometry(QtCore.QRect(0, 0, 331, 640))
self.SidebarMenuPage3.setObjectName("SidebarMenuPage3")
self.main_start = QtWidgets.QPushButton(self.SidebarMenuPage3)
self.main_start.setGeometry(QtCore.QRect(170, 60, 151, 23))
self.main_start.setObjectName("main_start")
self.main_remote_ctrl = QtWidgets.QCheckBox(self.SidebarMenuPage3)
self.main_remote_ctrl.setGeometry(QtCore.QRect(10, 30, 141, 17))
self.main_remote_ctrl.setTristate(True)
self.main_remote_ctrl.setObjectName("main_remote_ctrl")
self.main_stop = QtWidgets.QPushButton(self.SidebarMenuPage3)
self.main_stop.setGeometry(QtCore.QRect(10, 60, 151, 23))
self.main_stop.setObjectName("main_stop")
self.main_mission_state = QtWidgets.QCheckBox(self.SidebarMenuPage3)
self.main_mission_state.setGeometry(QtCore.QRect(10, 10, 101, 17))
self.main_mission_state.setTristate(True)
self.main_mission_state.setObjectName("main_mission_state")
self.SidebarMenu.addItem(self.SidebarMenuPage3, "")
self.tabWidget = QtWidgets.QTabWidget(self.centralwidget)
self.tabWidget.setGeometry(QtCore.QRect(330, 0, 961, 721))
self.tabWidget.setAutoFillBackground(True)
self.tabWidget.setObjectName("tabWidget")
self.tab = QtWidgets.QWidget()
self.tab.setObjectName("tab")
self.MotorGUI_ctrl_frame = QtWidgets.QFrame(self.tab)
self.MotorGUI_ctrl_frame.setGeometry(QtCore.QRect(0, 0, 321, 171))
self.MotorGUI_ctrl_frame.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.MotorGUI_ctrl_frame.setFrameShadow(QtWidgets.QFrame.Raised)
self.MotorGUI_ctrl_frame.setObjectName("MotorGUI_ctrl_frame")
self.label_5 = QtWidgets.QLabel(self.MotorGUI_ctrl_frame)
self.label_5.setGeometry(QtCore.QRect(140, 100, 71, 21))
self.label_5.setObjectName("label_5")
self.enableKeyboardRC_checkBox = QtWidgets.QCheckBox(self.MotorGUI_ctrl_frame)
self.enableKeyboardRC_checkBox.setGeometry(QtCore.QRect(150, 40, 141, 17))
self.enableKeyboardRC_checkBox.setObjectName("enableKeyboardRC_checkBox")
self.guiRC_Reverse_Cmd = QtWidgets.QPushButton(self.MotorGUI_ctrl_frame)
self.guiRC_Reverse_Cmd.setGeometry(QtCore.QRect(50, 110, 31, 31))
self.guiRC_Reverse_Cmd.setObjectName("guiRC_Reverse_Cmd")
self.guiRC_Forward_Cmd = QtWidgets.QPushButton(self.MotorGUI_ctrl_frame)
self.guiRC_Forward_Cmd.setGeometry(QtCore.QRect(50, 50, 31, 31))
self.guiRC_Forward_Cmd.setObjectName("guiRC_Forward_Cmd")
self.manualGuiSpeed_Cmd = QtWidgets.QLineEdit(self.MotorGUI_ctrl_frame)
self.manualGuiSpeed_Cmd.setGeometry(QtCore.QRect(220, 100, 51, 20))
self.manualGuiSpeed_Cmd.setObjectName("manualGuiSpeed_Cmd")
self.enableGuiRC_checkBox = QtWidgets.QCheckBox(self.MotorGUI_ctrl_frame)
self.enableGuiRC_checkBox.setGeometry(QtCore.QRect(150, 60, 111, 17))
self.enableGuiRC_checkBox.setObjectName("enableGuiRC_checkBox")
self.guiRC_Right_Cmd = QtWidgets.QPushButton(self.MotorGUI_ctrl_frame)
self.guiRC_Right_Cmd.setGeometry(QtCore.QRect(80, 80, 31, 31))
self.guiRC_Right_Cmd.setObjectName("guiRC_Right_Cmd")
self.guiRC_Left_Cmd = QtWidgets.QPushButton(self.MotorGUI_ctrl_frame)
self.guiRC_Left_Cmd.setGeometry(QtCore.QRect(20, 80, 31, 31))
self.guiRC_Left_Cmd.setObjectName("guiRC_Left_Cmd")
self.label_8 = QtWidgets.QLabel(self.MotorGUI_ctrl_frame)
self.label_8.setGeometry(QtCore.QRect(10, 10, 151, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_8.setFont(font)
self.label_8.setObjectName("label_8")
self.frame = QtWidgets.QFrame(self.tab)
self.frame.setGeometry(QtCore.QRect(0, 170, 321, 321))
self.frame.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame.setObjectName("frame")
self.label_9 = QtWidgets.QLabel(self.frame)
self.label_9.setGeometry(QtCore.QRect(10, 10, 201, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_9.setFont(font)
self.label_9.setObjectName("label_9")
self.label_21 = QtWidgets.QLabel(self.frame)
self.label_21.setGeometry(QtCore.QRect(10, 149, 91, 21))
self.label_21.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_21.setObjectName("label_21")
self.Robot_WheelRadius_Right = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_WheelRadius_Right.setGeometry(QtCore.QRect(109, 230, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_WheelRadius_Right.sizePolicy().hasHeightForWidth())
self.Robot_WheelRadius_Right.setSizePolicy(sizePolicy)
self.Robot_WheelRadius_Right.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_WheelRadius_Right.setToolTip("<html><head/><body>Wheel radius</body></html>")
self.Robot_WheelRadius_Right.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelRadius_Right.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelRadius_Right.setReadOnly(False)
self.Robot_WheelRadius_Right.setDecimals(4)
self.Robot_WheelRadius_Right.setMinimum(0.001)
self.Robot_WheelRadius_Right.setMaximum(0.99)
self.Robot_WheelRadius_Right.setSingleStep(0.001)
self.Robot_WheelRadius_Right.setProperty("value", 0.03)
self.Robot_WheelRadius_Right.setObjectName("Robot_WheelRadius_Right")
self.Robot_WheelRadius_Left = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_WheelRadius_Left.setGeometry(QtCore.QRect(109, 190, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_WheelRadius_Left.sizePolicy().hasHeightForWidth())
self.Robot_WheelRadius_Left.setSizePolicy(sizePolicy)
self.Robot_WheelRadius_Left.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_WheelRadius_Left.setToolTip("<html><head/><body>Wheel radius</body></html>")
self.Robot_WheelRadius_Left.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelRadius_Left.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelRadius_Left.setReadOnly(False)
self.Robot_WheelRadius_Left.setDecimals(4)
self.Robot_WheelRadius_Left.setMinimum(0.001)
self.Robot_WheelRadius_Left.setMaximum(0.99)
self.Robot_WheelRadius_Left.setSingleStep(0.001)
self.Robot_WheelRadius_Left.setProperty("value", 0.03)
self.Robot_WheelRadius_Left.setObjectName("Robot_WheelRadius_Left")
self.label_22 = QtWidgets.QLabel(self.frame)
self.label_22.setGeometry(QtCore.QRect(10, 118, 91, 23))
self.label_22.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_22.setObjectName("label_22")
self.Robot_GearRatio = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_GearRatio.setGeometry(QtCore.QRect(110, 120, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_GearRatio.sizePolicy().hasHeightForWidth())
self.Robot_GearRatio.setSizePolicy(sizePolicy)
self.Robot_GearRatio.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_GearRatio.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_GearRatio.setReadOnly(False)
self.Robot_GearRatio.setDecimals(3)
self.Robot_GearRatio.setMaximum(999.99)
self.Robot_GearRatio.setSingleStep(0.001)
self.Robot_GearRatio.setProperty("value", 9.68)
self.Robot_GearRatio.setObjectName("Robot_GearRatio")
self.label_24 = QtWidgets.QLabel(self.frame)
self.label_24.setGeometry(QtCore.QRect(10, 169, 121, 20))
self.label_24.setAlignment(QtCore.Qt.AlignCenter)
self.label_24.setObjectName("label_24")
self.label_127 = QtWidgets.QLabel(self.frame)
self.label_127.setGeometry(QtCore.QRect(50, 30, 51, 20))
self.label_127.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_127.setObjectName("label_127")
self.Robot_WheelBase = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_WheelBase.setGeometry(QtCore.QRect(110, 90, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_WheelBase.sizePolicy().hasHeightForWidth())
self.Robot_WheelBase.setSizePolicy(sizePolicy)
self.Robot_WheelBase.setToolTip("<html><head/><body>Distance between drivng wheels (0.155m)</body></html>")
self.Robot_WheelBase.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelBase.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelBase.setReadOnly(False)
self.Robot_WheelBase.setDecimals(4)
self.Robot_WheelBase.setMaximum(5.0)
self.Robot_WheelBase.setSingleStep(0.01)
self.Robot_WheelBase.setProperty("value", 0.155)
self.Robot_WheelBase.setObjectName("Robot_WheelBase")
self.label_128 = QtWidgets.QLabel(self.frame)
self.label_128.setGeometry(QtCore.QRect(50, 60, 51, 20))
self.label_128.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_128.setObjectName("label_128")
self.label_25 = QtWidgets.QLabel(self.frame)
self.label_25.setGeometry(QtCore.QRect(20, 90, 81, 21))
self.label_25.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_25.setObjectName("label_25")
self.RobotID = QtWidgets.QDoubleSpinBox(self.frame)
self.RobotID.setEnabled(True)
self.RobotID.setGeometry(QtCore.QRect(110, 31, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.RobotID.sizePolicy().hasHeightForWidth())
self.RobotID.setSizePolicy(sizePolicy)
self.RobotID.setToolTip("<html><head/><body>This is the ID of the robot [1..9999].To change the ID of robot: change ID, click "save", click "save on robot".Change ID to -1 to revert to factory settings. And then change ID back to right value</body></html>")
self.RobotID.setStatusTip("")
self.RobotID.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.RobotID.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.RobotID.setReadOnly(False)
self.RobotID.setSpecialValueText("")
self.RobotID.setSuffix("")
self.RobotID.setDecimals(0)
self.RobotID.setMinimum(-2.0)
self.RobotID.setMaximum(10000.0)
self.RobotID.setObjectName("RobotID")
self.RobotHW_type = QtWidgets.QDoubleSpinBox(self.frame)
self.RobotHW_type.setEnabled(True)
self.RobotHW_type.setGeometry(QtCore.QRect(110, 60, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.RobotHW_type.sizePolicy().hasHeightForWidth())
self.RobotHW_type.setSizePolicy(sizePolicy)
self.RobotHW_type.setToolTip("<html><head/><body>The hardware version1 = old version with no wifi nor line sensor plugs on main board and round power supply plug on main board.2 = more plugs and satellite PCB for power management and wifi.3 = power management and on-off switch on-board (big motor controller board)4 = same as 3, but with small motor controller board.5 = same as 2, but updated with new sattelite boards (power and wifi) etc.6 = hardware version 4 (motor controller integrated on board)To change: click edit change HW setting, click "save", click "save to flash".(ID must be > 0)</body></html>")
self.RobotHW_type.setStatusTip("")
self.RobotHW_type.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.RobotHW_type.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.RobotHW_type.setReadOnly(False)
self.RobotHW_type.setSpecialValueText("")
self.RobotHW_type.setSuffix("")
self.RobotHW_type.setDecimals(0)
self.RobotHW_type.setMinimum(-2.0)
self.RobotHW_type.setMaximum(10.0)
self.RobotHW_type.setObjectName("RobotHW_type")
self.Robot_PulsePerRev = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_PulsePerRev.setGeometry(QtCore.QRect(110, 150, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_PulsePerRev.sizePolicy().hasHeightForWidth())
self.Robot_PulsePerRev.setSizePolicy(sizePolicy)
self.Robot_PulsePerRev.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PulsePerRev.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PulsePerRev.setReadOnly(False)
self.Robot_PulsePerRev.setDecimals(0)
self.Robot_PulsePerRev.setMinimum(1.0)
self.Robot_PulsePerRev.setMaximum(1000.0)
self.Robot_PulsePerRev.setProperty("value", 48.0)
self.Robot_PulsePerRev.setObjectName("Robot_PulsePerRev")
self.label_26 = QtWidgets.QLabel(self.frame)
self.label_26.setGeometry(QtCore.QRect(10, 210, 121, 20))
self.label_26.setAlignment(QtCore.Qt.AlignCenter)
self.label_26.setObjectName("label_26")
self.Robot_RevEncoder_Right = QtWidgets.QCheckBox(self.frame)
self.Robot_RevEncoder_Right.setGeometry(QtCore.QRect(190, 90, 131, 21))
self.Robot_RevEncoder_Right.setObjectName("Robot_RevEncoder_Right")
self.Robot_RevMotor = QtWidgets.QCheckBox(self.frame)
self.Robot_RevMotor.setGeometry(QtCore.QRect(190, 33, 92, 16))
self.Robot_RevMotor.setObjectName("Robot_RevMotor")
self.Robot_RevEncoder_Left = QtWidgets.QCheckBox(self.frame)
self.Robot_RevEncoder_Left.setGeometry(QtCore.QRect(190, 60, 131, 21))
self.Robot_RevEncoder_Left.setObjectName("Robot_RevEncoder_Left")
self.Robot_BatteryIdleVolts = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_BatteryIdleVolts.setGeometry(QtCore.QRect(110, 290, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_BatteryIdleVolts.sizePolicy().hasHeightForWidth())
self.Robot_BatteryIdleVolts.setSizePolicy(sizePolicy)
self.Robot_BatteryIdleVolts.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_BatteryIdleVolts.setToolTip("<html><head/><body>Go idle when battery voltage is below this level.For a 3 cell LiPo battery it should go in idle mode at 3 x 3.2V = 9.6VAt 10V about 10% capacity is left before battery destruction, and in idle mode the robot uses about 30mA, so battey will selfdestruct after another about 3 hours.So remember to switch off and to recharge.</body></html>")
self.Robot_BatteryIdleVolts.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_BatteryIdleVolts.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_BatteryIdleVolts.setReadOnly(False)
self.Robot_BatteryIdleVolts.setDecimals(1)
self.Robot_BatteryIdleVolts.setMinimum(9.0)
self.Robot_BatteryIdleVolts.setMaximum(17.0)
self.Robot_BatteryIdleVolts.setSingleStep(0.1)
self.Robot_BatteryIdleVolts.setProperty("value", 9.9)
self.Robot_BatteryIdleVolts.setObjectName("Robot_BatteryIdleVolts")
self.label_222 = QtWidgets.QLabel(self.frame)
self.label_222.setGeometry(QtCore.QRect(20, 290, 81, 20))
self.label_222.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_222.setObjectName("label_222")
self.save_id_on_robot = QtWidgets.QPushButton(self.frame)
self.save_id_on_robot.setEnabled(False)
self.save_id_on_robot.setGeometry(QtCore.QRect(210, 230, 92, 23))
self.save_id_on_robot.setToolTip("<html><head/><body>Save ID and robot specific settings (this frame) on robot. NB! you need to press "save on robot" in configuration block to save into EE-prom (flash)</body></html>")
self.save_id_on_robot.setCheckable(False)
self.save_id_on_robot.setChecked(False)
self.save_id_on_robot.setObjectName("save_id_on_robot")
self.label_129 = QtWidgets.QLabel(self.frame)
self.label_129.setGeometry(QtCore.QRect(10, 265, 91, 21))
self.label_129.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_129.setObjectName("label_129")
self.Robot_BalanceOffset = QtWidgets.QDoubleSpinBox(self.frame)
self.Robot_BalanceOffset.setGeometry(QtCore.QRect(110, 263, 71, 21))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Minimum, QtWidgets.QSizePolicy.Fixed)
sizePolicy.setHorizontalStretch(2)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.Robot_BalanceOffset.sizePolicy().hasHeightForWidth())
self.Robot_BalanceOffset.setSizePolicy(sizePolicy)
self.Robot_BalanceOffset.setToolTip("<html><head/><body>Balance offset - where center of gravity (COG) is just above support point (wheels).Angle is in radians.Positive is tilt forward.</body></html>")
self.Robot_BalanceOffset.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_BalanceOffset.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_BalanceOffset.setReadOnly(False)
self.Robot_BalanceOffset.setDecimals(4)
self.Robot_BalanceOffset.setMinimum(-1.0)
self.Robot_BalanceOffset.setMaximum(1.0)
self.Robot_BalanceOffset.setSingleStep(0.001)
self.Robot_BalanceOffset.setObjectName("Robot_BalanceOffset")
self.Robot_edit = QtWidgets.QPushButton(self.frame)
self.Robot_edit.setEnabled(True)
self.Robot_edit.setGeometry(QtCore.QRect(210, 259, 92, 23))
self.Robot_edit.setToolTip("<html><head/><body>Switch to edit mode - rather than listen to robot messages - for these fields.</body></html>")
self.Robot_edit.setCheckable(False)
self.Robot_edit.setChecked(False)
self.Robot_edit.setObjectName("Robot_edit")
self.Robot_OnBattery = QtWidgets.QCheckBox(self.frame)
self.Robot_OnBattery.setGeometry(QtCore.QRect(190, 120, 111, 21))
self.Robot_OnBattery.setObjectName("Robot_OnBattery")
self.robot_cancel = QtWidgets.QPushButton(self.frame)
self.robot_cancel.setEnabled(False)
self.robot_cancel.setGeometry(QtCore.QRect(210, 288, 92, 23))
self.robot_cancel.setToolTip("<html><head/><body>Revert to values on robot</body></html>")
self.robot_cancel.setCheckable(False)
self.robot_cancel.setChecked(False)
self.robot_cancel.setObjectName("robot_cancel")
self.tabWidget_2 = QtWidgets.QTabWidget(self.tab)
self.tabWidget_2.setGeometry(QtCore.QRect(0, 490, 321, 211))
self.tabWidget_2.setObjectName("tabWidget_2")
self.tab_6 = QtWidgets.QWidget()
self.tab_6.setObjectName("tab_6")
self.groupBox = QtWidgets.QGroupBox(self.tab_6)
self.groupBox.setGeometry(QtCore.QRect(0, 0, 311, 181))
self.groupBox.setTitle("")
self.groupBox.setObjectName("groupBox")
self.Robot_CurrentLeft_Value = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_CurrentLeft_Value.setGeometry(QtCore.QRect(110, 100, 71, 21))
self.Robot_CurrentLeft_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_CurrentLeft_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_CurrentLeft_Value.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_CurrentLeft_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_CurrentLeft_Value.setReadOnly(True)
self.Robot_CurrentLeft_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_CurrentLeft_Value.setDecimals(2)
self.Robot_CurrentLeft_Value.setMinimum(-100000.0)
self.Robot_CurrentLeft_Value.setMaximum(100000.0)
self.Robot_CurrentLeft_Value.setObjectName("Robot_CurrentLeft_Value")
self.Robot_Distance = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_Distance.setGeometry(QtCore.QRect(110, 130, 71, 21))
self.Robot_Distance.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_Distance.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_Distance.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_Distance.setReadOnly(True)
self.Robot_Distance.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_Distance.setDecimals(3)
self.Robot_Distance.setMinimum(-999999.0)
self.Robot_Distance.setMaximum(999999.99)
self.Robot_Distance.setObjectName("Robot_Distance")
self.label_36 = QtWidgets.QLabel(self.groupBox)
self.label_36.setGeometry(QtCore.QRect(186, 24, 71, 16))
self.label_36.setAlignment(QtCore.Qt.AlignCenter)
self.label_36.setObjectName("label_36")
self.label_37 = QtWidgets.QLabel(self.groupBox)
self.label_37.setGeometry(QtCore.QRect(106, 24, 71, 16))
self.label_37.setAlignment(QtCore.Qt.AlignCenter)
self.label_37.setObjectName("label_37")
self.label_42 = QtWidgets.QLabel(self.groupBox)
self.label_42.setGeometry(QtCore.QRect(11, 100, 91, 16))
self.label_42.setMinimumSize(QtCore.QSize(0, 0))
self.label_42.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_42.setObjectName("label_42")
self.Robot_EncLeft_Value = QtWidgets.QSpinBox(self.groupBox)
self.Robot_EncLeft_Value.setGeometry(QtCore.QRect(110, 40, 71, 21))
self.Robot_EncLeft_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_EncLeft_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_EncLeft_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_EncLeft_Value.setReadOnly(True)
self.Robot_EncLeft_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_EncLeft_Value.setMinimum(-2147483647)
self.Robot_EncLeft_Value.setMaximum(2147483647)
self.Robot_EncLeft_Value.setObjectName("Robot_EncLeft_Value")
self.Robot_CurrentRight_Value = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_CurrentRight_Value.setGeometry(QtCore.QRect(190, 100, 71, 21))
self.Robot_CurrentRight_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_CurrentRight_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_CurrentRight_Value.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_CurrentRight_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_CurrentRight_Value.setReadOnly(True)
self.Robot_CurrentRight_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_CurrentRight_Value.setDecimals(2)
self.Robot_CurrentRight_Value.setMinimum(-100000.0)
self.Robot_CurrentRight_Value.setMaximum(100000.0)
self.Robot_CurrentRight_Value.setObjectName("Robot_CurrentRight_Value")
self.label_28 = QtWidgets.QLabel(self.groupBox)
self.label_28.setGeometry(QtCore.QRect(11, 40, 91, 16))
self.label_28.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_28.setObjectName("label_28")
self.label_38 = QtWidgets.QLabel(self.groupBox)
self.label_38.setGeometry(QtCore.QRect(9, 5, 121, 21))
font = QtGui.QFont()
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_38.setFont(font)
self.label_38.setObjectName("label_38")
self.label_32 = QtWidgets.QLabel(self.groupBox)
self.label_32.setGeometry(QtCore.QRect(10, 130, 91, 16))
self.label_32.setMinimumSize(QtCore.QSize(0, 0))
self.label_32.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_32.setObjectName("label_32")
self.label_29 = QtWidgets.QLabel(self.groupBox)
self.label_29.setGeometry(QtCore.QRect(1, 70, 101, 16))
self.label_29.setMinimumSize(QtCore.QSize(0, 0))
self.label_29.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_29.setObjectName("label_29")
self.Robot_EncRight_Value = QtWidgets.QSpinBox(self.groupBox)
self.Robot_EncRight_Value.setGeometry(QtCore.QRect(190, 40, 71, 21))
self.Robot_EncRight_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_EncRight_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_EncRight_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_EncRight_Value.setReadOnly(True)
self.Robot_EncRight_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_EncRight_Value.setMinimum(-2147483647)
self.Robot_EncRight_Value.setMaximum(2147483647)
self.Robot_EncRight_Value.setObjectName("Robot_EncRight_Value")
self.Robot_WheelRightSpeed_Value = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_WheelRightSpeed_Value.setGeometry(QtCore.QRect(190, 70, 71, 21))
self.Robot_WheelRightSpeed_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_WheelRightSpeed_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_WheelRightSpeed_Value.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelRightSpeed_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelRightSpeed_Value.setReadOnly(True)
self.Robot_WheelRightSpeed_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_WheelRightSpeed_Value.setDecimals(3)
self.Robot_WheelRightSpeed_Value.setMinimum(-100000.0)
self.Robot_WheelRightSpeed_Value.setMaximum(100000.0)
self.Robot_WheelRightSpeed_Value.setObjectName("Robot_WheelRightSpeed_Value")
self.Robot_WheelLeftSpeed_Value = QtWidgets.QDoubleSpinBox(self.groupBox)
self.Robot_WheelLeftSpeed_Value.setGeometry(QtCore.QRect(110, 70, 71, 21))
self.Robot_WheelLeftSpeed_Value.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_WheelLeftSpeed_Value.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_WheelLeftSpeed_Value.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_WheelLeftSpeed_Value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_WheelLeftSpeed_Value.setReadOnly(True)
self.Robot_WheelLeftSpeed_Value.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_WheelLeftSpeed_Value.setDecimals(3)
self.Robot_WheelLeftSpeed_Value.setMinimum(-100000.0)
self.Robot_WheelLeftSpeed_Value.setMaximum(100000.0)
self.Robot_WheelLeftSpeed_Value.setObjectName("Robot_WheelLeftSpeed_Value")
self.tabWidget_2.addTab(self.tab_6, "")
self.tab_7 = QtWidgets.QWidget()
self.tab_7.setObjectName("tab_7")
self.groupBox_3 = QtWidgets.QGroupBox(self.tab_7)
self.groupBox_3.setGeometry(QtCore.QRect(0, 0, 311, 181))
self.groupBox_3.setTitle("")
self.groupBox_3.setObjectName("groupBox_3")
self.Robot_TiltDegrees = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_TiltDegrees.setGeometry(QtCore.QRect(190, 131, 71, 21))
self.Robot_TiltDegrees.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_TiltDegrees.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_TiltDegrees.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_TiltDegrees.setReadOnly(True)
self.Robot_TiltDegrees.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_TiltDegrees.setDecimals(3)
self.Robot_TiltDegrees.setMinimum(-999999.0)
self.Robot_TiltDegrees.setMaximum(999999.99)
self.Robot_TiltDegrees.setObjectName("Robot_TiltDegrees")
self.label_76 = QtWidgets.QLabel(self.groupBox_3)
self.label_76.setGeometry(QtCore.QRect(190, 80, 71, 21))
self.label_76.setAlignment(QtCore.Qt.AlignCenter)
self.label_76.setObjectName("label_76")
self.label_45 = QtWidgets.QLabel(self.groupBox_3)
self.label_45.setGeometry(QtCore.QRect(10, 0, 271, 31))
font = QtGui.QFont()
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_45.setFont(font)
self.label_45.setObjectName("label_45")
self.Robot_PoseX = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_PoseX.setGeometry(QtCore.QRect(100, 40, 81, 21))
self.Robot_PoseX.setMinimumSize(QtCore.QSize(0, 0))
self.Robot_PoseX.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_PoseX.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PoseX.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PoseX.setReadOnly(True)
self.Robot_PoseX.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_PoseX.setDecimals(3)
self.Robot_PoseX.setMinimum(-999999.0)
self.Robot_PoseX.setMaximum(999999.99)
self.Robot_PoseX.setObjectName("Robot_PoseX")
self.Robot_PoseY = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_PoseY.setGeometry(QtCore.QRect(100, 71, 81, 21))
self.Robot_PoseY.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_PoseY.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PoseY.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PoseY.setReadOnly(True)
self.Robot_PoseY.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_PoseY.setDecimals(3)
self.Robot_PoseY.setMinimum(-999999.0)
self.Robot_PoseY.setMaximum(999999.99)
self.Robot_PoseY.setObjectName("Robot_PoseY")
self.Robot_PoseReset = QtWidgets.QPushButton(self.groupBox_3)
self.Robot_PoseReset.setGeometry(QtCore.QRect(190, 40, 71, 21))
self.Robot_PoseReset.setToolTip("Reset robot pose and distance to 0.0")
self.Robot_PoseReset.setObjectName("Robot_PoseReset")
self.label_44 = QtWidgets.QLabel(self.groupBox_3)
self.label_44.setGeometry(QtCore.QRect(20, 100, 71, 21))
self.label_44.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_44.setObjectName("label_44")
self.label_34 = QtWidgets.QLabel(self.groupBox_3)
self.label_34.setGeometry(QtCore.QRect(10, 40, 81, 21))
self.label_34.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_34.setObjectName("label_34")
self.label_35 = QtWidgets.QLabel(self.groupBox_3)
self.label_35.setGeometry(QtCore.QRect(20, 71, 71, 20))
self.label_35.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_35.setObjectName("label_35")
self.Robot_Tilt = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_Tilt.setGeometry(QtCore.QRect(100, 131, 81, 21))
self.Robot_Tilt.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_Tilt.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_Tilt.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_Tilt.setReadOnly(True)
self.Robot_Tilt.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_Tilt.setDecimals(3)
self.Robot_Tilt.setMinimum(-999999.0)
self.Robot_Tilt.setMaximum(999999.99)
self.Robot_Tilt.setObjectName("Robot_Tilt")
self.Robot_PoseTheta = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_PoseTheta.setGeometry(QtCore.QRect(100, 100, 81, 21))
self.Robot_PoseTheta.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_PoseTheta.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PoseTheta.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PoseTheta.setReadOnly(True)
self.Robot_PoseTheta.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_PoseTheta.setDecimals(3)
self.Robot_PoseTheta.setMinimum(-999999.0)
self.Robot_PoseTheta.setMaximum(999999.99)
self.Robot_PoseTheta.setObjectName("Robot_PoseTheta")
self.Robot_PoseDegrees = QtWidgets.QDoubleSpinBox(self.groupBox_3)
self.Robot_PoseDegrees.setGeometry(QtCore.QRect(190, 100, 71, 21))
self.Robot_PoseDegrees.setStyleSheet("background-color: rgb(230,230,230);")
self.Robot_PoseDegrees.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.Robot_PoseDegrees.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.Robot_PoseDegrees.setReadOnly(True)
self.Robot_PoseDegrees.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.Robot_PoseDegrees.setDecimals(3)
self.Robot_PoseDegrees.setMinimum(-999999.0)
self.Robot_PoseDegrees.setMaximum(999999.99)
self.Robot_PoseDegrees.setObjectName("Robot_PoseDegrees")
self.label_125 = QtWidgets.QLabel(self.groupBox_3)
self.label_125.setGeometry(QtCore.QRect(10, 130, 81, 21))
self.label_125.setMinimumSize(QtCore.QSize(0, 0))
self.label_125.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_125.setObjectName("label_125")
self.tabWidget_2.addTab(self.tab_7, "")
self.tabWidget.addTab(self.tab, "")
self.tab_2 = QtWidgets.QWidget()
self.tab_2.setObjectName("tab_2")
self.frame_2 = QtWidgets.QFrame(self.tab_2)
self.frame_2.setGeometry(QtCore.QRect(0, 0, 711, 421))
self.frame_2.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_2.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_2.setObjectName("frame_2")
self.label_10 = QtWidgets.QLabel(self.frame_2)
self.label_10.setGeometry(QtCore.QRect(10, 10, 101, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_10.setFont(font)
self.label_10.setObjectName("label_10")
self.enableLineSensor_checkBox = QtWidgets.QCheckBox(self.frame_2)
self.enableLineSensor_checkBox.setGeometry(QtCore.QRect(10, 30, 111, 17))
self.enableLineSensor_checkBox.setObjectName("enableLineSensor_checkBox")
self.frame_3 = QtWidgets.QFrame(self.frame_2)
self.frame_3.setGeometry(QtCore.QRect(10, 50, 421, 231))
self.frame_3.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_3.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_3.setObjectName("frame_3")
self.line_bar_1 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_1.setGeometry(QtCore.QRect(10, 20, 16, 181))
self.line_bar_1.setMaximum(1000)
self.line_bar_1.setProperty("value", 0)
self.line_bar_1.setOrientation(QtCore.Qt.Vertical)
self.line_bar_1.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_1.setObjectName("line_bar_1")
self.line_bar_4 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_4.setGeometry(QtCore.QRect(70, 20, 16, 181))
self.line_bar_4.setMaximum(1000)
self.line_bar_4.setProperty("value", 0)
self.line_bar_4.setOrientation(QtCore.Qt.Vertical)
self.line_bar_4.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_4.setObjectName("line_bar_4")
self.line_bar_6 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_6.setGeometry(QtCore.QRect(110, 20, 16, 181))
self.line_bar_6.setMaximum(1000)
self.line_bar_6.setProperty("value", 0)
self.line_bar_6.setOrientation(QtCore.Qt.Vertical)
self.line_bar_6.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_6.setObjectName("line_bar_6")
self.line_bar_5 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_5.setGeometry(QtCore.QRect(90, 20, 16, 181))
self.line_bar_5.setMaximum(1000)
self.line_bar_5.setProperty("value", 0)
self.line_bar_5.setOrientation(QtCore.Qt.Vertical)
self.line_bar_5.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_5.setObjectName("line_bar_5")
self.line_bar_3 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_3.setGeometry(QtCore.QRect(50, 20, 16, 181))
self.line_bar_3.setMaximum(1000)
self.line_bar_3.setProperty("value", 0)
self.line_bar_3.setOrientation(QtCore.Qt.Vertical)
self.line_bar_3.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_3.setObjectName("line_bar_3")
self.line_bar_7 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_7.setGeometry(QtCore.QRect(130, 20, 16, 181))
self.line_bar_7.setMaximum(1000)
self.line_bar_7.setProperty("value", 0)
self.line_bar_7.setOrientation(QtCore.Qt.Vertical)
self.line_bar_7.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_7.setObjectName("line_bar_7")
self.line_bar_2 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_2.setGeometry(QtCore.QRect(30, 20, 16, 181))
self.line_bar_2.setMaximum(1000)
self.line_bar_2.setProperty("value", 0)
self.line_bar_2.setOrientation(QtCore.Qt.Vertical)
self.line_bar_2.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_2.setObjectName("line_bar_2")
self.line_bar_8 = QtWidgets.QProgressBar(self.frame_3)
self.line_bar_8.setGeometry(QtCore.QRect(150, 20, 16, 181))
self.line_bar_8.setMaximum(1000)
self.line_bar_8.setProperty("value", 0)
self.line_bar_8.setOrientation(QtCore.Qt.Vertical)
self.line_bar_8.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.line_bar_8.setObjectName("line_bar_8")
self.label_11 = QtWidgets.QLabel(self.frame_3)
self.label_11.setGeometry(QtCore.QRect(10, 0, 151, 21))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_11.setFont(font)
self.label_11.setObjectName("label_11")
self.label_12 = QtWidgets.QLabel(self.frame_3)
self.label_12.setGeometry(QtCore.QRect(10, 200, 16, 21))
self.label_12.setLayoutDirection(QtCore.Qt.RightToLeft)
self.label_12.setObjectName("label_12")
self.label_13 = QtWidgets.QLabel(self.frame_3)
self.label_13.setGeometry(QtCore.QRect(30, 200, 16, 21))
self.label_13.setObjectName("label_13")
self.label_14 = QtWidgets.QLabel(self.frame_3)
self.label_14.setGeometry(QtCore.QRect(110, 200, 16, 21))
self.label_14.setObjectName("label_14")
self.label_15 = QtWidgets.QLabel(self.frame_3)
self.label_15.setGeometry(QtCore.QRect(50, 200, 16, 21))
self.label_15.setObjectName("label_15")
self.label_16 = QtWidgets.QLabel(self.frame_3)
self.label_16.setGeometry(QtCore.QRect(70, 200, 16, 21))
self.label_16.setObjectName("label_16")
self.label_17 = QtWidgets.QLabel(self.frame_3)
self.label_17.setGeometry(QtCore.QRect(90, 200, 20, 21))
self.label_17.setObjectName("label_17")
self.label_18 = QtWidgets.QLabel(self.frame_3)
self.label_18.setGeometry(QtCore.QRect(130, 200, 16, 21))
self.label_18.setObjectName("label_18")
self.label_19 = QtWidgets.QLabel(self.frame_3)
self.label_19.setGeometry(QtCore.QRect(150, 200, 16, 21))
self.label_19.setObjectName("label_19")
self.ls_right_side = QtWidgets.QDoubleSpinBox(self.frame_3)
self.ls_right_side.setGeometry(QtCore.QRect(300, 170, 61, 21))
self.ls_right_side.setToolTip("<html><head/><body>Position of right edge (cm)</body></html>")
self.ls_right_side.setStyleSheet("background-color: rgb(230,230,230);")
self.ls_right_side.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ls_right_side.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_right_side.setReadOnly(True)
self.ls_right_side.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.ls_right_side.setSpecialValueText("")
self.ls_right_side.setDecimals(3)
self.ls_right_side.setMinimum(-100.0)
self.ls_right_side.setMaximum(100.0)
self.ls_right_side.setObjectName("ls_right_side")
self.ls_left_side = QtWidgets.QDoubleSpinBox(self.frame_3)
self.ls_left_side.setGeometry(QtCore.QRect(230, 170, 61, 21))
self.ls_left_side.setToolTip("<html><head/><body>Position of left edge (cm)</body></html>")
self.ls_left_side.setStyleSheet("background-color: rgb(230,230,230);")
self.ls_left_side.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ls_left_side.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_left_side.setReadOnly(True)
self.ls_left_side.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.ls_left_side.setSpecialValueText("")
self.ls_left_side.setDecimals(3)
self.ls_left_side.setMinimum(-100.0)
self.ls_left_side.setMaximum(100.0)
self.ls_left_side.setObjectName("ls_left_side")
self.label_118 = QtWidgets.QLabel(self.frame_3)
self.label_118.setGeometry(QtCore.QRect(230, 40, 61, 18))
self.label_118.setAlignment(QtCore.Qt.AlignCenter)
self.label_118.setObjectName("label_118")
self.ls_left_edge = QtWidgets.QProgressBar(self.frame_3)
self.ls_left_edge.setEnabled(True)
self.ls_left_edge.setGeometry(QtCore.QRect(240, 60, 41, 101))
self.ls_left_edge.setMinimum(-3)
self.ls_left_edge.setMaximum(103)
self.ls_left_edge.setProperty("value", 24)
self.ls_left_edge.setTextVisible(False)
self.ls_left_edge.setOrientation(QtCore.Qt.Vertical)
self.ls_left_edge.setInvertedAppearance(False)
self.ls_left_edge.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.ls_left_edge.setObjectName("ls_left_edge")
self.ls_right_edge = QtWidgets.QProgressBar(self.frame_3)
self.ls_right_edge.setGeometry(QtCore.QRect(310, 60, 41, 101))
self.ls_right_edge.setMinimum(-3)
self.ls_right_edge.setMaximum(103)
self.ls_right_edge.setProperty("value", 24)
self.ls_right_edge.setTextVisible(False)
self.ls_right_edge.setOrientation(QtCore.Qt.Vertical)
self.ls_right_edge.setInvertedAppearance(False)
self.ls_right_edge.setObjectName("ls_right_edge")
self.label_121 = QtWidgets.QLabel(self.frame_3)
self.label_121.setGeometry(QtCore.QRect(300, 40, 61, 18))
self.label_121.setAlignment(QtCore.Qt.AlignCenter)
self.label_121.setObjectName("label_121")
self.label_20 = QtWidgets.QLabel(self.frame_3)
self.label_20.setGeometry(QtCore.QRect(180, 0, 101, 21))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_20.setFont(font)
self.label_20.setObjectName("label_20")
self.label_27 = QtWidgets.QLabel(self.frame_3)
self.label_27.setGeometry(QtCore.QRect(290, 20, 48, 20))
self.label_27.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_27.setObjectName("label_27")
self.label_171 = QtWidgets.QLabel(self.frame_3)
self.label_171.setGeometry(QtCore.QRect(176, 20, 48, 20))
self.label_171.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_171.setObjectName("label_171")
self.ls_crossing_cnt = QtWidgets.QLineEdit(self.frame_3)
self.ls_crossing_cnt.setGeometry(QtCore.QRect(344, 20, 54, 20))
self.ls_crossing_cnt.setStatusTip("")
self.ls_crossing_cnt.setStyleSheet("background-color: rgb(230,230,230);")
self.ls_crossing_cnt.setObjectName("ls_crossing_cnt")
self.ls_line_valid_cnt = QtWidgets.QLineEdit(self.frame_3)
self.ls_line_valid_cnt.setGeometry(QtCore.QRect(230, 20, 54, 20))
self.ls_line_valid_cnt.setStyleSheet("background-color: rgb(230,230,230);")
self.ls_line_valid_cnt.setObjectName("ls_line_valid_cnt")
self.groupBox_5 = QtWidgets.QGroupBox(self.frame_2)
self.groupBox_5.setGeometry(QtCore.QRect(430, 10, 251, 361))
self.groupBox_5.setObjectName("groupBox_5")
self.CalibrateWhite_Cmd = QtWidgets.QPushButton(self.groupBox_5)
self.CalibrateWhite_Cmd.setGeometry(QtCore.QRect(20, 310, 101, 31))
self.CalibrateWhite_Cmd.setObjectName("CalibrateWhite_Cmd")
self.label_217 = QtWidgets.QLabel(self.groupBox_5)
self.label_217.setGeometry(QtCore.QRect(20, 75, 21, 16))
self.label_217.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_217.setObjectName("label_217")
self.label_218 = QtWidgets.QLabel(self.groupBox_5)
self.label_218.setGeometry(QtCore.QRect(20, 117, 21, 16))
self.label_218.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_218.setObjectName("label_218")
self.label_219 = QtWidgets.QLabel(self.groupBox_5)
self.label_219.setGeometry(QtCore.QRect(20, 159, 21, 16))
self.label_219.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_219.setObjectName("label_219")
self.label_221 = QtWidgets.QLabel(self.groupBox_5)
self.label_221.setGeometry(QtCore.QRect(20, 96, 21, 16))
self.label_221.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_221.setObjectName("label_221")
self.label_223 = QtWidgets.QLabel(self.groupBox_5)
self.label_223.setGeometry(QtCore.QRect(20, 138, 21, 16))
self.label_223.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_223.setObjectName("label_223")
self.label_224 = QtWidgets.QLabel(self.groupBox_5)
self.label_224.setGeometry(QtCore.QRect(20, 33, 21, 16))
self.label_224.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_224.setObjectName("label_224")
self.label_225 = QtWidgets.QLabel(self.groupBox_5)
self.label_225.setGeometry(QtCore.QRect(20, 180, 21, 16))
self.label_225.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_225.setObjectName("label_225")
self.label_226 = QtWidgets.QLabel(self.groupBox_5)
self.label_226.setGeometry(QtCore.QRect(20, 54, 21, 16))
self.label_226.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_226.setObjectName("label_226")
self.ls_max_white_7 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_7.setGeometry(QtCore.QRect(50, 159, 50, 15))
self.ls_max_white_7.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_7.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_7.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_7.setObjectName("ls_max_white_7")
self.ls_max_white_8 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_8.setGeometry(QtCore.QRect(50, 180, 50, 15))
self.ls_max_white_8.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_8.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_8.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_8.setObjectName("ls_max_white_8")
self.ls_max_white_1 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_1.setGeometry(QtCore.QRect(50, 33, 50, 15))
self.ls_max_white_1.setMinimumSize(QtCore.QSize(0, 0))
self.ls_max_white_1.setAutoFillBackground(False)
self.ls_max_white_1.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_1.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_1.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_1.setObjectName("ls_max_white_1")
self.ls_max_white_3 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_3.setGeometry(QtCore.QRect(50, 75, 50, 15))
self.ls_max_white_3.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_3.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_3.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_3.setObjectName("ls_max_white_3")
self.ls_max_white_5 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_5.setGeometry(QtCore.QRect(50, 117, 50, 15))
self.ls_max_white_5.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_5.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_5.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_5.setObjectName("ls_max_white_5")
self.ls_max_white_2 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_2.setGeometry(QtCore.QRect(50, 54, 50, 15))
self.ls_max_white_2.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_2.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_2.setObjectName("ls_max_white_2")
self.ls_max_white_4 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_4.setGeometry(QtCore.QRect(50, 96, 50, 15))
self.ls_max_white_4.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_4.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_4.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_4.setObjectName("ls_max_white_4")
self.ls_max_white_6 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_white_6.setGeometry(QtCore.QRect(50, 138, 50, 15))
self.ls_max_white_6.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_white_6.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_white_6.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_white_6.setObjectName("ls_max_white_6")
self.ls_max_black_5 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_5.setGeometry(QtCore.QRect(150, 117, 50, 15))
self.ls_max_black_5.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_5.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_5.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_5.setObjectName("ls_max_black_5")
self.ls_max_black_4 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_4.setGeometry(QtCore.QRect(150, 96, 50, 15))
self.ls_max_black_4.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_4.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_4.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_4.setObjectName("ls_max_black_4")
self.ls_max_black_8 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_8.setGeometry(QtCore.QRect(150, 180, 50, 15))
self.ls_max_black_8.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_8.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_8.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_8.setObjectName("ls_max_black_8")
self.ls_max_black_1 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_1.setGeometry(QtCore.QRect(150, 33, 50, 15))
self.ls_max_black_1.setMinimumSize(QtCore.QSize(0, 0))
self.ls_max_black_1.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_1.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_1.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_1.setObjectName("ls_max_black_1")
self.ls_max_black_2 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_2.setGeometry(QtCore.QRect(150, 54, 50, 15))
self.ls_max_black_2.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_2.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_2.setObjectName("ls_max_black_2")
self.ls_max_black_3 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_3.setGeometry(QtCore.QRect(150, 75, 50, 15))
self.ls_max_black_3.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_3.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_3.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_3.setObjectName("ls_max_black_3")
self.ls_max_black_6 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_6.setGeometry(QtCore.QRect(150, 138, 50, 15))
self.ls_max_black_6.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_6.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_6.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_6.setObjectName("ls_max_black_6")
self.ls_max_black_7 = QtWidgets.QLabel(self.groupBox_5)
self.ls_max_black_7.setGeometry(QtCore.QRect(150, 159, 50, 15))
self.ls_max_black_7.setFrameShape(QtWidgets.QFrame.Panel)
self.ls_max_black_7.setFrameShadow(QtWidgets.QFrame.Plain)
self.ls_max_black_7.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_max_black_7.setObjectName("ls_max_black_7")
self.label_227 = QtWidgets.QLabel(self.groupBox_5)
self.label_227.setGeometry(QtCore.QRect(50, 14, 51, 16))
self.label_227.setAlignment(QtCore.Qt.AlignCenter)
self.label_227.setObjectName("label_227")
self.CalibrateBlack_Cmd = QtWidgets.QPushButton(self.groupBox_5)
self.CalibrateBlack_Cmd.setGeometry(QtCore.QRect(130, 310, 101, 31))
self.CalibrateBlack_Cmd.setObjectName("CalibrateBlack_Cmd")
self.label_228 = QtWidgets.QLabel(self.groupBox_5)
self.label_228.setGeometry(QtCore.QRect(150, 15, 51, 16))
self.label_228.setAlignment(QtCore.Qt.AlignCenter)
self.label_228.setObjectName("label_228")
self.ls_swap_left_right = QtWidgets.QCheckBox(self.groupBox_5)
self.ls_swap_left_right.setGeometry(QtCore.QRect(140, 220, 81, 20))
self.ls_swap_left_right.setObjectName("ls_swap_left_right")
self.ls_show_normalized = QtWidgets.QCheckBox(self.groupBox_5)
self.ls_show_normalized.setGeometry(QtCore.QRect(30, 220, 101, 17))
self.ls_show_normalized.setObjectName("ls_show_normalized")
self.label_23 = QtWidgets.QLabel(self.groupBox_5)
self.label_23.setGeometry(QtCore.QRect(130, 240, 75, 21))
self.label_23.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_23.setObjectName("label_23")
self.line_disp_max_value = QtWidgets.QSpinBox(self.groupBox_5)
self.line_disp_max_value.setGeometry(QtCore.QRect(30, 260, 75, 20))
self.line_disp_max_value.setToolTip("<html><head/><body>Maximum value used as 100% in bar display.</body></html>")
self.line_disp_max_value.setStatusTip("")
self.line_disp_max_value.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.line_disp_max_value.setMaximum(4250)
self.line_disp_max_value.setSingleStep(256)
self.line_disp_max_value.setProperty("value", 4096)
self.line_disp_max_value.setObjectName("line_disp_max_value")
self.label_156 = QtWidgets.QLabel(self.groupBox_5)
self.label_156.setGeometry(QtCore.QRect(34, 240, 61, 20))
self.label_156.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.label_156.setObjectName("label_156")
self.ls_crossing_detect = QtWidgets.QDoubleSpinBox(self.groupBox_5)
self.ls_crossing_detect.setGeometry(QtCore.QRect(130, 260, 75, 20))
self.ls_crossing_detect.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ls_crossing_detect.setDecimals(1)
self.ls_crossing_detect.setMinimum(2.0)
self.ls_crossing_detect.setMaximum(8.0)
self.ls_crossing_detect.setSingleStep(0.1)
self.ls_crossing_detect.setProperty("value", 4.5)
self.ls_crossing_detect.setObjectName("ls_crossing_detect")
self.line_2 = QtWidgets.QFrame(self.groupBox_5)
self.line_2.setGeometry(QtCore.QRect(20, 200, 211, 21))
self.line_2.setFrameShape(QtWidgets.QFrame.HLine)
self.line_2.setFrameShadow(QtWidgets.QFrame.Sunken)
self.line_2.setObjectName("line_2")
self.line_3 = QtWidgets.QFrame(self.groupBox_5)
self.line_3.setGeometry(QtCore.QRect(20, 290, 211, 21))
self.line_3.setFrameShape(QtWidgets.QFrame.HLine)
self.line_3.setFrameShadow(QtWidgets.QFrame.Sunken)
self.line_3.setObjectName("line_3")
self.ls_power_high = QtWidgets.QCheckBox(self.frame_2)
self.ls_power_high.setGeometry(QtCore.QRect(230, 30, 91, 17))
self.ls_power_high.setObjectName("ls_power_high")
self.ls_line_white = QtWidgets.QCheckBox(self.frame_2)
self.ls_line_white.setGeometry(QtCore.QRect(140, 30, 81, 17))
self.ls_line_white.setObjectName("ls_line_white")
self.tabWidget.addTab(self.tab_2, "")
self.tab_8 = QtWidgets.QWidget()
self.tab_8.setObjectName("tab_8")
self.frame_37 = QtWidgets.QFrame(self.tab_8)
self.frame_37.setGeometry(QtCore.QRect(0, 0, 521, 291))
self.frame_37.setMinimumSize(QtCore.QSize(250, 0))
self.frame_37.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_37.setFrameShadow(QtWidgets.QFrame.Sunken)
self.frame_37.setObjectName("frame_37")
self.label_56 = QtWidgets.QLabel(self.frame_37)
self.label_56.setGeometry(QtCore.QRect(10, 10, 104, 16))
font = QtGui.QFont()
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_56.setFont(font)
self.label_56.setObjectName("label_56")
self.ir_bar_1 = QtWidgets.QProgressBar(self.frame_37)
self.ir_bar_1.setGeometry(QtCore.QRect(320, 200, 171, 41))
self.ir_bar_1.setMaximum(130000)
self.ir_bar_1.setProperty("value", 0)
self.ir_bar_1.setOrientation(QtCore.Qt.Horizontal)
self.ir_bar_1.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.ir_bar_1.setFormat("")
self.ir_bar_1.setObjectName("ir_bar_1")
self.ir_bar_2 = QtWidgets.QProgressBar(self.frame_37)
self.ir_bar_2.setGeometry(QtCore.QRect(280, 60, 41, 141))
self.ir_bar_2.setMaximum(130000)
self.ir_bar_2.setProperty("value", 0)
self.ir_bar_2.setOrientation(QtCore.Qt.Vertical)
self.ir_bar_2.setTextDirection(QtWidgets.QProgressBar.TopToBottom)
self.ir_bar_2.setFormat("")
self.ir_bar_2.setObjectName("ir_bar_2")
self.ir_d1_meters = QtWidgets.QLineEdit(self.frame_37)
self.ir_d1_meters.setGeometry(QtCore.QRect(400, 170, 81, 20))
self.ir_d1_meters.setStyleSheet("background-color: rgb(230,230,230);")
self.ir_d1_meters.setAlignment(QtCore.Qt.AlignCenter)
self.ir_d1_meters.setObjectName("ir_d1_meters")
self.label_279 = QtWidgets.QLabel(self.frame_37)
self.label_279.setGeometry(QtCore.QRect(360, 170, 41, 20))
self.label_279.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_279.setObjectName("label_279")
self.label_281 = QtWidgets.QLabel(self.frame_37)
self.label_281.setGeometry(QtCore.QRect(330, 140, 71, 18))
self.label_281.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_281.setObjectName("label_281")
self.ir_d1_raw = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d1_raw.setGeometry(QtCore.QRect(400, 140, 81, 20))
self.ir_d1_raw.setToolTip("<html><head/><body>Raw measured value 2Y0A21F sensor from A/D converter</body></html>")
self.ir_d1_raw.setStyleSheet("background-color: rgb(230,230,230);")
self.ir_d1_raw.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d1_raw.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d1_raw.setReadOnly(True)
self.ir_d1_raw.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.ir_d1_raw.setSpecialValueText("")
self.ir_d1_raw.setDecimals(0)
self.ir_d1_raw.setMinimum(0.0)
self.ir_d1_raw.setMaximum(1000000.0)
self.ir_d1_raw.setObjectName("ir_d1_raw")
self.label_287 = QtWidgets.QLabel(self.frame_37)
self.label_287.setGeometry(QtCore.QRect(490, 170, 16, 16))
self.label_287.setObjectName("label_287")
self.ir_d2_meters = QtWidgets.QLineEdit(self.frame_37)
self.ir_d2_meters.setGeometry(QtCore.QRect(402, 90, 81, 20))
self.ir_d2_meters.setStyleSheet("background-color: rgb(230,230,230);")
self.ir_d2_meters.setAlignment(QtCore.Qt.AlignCenter)
self.ir_d2_meters.setObjectName("ir_d2_meters")
self.label_280 = QtWidgets.QLabel(self.frame_37)
self.label_280.setGeometry(QtCore.QRect(360, 90, 51, 20))
sizePolicy = QtWidgets.QSizePolicy(QtWidgets.QSizePolicy.Preferred, QtWidgets.QSizePolicy.Maximum)
sizePolicy.setHorizontalStretch(0)
sizePolicy.setVerticalStretch(0)
sizePolicy.setHeightForWidth(self.label_280.sizePolicy().hasHeightForWidth())
self.label_280.setSizePolicy(sizePolicy)
self.label_280.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_280.setObjectName("label_280")
self.ir_d2_raw = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d2_raw.setGeometry(QtCore.QRect(400, 60, 81, 20))
self.ir_d2_raw.setToolTip("<html><head/><body>Raw measured value 2Y0A21F sensor from A/D converter</body></html>")
self.ir_d2_raw.setStyleSheet("background-color: rgb(230,230,230);")
self.ir_d2_raw.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d2_raw.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d2_raw.setReadOnly(True)
self.ir_d2_raw.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.ir_d2_raw.setSpecialValueText("")
self.ir_d2_raw.setDecimals(0)
self.ir_d2_raw.setMinimum(0.0)
self.ir_d2_raw.setMaximum(1000000.0)
self.ir_d2_raw.setObjectName("ir_d2_raw")
self.label_282 = QtWidgets.QLabel(self.frame_37)
self.label_282.setGeometry(QtCore.QRect(330, 60, 71, 18))
self.label_282.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_282.setObjectName("label_282")
self.label_288 = QtWidgets.QLabel(self.frame_37)
self.label_288.setGeometry(QtCore.QRect(490, 90, 16, 16))
self.label_288.setObjectName("label_288")
self.ir_d2_80cm = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d2_80cm.setGeometry(QtCore.QRect(170, 200, 85, 20))
self.ir_d2_80cm.setToolTip("<html><head/><body>Raw value at 50 cm from base of sensor (about 24000)</body></html>")
self.ir_d2_80cm.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d2_80cm.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d2_80cm.setReadOnly(False)
self.ir_d2_80cm.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.ir_d2_80cm.setSpecialValueText("")
self.ir_d2_80cm.setDecimals(0)
self.ir_d2_80cm.setMinimum(0.0)
self.ir_d2_80cm.setMaximum(128000.0)
self.ir_d2_80cm.setProperty("value", 24100.0)
self.ir_d2_80cm.setObjectName("ir_d2_80cm")
self.ir_d1_80cm = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d1_80cm.setGeometry(QtCore.QRect(170, 148, 85, 20))
self.ir_d1_80cm.setToolTip("<html><head/><body>Raw value at 50cm from base of sensor (about 24000)</body></html>")
self.ir_d1_80cm.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d1_80cm.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d1_80cm.setReadOnly(False)
self.ir_d1_80cm.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.ir_d1_80cm.setSpecialValueText("")
self.ir_d1_80cm.setDecimals(0)
self.ir_d1_80cm.setMinimum(0.0)
self.ir_d1_80cm.setMaximum(128000.0)
self.ir_d1_80cm.setProperty("value", 24100.0)
self.ir_d1_80cm.setObjectName("ir_d1_80cm")
self.ir_d2_20cm = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d2_20cm.setGeometry(QtCore.QRect(170, 174, 85, 20))
self.ir_d2_20cm.setToolTip("<html><head/><body>Raw value at 13 cm from base of sensor (about 80000)</body></html>")
self.ir_d2_20cm.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d2_20cm.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d2_20cm.setReadOnly(False)
self.ir_d2_20cm.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.ir_d2_20cm.setSpecialValueText("")
self.ir_d2_20cm.setDecimals(0)
self.ir_d2_20cm.setMinimum(0.0)
self.ir_d2_20cm.setMaximum(128000.0)
self.ir_d2_20cm.setProperty("value", 72300.0)
self.ir_d2_20cm.setObjectName("ir_d2_20cm")
self.label_284 = QtWidgets.QLabel(self.frame_37)
self.label_284.setGeometry(QtCore.QRect(32, 148, 132, 20))
self.label_284.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_284.setObjectName("label_284")
self.label_286 = QtWidgets.QLabel(self.frame_37)
self.label_286.setGeometry(QtCore.QRect(32, 174, 132, 20))
self.label_286.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_286.setObjectName("label_286")
self.label_285 = QtWidgets.QLabel(self.frame_37)
self.label_285.setGeometry(QtCore.QRect(32, 200, 132, 20))
self.label_285.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_285.setObjectName("label_285")
self.label_283 = QtWidgets.QLabel(self.frame_37)
self.label_283.setGeometry(QtCore.QRect(32, 122, 132, 20))
self.label_283.setMaximumSize(QtCore.QSize(16777215, 20))
self.label_283.setObjectName("label_283")
self.ir_d1_20cm = QtWidgets.QDoubleSpinBox(self.frame_37)
self.ir_d1_20cm.setGeometry(QtCore.QRect(170, 122, 85, 20))
self.ir_d1_20cm.setToolTip("<html><head/><body>Raw value at 13 cm from base of sensor (about 80000)</body></html>")
self.ir_d1_20cm.setLocale(QtCore.QLocale(QtCore.QLocale.C, QtCore.QLocale.AnyCountry))
self.ir_d1_20cm.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.ir_d1_20cm.setReadOnly(False)
self.ir_d1_20cm.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.ir_d1_20cm.setSpecialValueText("")
self.ir_d1_20cm.setDecimals(0)
self.ir_d1_20cm.setMinimum(0.0)
self.ir_d1_20cm.setMaximum(128000.0)
self.ir_d1_20cm.setProperty("value", 72000.0)
self.ir_d1_20cm.setObjectName("ir_d1_20cm")
self.checkBox_ir_use = QtWidgets.QCheckBox(self.frame_37)
self.checkBox_ir_use.setGeometry(QtCore.QRect(30, 70, 132, 17))
self.checkBox_ir_use.setObjectName("checkBox_ir_use")
self.checkBox_ir_installed = QtWidgets.QCheckBox(self.frame_37)
self.checkBox_ir_installed.setGeometry(QtCore.QRect(30, 44, 111, 17))
self.checkBox_ir_installed.setObjectName("checkBox_ir_installed")
self.label_277 = QtWidgets.QLabel(self.frame_37)
self.label_277.setGeometry(QtCore.QRect(30, 100, 132, 13))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_277.setFont(font)
self.label_277.setObjectName("label_277")
self.ir_apply = QtWidgets.QPushButton(self.frame_37)
self.ir_apply.setGeometry(QtCore.QRect(170, 230, 85, 23))
self.ir_apply.setObjectName("ir_apply")
self.label_278 = QtWidgets.QLabel(self.frame_37)
self.label_278.setGeometry(QtCore.QRect(310, 30, 171, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.label_278.setFont(font)
self.label_278.setObjectName("label_278")
self.ir_edit = QtWidgets.QPushButton(self.frame_37)
self.ir_edit.setGeometry(QtCore.QRect(30, 230, 132, 23))
self.ir_edit.setObjectName("ir_edit")
self.ir_cancel = QtWidgets.QPushButton(self.frame_37)
self.ir_cancel.setGeometry(QtCore.QRect(260, 230, 51, 23))
self.ir_cancel.setObjectName("ir_cancel")
self.tabWidget.addTab(self.tab_8, "")
self.tab_3 = QtWidgets.QWidget()
self.tab_3.setObjectName("tab_3")
self.frame_4 = QtWidgets.QFrame(self.tab_3)
self.frame_4.setGeometry(QtCore.QRect(0, 0, 711, 691))
self.frame_4.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_4.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_4.setObjectName("frame_4")
self.EnableCamera_checkBox = QtWidgets.QCheckBox(self.frame_4)
self.EnableCamera_checkBox.setGeometry(QtCore.QRect(20, 20, 121, 17))
self.EnableCamera_checkBox.setObjectName("EnableCamera_checkBox")
self.frame_5 = QtWidgets.QFrame(self.frame_4)
self.frame_5.setGeometry(QtCore.QRect(10, 430, 461, 211))
self.frame_5.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_5.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_5.setObjectName("frame_5")
self.Camera_Record_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_Record_Cmd.setGeometry(QtCore.QRect(240, 10, 91, 23))
self.Camera_Record_Cmd.setObjectName("Camera_Record_Cmd")
self.Camera_StopRecord_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_StopRecord_Cmd.setGeometry(QtCore.QRect(340, 10, 91, 23))
self.Camera_StopRecord_Cmd.setObjectName("Camera_StopRecord_Cmd")
self.Camera_Snapshot_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_Snapshot_Cmd.setGeometry(QtCore.QRect(10, 10, 111, 23))
self.Camera_Snapshot_Cmd.setObjectName("Camera_Snapshot_Cmd")
self.Camera_SnapFilename = QtWidgets.QLineEdit(self.frame_5)
self.Camera_SnapFilename.setGeometry(QtCore.QRect(240, 70, 191, 20))
self.Camera_SnapFilename.setObjectName("Camera_SnapFilename")
self.label_55 = QtWidgets.QLabel(self.frame_5)
self.label_55.setGeometry(QtCore.QRect(240, 50, 121, 20))
self.label_55.setObjectName("label_55")
self.Camera_SnapSave_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_SnapSave_Cmd.setGeometry(QtCore.QRect(340, 100, 91, 23))
self.Camera_SnapSave_Cmd.setObjectName("Camera_SnapSave_Cmd")
self.Snapshot_Label = QtWidgets.QLabel(self.frame_5)
self.Snapshot_Label.setGeometry(QtCore.QRect(10, 40, 221, 161))
self.Snapshot_Label.setLayoutDirection(QtCore.Qt.LeftToRight)
self.Snapshot_Label.setAutoFillBackground(True)
self.Snapshot_Label.setFrameShape(QtWidgets.QFrame.Box)
self.Snapshot_Label.setFrameShadow(QtWidgets.QFrame.Raised)
self.Snapshot_Label.setLineWidth(-2)
self.Snapshot_Label.setText("")
self.Snapshot_Label.setScaledContents(True)
self.Snapshot_Label.setIndent(0)
self.Snapshot_Label.setObjectName("Snapshot_Label")
self.VideoRecording_Label = QtWidgets.QLabel(self.frame_5)
self.VideoRecording_Label.setGeometry(QtCore.QRect(240, 30, 71, 16))
font = QtGui.QFont()
font.setBold(True)
font.setWeight(75)
self.VideoRecording_Label.setFont(font)
self.VideoRecording_Label.setObjectName("VideoRecording_Label")
self.Camera_ClearSnap_Cmd = QtWidgets.QPushButton(self.frame_5)
self.Camera_ClearSnap_Cmd.setGeometry(QtCore.QRect(130, 10, 91, 23))
self.Camera_ClearSnap_Cmd.setObjectName("Camera_ClearSnap_Cmd")
self.frame_6 = QtWidgets.QFrame(self.frame_4)
self.frame_6.setGeometry(QtCore.QRect(10, 40, 561, 381))
self.frame_6.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_6.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_6.setObjectName("frame_6")
self.Stream_Label = QtWidgets.QLabel(self.frame_6)
self.Stream_Label.setGeometry(QtCore.QRect(10, 0, 541, 371))
self.Stream_Label.setText("")
self.Stream_Label.setScaledContents(True)
self.Stream_Label.setObjectName("Stream_Label")
self.tabWidget.addTab(self.tab_3, "")
self.tab_4 = QtWidgets.QWidget()
self.tab_4.setObjectName("tab_4")
self.frame_7 = QtWidgets.QFrame(self.tab_4)
self.frame_7.setGeometry(QtCore.QRect(0, 0, 541, 631))
self.frame_7.setFrameShape(QtWidgets.QFrame.StyledPanel)
self.frame_7.setFrameShadow(QtWidgets.QFrame.Raised)
self.frame_7.setObjectName("frame_7")
self.label_30 = QtWidgets.QLabel(self.frame_7)
self.label_30.setGeometry(QtCore.QRect(10, 0, 151, 21))
font = QtGui.QFont()
font.setPointSize(10)
font.setBold(True)
font.setWeight(75)
self.label_30.setFont(font)
self.label_30.setObjectName("label_30")
self.enableServo1_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo1_checkBox.setGeometry(QtCore.QRect(10, 30, 111, 17))
self.enableServo1_checkBox.setObjectName("enableServo1_checkBox")
self.enableServo2_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo2_checkBox.setGeometry(QtCore.QRect(10, 150, 111, 17))
self.enableServo2_checkBox.setObjectName("enableServo2_checkBox")
self.enableServo3_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo3_checkBox.setGeometry(QtCore.QRect(10, 270, 111, 17))
self.enableServo3_checkBox.setObjectName("enableServo3_checkBox")
self.Servo1_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo1_Pos_dial.setGeometry(QtCore.QRect(210, 50, 81, 81))
self.Servo1_Pos_dial.setMinimum(-920)
self.Servo1_Pos_dial.setMaximum(920)
self.Servo1_Pos_dial.setTracking(False)
self.Servo1_Pos_dial.setObjectName("Servo1_Pos_dial")
self.label_31 = QtWidgets.QLabel(self.frame_7)
self.label_31.setGeometry(QtCore.QRect(10, 50, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_31.setFont(font)
self.label_31.setObjectName("label_31")
self.servo1_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo1_current_pos.setGeometry(QtCore.QRect(100, 50, 91, 20))
self.servo1_current_pos.setStatusTip("")
self.servo1_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo1_current_pos.setObjectName("servo1_current_pos")
self.label_33 = QtWidgets.QLabel(self.frame_7)
self.label_33.setGeometry(QtCore.QRect(10, 80, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_33.setFont(font)
self.label_33.setObjectName("label_33")
self.manualServoPos1_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos1_Val.setGeometry(QtCore.QRect(100, 80, 91, 20))
self.manualServoPos1_Val.setObjectName("manualServoPos1_Val")
self.manualServoPos1_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos1_Cmd.setGeometry(QtCore.QRect(100, 110, 91, 23))
self.manualServoPos1_Cmd.setObjectName("manualServoPos1_Cmd")
self.label_39 = QtWidgets.QLabel(self.frame_7)
self.label_39.setGeometry(QtCore.QRect(220, 30, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_39.setFont(font)
self.label_39.setObjectName("label_39")
self.Servo2_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo2_Pos_dial.setGeometry(QtCore.QRect(210, 170, 81, 81))
self.Servo2_Pos_dial.setMinimum(-920)
self.Servo2_Pos_dial.setMaximum(920)
self.Servo2_Pos_dial.setTracking(False)
self.Servo2_Pos_dial.setObjectName("Servo2_Pos_dial")
self.label_40 = QtWidgets.QLabel(self.frame_7)
self.label_40.setGeometry(QtCore.QRect(220, 150, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_40.setFont(font)
self.label_40.setObjectName("label_40")
self.manualServoPos2_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos2_Val.setGeometry(QtCore.QRect(100, 200, 91, 20))
self.manualServoPos2_Val.setObjectName("manualServoPos2_Val")
self.servo2_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo2_current_pos.setGeometry(QtCore.QRect(100, 170, 91, 20))
self.servo2_current_pos.setStatusTip("")
self.servo2_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo2_current_pos.setObjectName("servo2_current_pos")
self.label_41 = QtWidgets.QLabel(self.frame_7)
self.label_41.setGeometry(QtCore.QRect(10, 200, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_41.setFont(font)
self.label_41.setObjectName("label_41")
self.manualServoPos2_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos2_Cmd.setGeometry(QtCore.QRect(100, 230, 91, 23))
self.manualServoPos2_Cmd.setObjectName("manualServoPos2_Cmd")
self.label_43 = QtWidgets.QLabel(self.frame_7)
self.label_43.setGeometry(QtCore.QRect(10, 170, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_43.setFont(font)
self.label_43.setObjectName("label_43")
self.Servo3_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo3_Pos_dial.setGeometry(QtCore.QRect(210, 290, 81, 81))
self.Servo3_Pos_dial.setMinimum(-920)
self.Servo3_Pos_dial.setMaximum(920)
self.Servo3_Pos_dial.setTracking(False)
self.Servo3_Pos_dial.setObjectName("Servo3_Pos_dial")
self.label_46 = QtWidgets.QLabel(self.frame_7)
self.label_46.setGeometry(QtCore.QRect(220, 270, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_46.setFont(font)
self.label_46.setObjectName("label_46")
self.manualServoPos3_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos3_Val.setGeometry(QtCore.QRect(100, 320, 91, 20))
self.manualServoPos3_Val.setObjectName("manualServoPos3_Val")
self.servo3_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo3_current_pos.setGeometry(QtCore.QRect(100, 290, 91, 20))
self.servo3_current_pos.setStatusTip("")
self.servo3_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo3_current_pos.setObjectName("servo3_current_pos")
self.label_47 = QtWidgets.QLabel(self.frame_7)
self.label_47.setGeometry(QtCore.QRect(10, 320, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_47.setFont(font)
self.label_47.setObjectName("label_47")
self.manualServoPos3_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos3_Cmd.setGeometry(QtCore.QRect(100, 350, 91, 23))
self.manualServoPos3_Cmd.setObjectName("manualServoPos3_Cmd")
self.label_48 = QtWidgets.QLabel(self.frame_7)
self.label_48.setGeometry(QtCore.QRect(10, 290, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_48.setFont(font)
self.label_48.setObjectName("label_48")
self.enableServo4_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo4_checkBox.setGeometry(QtCore.QRect(10, 390, 111, 17))
self.enableServo4_checkBox.setObjectName("enableServo4_checkBox")
self.Servo4_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo4_Pos_dial.setGeometry(QtCore.QRect(210, 410, 81, 81))
self.Servo4_Pos_dial.setMinimum(-920)
self.Servo4_Pos_dial.setMaximum(920)
self.Servo4_Pos_dial.setTracking(False)
self.Servo4_Pos_dial.setObjectName("Servo4_Pos_dial")
self.label_49 = QtWidgets.QLabel(self.frame_7)
self.label_49.setGeometry(QtCore.QRect(220, 390, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_49.setFont(font)
self.label_49.setObjectName("label_49")
self.manualServoPos4_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos4_Val.setGeometry(QtCore.QRect(100, 440, 91, 20))
self.manualServoPos4_Val.setObjectName("manualServoPos4_Val")
self.servo4_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo4_current_pos.setGeometry(QtCore.QRect(100, 410, 91, 20))
self.servo4_current_pos.setStatusTip("")
self.servo4_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo4_current_pos.setObjectName("servo4_current_pos")
self.label_50 = QtWidgets.QLabel(self.frame_7)
self.label_50.setGeometry(QtCore.QRect(10, 440, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_50.setFont(font)
self.label_50.setObjectName("label_50")
self.manualServoPos4_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos4_Cmd.setGeometry(QtCore.QRect(100, 470, 91, 23))
self.manualServoPos4_Cmd.setObjectName("manualServoPos4_Cmd")
self.label_51 = QtWidgets.QLabel(self.frame_7)
self.label_51.setGeometry(QtCore.QRect(10, 410, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_51.setFont(font)
self.label_51.setObjectName("label_51")
self.enableServo5_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.enableServo5_checkBox.setGeometry(QtCore.QRect(10, 510, 111, 17))
self.enableServo5_checkBox.setObjectName("enableServo5_checkBox")
self.Servo5_Pos_dial = QtWidgets.QDial(self.frame_7)
self.Servo5_Pos_dial.setGeometry(QtCore.QRect(210, 530, 81, 81))
self.Servo5_Pos_dial.setMinimum(-920)
self.Servo5_Pos_dial.setMaximum(920)
self.Servo5_Pos_dial.setTracking(False)
self.Servo5_Pos_dial.setObjectName("Servo5_Pos_dial")
self.label_52 = QtWidgets.QLabel(self.frame_7)
self.label_52.setGeometry(QtCore.QRect(220, 510, 61, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_52.setFont(font)
self.label_52.setObjectName("label_52")
self.manualServoPos5_Val = QtWidgets.QLineEdit(self.frame_7)
self.manualServoPos5_Val.setGeometry(QtCore.QRect(100, 560, 91, 20))
self.manualServoPos5_Val.setObjectName("manualServoPos5_Val")
self.servo5_current_pos = QtWidgets.QLineEdit(self.frame_7)
self.servo5_current_pos.setGeometry(QtCore.QRect(100, 530, 91, 20))
self.servo5_current_pos.setStatusTip("")
self.servo5_current_pos.setStyleSheet("background-color: rgb(230,230,230);")
self.servo5_current_pos.setObjectName("servo5_current_pos")
self.label_53 = QtWidgets.QLabel(self.frame_7)
self.label_53.setGeometry(QtCore.QRect(10, 560, 91, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_53.setFont(font)
self.label_53.setObjectName("label_53")
self.manualServoPos5_Cmd = QtWidgets.QPushButton(self.frame_7)
self.manualServoPos5_Cmd.setGeometry(QtCore.QRect(100, 590, 91, 23))
self.manualServoPos5_Cmd.setObjectName("manualServoPos5_Cmd")
self.label_54 = QtWidgets.QLabel(self.frame_7)
self.label_54.setGeometry(QtCore.QRect(10, 530, 81, 21))
font = QtGui.QFont()
font.setBold(False)
font.setWeight(50)
self.label_54.setFont(font)
self.label_54.setObjectName("label_54")
self.Servo1Steering_checkBox = QtWidgets.QCheckBox(self.frame_7)
self.Servo1Steering_checkBox.setGeometry(QtCore.QRect(110, 30, 111, 17))
self.Servo1Steering_checkBox.setObjectName("Servo1Steering_checkBox")
self.val_servo1_offset = QtWidgets.QDoubleSpinBox(self.frame_7)
self.val_servo1_offset.setGeometry(QtCore.QRect(300, 50, 81, 18))
self.val_servo1_offset.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.val_servo1_offset.setReadOnly(False)
self.val_servo1_offset.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.val_servo1_offset.setSpecialValueText("")
self.val_servo1_offset.setDecimals(0)
self.val_servo1_offset.setMinimum(-200.0)
self.val_servo1_offset.setMaximum(200.0)
self.val_servo1_offset.setSingleStep(0.01)
self.val_servo1_offset.setProperty("value", 10.0)
self.val_servo1_offset.setObjectName("val_servo1_offset")
self.label_servo_6 = QtWidgets.QLabel(self.frame_7)
self.label_servo_6.setGeometry(QtCore.QRect(310, 70, 75, 13))
self.label_servo_6.setAlignment(QtCore.Qt.AlignCenter)
self.label_servo_6.setObjectName("label_servo_6")
self.val_servo1_scale = QtWidgets.QDoubleSpinBox(self.frame_7)
self.val_servo1_scale.setGeometry(QtCore.QRect(300, 130, 81, 18))
self.val_servo1_scale.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.val_servo1_scale.setReadOnly(False)
self.val_servo1_scale.setButtonSymbols(QtWidgets.QAbstractSpinBox.UpDownArrows)
self.val_servo1_scale.setSpecialValueText("")
self.val_servo1_scale.setDecimals(1)
self.val_servo1_scale.setMinimum(-180.0)
self.val_servo1_scale.setMaximum(180.0)
self.val_servo1_scale.setSingleStep(1.0)
self.val_servo1_scale.setProperty("value", 160.0)
self.val_servo1_scale.setObjectName("val_servo1_scale")
self.label_servo_9 = QtWidgets.QLabel(self.frame_7)
self.label_servo_9.setGeometry(QtCore.QRect(300, 110, 87, 13))
self.label_servo_9.setAlignment(QtCore.Qt.AlignCenter)
self.label_servo_9.setObjectName("label_servo_9")
self.label_servo_8 = QtWidgets.QLabel(self.frame_7)
self.label_servo_8.setGeometry(QtCore.QRect(303, 31, 81, 13))
self.label_servo_8.setAlignment(QtCore.Qt.AlignCenter)
self.label_servo_8.setObjectName("label_servo_8")
self.val_steer_distance = QtWidgets.QDoubleSpinBox(self.frame_7)
self.val_steer_distance.setGeometry(QtCore.QRect(300, 90, 81, 18))
self.val_steer_distance.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter)
self.val_steer_distance.setReadOnly(False)
self.val_steer_distance.setButtonSymbols(QtWidgets.QAbstractSpinBox.NoButtons)
self.val_steer_distance.setSpecialValueText("")
self.val_steer_distance.setDecimals(3)
self.val_steer_distance.setMinimum(-1.0)
self.val_steer_distance.setMaximum(1.0)
self.val_steer_distance.setSingleStep(0.01)
self.val_steer_distance.setProperty("value", 0.16)
self.val_steer_distance.setObjectName("val_steer_distance")
self.tabWidget.addTab(self.tab_4, "")
self.tab_5 = QtWidgets.QWidget()
self.tab_5.setObjectName("tab_5")
self.tabWidget.addTab(self.tab_5, "")
MainWindow.setCentralWidget(self.centralwidget)
self.menubar = QtWidgets.QMenuBar(MainWindow)
self.menubar.setGeometry(QtCore.QRect(0, 0, 1122, 21))
self.menubar.setObjectName("menubar")
self.menuFile = QtWidgets.QMenu(self.menubar)
self.menuFile.setObjectName("menuFile")
self.menuHelp = QtWidgets.QMenu(self.menubar)
self.menuHelp.setObjectName("menuHelp")
self.menuTools = QtWidgets.QMenu(self.menubar)
self.menuTools.setObjectName("menuTools")
MainWindow.setMenuBar(self.menubar)
self.statusbar = QtWidgets.QStatusBar(MainWindow)
self.statusbar.setObjectName("statusbar")
MainWindow.setStatusBar(self.statusbar)
self.menubar.addAction(self.menuFile.menuAction())
self.menubar.addAction(self.menuTools.menuAction())
self.menubar.addAction(self.menuHelp.menuAction())

self.retranslateUi(MainWindow)
self.tabWidget.setCurrentIndex(2)
self.tabWidget_2.setCurrentIndex(0)
QtCore.QMetaObject.connectSlotsByName(MainWindow)

def retranslateUi(self, MainWindow):
_translate = QtCore.QCoreApplication.translate
MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow"))
self.label_2.setText(_translate("MainWindow", "Network Status: "))
self.Disconnect_Cmd.setText(_translate("MainWindow", "Disconnect"))
self.connect_Cmd.setText(_translate("MainWindow", "Connect"))
self.label.setText(_translate("MainWindow", "IP adress:"))
self.IP_NetworkStatus_label.setText(_translate("MainWindow", "Status"))
self.label_7.setText(_translate("MainWindow", "INTERNET CONNECTION"))
self.SidebarMenu.setItemText(self.SidebarMenu.indexOf(self.SidebarMenuPage1), _translate("MainWindow", "Robot Connection"))
self.send_Cmd.setText(_translate("MainWindow", "Send"))
self.TX_checkBox.setText(_translate("MainWindow", "TX"))
self.RX_checkBox.setText(_translate("MainWindow", "RX"))
self.ConsoleHelp_Cmd.setText(_translate("MainWindow", "Help"))
self.ConsoleClear_Cmd.setText(_translate("MainWindow", "Clear"))
self.SidebarMenu.setItemText(self.SidebarMenu.indexOf(self.SidebarMenuPage2), _translate("MainWindow", "Debugger"))
self.main_start.setText(_translate("MainWindow", "Start"))
self.main_remote_ctrl.setToolTip(_translate("MainWindow", "<html><head/><body>Remote control is active,and mission paused.</body></html>"))
self.main_remote_ctrl.setText(_translate("MainWindow", "Enable Remote-Control"))
self.main_stop.setText(_translate("MainWindow", "Stop"))
self.main_mission_state.setToolTip(_translate("MainWindow", "<html><head/><body>Shows that a mission is active.Number is0: not active1: waiting 1 second to start2: running mission line6,7: listening to button (more button presses)8: stop mission</body></html>"))
self.main_mission_state.setText(_translate("MainWindow", "Enable Mission"))
self.SidebarMenu.setItemText(self.SidebarMenu.indexOf(self.SidebarMenuPage3), _translate("MainWindow", "Mission"))
self.label_5.setText(_translate("MainWindow", "Manual Speed:"))
self.enableKeyboardRC_checkBox.setToolTip(_translate("MainWindow", "<html><head/><body>Enable Remote Keyboard control</body></html>"))
self.enableKeyboardRC_checkBox.setText(_translate("MainWindow", "Enable keyboard control"))
self.guiRC_Reverse_Cmd.setText(_translate("MainWindow", "V"))
self.guiRC_Forward_Cmd.setText(_translate("MainWindow", "^"))
self.enableGuiRC_checkBox.setToolTip(_translate("MainWindow", "<html><head/><body>Enable Remote - GUI control </body></html>"))
self.enableGuiRC_checkBox.setText(_translate("MainWindow", "Enable GUI control"))
self.guiRC_Right_Cmd.setText(_translate("MainWindow", ">"))
self.guiRC_Left_Cmd.setText(_translate("MainWindow", "<"))
self.label_8.setText(_translate("MainWindow", "Motor Remote GUI control"))
self.label_9.setText(_translate("MainWindow", "Robot Motor hardware parameters"))
self.label_21.setToolTip(_translate("MainWindow", "<html><head/><body>Number of encoder pulses on each motor revolution (normally 48)</body></html>"))
self.label_21.setText(_translate("MainWindow", "Enc. pulse per rev:"))
self.label_22.setText(_translate("MainWindow", "Gear ratio [N]:"))
self.Robot_GearRatio.setToolTip(_translate("MainWindow", "<html><head/><body>Gear ratio from motor to wheel axle (9.68).</body></html>"))
self.label_24.setText(_translate("MainWindow", "Wheel radius (Left) [m]:"))
self.label_127.setText(_translate("MainWindow", "Robot ID:"))
self.label_128.setText(_translate("MainWindow", "Hardware:"))
self.label_25.setToolTip(_translate("MainWindow", "<html><head/><body>Wheel base is distance between wheels (mid-wheel to mid-wheel) for turn calculations.</body></html>"))
self.label_25.setText(_translate("MainWindow", "Wheel-base [m]:"))
self.Robot_PulsePerRev.setToolTip(_translate("MainWindow", "<html><head/><body>Number of encoder ticks per motor revolution (48)</body></html>"))
self.label_26.setText(_translate("MainWindow", "Wheel radius (Right) [m]:"))
self.Robot_RevEncoder_Right.setToolTip(_translate("MainWindow", "<html><head/><body>Checking this makes odometry go the other way, i.e. encoder pins swapped (right motor).This should NOT be checked on normal REGBOT.</body></html>"))
self.Robot_RevEncoder_Right.setText(_translate("MainWindow", "Reverse Right Encoder"))
self.Robot_RevMotor.setToolTip(_translate("MainWindow", "<html><head/><body>Checking this makes wheels go the other way when orderd forward.This should NOT be checked on normal REGBOT.</body></html>"))
self.Robot_RevMotor.setText(_translate("MainWindow", "Reverse Motor"))
self.Robot_RevEncoder_Left.setToolTip(_translate("MainWindow", "<html><head/><body>Checking this makes odometry go the other way, i.e. encoder pins swapped.This should NOT be checked on normal REGBOT.</body></html>"))
self.Robot_RevEncoder_Left.setText(_translate("MainWindow", "Reverse Left Encoder"))
self.label_222.setToolTip(_translate("MainWindow", "<html><head/><body>Battery is assumed to be of type LiPo 3cell. Any battery with a normal voltage between 9 and 15V should do.</body></html>"))
self.label_222.setText(_translate("MainWindow", "Go idle at [V]:"))
self.save_id_on_robot.setText(_translate("MainWindow", "Save"))
self.label_129.setText(_translate("MainWindow", "Tilt offset [Rad]:"))
self.Robot_edit.setText(_translate("MainWindow", "Edit"))
self.Robot_OnBattery.setText(_translate("MainWindow", "Robot On Battery"))
self.robot_cancel.setText(_translate("MainWindow", "Cancel"))
self.label_36.setText(_translate("MainWindow", "Right"))
self.label_37.setText(_translate("MainWindow", "Left"))
self.label_42.setText(_translate("MainWindow", "Current [A]:"))
self.label_28.setText(_translate("MainWindow", "Encoder:"))
self.label_38.setText(_translate("MainWindow", "Drive information"))
self.label_32.setText(_translate("MainWindow", "Distance [m]:"))
self.label_29.setText(_translate("MainWindow", "Wheel Speed [m/s]:"))
self.tabWidget_2.setTabText(self.tabWidget_2.indexOf(self.tab_6), _translate("MainWindow", "Drive Info"))
self.label_76.setText(_translate("MainWindow", "degrees"))
self.label_45.setText(_translate("MainWindow", "Pose (relative to starting/reset point)"))
self.Robot_PoseReset.setText(_translate("MainWindow", "reset"))
self.label_44.setText(_translate("MainWindow", "Heading [rad]:"))
self.label_34.setText(_translate("MainWindow", "X (forward) [m]:"))
self.label_35.setText(_translate("MainWindow", "Y (left) [m]:"))
self.label_125.setText(_translate("MainWindow", "Tilt [rad]:"))
self.tabWidget_2.setTabText(self.tabWidget_2.indexOf(self.tab_7), _translate("MainWindow", "Pose Info"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab), _translate("MainWindow", "Motor"))
self.label_10.setText(_translate("MainWindow", "Line Sensor"))
self.enableLineSensor_checkBox.setText(_translate("MainWindow", "Enable Line Sensor"))
self.line_bar_1.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_4.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_6.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_5.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_3.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_7.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_2.setFormat(_translate("MainWindow", "%p%"))
self.line_bar_8.setFormat(_translate("MainWindow", "%p%"))
self.label_11.setText(_translate("MainWindow", "Sensor readings:"))
self.label_12.setText(_translate("MainWindow", "L1"))
self.label_13.setText(_translate("MainWindow", "L2"))
self.label_14.setText(_translate("MainWindow", "L6"))
self.label_15.setText(_translate("MainWindow", "L3"))
self.label_16.setText(_translate("MainWindow", "L4"))
self.label_17.setText(_translate("MainWindow", "L5"))
self.label_18.setText(_translate("MainWindow", "L7"))
self.label_19.setText(_translate("MainWindow", "L8"))
self.label_118.setText(_translate("MainWindow", "Left edge"))
self.ls_left_edge.setToolTip(_translate("MainWindow", "<html><head/><body>Left edge positionGray if an edge is not detected</body></html>"))
self.ls_right_edge.setToolTip(_translate("MainWindow", "<html><head/><body>Right edge positionEither no edge is valid or both are valid.</body></html>"))
self.label_121.setText(_translate("MainWindow", "Right edge"))
self.label_20.setText(_translate("MainWindow", "Edge Analysis:"))
self.label_27.setText(_translate("MainWindow", "Crossing"))
self.label_171.setText(_translate("MainWindow", "Line valid"))
self.ls_crossing_cnt.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.ls_line_valid_cnt.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 5.0 = no line, 5=line seen in 5ms. </body></html>"))
self.groupBox_5.setTitle(_translate("MainWindow", "Sensor Calibration"))
self.CalibrateWhite_Cmd.setText(_translate("MainWindow", "Calibrate White"))
self.label_217.setText(_translate("MainWindow", "L3"))
self.label_218.setText(_translate("MainWindow", "L5"))
self.label_219.setText(_translate("MainWindow", "L7"))
self.label_221.setText(_translate("MainWindow", "L4"))
self.label_223.setText(_translate("MainWindow", "L6"))
self.label_224.setText(_translate("MainWindow", "L1"))
self.label_225.setText(_translate("MainWindow", "L8"))
self.label_226.setText(_translate("MainWindow", "L2"))
self.ls_max_white_7.setText(_translate("MainWindow", "L1"))
self.ls_max_white_8.setText(_translate("MainWindow", "L1"))
self.ls_max_white_1.setText(_translate("MainWindow", "L1"))
self.ls_max_white_3.setText(_translate("MainWindow", "L1"))
self.ls_max_white_5.setText(_translate("MainWindow", "L1"))
self.ls_max_white_2.setText(_translate("MainWindow", "L1"))
self.ls_max_white_4.setText(_translate("MainWindow", "L1"))
self.ls_max_white_6.setText(_translate("MainWindow", "L1"))
self.ls_max_black_5.setText(_translate("MainWindow", "L1"))
self.ls_max_black_4.setText(_translate("MainWindow", "L1"))
self.ls_max_black_8.setText(_translate("MainWindow", "L1"))
self.ls_max_black_1.setText(_translate("MainWindow", "L1"))
self.ls_max_black_2.setText(_translate("MainWindow", "L1"))
self.ls_max_black_3.setText(_translate("MainWindow", "L1"))
self.ls_max_black_6.setText(_translate("MainWindow", "L1"))
self.ls_max_black_7.setText(_translate("MainWindow", "L1"))
self.label_227.setText(_translate("MainWindow", "White"))
self.CalibrateBlack_Cmd.setText(_translate("MainWindow", "Calibrate Black "))
self.label_228.setText(_translate("MainWindow", "Black"))
self.ls_swap_left_right.setToolTip(_translate("MainWindow", "<html><head/><body>Swap the detected left and right edge.Check this if LED\'s are (mostly) in the forward (X) direction.(Normally the LED end of the line sensor PCB is closer to driving wheel axis).</body></html>"))
self.ls_swap_left_right.setText(_translate("MainWindow", "Swap Sides"))
self.ls_show_normalized.setToolTip(_translate("MainWindow", "<html><head/><body>If line is not white (bright), then assumed black (dark)The normalized value should be shown as 75%.</body></html>"))
self.ls_show_normalized.setText(_translate("MainWindow", "Show normalized"))
self.label_23.setText(_translate("MainWindow", "Crossing det."))
self.label_156.setText(_translate("MainWindow", "Max RAW"))
self.ls_crossing_detect.setToolTip(_translate("MainWindow", "<html><head/><body>Distance between left and right edge in a 0 to 8 scale.Value depends distance between edges with an extra value if width increse fast. Therefore the width can temorarily be higher than 6 (edge values are from -3 to 3)this should make it easier to detect a crossing.For Regbot in balance (and normal tape (38mm)), then about 4.8 cm to trigger crossing line.For Robobot this could be as low as 3-4, as the linesensor is wider.</body></html>"))
self.ls_power_high.setToolTip(_translate("MainWindow", "<html><head/><body>If not in auto mode, should the line sensor LED power be high (ca. 30mA), else low (ca 10mA)</body></html>"))
self.ls_power_high.setText(_translate("MainWindow", "High power"))
self.ls_line_white.setToolTip(_translate("MainWindow", "<html><head/><body>If line is not white (bright), then assumed black (dark)</body></html>"))
self.ls_line_white.setText(_translate("MainWindow", "White line"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_2), _translate("MainWindow", "Edge Detection"))
self.label_56.setText(_translate("MainWindow", "Distance sensor"))
self.label_279.setText(_translate("MainWindow", "D1 right"))
self.label_281.setText(_translate("MainWindow", "D1 right RAW"))
self.label_287.setText(_translate("MainWindow", "m"))
self.label_280.setText(_translate("MainWindow", "D2 front"))
self.label_282.setText(_translate("MainWindow", "D2 front RAW"))
self.label_288.setText(_translate("MainWindow", "m"))
self.label_284.setText(_translate("MainWindow", "D1 right (50cm)"))
self.label_286.setText(_translate("MainWindow", "D2 front (13cm)"))
self.label_285.setText(_translate("MainWindow", "D2 front (50cm)"))
self.label_283.setText(_translate("MainWindow", "D1 right (13cm)"))
self.checkBox_ir_use.setToolTip(_translate("MainWindow", "<html><head/><body>Should be turned on at power on (will be turned on automatically in missions that use the sensor - if installed). If not installed, then distance is 10m</body></html>"))
self.checkBox_ir_use.setText(_translate("MainWindow", "Sensor Enable"))
self.checkBox_ir_installed.setToolTip(_translate("MainWindow", "<html><head/><body>Is the sensor installed (can not be turned on if not installed)</body></html>"))
self.checkBox_ir_installed.setText(_translate("MainWindow", "Sensor Installed"))
self.label_277.setText(_translate("MainWindow", "Calibrate values (RAW)"))
self.ir_apply.setText(_translate("MainWindow", "Apply"))
self.label_278.setText(_translate("MainWindow", "Distance values (measured)"))
self.ir_edit.setText(_translate("MainWindow", "Edit"))
self.ir_cancel.setText(_translate("MainWindow", "Cancel"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_8), _translate("MainWindow", "IR Distance"))
self.EnableCamera_checkBox.setToolTip(_translate("MainWindow", "<html><head/><body>To enable Camera stream, install mjpeg-streamer and run follwing commands: cd mjpg-streamer/mjpg-streamer-experimental/ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"</body></html>"))
self.EnableCamera_checkBox.setText(_translate("MainWindow", "Enable Camera"))
self.Camera_Record_Cmd.setText(_translate("MainWindow", "Record"))
self.Camera_StopRecord_Cmd.setText(_translate("MainWindow", "Stop Recording"))
self.Camera_Snapshot_Cmd.setText(_translate("MainWindow", "Take Snapshot"))
self.label_55.setText(_translate("MainWindow", "Snapshot Save Filename"))
self.Camera_SnapSave_Cmd.setText(_translate("MainWindow", "Save"))
self.VideoRecording_Label.setText(_translate("MainWindow", "<html><head/><body>RECORDING...</body></html>"))
self.Camera_ClearSnap_Cmd.setText(_translate("MainWindow", "Clear Snap"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_3), _translate("MainWindow", "Camera"))
self.label_30.setText(_translate("MainWindow", "Servo Controls"))
self.enableServo1_checkBox.setText(_translate("MainWindow", "Enable Servo 1"))
self.enableServo2_checkBox.setText(_translate("MainWindow", "Enable Servo 2"))
self.enableServo3_checkBox.setText(_translate("MainWindow", "Enable Servo 3"))
self.label_31.setText(_translate("MainWindow", "Current Position"))
self.servo1_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_33.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos1_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_39.setText(_translate("MainWindow", "Position Dial"))
self.label_40.setText(_translate("MainWindow", "Position Dial"))
self.servo2_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_41.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos2_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_43.setText(_translate("MainWindow", "Current Position"))
self.label_46.setText(_translate("MainWindow", "Position Dial"))
self.servo3_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_47.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos3_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_48.setText(_translate("MainWindow", "Current Position"))
self.enableServo4_checkBox.setText(_translate("MainWindow", "Enable Servo 4"))
self.label_49.setText(_translate("MainWindow", "Position Dial"))
self.servo4_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_50.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos4_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_51.setText(_translate("MainWindow", "Current Position"))
self.enableServo5_checkBox.setText(_translate("MainWindow", "Enable Servo 5"))
self.label_52.setText(_translate("MainWindow", "Position Dial"))
self.servo5_current_pos.setToolTip(_translate("MainWindow", "<html><head/><body>Value from 0 to 20.0 = no crossing, 20 = crossing detected in 20ms. Counts +1 when detection is above "Crossing det." value. else -1. </body></html>"))
self.label_53.setText(_translate("MainWindow", "Command Position"))
self.manualServoPos5_Cmd.setText(_translate("MainWindow", "Apply"))
self.label_54.setText(_translate("MainWindow", "Current Position"))
self.Servo1Steering_checkBox.setText(_translate("MainWindow", "Enable Steering"))
self.val_servo1_offset.setToolTip(_translate("MainWindow", "<html><head/><body>Steering offset to align steering wheel with straight forward.Units is control units, i.e. about 50 is 5 degrees.Zero offset means forward is 1.5ms pulse (center for servo)</body></html>"))
self.label_servo_6.setText(_translate("MainWindow", "distance"))
self.val_servo1_scale.setToolTip(_translate("MainWindow", "<html><head/><body>Servo angle change from 1ms to 2ms PWM pulse to servo.(typically 160 deg for hitec7235 and 90 deg for "old" servos)</body></html>"))
self.label_servo_9.setText(_translate("MainWindow", "scale"))
self.label_servo_8.setText(_translate("MainWindow", "Offset"))
self.val_steer_distance.setToolTip(_translate("MainWindow", "<html><head/><body>Distance from drive axle to steering wheel.Positive is forward (front wheel). In meters.Typically 0.16 m for ROBOBOT.</body></html>"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_4), _translate("MainWindow", "Servo"))
self.tabWidget.setTabText(self.tabWidget.indexOf(self.tab_5), _translate("MainWindow", "Mission"))
self.menuFile.setTitle(_translate("MainWindow", "File"))
self.menuHelp.setTitle(_translate("MainWindow", "Help"))
self.menuTools.setTitle(_translate("MainWindow", "Tools"))

if __name__ == "__main__":
import sys
app = QtWidgets.QApplication(sys.argv)
MainWindow = QtWidgets.QMainWindow()
ui = Ui_MainWindow()
ui.setupUi(MainWindow)
MainWindow.show()
sys.exit(app.exec_())

User Interface Code

2020-12-06T17:39:55Z

S192320: Blanked the page

User Interface Code

2020-12-06T17:39:27Z

S192320: Created page with " # -*- coding: utf-8 -*- # Form implementation generated from reading ui file 'Regbot_V1_9.ui' # # Created by: PyQt5 UI code generator 5.13.2 # # WARNING! All changes m..."

Source Code

2020-12-06T17:38:15Z

S192320: /* User Interface Code */

==User Interface Code==

[[User Interface Code]]

==Main Application Code==

==Linesensor==

==IR distance sensors==

==Servo==

==Video Threads==

Source Code

2020-12-06T17:37:56Z

S192320: /* Source Code */

==User Interface Code==

==Main Application Code==

==Linesensor==

==IR distance sensors==

==Servo==

==Video Threads==

Source Code

2020-12-06T17:34:47Z

S192320: Created page with "==Source Code=="

==Source Code==

Python interface

2020-12-06T17:34:31Z

S192320: /* Source Code */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
The Full Source code can be seen in the following link ->

[[Source Code]]

Python interface

2020-12-06T17:34:02Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== Source Code ==
[[Source Code]]

Python interface

2020-12-06T17:26:56Z

S192320: /* The Source Files */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

Python interface

2020-12-06T17:20:53Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

The actual Video recording class that controls the video recording can be seen below. The function ''out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))'' defines the output object and the ''cv_img = cap.read()'' and ''out.write(cv_img) '' gets the actual image from the cv2.VideoCapture() and then writes the image to the actual output video file.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream')
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-06T17:07:27Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

Meanwhile, the recording is happening, the toogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

Similarly, the video_StopRecording() function stops the video recording by stopping the video thread and setting video_recording flag to False.

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') # maybe replace with something else?
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-06T17:05:55Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to ensure the video starting/stopping and saving the video are the following. video_StartRecording() functions connects the update_image() function with the video reocrding thread and starts it. The video_recording flag is True. Meanwhile, the recording is happening, the roogleRecLabel(), makes the recirding status label visible, which indicates that the recoding is happening.

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') # maybe replace with something else?
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-06T16:50:19Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.
The key two functions to enure the actual video saving an

def video_StartRecording(self):
self.video_recording = True
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread = VideoRecordingThread()
self.VideoRecordingThread.change_pixmap_signal.connect(self.update_image)
self.VideoRecordingThread.start()
self.ConsoleMessage += "*Recording...\n"
self.ConsoleMessagePush = True

def toogleRecLabel(self):
self.toogleSignalON = not self.toogleSignalON
if self.toogleSignalON:
self.ui.VideoRecording_Label.setHidden(True)
else:
self.ui.VideoRecording_Label.setHidden(False)

def video_StopRecording(self):
if self.video_recording and self.wifiConnected:
self.VideoRecordingThread.stop()
self.ConsoleMessage += "*Recording Stopping...\n"
self.ConsoleMessagePush = True
self.ui.VideoRecording_Label.setHidden(True)
self.video_recording = False

out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
out.write(cv_img)

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') # maybe replace with something else?
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-06T16:46:23Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

To record the video from the video stream, the other class was implemented which took care of the video recording functionality.

class VideoRecordingThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') # maybe replace with something else?
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
out = cv2.VideoWriter('PiVideo_1.avi',cv2.VideoWriter_fourcc('M','J','P','G'), 10, (frame_width,frame_height))
while self._run_flag:
ret, cv_img = cap.read()
if ret:
out.write(cv_img)
cap.release()
out.release()
def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-06T16:43:06Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually display the image onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output image of the mjpeg streamer and converts it to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

Current setup for the Camera includes also the functionality to take a picture and record a video with the camera. It is a little more simple with taking a picture from the stream because we are just taking last snapped image from the update_image() function and save the image under the entered filename using snapshot_save() function. As it can be seen below, the picture by default saves the ".jpg" file with the current date and time. Otherwise, an user can enter the filename that he would like to use and save it under that. And using the opencv function cv2.imwrite() we save the picture.

def snapshot_save(self):
t = time.gmtime()
if self.ui.Camera_SnapFilename.text() == "":
filename = 'snapshot_' + str(t.tm_year) + str(t.tm_mon) + str(t.tm_mday) + "_" + str(t.tm_hour) + str(t.tm_min) + str(t.tm_sec) +'.jpg'
else:
filename = str(self.ui.Camera_SnapFilename.text() + '.jpg')
cv2.imwrite(filename, self.snapshot_image)

== The Source Files ==

Python interface

2020-12-02T18:59:11Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

The following functions are crucial to actually displaying immage onto the label of the Qlabel object. The covert_cv_qt(cv_img) takes the output immage of the mjpeg streamer and converts to the qt format so it can be displayed in our Qt Gui application.

def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

The update_image() actually updates the image and displays it to the Qlabel. As you can notice, this function uses the convert_cv_qt function to transform the received image in order to display it properly.

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False

== The Source Files ==

Python interface

2020-12-02T18:52:21Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

def update_image(self, cv_img):
if not self.camera_enabled:
cv_img = cv2.imread('nosignal.png')
qt_img = self.convert_cv_qt(cv_img)
self.ui.Stream_Label.setPixmap(qt_img)
if self.camera_enabled and self.take_snapshot:
qt_img = self.convert_cv_qt(cv_img)
self.snapshot_image = cv_img
self.ui.Snapshot_Label.setPixmap(qt_img)
self.take_snapshot = False
def convert_cv_qt(self, cv_img):
rgb_image = cv2.cvtColor(cv_img, cv2.COLOR_BGR2RGB)
h, w, ch = rgb_image.shape
bytes_per_line = ch * w
convert_to_Qt_format = QtGui.QImage(rgb_image.data, w, h, bytes_per_line, QtGui.QImage.Format_RGB888)
p = convert_to_Qt_format.scaled(500, 500, Qt.KeepAspectRatio)
return QPixmap.fromImage(p)

== The Source Files ==

Python interface

2020-12-02T18:48:51Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

As the Video stream will need to happen simultaneously to other robot actions, we pass the ''Qthread'' as an argument for this class, which will treat this function as a parallel thread to the main function.
Using the open-cv library we are able to capture a video stream and converting it to the Qt format in which we will be able to display the actual image to the ''Qlabel'' object.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-02T18:36:38Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

$ pip install opencv-python

import cv2

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-02T18:35:29Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

To implement the mjpeg-streamer also requires ''open cv'' library installed on your machine. This way the image can be displayed using the open-cv framework.

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True
def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') #
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()
def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-02T18:33:33Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

class VideoThread(QThread):
change_pixmap_signal = pyqtSignal(np.ndarray)
def __init__(self):
super().__init__()
self._run_flag = True

def run(self):
cap = cv2.VideoCapture('http://cleo.local:8080/?action=stream') # maybe replace with something else?
while self._run_flag:
ret, cv_img = cap.read()
if ret:
self.change_pixmap_signal.emit(cv_img) #cv_img
cap.release()

def stop(self):
self._run_flag = False
self.wait()

== The Source Files ==

Python interface

2020-12-02T18:29:23Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot

== The Source Files ==

Python interface

2020-12-02T18:28:45Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

To get a snapshot of the current stream use this:

http://'robotname'.local:8080/?action=snapshot
http://'robotname'.local:8080/?action=stream

== The Source Files ==

Python interface

2020-12-02T18:27:27Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

It is possible to also see the stream using the Web-browser using the host address. This is convinient way how to test if the streamer works correctly.

http://'robotname'.local:8080/?action=stream

== The Source Files ==

Python interface

2020-12-02T18:24:48Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

If the picture is reversed, use this function to display to run the stream:

$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -hf"

== The Source Files ==

Python interface

2020-12-02T18:23:53Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

Now everything is set to runt the streamer:

*If you are running from the root folder, remember to go to the folder ->
$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ ./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so"

== The Source Files ==

Python interface

2020-12-02T18:21:43Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

To download the project you will need a source control system called git. It may not be installed on a fresh image. I know it’s not on the lite image. So you may need to install it.
$ sudo apt-get install git

Now that you have git installed, use it to clone a copy of the mjpg-streamer to your Pi.

$ git clone https://github.com/jacksonliam/mjpg-streamer.git

After the cloned copy of the mjpeg-stream has been coppied to the raspberry-pi, follow these steps.

$ cd mjpg-streamer/
$ cd mjpg-streamer-experimental/
$ sudo apt-get install cmake
$ sudo apt-get install python-imaging
$ sudo apt-get install libjpeg-dev
$ make CMAKE_BUILD_TYPE=Debug
$ sudo make install

When all the necessary libraries have been installed, it is necessary to run the following script to make this setting permanent.

$ export LD_LIBRARY_PATH=.

== The Source Files ==

Python interface

2020-12-02T18:07:44Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System. *Note that you have to be connected to the network in order to update the software. In some cases, it might be successful, as the robot has not got internet access even though it should have been connected to the internet wirelessly - the best and reliable solution is to use an Ethernet cable and connect to the DTU network using some ports, that can be found around the Campus.
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

== The Source Files ==

Python interface

2020-12-02T18:04:39Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, that we are going to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

'''How to set up mjpeg streamer on Raspbery-Pi'''
Connect to your Raspberry-Pi
$ ssh local@IP(or robot name)

Update the Operating System
$ sudo apt-get update
Enable the camera and Launch raspi-config:

$ sudo raspi-config
From the menu options:

Select:
6. Enable Camera
Click: Yes
Click: OK
Click: Finish

== The Source Files ==

Python interface

2020-12-02T17:55:41Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.

The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
The designed Camera window has been designed as in the following picture.
The key tip designing the mjpeg-streamer in Qt-designer, is to use ''QLabel'' object for the actual frame where to display the actual live-video footage.

[[File:Camera.JPG | 600px]]

== The Source Files ==

Python interface

2020-12-02T17:51:56Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.
The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.

[[File:Camera.JPG | 600px]]

== The Source Files ==

Python interface

2020-12-02T17:51:47Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.
The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.

[[File:Camera.JPG | 900px]]

== The Source Files ==

Python interface

2020-12-02T17:50:23Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.
The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.

[[File:Camera.JPG]]

== The Source Files ==

File:Camera.JPG

2020-12-02T17:50:07Z

S192320:

Python interface

2020-12-02T17:50:02Z

S192320: /* Implementation of the Video Stream and Camera */

== Qt GUI interface design ==

The Regbot desktop application is designed using the Qt-Designer app which uses the widgets from the Qt GUI framework. The application gives a possibility to quickly build interfaces using the drag-and-drop feature for placing necessary components in your interface.
The Qt designer produces .ui files that can be translated to C++ or Python interface code.
This user interface was designed using Python 3.7.9 and Qt version 5.
''To some extent, it was possible to use Visual Studio, but I ran into some problems with finding the necessary libraries for some tasks I wanted to use (like cv2, keyboard and some PyQT libraries). A Safe choice was to use one Python development platform on your PC, for example, Spyder (as it was necessary to use this for other courses). ''

[[File:UI.JPG | 900px]]

The translated UI file will be autogenerated with all the parameters that were set in Qt-Designer app like - object name, size, and many other parameters that correspond to each specific object.
At first, to use Qt-Designer version 5 with python it is necessary to install the necessary python packages:
# Installing PyQt5 package
pip install pyqt5-tools
pip install pyqt5

After installing and designing your user-interface it is necessary to convert your user interface .ui file to a .py file, that can be run in your program.
The translated .ui files can be auto-generated using the following commands:
# Generating Python interface file from Qt-Designer .ui file
pyuic5 -x "filename".ui -o "filename".py

Note that this file is generated in the directory where the terminal was opened, so make sure that you know where this file is located after using this command.

In the picture below, you can see an example of an auto-generated python interface file, that can later be included in the rest of your application as a user interface.
By knowing the name of the object, the actual user interface now can also be tweaked by editing the Python code.

[[File:UIpy.JPG | 400px]]

If this auto-generated python file is run on its own, the user interface is executed as an output of this program.

[[File:gui.JPG | 700px]]

To add functionality to the object of the user interface (like buttons, checkboxes, text fields, etc.), we can attach a function to the interface object to be executed, for example, if the check box is checked, or button is pressed. This can be done by taking the user interface object in the python .py file and passing your function to it as an argument.

By running the python class script, the first function that is run is "__init__". In "__init__" usually we define all other python classes that need to be initialized and defined.
The following code is a template example of how to connect the user-interface object with the function in your program. The specific object-related function (clicked, released, pressed) varies from object to object.
# self."user-interface-file"."object-name"."function".connect(self."function name")

See the example below:

[[File:code1.JPG | 600px]]

== Programming GUI ==

As the whole Regbot firmware and bridge software is already programmed (see section "Navigation box software" on the main robobot page). And accessing the local@IP port 24001 we are able to receive and send byte-type messages to the network socket. Thus, by building a GUI, it is possible to control the robot remotely, read/test sensor values, read and write robot system parameters to its configuration.

The first thing that is needed to do is install the socket library and define/initialize is the network socket (internet connection) to the robobot bridge.

# import socket
class GUIMainWindow(QtWidgets.QMainWindow):
socket = socket.socket()

When that is done, we connect the actual "connect" button to the connectClient() function do all the necessary steps to establish a connection with the chosen IP address.
The Connect client functions
self.ui.connect_Cmd.clicked.connect(self.connectClient)

[[File:Connectclient.JPG]]

The connectClient() function implementation can be seen below. To make a stable socket connection, at first, it is required to get the address information parameters like - Adress Family, Socket Type and the Adress Info (IP and port). Then using the previously extracted parameters we define the actual network socket and connect with the IP adress using ''socket.socket.connect(IP)''. If the attempt to make a connection is faulted, it is necessary to close a connection using ''socket.close()'' function. In the code below some extra functions are implemented aswell, for example to change the frame color or text in the status label to indicate that the connection has successfully been made or not.

def connectClient(self):
if not self.wifiConnected:
IP = str('192.168.43.101') # Connect automatically to this address
#IP = str(self.ui.IP_Adress_field.text()) # Manual IP adress input
# Getting address info for the connection
for addressInfo in socket.getaddrinfo(IP, 24001, socket.AF_UNSPEC, socket.SOCK_STREAM):
print("Socket Info " + str(addressInfo)) # Printing the whole adress info
AddresFamily = addressInfo[0] # Extracting the Adress Family type
print("# Adress Family ", AddresFamily)
SocketType = addressInfo[1] # Extracting the Socket Type
print("# Socket Type " , SocketType)
IP = addressInfo[4] # both IP and port number
print("# IP adress and port ", IP)
try:
self.socket = socket.socket(AddresFamily, SocketType, 0) # Making a actual networ socket with necessary adress parameters.
self.socket.settimeout(5)
except OSError as msg:
print("# Network Connection timeout - retry")
try:
self.ConsoleMessage += "Connecting to Client..."
self.ConsoleMessagePush = True
print("Connecting to Client...")
self.socket.connect(IP) # Connecting to the adress
self.wifiConnected = True
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(0,255,0,255))
self.ui.IP_NetworkStatus_label.setText('Connected')
print("Connected")
self.ConsoleMessage += "Network is Connected\n"
self.ConsoleMessage += "Socket Info " + str(addressInfo) + "\n"
self.ConsoleMessagePush = True
except OSError as msg:
self.socket.close() # closing the connection if faulted
self.ui.IP_NetworkStatus_label.setText('Faulted')
'''To avoid the confusion, why the connection with the Raspberry Pi suddenly faulted, it is required to continuously send "alive" or heartbeat command"u5". The Pi is configured in a way, if it doesn't receive anything in the connection socket for couple of seconds, the connection is terminated.'''

In a similar fashion the ''Disconnect'' button in the GUI is connected with the ''disconnectClient()'' function. To correctly close down the connection, it is required to shut down the socket using ''socket.shutdow()'' and then close the connection using ''socket.close()''. As one can notice, other different functions are called in ''disconnectClient()'' function, which stops other features in the programm.

def disconnectClient(self):
if self.wifiConnected:
print("Disconnecting...")
self.ui.IP_NetworkStatus_label.setText('Disconnecting')
self.wifiConnected = False
self.video_StopRecording()
self.socket.shutdown(2) # Shutting down the socket
self.socket.close() # Closing down the connection
self.setConnectFrameColor(self.ui.IP_Connect_frame,QtGui.QColor(200,0,0,255))
self.ui.IP_NetworkStatus_label.setText('Disconnected')
print("Disconnected")
self.ConsoleMessage += "Network is Disconnected\n"
self.ConsoleMessagePush = True

The messages in the code to the socket can be written using the socket function ''socket.send("MESSAGE-TEXT")''. The most important thing that the string message has to be converted to a type byte to be able to successfully send the message to the socket.

def connectionWrite(self,MESSAGE):
MESSAGE = bytes(MESSAGE, 'utf-8')
if self.wifiConnected:
self.socket.send(MESSAGE)
if (self.wifiConnected):
if (self.ui.TX_checkBox.isChecked()):
self.ConsoleMessage += "(tx)->"+ str(MESSAGE) +'\n'
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "Network not connected, could not send message -" + str(MESSAGE)
pass

Here is visible the screenshot of the debugging message window in the GUI. Here using TX and RX checkboxes we can see what messages are sent to the connection and received back. As described before, the GUI button "Send" is connected with the text input field and function ''connectionWrite()''.
By pressing the Help button, the "help" message is sent to the socket, and then as the reply, we are receiving back the reply from the connection. This reply consists of a list of function messages that can be written in the socket in order to receive actual data from the robobot bridge.

[[File:Debugger.JPG]]

By sending the specific predifined command to the socket, the reply will be provided that later can be used as an actual value for editing the robot configuration, displaying the sensor values etc. However, the reply is sent in a form one string, so the decoding function is needed to be implemented to seperate the values. The list of these messages can be seen here:
(rx)-> # help for special second parameter:
(rx)-> # get Gets the value of the tata item
(rx)-> # meta Gets 'key meta r vs s p description': r=responder, vs: 0=val 1=seq, s=source, p=priority
(rx)-> # subscribe p Set subscription priority 0=none, 1=fast (10ms), 5 = slow (6sec), 6=all updates
(rx)-> # status Sends status 'key status c T n p p p p ...' c: update count, T: since last (sec), n=clients slots, p client priority
(rx)-> # name xxx Sets name or description for data item
(rx)-> # logopen Opens a (new) logfile and log all updates with timestamp (key.txt)
(rx)-> # logclose Closes logfile (if open)
(rx)-> # h This help
(rx)->
(rx)-> # NB! you now subscribed to all '#' messages
(rx)-> # (use 'robot silent' or '# subscribe 0' to undo)
(rx)-> # Robot connection has these subcommand options:
(rx)-> # clogopen open communication log for all traffic to and from Regbot
(rx)-> # clogclose close these logfiles
(rx)-> # help This help text
(rx)-> # Regbot communication open=1
(rx)-> # RegBot commands ($Id: command.cpp 1175 2020-01-04 15:40:53Z jcan $)
(rx)-> # M=N Select mission 0=user mission, 1.. = hard coded mission (M=0)
(rx)-> # i=1 Interactive: 0: GUI info, 1: use local echo of all commands, -1:no info (i=0)
(rx)-> # s=N A Log interval in milliseconds (s=5) and allow A=1 (is 1)
(rx)-> # lcl 0 0 0 0 0 0 0 0 0 Log control: vel,head,pos,edge,walldist,fwd-dist,bal,bal-vel,bal-pos (is 100000000)
(rx)-> # log start Start logging to 70kB RAM (is=0, logged 0/1458 lines)
(rx)-> # log get Transfer log to USB (active=0)
(rx)-> # motw m1 m2 Set motor PWM -1024..1024 (is=0 0)
(rx)-> # motv m1 m2 Set motor voltage -6.0 .. 6.0 (is=0.00 0.00)
(rx)-> # mote m1 m2 Set motor enable (left right) (is=0 0)
(rx)-> # u0..u8 Status: u0:ver,u1:measure,u2:mission, u3:log,u4:robot,u5:heartbeat,u6:mag,u8:motorVA
(rx)-> # u9..u14 Status: Line sensor u9=limit w, u10=limits B, u11=value, u12=raw, u13=pos and x, u14=ADC
(rx)-> # u15..u24 Status: 15=acc, 16=gyro, 17=gyro offs, 18=motor (A), 19=enc, 20=vel(m/s), 21=pos(m), 22=pose, 23=button, 24=voltage (V)
(rx)-> # u25..u31 Status: 25,26 encoder calibrate raw values (cpu clock units), u27,28 calibrate factors, u29 enc status, u30 line sensor, u31 line sensor gain
(rx)-> # v0..2 Status: v0:IR sensor data, v1:wifi status, v2 wifi clients
(rx)-> # xID Get Control params, ID: cvel=wheel vel, ctrn=turn, cwve=wall vel, cwth=wall turn, cpos=position,
(rx)-> # cedg=edge, cbal=balance, cbav=bal vel, ctrl=rate limit
(rx)-> # ID x x ... Set controler parameters (ID use Kp ... see format below)
(rx)-> # format: ID use kp iuse itau ilim Lfuse LfNum LfDen Lbuse LbNum LbDen preUse preNum preDen
(rx)-> # preIuse preItau preIlim ffUse ffKp ffFuse ffFnum ffFden LimUse Lim
(rx)-> # rid=d d d d d d d d d d Robot ID set: ID wheelBase gear PPR RadLeft RadRight balanceOffset flags batt_low HW_version
(rx)-> # eew Save configuration to EE-Prom
(rx)-> # eeW Get configuration as string
(rx)-> # eer Read configuration from EE-Prom
(rx)-> # eeR=X Read config and mission from hard coded set X=0: empty, X=1 follow wall
(rx)-> # sub s p m Subscribe s=1, unsubscribe s=0, p=priority 0 (high) .. 4(low), m=msg:0=hbt, 1=pose,2=IR,3=edge,4=acc,5=gyro,6=current,7=vel,8=mission
(rx)-> # sut t p msg interval for priority, t=time in ms 1.., p=priority 0..4 (is 0=1 1=5 2=25 3=125 4=625)
(rx)-> # posec Reset pose and position
(rx)-> # gyroo Make gyro offset calibration
(rx)-> # mem Some memory usage info
(rx)-> # start start mission now
(rx)-> # stop terminate mission now
(rx)-> # rc=A V T [B] Remote control A=0/1 (is 0), V=m/s (is 0), T=vel difference (is 0), B=1 try keep balance
(rx)-> # <add user-mission-line> add a user mission line (3 lines loaded in 1 threads)
(rx)-> # <mod T L user-mission-line Modify line L in thread T to new user-mission-line
(rx)-> # <clear> Clear all user mission lines
(rx)-> # <get> Get all user mission lines
(rx)-> # <event=X> Make an event number X (from 0 to 31)
(rx)-> # <token> Get all user mission lines as tokens
(rx)-> # :xxxx Send data (AT commands) to wifi board (all except the ':') \r\n is added
(rx)-> # link=L,data Send data to socket link L
(rx)-> # wifi use port SSID PW Wifi setup (e.g. 'wifi 1 24001 "device" ""')
(rx)-> # wifi e/n Echo all received from 8266 to USB link
(rx)-> # halt Turn 12V power off (on by halt=0) (is 0)
(rx)-> # alive Alive command - reply: <alive last="0.0xxx"/>
(rx)-> # iron 1 IR sensor on (1 = on, 0 = off) is=0
(rx)-> # irc 2 8 2 8 u i IR set 20cm 80cm 20cm 80cm on installed
(rx)-> # servo N P V Set servo N=1..3 (4,5) P (position):-512..+512 (>1024 = disable), V (velocity): 0=full, 1..10 P-values/per ms
(rx)-> # (status: (servo 1,2,3) enabled 0,0,0, P= 0, 0, 0, V= 2, 2, 2)
(rx)-> # servo -1 frw Set PWM frequency
(rx)-> # svo Get servo status (same format as below)
(rx)-> # svo s1 p1 v1 s2 p2 v2 s3 p3 v3 s4 p4 v4 s5 p5 v5 Set servo status sx=enable px=position (-1000..1000), vx velocity 0 (full) 1..10 (slower)
(rx)-> # encc, enci Encoder calibrate: enc: calibrate now, eni: Find calibration index now
(rx)-> # eneX=1 enec=1/0 enable timing collect, eneu=1/0 use calibration, enea=1/0 auto calibrate
(rx)-> # silent=1 Should USB be silent, if no communication (1=auto silent)
(rx)-> # help This help text.
(rx)-> # ADC use=1, seq=0, resetcnt=0, reset

== Receiving, Decoding and Translating received messages ==

When the correct message is sent, and the bridge has understood the message, it provides a reply. The reply usually is in the form as show in the following string - "keyword parameter1 parameter2 parameter3 ... parameter_n".
For example, sending the message "u5", we receive :
''(rx)-> hbt 6542.6 11.5319 0 0 0 221''
where [0] element - is the keyword
[1] element - regbot time since startup
[2] element - battery voltage
[3] element - control active flag
[4] element - mission state
[5] element - remote control flag
[6] element - cpu load

The translation and the meaning of each element can be found in the previous "help" section.

It is possible to get a continuous stream of data of the selected parameter of by sending a subscription command - "keyword subscribe x", where x is the priority of the subscribtion stream.
For example sending a subscribrion command to subscribe to linesensor values:
liv subscribe 6
(rx)-> liv 3035 3279 3067 3178 3322 2683 3045 2761
*RAW values of the line sensor

To decode the received message, it is necessary to create a function that separates the values of the received string.
Using the readReply() function, firstly make sure that we have a valid, stable connection and then with ''socket.recv(1).decode()'' we read the reply from the socket chatacter-by-character and putting them all together to get a string message. When the full message has been stacked together, we pass the "message" in the function decodeCommand() which will seperate the necessary values.
def readReply(self):
c = '\0'
self.socket.settimeout(1)
while (not self.stop.is_set()):
if self.wifiConnected:
n = 0
if (c == '\n'):
MESSAGE = ""
c = '\0'
try:
while (c != '\n' and self.wifiConnected):
c = self.socket.recv(1).decode()
if (len(c) > 0):
if (c >= ' ' or c == '\n'):
MESSAGE = MESSAGE + str(c)
n = len(MESSAGE)
except:
time.sleep(0.01)
if (n > 0):
self.decodeCommand(MESSAGE, n)
else:
time.sleep(0.1)
pass

The "message" then is passed around, until it reaches the function that recognizes the initial keyword. *The Function also is printed on the GUI message terminal.
For example, if the decodeCommand() function recognizes that the message belongs to linesensor.readData(), the message is treated further in this function.

def decodeCommand(self, MESSAGE, n):
if n > 0:
if (self.ui.RX_checkBox.isChecked()):
self.ConsoleMessage += "(rx)-> " + str(MESSAGE)
self.ConsoleMessagePush = True
if n > 1:
gg = MESSAGE.split()

if gg[0] == "hbt":
print(gg, 'hbt')
elif self.info.readData(gg):
pass
elif self.lineSensor.readData(gg): <------------------
pass
elif self.servo.readData(gg):
pass
if self.irdist.readData(gg):
pass

As we received the message that belongs to linesensor.readData() function, we are able to split the array of string parameters and assign them to our own variables, that we can pass to our GUI objects.

# readData function from linesensor class
def readData(self, MESSAGE):
used = True
try:
if MESSAGE[0] == "liv":
self.dataReadLiv = True
if self.ui.ls_show_normalized.isChecked():
maxraw = self.ui.line_disp_max_value.value()
self.lineValue[0] = maxraw * (int(MESSAGE[1],0) - self.lineMaxBlack[0])/(self.lineMaxWhite[0] - self.lineMaxBlack[0])
self.lineValue[1] = maxraw * (int(MESSAGE[2],0) - self.lineMaxBlack[1])/(self.lineMaxWhite[1] - self.lineMaxBlack[1])
self.lineValue[2] = maxraw * (int(MESSAGE[3],0) - self.lineMaxBlack[2])/(self.lineMaxWhite[2] - self.lineMaxBlack[2])
self.lineValue[3] = maxraw * (int(MESSAGE[4],0) - self.lineMaxBlack[3])/(self.lineMaxWhite[3] - self.lineMaxBlack[3])
self.lineValue[4] = maxraw * (int(MESSAGE[5],0) - self.lineMaxBlack[4])/(self.lineMaxWhite[4] - self.lineMaxBlack[4])
self.lineValue[5] = maxraw * (int(MESSAGE[6],0) - self.lineMaxBlack[5])/(self.lineMaxWhite[5] - self.lineMaxBlack[5])
self.lineValue[6] = maxraw * (int(MESSAGE[7],0) - self.lineMaxBlack[6])/(self.lineMaxWhite[6] - self.lineMaxBlack[6])
self.lineValue[7] = maxraw * (int(MESSAGE[8],0) - self.lineMaxBlack[7])/(self.lineMaxWhite[7] - self.lineMaxBlack[7])
else:
self.lineValue[0] = int(MESSAGE[1],0)
self.lineValue[1] = int(MESSAGE[2],0)
self.lineValue[2] = int(MESSAGE[3],0)
self.lineValue[3] = int(MESSAGE[4],0)
self.lineValue[4] = int(MESSAGE[5],0)
self.lineValue[5] = int(MESSAGE[6],0)
self.lineValue[6] = int(MESSAGE[7],0)
self.lineValue[7] = int(MESSAGE[8],0)

These values can be stored and assigned to line sensor GUI object element for displaying its value.
By subscribing to different parameters like - liv,lib lip, we can get the calibrated values and configuration parameters for edge detection, crossing detection etc.

[[File:Linesensor.JPG]]

== Manual Robot Control from the GUI ==

Just like it is possible to start the robot mission and control it remotely from the game-controller, it is possible to manually control the motor movement remotely from the user-interface or computer keyboard.
As it can be seen in the picture below, the two checkboxes correspond to each mode - GUI control, where pressing the arrow buttons in the user-interface, will control the robot with the speed specified in the text-field.

Similarly, the robot can be controlled using the computer keyboard arrows, with ''keyboard control'' enabled and manual speed specified in the manual speed text-field. *If the manual speed is not specified, the remote control is not possible.

[[File:Rc.JPG]]

The function that controls the remote manual control from the computer keyboard is as follows. This requires to installation and import python ''keyboard'' library.
The basic principle of controlling the robot is as discussed before - by sending the "rc = enable fwd_speed turn_speed" string to the IP address port 24001 connection.
>>>pip install keyboard

import keyboard
.
.
.
def keyboardRC_enable(self):
self.keyboardRC_enabled = not self.keyboardRC_enabled
if self.keyboardRC_enabled:
self.ConsoleMessage += "*Keyboard Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*Keyboard Remote control disabled\n"
self.ConsoleMessagePush = True
self.keyboardRC()
.
.
def keyboardRC(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.keyboardRC_enabled and speed != "" and float(speed):
if keyboard.is_pressed('up arrow'): # if key 'up' is pressed
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('down arrow'): # if key 'down' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('left arrow'): # if key 'left' is pressed
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('right arrow'): # if key 'right' is pressed
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)
elif keyboard.is_pressed('esc'): # if key 'esc' is pressed
self.ui.enableKeyboardRC_checkBox.setChecked(False)
self.keyboardRC_enable()
else:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

For the robot to follow the input from the user as the action has been done without huge time delays, it is necessary for some conditions to be checked periodically and continuously. That's why the ''timer'' is used.
The timer is set to start counting and until the defined value in milliseconds has been reached and the timeout signal has been provided which runs the assigned functions periodically.

self.timer = QtCore.QTimer(self)
self.timer.start(50) # Start's the timer with 50 ms interval
self.timer.timeout.connect(self.keyboardRC) # After each timout, the function is executed.
self.timer.timeout.connect(self.timerUpdate)

The functions for remote control using GUI, is developed in similar fashion, by sending the manual control string message to the IP port connection. The code can be seen below.

# Assigning the button the function for remote control.
self.ui.guiRC_Forward_Cmd.pressed.connect(self.guiRC_fwd)
self.ui.guiRC_Forward_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Reverse_Cmd.pressed.connect(self.guiRC_rev)
self.ui.guiRC_Reverse_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Right_Cmd.pressed.connect(self.guiRC_right)
self.ui.guiRC_Right_Cmd.released.connect(self.guiRC_idle)
self.ui.guiRC_Left_Cmd.pressed.connect(self.guiRC_left)
self.ui.guiRC_Left_Cmd.released.connect(self.guiRC_idle)

def guiRC_enable(self):
self.guiRC_enabled = not self.guiRC_enabled
if self.guiRC_enabled:
self.ConsoleMessage += "*GUI Remote control enabled\n"
self.ConsoleMessagePush = True
else:
self.ConsoleMessage += "*GUI Remote control disabled\n"
self.ConsoleMessagePush = True

def guiRC_fwd(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 '+speed+' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_rev(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 '+ str(rev) +' 0\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_right(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
rev = float(speed)*(-1)
MESSAGE = bytes('rc=1 0 '+ str(rev) +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_left(self):
speed = self.ui.manualGuiSpeed_Cmd.text()
if self.guiRC_enabled and speed != "" and float(speed):
MESSAGE = bytes('rc=1 0 '+ speed +'\n', 'utf-8')
self.socket.send(MESSAGE)

def guiRC_idle(self):
if self.guiRC_enabled:
MESSAGE = b'rc=1 0 0\n'
self.socket.send(MESSAGE)

== Implementation of the Video Stream and Camera ==

In this section, it is going to be described how to implement the video stream in your python Qt Gui application using mjpeg-streamer.
The first thing to do is to design the layout for your mjpeg-streamer. The mjpg-streamer is a command-line application that copies JPEG frames from one or more input plugins to multiple output plugins.
[[File:Camera.JPG]]

== The Source Files ==