General Information

Camera Microprocessor

Raspberry Pi (RPi) 3B

This model of RPi is the choice of our SBC for the camera system of our robot, because of its low price and high computational power. Furthermore, it has a well established community online which has been helpful when we faced issues. Aside from RPi 3B, we have also considered other alternatives.

Alternatives considered

Raspberry Pi 4B

The newer model of RPi was released in June 2019. By then, we had purchased a few RPi 3Bs already, thus it will be a waste to buy new RPi 4Bs. Furthermore, since the RPi 4B had just been released, there might be hardware or software issues. We did not want to take the risk of running into such problems, so we took the safe route of sticking with RPi 3B.

Other SBCs

The $99 price tag of a Jetson Nano is around 3 times that of a RPi. This high price is similar in other SBCs like the ODroid or Udoo lineup. Furthermore, we were unsure of their stability and ease of use due to a relatively small community and support available.

We did purchase an Asus Tinkerboard to experiment with it since we already had RPi cameras which could theoretically work with this board. However, we could not activate the GPU to process the images from the camera and tune values like white balance, giving us unusable images. This was made worse by the fact that there was little online support and documentation for this board.

While these SBCs could potentially have greater performance benefits, we chose settle on RPi which was “good enough” so that we can invest more time in other parts of the robot.

RPi Compute Module

While the compute module could potentially make way for a more compact design of our robot, we did not use it due to the unreliability of our PCB during the time of consideration (when we first started designing PCBs). Also, similar to the RPi 4B, we did not want to put the existing RPi 3Bs in our possession to waste so we did not convert to compute modules. Lastly, we thought that with the Compute Module 4 in the horizon, maybe we could purchase one then rather than a Compute Module 3 now for better potential performance.

OpenMV

OpenMV was an enticing product for us as it had everything in place, from a client-side interface to the algorithms itself. However, we decided not to use it as it was extremely restrictive in terms of the algorithms. Furthermore, in the aspect of performance, OpenMV is slower than the RPi.

Camera

RPi Camera V1

Currently, we are using RPi camera V1 which is able to capture frames at 640x480p and 90 FPS. Due to the inferior image sensor (OV5647) compared to its successor, we had to tune various camera parameters such as white balance, saturation, etc. to exaggerate certain colours on the field to compensate for the lower quality images.

Alternatives considered

RPi Camera V2

While this model boasts a better image sensor (IMX219), the FPS cap for full FOV streaming was a dealbreaker (45FPS). We tried to circumvent this problem by utilising the raspiraw library, but we faced some issues and abandoned this plan eventually as it was taking longer than expected. We found an alternative solution to the issue of bad images by simply adjusting the parameters of the V1 camera.

ArduCam

We are aware of alternative MIPI cameras that could interface with the RPi such as the offerings from ArduCam. However, from our research they could only utilise 1-lane MIPI, and we were afraid that due to limited support we could face compatibility or bottleneck issues.

Operating System (OS)

Arch Linux

With Arch Linux, the start up time of our RPi is around 12 seconds and on idle, the CPU and RAM utilisation is close to 0%. To achieve that we only installed essential components and run the RPi headless.

This was how we set up our RPi with Arch Linux

1. Write Arch Linux Arm for Raspberry Pi 3 image into an SD card
2. Set-up autologin
3. Install a desktop environment (preferably a lightweight one (LXDE) due to the limited RAM of the RPi) (Optional)
4. Obtain SSH public keys on both devices (laptop and RPi)
5. Set up a DHCP server on the RPi (Allow RPi to assign the laptop a static IP address)
6. Install OpenCV with optimization (ARM NEON and VFPV3 enabled)
7. Also install the following packages: libssh and raspicam (modified)

Alternatives considered

Raspbian (Lite)

Initially when we first started using RPi, we used the Raspbian OS (now Raspberry Pi OS). However, we thought the startup time was too long for us (>30s) and even at idle there were many background processes running, thus draining the limited computational resources.

Other OSes

While we are aware that a number of other lightweight OSes exist (e.g. DietPi), we started by trying Arch Linux because we wanted to eventually create a read-only FS that cannot be corrupted from sudden power losses. We came across this guide and decided to try it out. Since Arch Linux worked and we did not experience any performance issues, we adopted it as our preferred OS.

Buildroot / Bare Metal

We were very interested to cut down the boot time of our OS further by stripping unneeded programs / processes using Builtroot or by simply going bare metal and running our programs without an OS. However these seemed very difficult with little support online, and we thought we could spend more time on other aspects of our robot.

Backing Up

Given the possibility of our SD card being corrupted, we would regularly backup an image of the whole OS. Furthermore, we can revert any changes made easily if we accidentally break the system. However, since we are using a 32GB SD card, the image of the OS will be 32GB even though the actual space used is less than 10GB. Therefore, the task of backing up was slow and a waste of space. We found out about pishrink which shrinks pi images such that it doesn’t include unused space in the image. With this, we were able to effectively backup the raspi.

Programming Language

All of our code is written in C++ as it is more suited for our use of running programs in realtime. In previous years, we used Python as most tutorials online use it. However, after uncovering problems such as slower computational speed and the Global Interpreter Lock, we made a switch to C++.

Libraries Used

These are some libraries that we used in the development of our camera program and GUI.

“OpenCv”

OpenCV was used for image transformation and processing as it contains a wide range of well documented computer vision algorithms. We built the OpenCV with all available optimisations for RPi. With these optimisations, most functions are 4 times faster.

“LibSSH” (modified)

LibSSH was used to access the files in the raspi from the laptop. It played an essential part in the automation of sending files, running programs and running commands on the terminal of the RPi in our GUI.

“Threading”

Task parallelisation was achieved with this library. We used atomic variables and conditions to control and synchronise multiple threads. Threading in C++ allowed us to achieve true task parallelisation instead of rapid task switching in Python.

“QT5”

Our GUI was created using this library - from the design to functions.