Tracking Algorithm
Overview
The camera program is written in C++. It is separated into two sections, the main processing and debugging. These two sections can be found in camera.cpp and debug.cpp respectively. Calibration values can be found in config.txt which can be edited without recompiling the program.
Our program will launch on startup and enter either competition or debug mode upon selection by the user. The code behind the tracking program will be explained in this article while debugging will be covered under GUI (next article).
Tracking
Considerations
Speed and latency as well as accuracy and reliability were some considerations while we coded our tracking program.
Speed & Latency
We wanted a program which could track objects above 60 FPS with minimal latency. Since our robot is moving at high speeds throughout the game, it is important to have high refresh rates so that the movement of the robot will be fluid. It is also essential for the robot to have minimal latency to avoid low responsiveness and overshooting the ball while tracking it. From past year’s experience, our robot would overshoot while moving towards the ball which could be explained by the low FPS and high latency of the program as we were using python back then.
Accuracy & Reliability
We also wanted a program which can track the objects on the field accurately while doing so reliably. Given that we are using an omnidirectional mirror, the images captured will be distorted and minimised. Furthermore, our mirror is not completely smooth, thus the image will be slightly blurred. To compensate for these problems arising from the choice of an omnidirectional mirror, we had to create a program which has high accuracy and reliability at detecting objects. Moreover, we had to discern between the objects in the field and unwanted objects outside of the field accurately to prevent detecting false positives.
Algorithms
We used several algorithms and techniques to fulfill the considerations we had.
Circular buffer
To avoid race conditions where one thread is reading the image while another is writing into the same image array, we implemented the circular buffer. With the circular buffer, each thread is not hindered by another thread while trying to access information. As such, race conditions can be prevented and our program is more efficient.
Image Processing
Kalman Filter
In order to provide an accurate prediction of the ball in situations where it is obscured by other elements on the field, we employed the kalman filter. With the kalman filter, we can have a better idea of where the ball was based on the trajectory of the ball. However, due to the non-linear nature of the image from our curved mirror, we mapped the ball from the image to real world coordinates before applying the kalman filter. This way, the kalman filter is applied effectively.
Threading
Since we are tracking multiple different objects on the field, ball, goals, area of the field, we utilised threading to improve the performance of our program.
Precautions
The most common issue with threading is race condition where a variable is accessed while it is being edited, corrupting the data. Hence, we took extra precautions in our program to prevent this fatal error. Through the use of the atomic variables and locks, we created a program free of race conditions. Another precaution we took was the possibility of locks decreasing the performance of our program. We minimise the impacts of locks on our program by implementing the circular buffer as described above.
Dynamic priority
Keeping in mind not all the elements need to be tracked at the highest frame rate possible all the time, we effectively allocate the available computing resources with dynamic priority implementation. Priorities for different objects can be controlled by the main microcontroller. For example, if the robot is not in possession of the ball, it will run the ball tracking thread at the highest possible performance while other elements like the field and the goals can be placed at a lower priority.