When displaying video back to the user, many robots and drones suffer from shaky camera footage. Often, gimbals take up too much room or power. Sundance looked for a solution to stabilise their footage on their robotics platform.
We proposed using our A-IoT edge solution to build real-time digital image stabilisation on their robotics platform. The platform could then communicate with the Cloud to allow users to monitor data from a remote control panel.
By using ClickCV hardware-accelerated modules, we were able to achieve real-time video stabilisation at 1080p 30 FPS, with minimal latency. Our A-IoT Edge solution also enables real-time monitoring of the video stabilisation to ensure a consistent, smooth video is maintained.
“Beetlebox met two critical criteria for the edge. Firstly, it enabled heavy amounts of computation to run at lower power than conventional CPUs. It also ran the video stabilisation at a consistent, low latency, something that GPUs have traditionally struggled with.”
Sundance Microprocessor Technologies specialise in the production of Commercial-Off-The-Shelf boards for robotics and visual applications, many of which are powered through FPGAs.
Often, the cameras they deploy on robots and drones suffer from shaky footage caused by uneven terrain, high winds or motors. Gimbals are prohibitively large or consume too much power, and the cameras that Sundance uses have no built-in stabilisation features.
Partnering with us, Sundance wanted to develop a video stabilisation system that would run entirely on the boards. The technical challenge was to analyse high definition footage at 30 FPS without the user noticing any latency, whilst simultaneously sending information to and from the Cloud.
Our A-IoT Edge solution was both able to provide the high-performance video analytics and the data management to successfully communicate with the Cloud. We developed real-time electronic video stabilisation using ClickCV modules and achieved 1080p at 30 FPS, whilst minimising delay on the video.
To perform video stabilisation, we first got an estimate of the movement of the camera, which we did by tracking static features in the video. For instance, we could detect movement of the mountain in the background from one frame to the next. Since mountains remain stationary, we knew any movement must have been caused by the camera shaking. We could then map these shakes onto a graph, plotting its movement.
The final step was to remove the shakes and create a smooth version of the movements in real-time. To ensure that the video remained smooth, we provided camera movement data to the Cloud for monitoring. In this case, we provided the monitoring by connecting the device using AWS IoT Greengrass, which then sent data to the AWS cloud. The stabilisation could then be measured without needing to extract video from the device itself, significantly saving on networking and cloud processing costs.
The Beetlebox A-IoT Edge solution was not only used to communicate with the Cloud, but also managed the entire development flow of the video stabilisation system.
Our solution provided pre-set hardware and platforms, allowing us to concentrate on the software development of the video stabilisation system. Our A-IoT edge solution allowed us to implement Continuous Integration (CI), simplifying code and functional coverage tests, which enabled faster iteration, better team collaboration and reduced project risk.
Through the Beetlebox A-IoT edge solution, we were able to build, test and monitor real-time video stabilisation running in high definition at 30 FPS. The next step for our partnership with Sundance will be connecting their robots to the Cloud, providing customers with advanced monitoring. Using our A-IoT edge solution, we will provide the same rapid iteration and reduced risk to the deployment of these robots.
1-3 Brixton Road,
Beetlebox Limited is a
company registered in
England & Wales with
Company Number 11215854
and VAT no. GB328268288
2020 Beetlebox Limited