Welcome to part 4! Over at Beetlebox we are excited for the release of Vitis, which is the unification of all of Xilinx’s previous software into a single package. We have been working hard on computer vision using this platform and thought that we could provide some help to others wanting to get started on Xilinx’s development boards. This tutorial will be a multi-part series covering the basics of getting started with computer vision and Vitis and will be covering:
We hope these tutorials will be useful for anyone looking to get into computer vision on FPGAs.
Vitis Vision Library is an accelerated Computer Vision library based on OpenCV and was formerly known as xfOpenCV that worked on the old Xilinx reVision platform. Xilinx is now making a push to provide accelerated versions of standard libraries, so that any software developer familiar with them can benefit from using FPGAs. Today we are going to look at getting started with the Vitis Vision library using the examples that Xilinx provides and running them in the Vitis IDE.
Unfortunatly as of 2019.2 the Vitis Vision library is not included in the Vitis IDE as standard, so we will need to put in some effort for it to compile and run properly.
Instead of including the entire library we are going to look at including an example design from Vitis Vision Library, which will be FAST Corner Detection. FAST is a feature detector that looks for corners in images and is commonly used in application such as image stitching, motion tracking and SLAM.
Our particular system:
petalinux-build petalinux-config -c rootfs
cd images/linux petalinux-build --sdk ./sdk.sh
git clone --branch 2019.2 https://github.com/Xilinx/Vitis_Libraries.git
ERROR: Cannot open image
INFO: Running OpenCL section. Found Platform Platform Name: Xilinx INFO: Device found - xilinx:pcie-hw-em:7v3:1.0 XCLBIN File Name: krnl_fast INFO: Importing ./binary_container_1.xclbin Loading: './binary_container_1.xclbin' ocvpoints:1189= INFO: Verification results: Common = 1189 Success = 100 Loss = 0 Gain = 0
connect -url tcp:localhost:1440
tfile copy -to-host /mnt/output_hls.png <host_directory>/ output_hls.png
ERROR: Cannot Open Image
INFO: Running OpenCL section. Found Platform Platform Name: Xilinx INFO: Device found - zcu104_base XCLBIN File Name: krnl_fast INFO: Importing ./binary_container_1.xclbin Loading: './binary_container_1.xclbin' ocvpoints:1189= INFO: Verification results: Common = 1189 Success = 100 Loss = 0 Gain = 0
In this tutorial, we have managed to run a hardware accelerated kernel in the Vitis Vision library. These kernels will form the low level basis of any Computer Vision System we wish to develop. Looking at these kernels can also effectivly teach us how to to use Vivado HLS and develop our own vision modules to be accelerated on the FPGA, which we will be looking at next time!
Previous: Using OpenCV and File Transfers
Next: Accelerating Computer Vision using XRT and Kernels
If you have enjoyed this tutorial but are in current need of talent to build advanced Computer Vision systems on FPGAs, consider joining our ClickCV Early Access programme. ClickCV Early Access provides bespoke service and support for developing advanced Computer Vision systems on FPGAs. We use our own proprietary Computer Vision Library, ClickCV, to provide our clients the cutting edge in low latency, high definition processing. Contact us today to find out how we could build your next-generation system.
About the Author: Andrew Swirski is the founder and managing director of Beetlebox, a Computer Vision Acceleration specialist. The company develops the Computer Vision Acceleration library called ClickCV, which is designed to fully utilise the hardware adaptability and performance of FPGA chips. Beetlebox is currently running an Early Access programme, where the company provides bespoke service and support to develop client’s Computer Vision systems on FPGAs. Before Beetlebox, Andrew Swirski previously worked at Intel (formerly Altera) in FPGA encoding and decoding. He completed a Masters’ in Electrical and Electronic Engineering from Imperial College London in 2017.