68% of FPGA projects are completed behind schedule with 51% of project time dominated by verification. Yet many FPGA engineers resign themselves to running every project the same way. They manually run through all the steps, every time they want to test the system. When there is a lot of time pressure, running over these steps repeatedly is a sure way of derailing any schedule. So here at Beetlebox, we decided to put together a little experiment. We are going to show how to have a 43% time save by integrating CI/CD into the FPGA design process.
Setting up the CI/CD experiment
We are going to run a large FPGA-based, high level synthesis project called colour detect. This project is made of three separate smaller modules: dilate, erode and cvtcolour. All three submodules will be separately built and tested, before the entire system is put together and tested. We build and test everything three times. The first is a software emulation build and test that will run our code as a simple C program. The second is a hardware emulation build and test that can emulate the actual hardware logic. Finally we build the hardware itself to run on-chip. For simplicity, we will not build a test after the on-chip build.
In total, we have nine build and six tests on three sub-modules and then three builds and two tests for the main top module. Now we will perform these builds and tests with and without BeetleboxCI. Without BeetleboxCI, an engineer will need to manually run through every build and test individually. On top of this, most engineers are stuck using their work computer meaning they can’t run more than one build and test at a time. With BeetleboxCI, an engineer needs to create a configuration file. After that though, the entire build and test process is managed by the server.
Results
Our results show that without using BeetleboxCI, an engineer takes 281 minutes to complete the entire build and test process whilst using BeetleboxCI, an engineer will take 159 minutes. This provides a speed up of 43%.
Why CI/CD is so powerful for FPGA design and test
Using CI/CD can provide significant saving to an engineer’s time because of the ability to run their work in parallel. When an engineer manually runs through the entire process, they are not adding value to the project. FPGA engineers should be either exploring new designs or building more comprehensive tests. They should not be stuck waiting for their computer to be available again. Many engineers resort to building their own scripts, but these are still often linear, running a single task one after another.
What makes CI/CD so powerful is the ability to take these stages that run for multiple hours and run them in parallel. BeetleboxCI can take this one step further and run these stages on a separate server, meaning that the engineer’s computer will not even be locked during the process.
Let us takes this example and crudely scale it up over a one-year project for a team of ten engineers. Each engineer spends 51% of their time on testing, which is 122 days of their project per developer. Across the team that is 1220 workdays per year. Using this method, we can save 43% of that time, so testing time can be cut down to 695 workdays per year.
Conclusion
CI/CD could help reduce the time an engineer takes building and testing by up to 41% by ensuring their tests run in parallel. We have demonstrated this through a simple colour detection system. This system comprised of several builds and tests. By lowering time taken, we can enable engineers to explore more designs and build more comprehensive tests. This ultimately ensuring that project requirements are met on time.
If you would like to try BeetleboxCI, contact our sales team here.
