How do you know the product you are building is going to work in the real world? How do you as an engineer ensure that the software you are building for an electronic device will work reliably in the real world, on actual hardware? Well the answer to that is today’s topic with hardware in the loop testing.
In this article, we will explain Hardware in the loop or HIL testing, looking at why we need it, how it works, the benefits as well as the challenges it brings. We will look at few real-world applications and then finally mention our own work within the sector in terms of how hardware in the loop testing fits into continuous integration and continuous testing.
What is hardware in the loop (HIL) testing?
Hardware in the loop testing is a technique where an embedded system is placed into a testing process that simulates the operation of the product under real world conditions. It allows interaction between the embedded system and real hardware components or high-fidelity simulators. Simulators can replace real hardware with simulated components to test control algorithms in real-time.
It was developed as a response to the increasing complexity of our electronic products. In the past, when a device only had to perform one or two functions, you didn’t need individual testing rigs to check specific systems. You would build a complete prototype and then give it to either your QA team or to an early customer and have them test the product to see where the problems were.
Now consider the modern complexity of the average car and how difficult it is to produce even a single prototype. A lot of new features in modern cars are now delivered through software. If you need to wait for a prototype car to be fully assembled to test a new feature, it would make software development infeasibly slow and expensive. That doesn’t even account for if there are multiple teams who may all want to be testing their new features on the same car.
We don’t want to be testing through field trials alone, so why don’t we just test though software simulations? You can run hundreds of tests easily that way. At Beetlebox, we always recommend using software simulations as a first step, but hardware in the loop testing just provides that more realistic interaction with physical hardware that you don’t get when using a simulation.
It is very easy for a software simulation to overlook certain hardware constraints like signal integrity, electrical noise, thermal effects or even processing limitations. Combining software simulation with hardware in the loop provides a really effective testing combination that we can look at later.
How does hardware in the loop testing work?
As an example, let’s talk about that average modern car. Modern cars are great candidates for hardware in the loop testing because of just how impractical manual tests are for them. It’s very time consuming to need to load software into an actual vehicle and drive around a test track every time you make a small change to the code.
Specific functions in a car are controlled by small devices in the vehicle’s body called Electronic Control Units or ECUs and a vehicle may have greater than 100 of these ECUs. Each ECU contains a dedicated chip that has both its own software and firmware loaded on. These ECUs then receive real time information from sensors in the vehicle like engine timing or vehicle speed. They then process this information and send output signals to the various components in the car to perform physical actions.
In this hardware in the loop setup, the ECU responsible for blind spot detection is connected to simulators for radar and video sensors, receiving simulated vehicle speed data. The ECU processes this simulated input to determine if an object is near the vehicle, classify it as a car, and decide whether to alert the user based on speed. The success of the test is confirmed if the simulated dashboard receives the alert signal.
The testbench would work by mimicking the real-world conditions. For instance, we could run a test by having a physical object next to the sensor, then feeding a video to the camera of a close by car and finally telling the ECU via the data link that the car is travelling at 50 miles per hour. The test is then a success if it picks up that the ECU has sent a signal to alert the simulated dashboard.
Benefits of hardware in the loop Testing
This test system brings several advantages to the manufacturer. The first is that it detects errors early in the development process where they are far cheaper to fix rather than late in the development process where they are more costly.
On top of this it also allows engineers to get data on tests that would be too dangerous to do on a prototype. For instance, how does the system react when a car is travelling at 200 miles per hour.
It also allows far more rapid prototyping of new features as different engineering teams won’t all be tripping over themselves to get all the new features onto a single prototype.
It also increases the safety and reliability of testing more than just having software simulations alone. In fact, in our blind spot detection example, it is imperative to lower the amount of false positives, for instance when the radar detects something but it isn’t a car. If there are too many, it trains the driver to ignore the warnings and the safety feature becomes useless. By using this hardware in the loop system, engineers can run data analytics to estimate the number of false positives and see how adjustments can reduce them.
Challenges
As you have probably already worked by now though, there are problems to this method. Setting up hardware in the loop testing as sophisticated as we talked about can be time consuming and complex. You’ve got to build a system that can create real world data that is effective enough to actually test the system. That was for a single system as well and a car might have a hundred of these ECUs. Can manufacturers afford the time needed to setup 100 of these?
There are also significant challenges in simulating real world data. How do you create a controllable variable for things like temperature or vibrations?
Also setting up these tests is in itself a technical challenge. You need the expertise to script these tests, so you are looking at involving embedded software engineers, DevOps engineers and quality assurance engineers. Many companies may need to go on a recruitment drive to get the talent.
Another challenge is once you have built your testing rig, it is not enough to simply test your code once. It must be continuously tested any time a change is made, so developers need a system that can continuously take their code changes and run it through. Here at Beetlebox we have developed our own platform to solve this challenge that we call BeetleboxCI.
BeetleboxCI and HIL Testing
BeetleboxCI simplifies hardware in the loop testing by automating workflows and smoothly integrating with hardware components. It allows teams to run real-time tests, reducing manual effort and ensuring continuous, accurate testing throughout development.
BeetleboxCI is what is called a continuous integration and continuous testing platform. Continuous Integration is the practise of developers frequently merging their code into a single central repository, so a tool like GitHub or GitLab. The code from that repository is then frequently built and tested in a systemic manner.
So let’s say one of the engineers for the blind spot detection has just updated their code. This can trigger an automatic pipeline that takes those code changes and runs through the compile process. We can then run unit tests on software emulators that can alert us to any basic errors. Afterwards the code is ready to be run through hardware in the loop.
BeetleboxCI has a dedicated devices feature which makes it easy to integrate your hardware into your automated flows and allows users to execute tests on device. The testing rig can be used to create tests in various different conditions, such as what happens at 50 mph vs 20 mph. If those tests fail the engineers can then access the test logs through BeetleboxCI and make the necessary modifications.
If our hardware in the loop tests pass, we can then take that through to the continuous deployment phase, where the software is packaged so that it can be installed on a prototype vehicle.
Conclusion
In this article, we have explored what hardware in the loop testing is using an example from the automotive industry for blind spot detection. We have also highlighted the importance of continuous integration and continuous testing in hil testing. If you want try BeetleboxCI for yourself, you can sign up for a free trial of BeetleboxCI.
Sources
https://www.aptiv.com/en/insights/article/what-is-an-electronic-control-unit
https://www.aptiv.com/en/insights/article/what-is-hardware-in-the-loop-testing
https://roboticsknowledgebase.com/wiki/system-design-development/In-Loop-Testing/
https://www.linkedin.com/pulse/electronic-control-unit-ecu-vijay-tharad/