Testing Code for Embedded Application

Question

Background:

I am developing a largish project using at Atmel AVR atmega2560. This project contains a lot of hardware based functions (7 SPI devices, 2 I2C, 2 RS485 MODBUS ports, lots of Analogue and Digital I/O). I have developed "drivers" for all of these devices which provide the main application loop with an interface to access the required data.

Question:

The project I am developing will eventually have to meet SIL standards.

I would like to be able to test the code and provide a good level of code coverage. However I am unable to find any information to get me started on how such a testing framework should be set up.

The idea is that I can have a suite of automated tests which will allow future bug fixes and feature additions to be tested to see if they break the code. The thing is I don't understand how the code can be tested on chip.

Do I require hardware to monitor the I/O on the device and emulate externally connected devices? Any pointers that could be provided would be highly appreciated.

--Steve

Answer 1

This is a very good question - a common concern for embedded developers. Unfortunately, most embedded developers aren't as concerned as you are and only test the code on real hardware. But as another answer pointed out, this can basically test just the nominal functionality of the code and not the corner/error cases.

There is no single and simple solution to this problem. Some guidelines and techniques exist, however, to do a relatively good job.

First, separate your code into layers. One layer should be "hardware agnostic" - i.e. function calls. Do not ask the user to write into HW registers directly. The other (lower) layer deals with the HW. This layer can be "mocked" in order to test the higher level. The lower level can not be really tested without the HW, but it's not going to change often and needs deep HW integration, so it's not an issue.

A "test harness" will be all your high-level HW agnostic code with a "fake" lower level specifically for testing. This can simulate the HW devices for correct and incorrect functionality and thus allow you to run automated tests on the PC.

Answer 2

Never run unit tests on or against the real hardware. Always mock your I/O interfaces. Otherwise, you can't simulate error conditions and, more importantly, you can't rely on the test to succeed.

So what you need is to split your app into various pieces that you can test independently. Simulator (or mock) all hardware that you need for those tests and run them on your development PC.

That should cover most of your code and leaves you with the drivers. Try to make as much of the driver code as possible work without the hardware. For the rest, you'll have to figure a way to make the code run on the hardware. This usually means you must create a test bed with external devices which respond to signals, etc. Since this is brittle (as in "your tests can't make this work automatically"), you must run these tests manually after preparing the hardware.