Unit Testing Arduino Code on the Host Machine using PlatformIO

Dec 23, 2023
how-to arduino testing platformio

Unit testing, or more general, automated tests and Test Driven Development (TDD) is unfortunately not widely spread in the embedded software domain. I’m way more productive and confident in my code when it’s properly unit and integration-tested. Especially when programming Arduinos, I experienced a significant productivity boost: Instead of uploading the Firmware to the Arduino board, using Serial, buttons, or other connected devices to trigger the desired functionality to get feedback, I can just write a unit test. The feedback cycle went from several minutes down to seconds. Faster feedback, faster iterations, fewer bugs.

Not all functionality can be unit-tested. Automated tests don’t replace manual testing using the actual hardware. However, defects can be avoided more than 90% of the time in my experience.

Let me show you my setup:

Setup

  • Platform.IO IDE
  • MacBook Pro 2023 (but should work on Windows and Linux as well)
  • Arduino Nano board (also tested with an ESP32 Wroom board, and should work with other hardware as well)

In the platformio.ini, I configure 2 environments. One for the actual target hardware, and one for the desktop computer to run the unit tests:

[common]
build_flags =
    -std=c++11
build_src_filter =
    +<common/**>
test_filter =
    common/*

[env:nano]
platform = atmelavr
board = nanoatmega328
framework = arduino
build_src_filter =
    ${common.build_src_filter}
    +<nano/**>
test_filter =
    ${common.test_filter}
    nano/*
build_flags =
    ${common.build_flags}
    -DPIO_ENV_NANO
build_type = debug

; For running tests on native host machine
[env:desktop]
platform = native
build_src_filter =
    ${common.build_src_filter}
    +<desktop/**>
test_filter =
    ${common.test_filter}
    desktop/*
build_flags =
    ${common.build_flags}
    -DPIO_ENV_DESKTOP
build_type = debug

The environment for the target hardware, I usually name after the hardware (e.g. nano for Arduino Nano or esp32 for ESP32 Wroom). The configured build_src_filter and test_filter allow to split the source files for each target. On the desktop, there is no Arduino.h and no setup or loop function. On the other hand, on the nano we don’t want to implement our own main function. Therefore we place the source files for each platform into dedicated folders:

src/nano/main.cpp:

#include <Arduino.h>

void setup()
{
}

void loop()
{
}

src/desktop/main.cpp:

// Here is no desktop app.
// This file is only necessary, so building the project for `desktop` environment doesn't fail.

int main() {
    return 0;
}

If you have shared code, you can place it in the src/common folder. But we’ll talk about code that runs on both environments soon.

Test your setup by running:

pio run

The command should succeed and the output look like this:

Environment    Status    Duration
-------------  --------  ------------
nano           SUCCESS   00:00:00.314
desktop        SUCCESS   00:00:00.288
========================= 2 succeeded in 00:00:00.603 =========================

Setup Tests

Let’s set up our test environment. We only want to run tests on desktop. That’s why we create the folder test/desktop where we’ll place our test that runs on the desktop.

Platform IO only looks for folders with a test_ prefix and treats them as independent test applications that need a dedicated main() function (on desktop, or setup() and loop() on Arduino). See the official Test Hierarchy documentation for more details.

Let’s create test/desktop/test_desktop/test_main.cpp:

#include <unity.h>

int main()
{
    UNITY_BEGIN();
    return UNITY_END();
}

Unity is the built-in, default test framework of PlatformIO.

You can run pio test to check if this basic test works:

Collected 1 tests

Processing desktop/test_desktop in desktop environment
--------------------------------------------------------------------------------
Building...
Testing...
----------- desktop:desktop/test_desktop [PASSED] Took 0.35 seconds -----------

================== 0 test cases: 0 succeeded in 00:00:00.351 ==================

It succeeds! So far so good!

Note: pio test runs the test of all environments! However, since we only defined a test in test/desktop/…, only the desktop tests are executed. To limit the environment explicitly, you can run pio test --environment desktop.

Add a Test

You can either have all tests in one file, or you can spread it into multiple files. Furthermore, you can split them into multiple test_* folders.

PlatformIO, by default, doesn’t include the main sources in src in the tests. Instead, they recommend to split the program into separate libraries in lib. So src will eventually only contain the main() or setup()/loop() functions, and a bit of initialization code. If you still want to test your main sources you still have the option to enable it via test_build_src = true. I never tried it though.

Let’s follow PlatformIOs recommendation and create a separate library. We’ll implement a very simple fuel gauge.

Let’s do it TDD style!

We start with a new test file in a test module:

test/desktop/test_fuelgauge/test_fuelgauge.cpp:

#include <unity.h>

#include <FuelGauge.h>

void fuelGuage_is_initially_empty()
{
    FuelGauge fuelGauge;
    TEST_ASSERT(fuelGauge.isAlmostEmpty());
}

int main()
{
    UNITY_BEGIN();
    RUN_TEST(fuelGuage_is_initially_empty);
    return UNITY_END();
}

Obviously, this test will fail, since FuelGauge doesn’t exist: error: unknown type name 'FuelGauge'. But that’s intentional in TDD: Write the test first, it will fail, then write just enough production code to make it green. Repeat.

Compilation errors are also considered a failing test. So let’s write just enough production code to fix it:

We create a new library by creating the folder lib/fuelgauge. The source code of the library must be placed in a subfolder called src.

lib/fuelgauge/src/FuelGauge.h:

#ifndef FUELGAUGE_H
#define FUELGAUGE_H

class FuelGauge
{
public:
    bool isAlmostEmpty();
};

#endif

lib/fuelgauge/src/FuelGauge.cpp:

#include <FuelGauge.h>

bool FuelGauge::isAlmostEmpty()
{
    return true;
}

Now run pio test and see a green test:

Processing desktop/test_fuelgauge in desktop environment
----------------------------------------------------------------------------------------------------------
Building...
Testing...
test/desktop/test_fuelgauge/test_fuelguage.cpp:14: fuelGuage_is_initially_empty	[PASSED]
----------------------- desktop:desktop/test_fuelgauge [PASSED] Took 0.47 seconds -----------------------

================================================ SUMMARY ================================================
Environment    Test                    Status    Duration
-------------  ----------------------  --------  ------------
desktop        desktop/test_fuelgauge  PASSED    00:00:00.474
=============================== 1 test cases: 1 succeeded in 00:00:01.022 ===============================

This doesn’t do much, so let’s be more precise:

Change test/desktop/test_fuelgauge/test_fuelgauge.cpp:

 #include <unity.h>

 #include <FuelGauge.h>

 void fuelGuage_is_initially_empty()
 {
     FuelGauge fuelGauge;
     TEST_ASSERT(fuelGauge.isAlmostEmpty());
+    TEST_ASSERT_EQUAL(0, fuelGauge.getPercentage());
 }

 int main()
 {
     UNITY_BEGIN();
     RUN_TEST(fuelGuage_is_initially_empty);
     return UNITY_END();
 }

This will require these code changes: lib/fuelgauge/src/FuelGauge.h:

 #ifndef FUELGAUGE_H
 #define FUELGAUGE_H

 class FuelGauge
 {
 public:
     bool isAlmostEmpty();
+    int getPercentage();
 };

 #endif

lib/fuelgauge/src/FuelGauge.cpp:

 #include <FuelGauge.h>

 bool FuelGauge::isAlmostEmpty()
 {
     return true;
 }
+
+int FuelGauge::getPercentage() {
+    return 0;
+}

And we’re green again!

I like to formulate the test names are requirements to be fulfilled. The next requirement is that the percetage is 100 when full:

test/desktop/test_fuelgauge/test_fuelgauge.cpp:

 #include <unity.h>

 #include <FuelGauge.h>

 void fuelGuage_is_initially_empty()
 {
     FuelGauge fuelGauge;
     TEST_ASSERT(fuelGauge.isAlmostEmpty());
     TEST_ASSERT_EQUAL(0, fuelGauge.getPercentage());
 }

+void fuelGuage_percentage_is_100_when_full()
+{
+    FuelGauge fuelGauge;
+    fuelGauge.setValue(1024);
+    TEST_ASSERT_FALSE(fuelGauge.isAlmostEmpty());
+    TEST_ASSERT_EQUAL(100, fuelGauge.getPercentage());
+}
+
 int main()
 {
     UNITY_BEGIN();
     RUN_TEST(fuelGuage_is_initially_empty);
+    RUN_TEST(fuelGuage_percentage_is_100_when_full);
     return UNITY_END();
 }

lib/fuelgauge/src/FuelGauge.h:

 #ifndef FUELGAUGE_H
 #define FUELGAUGE_H

 class FuelGauge
 {
 public:
     bool isAlmostEmpty();
     int getPercentage();
+    void setValue(int value);
+private:
+    int value;
 };

 #endif

lib/fuelgauge/src/FuelGauge.cpp:

 #include <FuelGauge.h>

 bool FuelGauge::isAlmostEmpty()
 {
-    return true;
+    return getPercentage() <= 5;
 }
 
 int FuelGauge::getPercentage() {
-    return 0;
+    return value * 100 / 1024;
+}
+
+void FuelGauge::setValue(int value) {
+    this->value = value;
 }

So far so good. Let’s test the boundary when isAlmostEmpty changes state. After a few iterations, we have this:

test/desktop/test_fuelgauge/test_fuelgauge.cpp:

 #include <unity.h>

 #include <FuelGauge.h>

 void fuelGuage_is_initially_empty()
 {
     FuelGauge fuelGauge;
     TEST_ASSERT(fuelGauge.isAlmostEmpty());
     TEST_ASSERT_EQUAL(0, fuelGauge.getPercentage());
 }

 void fuelGuage_percentage_is_100_when_full()
 {
     FuelGauge fuelGauge;
     fuelGauge.setValue(1024);
     TEST_ASSERT_FALSE(fuelGauge.isAlmostEmpty());
     TEST_ASSERT_EQUAL(100, fuelGauge.getPercentage());
 }

+void fuelGuage_not_isAlmostEmpty_at_6_percent()
+{
+    FuelGauge fuelGauge;
+    fuelGauge.setValue(62); // 62/1024 = 6.05%
+    TEST_ASSERT_FALSE(fuelGauge.isAlmostEmpty());
+    TEST_ASSERT_EQUAL(6, fuelGauge.getPercentage());
+}
+
+void fuelGuage_isAlmostEmpty_at_5_percent()
+{
+    FuelGauge fuelGauge;
+    fuelGauge.setValue(61); // 61/1024 = 5.96%
+    TEST_ASSERT(fuelGauge.isAlmostEmpty());
+    TEST_ASSERT_EQUAL(5, fuelGauge.getPercentage());
+}
+
 int main()
 {
     UNITY_BEGIN();
     RUN_TEST(fuelGuage_is_initially_empty);
     RUN_TEST(fuelGuage_percentage_is_100_when_full);
+    RUN_TEST(fuelGuage_not_isAlmostEmpty_at_6_percent);
+    RUN_TEST(fuelGuage_isAlmostEmpty_at_5_percent);
     return UNITY_END();
 }

There are a few more tests that we could write: For example, what should happen when setValue is called with a value greater than 1024 or what should happen with negative values, etc. But for this example it’s enough. Let’s see how this can be used in Arduino code.

Use Tested Code on Arduino

Let’s use our newly created FuelGauge on our Arduino!

Edit src/nano/main.cpp to:

#include <Arduino.h>
#include <FuelGauge.h>

const int analogPin = A3;
const int almostEmptyLedPin = 13;
FuelGauge fuelGauge;

void setup()
{
    Serial.begin(9600);
    pinMode(almostEmptyLedPin, OUTPUT);
}

void loop()
{
    fuelGauge.setValue(analogRead(analogPin));

    if (fuelGauge.isAlmostEmpty()) {
        digitalWrite(almostEmptyLedPin, HIGH);
    } else {
        digitalWrite(almostEmptyLedPin, LOW);
    }

    Serial.println(fuelGauge.getPercentage());
    delay(1000);
}

Do you see how this code is only concerned with “gluing” Arduino-specific IO with environment-independent business logic? This glue-code is not covered by automated tests, but the business logic is. This can already reduce the potential errors a lot!

Conclusion

By testing code directly on the desktop computer, we can eliminate many errors before flashing the firmware on an Arduino board. The feedback cycles are faster and the total time spent is shorter.

Testing Arduino-specific code (like digitalWrite, etc.) is also possible, but more on this in a future post.

Also have a look at the official examples: PlatformIO Unit Test Examples.

Stay safe, do TDD!

You might also like

© 2024 xa1.at