How can Arduino source code be tested automatically (continuous integration)?

Newest version of Arduino ide has a command line interface to build and upload code. But you can obviously do it by makefile and avrdude. Now, you have compiled your code BUT you need testing. As simulator are complex,incomplete, expansive and.. Just a simulation, and because the chip are relatively cheap, building a board witch will make HW interaction AND check the results its the fastest and easiest way. On that "special" board you may upload something like an interpreter witch take a test from the PC and execute it, something similar to firmata for arduino. At least that is how I would build it. And as far as I know there are no know implementation of this, even if I'm quite sure many industries should and probably do that.

I would recommend you to look into PlatformIO, cross-platform code builder and the missing library manager. It can build the same code for the famous embedded development platforms and the most popular embedded boards.

PlatformIO can be be integrated with the popular Continuous Integration (CI) systems (or your own). See documentation with examples.

Let's look into .travis.yml config/template for Travis CI:

language: python
python:
    - "2.7"

env:
    - PLATFORMIO_CI_SRC=path/to/source/file.c
    - PLATFORMIO_CI_SRC=path/to/source/file.ino
    - PLATFORMIO_CI_SRC=path/to/source/directory

install:
    - python -c "$(curl -fsSL https://raw.githubusercontent.com/platformio/platformio/master/scripts/get-platformio.py)"

script:
    - platformio ci --board=TYPE_1 --board=TYPE_2 --board=TYPE_N

Example

Integration for USB_Host_Shield_2.0 project. The .travis.yml configuration file:

language: python
python:
    - "2.7"

env:
    - PLATFORMIO_CI_SRC=examples/acm/acm_terminal
    - PLATFORMIO_CI_SRC=examples/Bluetooth/WiiIRCamera PLATFORMIO_BUILD_FLAGS="-DWIICAMERA"
    - PLATFORMIO_CI_SRC=examples/ftdi/USBFTDILoopback
    - PLATFORMIO_CI_SRC=examples/Xbox/XBOXUSB
    # - ...

install:
    - python -c "$(curl -fsSL https://raw.githubusercontent.com/platformio/platformio/master/scripts/get-platformio.py)"

    # Libraries from PlatformIO Library Registry
    # http://platformio.org/#!/lib/show/416/TinyGPS
    # http://platformio.org/#!/lib/show/417/SPI4Teensy3
    - platformio lib install 416 417

script:
    - platformio ci --board=uno --board=teensy31 --board=due --lib="."

• Configuration file: https://github.com/felis/USB_Host_Shield_2.0/blob/master/.travis.yml
• Build History: https://travis-ci.org/felis/USB_Host_Shield_2.0

One example of setting up Jenkins continuous integration for Arduino project can be found here: Continuous integration for embedded systems

The example shows how to build and upload image to Arduino and execute Selenium web tests (the system under test is Arduino-based web-server).

I would advice to use the arduino eclipse plugin named sloeber sloeber.io of which I'm the project lead.
It integrates with version control and allows building on multiple platforms as it supports multiple configurations.
Though it is not yet idiot proof I have documented and demonstrated using unit testing of arduino code on the local pc using the google test framework.
Here is a link to a blog containing a presentation of how you can do it. http://blog.baeyens.it/#post25