Published on

Test Approach

The ecosystem of UI frameworks/libraries, cloud services, and turnkey apps offers developers and engineers plenty of choice.

Objective comparisons are done at a feature level to allow the comparison filters to function, but I understand that products have differences in design goals and the target audience for software will have different needs, i.e.

  • Hobby users don't care about localisation features as much as cost,
  • A pro will prioritise branding and distribution control over a low-code development experience,

So I'll use a reasonably simple baseline test using features that most options should support, and then dig into more advanced functionality if I find it interesting.

Basic Functionality

This test aims to provide an 'out of the box' comparison between the different options.

Widespread community support and low cost make the ESP32 one of the most supported microcontroller+radio combinations in the DIY community, and therefore it represents the baseline target used for all of these tests.

I've also connected a LSM6DS0 IMU via I2C to act a source of real-world data.

ESP32 module with Sparkfun Micromod carrier board, IMU breakout board connected via I2C

I'll keep it simple by taking the perspective of a beginner, and attempt a few simple tasks which are common across DIY projects, robotics, and general embedded systems development:

  • Connect to hardware over serial, WiFi, or Bluetooth
  • Receive data sent from hardware
  • Render a variable as text
  • Plot a chart
  • Allow the user to change a variable from the UI

Advanced Functionality

Moving up from a basic comparison, most tools should be able to handle common tasks like data-logging, more complex visualisations, and post-processing computation that's not feasible on the embedded device.

For these tasks, I'll take the perspective of an experienced hobbyist or ECE student:

  • Logging data to file/database, export formats, etc

  • Realtime charting options, chart types, 3D visualisations

  • User input components, flexibilty of input methods

  • Rendering maps, images and video feeds

"Production Ready" Tests

For community projects or commercial products which need the deployed to end-users, the publishing and maintenance cycle matters.

Where applicable I'll try to assess the practicality and functionality of exporting and sharing 'finished' software in the context of a professional use-case:

  • Visual customisation, branding and control over specific components,
  • Building binaries/apps without needing the development environment,
  • Cross-platform support,
  • Deployment options, documentation, additional costs.