Thursday, August 13, 2015

Why I Still Hate The Arduino IDE; or "How Come The NMU-7700 Software Isn't Ready Yet?"

I've been adding stuff to the NMU-7700 page over the last week or two - I wanted to quickly wrap it up for now so it's done, ready to build, & I could move onto other things. So far all the hardware stuff's there; what's missing is the firmware/software.

Which is where the post title comes in…

Up 'til now all the software development has been done in embedXcode/Xcode (Apple haters: think Visual Studio + VisualMicro, or an Atmel Studio that supports other micros). Since it uses the existing Arduino libraries it's 99% compatible, and where it's not it's usually because it's stricter about insisting things be C/C++ like. The upshot is you can usually do all the development in a nice IDE, then drop the source straight into the Arduino IDE to double-check it before release.

With the NMU it hasn't worked out that simply.

The problem is the Arduino environment. Check out this (condensed) set of messages I get from the Arduino compilation/linking step when building the NMU software...

Arduino: 1.6.5 (Mac OS X), Board: "NMU-7700 Dev"

Using library SPI in folder: /Applications/
Using library Wire in folder: /Applications/
Using library NMU Interface in folder: /Users/[username]/Documents/Arduino/libraries/NMU_Interface
Using library AD9833 SPI in folder: /Users/[username]/Documents/Arduino/libraries/AD9833_SPI
Using library FreqCounter in folder: /Users/[username]/Documents/Arduino/libraries/FreqCounter
Using library Robot Control in folder: /Applications/

< …compilation stuff removed… > 

/Applications/ In constructor 'RobotControl::RobotControl()':
/Applications/ error: 'LCD_CS' was not declared in this scope
< …bunch of similar errors removed… > 

/Applications/ error: 'Serial1' was not declared in this scope

Error compiling.

Sorry, what?! What the hell has Robot_Control got to do with anything?

If you Google around you'll see plenty of similar complaints. I know it's been cropping up ever since those robot libraries were introduced back around v.1.0.4 or 1.0.5, and in most cases seems to involve the SPI library somehow getting tangled up with Robot_Control. The 'solution', if you can call it that, offered by the Arduino devs & community is basically "delete the Robot_Control library from the Arduino system library folder".

Which is fine for me - I'm never likely to use it - and deleting it solved the problem back then. But this is now, and telling people "oh no, if you want to use my device then you have to delete system libraries to make it work" isn't exactly the right thing to do.

Besides, this time the error was subtly different - there's no mention of SPI amongst the Robot_Control stuff. Something else was going on…

Eventually I figured it out. Have a look at this bit of the NMU code:

// Include application, user and local libraries
#include <SPI.h>
#include <Wire.h>
// NMU-7700 Interface Board library
#include <NMU_Interface.h>
// AD9833 SPI library
#include <AD9833_SPI.h>
// Martin Nawrath's Frequency Counter library
#include <FreqCounter.h>
// Note: These 2 headers/libraries live in the sketch directory, hence "..."
// not <...>
// Logical pin names
#include "NMU_Pins_v4.h"
// Modified Apex Logic I2C EEPROM library
#include "EEPROM_I2C.h"

(Clever people who have previously spent time among the Arduino system libraries may already see the problem…)

Now, read this bit about the #include C pre-processor directive from the GCC compiler documentation:

  • #include <file> - This variant is used for system header files. It searches for a file named file in a list of directories specified by you, then in a standard list of system directories.
  • #include "file" - This variant is used for header files of your own program. It searches for a file named file first in the current directory, then in the same directories used for system header files.

In other words when using "…" the Arduino compiler should first look for the header file in the current sketch directory - and only look elsewhere if it can't be found.

Turns out that for historical reasons of compatibility with pre-1.0 versions the Arduino build scripts don't work like that. I haven't bothered to look and see exactly what they do do, but the upshot of whatever it is is that they find a file also named EEPROM_I2C.h inside the Robot_Control library - and choose to use that instead of the one I told it to use.

The solution, then, is simple. Just make sure none of your libraries share their name with any past, present, or future libraries!

That's dumb. The system shouldn't be making decisions on what library to use - it should respect exactly what you've told it to do, and if you told it to do something that is ambiguous or causes a clash then it should spit a suitable error message. It doesn't do that, and the only reason why is because the Arduino developers broke things from the start & now need to keep them broken to ensure backwards compatibility.

Which is a pity. Although I wasn't a fan of the earlier 1.0.x versions - mainly because they weren't much more than front end to a shitty text editor, an even shittier terminal emulator, and set of inconsistently-obtuse build scripts - I was actually starting to like the user-facing bits of the latest v1.6.x versions. While it's still nothing like an actual IDE, it's now starting to gain enough features to make it a good option for non-programmers - like:
  • code-folding - you can now collapse comments & sections/sub-sections of code, replacing the hidden part with a single line that serves as an “overview”. It's particularly nice for long sketches, since you can tuck away the bits that are complete & concentrate on the bit you're working on. 
  • library management - it's now simple to add 3rd-party custom/contributed libraries. There's even a manifest which provides extended information, hardware compatibility, and hints of the beginning of version control. It's still a bit rough around the edges, but it's there & being worked on. 
  • tabs (can) mean tabs! - a personal bitch of mine, and I suspect of anybody who grew up understanding actual typewriters before encountering computers. Unlike the early computer programmers (and the sad deluded people who still follow them ;))*, we know that tabs don't mean 8 (or 2, or 4, or sometimes even 3 or 5 or whatever) spaces, or even 'a character that says jump ahead 8 (or 2, or 4, or sometimes even 3 or 5 or whatever) spaces'. The tab character (ASCII 09) means 'jump to the next tab stop wherever it's set to', and thankfully the Arduino IDE can now be configured** to do exactly that…
It's also got the beginnings of support for user-designed custom boards. Previously, adding a non-Arduino-supported board meant digging around in the guts to edit files and whatnot; now they've come up with a simple bundle / manifest system for updating the list of supported boards. Currently it's a bit immature & doesn't support some of the stuff needed for the NMU-7700 (e.g. custom bootloaders, distributable packages, etc.), but it seems they plan to add that in the future. Good move, Arduino-guys!

Pity about the rest of it…

(† edit, 15/8: and now I see the as-yet-unreleased v1.6.6 makes changes to that which will break any custom board package I build for the current version. Looks like waiting a bit longer is in the wings…)

My little comment on the ongoing "Arduino vs Arduino" spat: Despite my personal opinion of the Arduino environment, I'm inclined to side with the original Arduino guys i.e. Arduino SRL,, & IDE v1.6.5. For that reason I haven't even looked at the 'opposition' Arduino v1.7.0 - I assume my code itself will be fine, but beyond that who knows?

There's also the slight problem that embedXcode supports neither at the moment - in essence, the developer is waiting until support tools for either turn up before incorporating the considerable changes between the original IDE versions and both newer versions. That hasn't been a problem for me so far - up until now I've been developing on my old Macbook with Arduino v1.0.6 - but I've recently updated to a new MBP and Mac Mini desktop, & installed Arduino v1.6.5 to take advantage of the improved library / board management.

For obvious reasons, until things shake out a bit on both fronts development will remain slow…

To wrap that up: I now have it all working under the Arduino IDE, and in the next day or so I'll add the NMU software & libraries to the project page. There's still some more I want to do - replace the standard I2C library to get past a limitation that cramps the memory storage routines (which will then require a rewrite of the EEPROM library and major-ish mods to the Interface library), swap the frequency counter library I've used up until now (Martin Nawrath's FreqCounter) for the the officially-endorsed FreqCount library, and add end-user calibration to the the F.Tune control - but that can wait until I sort out whether I can use the custom board support to add the NMU-7700 to the Arduino's board list…

Oh, and you probably also noticed that I've been playing with the blog theme. That'll keep changing for a while until I'm happy with it. Let me have my fun ;)

(* Yes, I know, it's one of programming's great holy wars. Let's just say that typewriter manufacturers elegantly solved that particular issue nearly 100 years before computer programmers decided to recreate the wheel as a series of concentric cubes…)
(** Locate your Arduino preferences.txt file and change the following lines:
editor.tabs.size=4:    // Or 8 or 2 or sometimes even 3 or 5 or whatever…
Now, see how simple it makes things? ;))

No comments:

Post a Comment