[sdiy] Comparing two compiled hex files

[rsdio at audiobanshee.com]

This should be on-topic for synth firmware.

The answer has several levels.

1) These hex files are ASCII representations of binary files. It’s possible for two different ASCII representations to create the same binary.

2) The binary files represent machine code and data. It’s somewhat feasible that two different binary images could execute equivalent code, such as when the linker places the same code at different addresses and in different order.

3) The compiler versions could be different. I doubt that the different machines or different operating systems would matter, but different compiler versions could easily produce different results. There are also simple things like Debug versus Release build options that could explain the differences.

4) Sometimes all of the code matches, but there are some differences in non-code memory ranges that make the hex files differ. These differences can be the CONFIG or EEPROM values. It helps to know your target processor and how to edit hex files to delete or merge these memory sections.

I’ve been working on firmware long enough that I’ve built several tools to answer these sorts of questions. A hex decoder can build a binary image of the target address space, and then compare this against what is produced by another hex file (1 above). A disassembler can turn machine code and data into assembly language source that is a bit easier to compare by a human when looking for differences. My most recent disassembler actually creates textual labels for target addresses and removes all other incidental addresses. This latter feature, combined with some options for skipping particular automatic label names for certain hex files, allows me to get a file that will compare as “identical” even though the hex files are technically different. Of course, this sort of comparison requires a lot of specific knowledge of the firmware. I actually used it to find a bug that crept into the firmware over revisions that did not literally match. The process of weeding out all of the code that didn’t change (but had moved to slightly different addresses) was what required the magic.

None of my tools are available because they’re proprietary. I’m describing them here to illustrate the subtleties of comparing hex files or binary images of firmware builds.

Short answer: Your theory isn’t flawed, but there are still a lot of places to look before you can know whether these differences are expected or if they represent serious build problems that could result in firmware bugs because the right code isn’t being produced.

Brian Willoughby
Sound Consulting


On Jun 19, 2018, at 1:40 AM, Justin Owen <juzowen at gmail.com> wrote:
> Hello - I hope this isn’t too OT...
> 
> I’ve recently set up a gitlab repo to give a few different devs access to code assets. I’m not a coder - so this is all still new to me.
> 
> I thought a good proof of concept that the new, cloned repos on other devs machines were working as expected would be for them to compile/build a hex file from existing code and for me to compare their file against the ‘master’ production file using an online utility e.g. https://www.diffnow.com/
> 
> One of the compiled files comes up as a match using this method - but the file I’ve built doesn’t. However, the file I’ve built uploads to a test device in a couple of ways and seems to work just fine.
> 
> So now I’m trying to troubleshoot this.
> 
> Is my theory flawed? Is it possible that hex files from the same code built on different machines/OS/other variables will work but won’t be an exact file <-> file match?
> 
> Is there a reasonably simple way of isolating what the differences are - currently I just have big chunks of hex and I have no way of knowing if these are serious differences or something negligible.
> 
> Thanks!
> 
> / Justin