[sdiy] Techniques for Multiplying MIDI Clock frequency?

[Brian Willoughby]

On Dec 23, 2021, at 19:06, mskala at northcoastsynthesis.com wrote:
> On Thu, 23 Dec 2021, Brian Willoughby wrote:
>> I'm not sure what this "high bandwidth" mechanism is, other than the
>> fact that Full Speed is a lot faster than 31.25 kbaud.
> 
> I've seen over 40 poll/NAK sequences per frame (maybe over 70; both
> numbers are in my memory but I'm not sure the faster one was actually with
> working code) between my host and USB-MIDI keyboard when the host had
> nothing better to do than poll as fast as it could.  I ended up
> implementing a token-bucket rate limiter because I wanted to allow it to
> poll more than once per frame, but not at an unlimited rate.

What operating system was serving as the USB Host?

I primarily use Mac OS X, so I have no control over how often the Host does Bulk polling because my application is designed to communicate directly with CoreMIDI instead of the USB Host driver. You can't really control the Host from your Device firmware anyway, and perhaps you don't want to try limiting things. So long as the Device responds saying "I don't have any Bulk data for you right now," then everything should work normally.

> PIC24 16-bit
> microcontroller, 16MHz instruction clock; not a hugely powerful computer
> there.

My USB-MIDI Device firmware experience is with the PIC18 8-bit MCU with a 48 MHz clock.

There are two USB firmware stacks from Microchip for the PIC18 (at the time I worked on this). One is for the normal firmware, the alternate stack is for the firmware updates that are supported over USB. The latter is slightly stripped down because it needs to fit within a finite section of Flash code memory, and it doesn't usually need to support product-specific features. I noticed that each of these two stacks had a history of bug fixes across releases, and while they were fairly well documented, it wasn't entirely clear that all bugs had been fixed in both. So, I painstakingly mapped out all of the bug fixes in the various revisions, and made sure that they were applied to both stacks. I may have also added some features to the "firmware update over USB" code because I wanted to enable switching into and out of firmware update mode without forcing the user to manually power cycle. I don't think Microchip supported the USB DFU standard at the time I was working (probably because DFU may not have even been finalized yet). So I recall spending some time implementing DFU, although I can't recall whether it was successful in a compatible way.

These Microchip USB stacks are all in C, and the chip has hardware support for the timing-critical stuff anyway. All you need to do is implement the details that make your USB Device unique compared to the average USB Device. They didn't provide a USB-MIDI driver at the time, but USB-MIDI is a very simple layer on top of the generic USB Device. Just set up Bulk endpoints for IN and OUT, group data into 4-byte USB-MIDI messages, and that's it. The rest of the code is basic MIDI (although I always thought it was a severe oversight that the USB-MIDI message has the same 4-bit command encoded twice in every message, making it rather difficult to know which one to honor if they happen to be different).

> Of course, that is with no real MIDI data being sent - it couldn't
> be transferring full 64-byte packets at that rate because it would exceed
> the bus bandwidth, and the host would also hit some limit on how fast it
> could actually do anything useful with the MIDI data.  But I was impressed
> that the keyboard had no trouble keeping up with the host.

I believe that USB allows multiple 64-byte Bulk packets per USB frame. It's up to the Host to schedule these appropriately while accounting for the minimum required Isochronous transfers plus other endpoint types. There's a table in the USB spec that shows the maximum number of Bulk transfers of various sizes, and you can see how the advertised USB bandwidth can never really be reached. It's also obvious that smaller packets reduce the bandwidth because each packet has overhead.

The way the Microchip PIC18 USB stack worked is that the Device code would queue up Bulk data ahead of time, and then whenever the Host scheduled a Bulk transfer the PIC18 firmware would set up DMA for the Bulk data. Various firmware settings made it easy to avoid going over 64 bytes per packet, or responding to a poll when there was nothing to send, and also handled Bulk packets that were shorter than the maximum (64-byte) size.

I recall doing some sort of benchmark. Either I counted packets per frame and came up with "3" or I put the frame number into the USB-MIDI data stream and noticed that there were 3 USB-MIDI messages in the same frame.

I assume that I could have optimized this by building a packet queue, where each new USB-MIDI message was added to a given packet until there were 16 USB-MIDI messages in the packet, and then moving on to fill additional packets. I could have used that table from the USB spec to see the maximum number of packets that might be needed in a single frame, plus maybe a little extra to allow queuing USB-MIDI messages for the next frame.

As it was, my code was throttled to 16 USB-MIDI messages so that the firmware was only working on one packet at a time. There was some sort of handshaking so that when a packet was sent to the Host, the queue of 16 was reset to allow more USB-MIDI messages to be added. I forget the details, but I assume that 3 messages of actual USB-MIDI data payload was probably not the actual maximum.

>> My most recent project was to write USB Host firmware without an
>> operating system of any kind. It was somewhat annoying because even
> 
> On what chip?

Texas Instruments TM4C1294 ARM, 32-bit, 120 MHz.

Texas Instruments provides several firmware examples including isochronous audio and USB Hub support. I didn't have to write the Hub support myself, but I did have to figure out how to combine multiple examples into a single piece of firmware that both supported a Hub and my Device.

My goal for supporting a Hub was to provide cheap isolation (I'd rather burn out a $4.99 Hub than my custom USB hardware), and also to allow for some way to save and restore custom settings on a USB memory stick without literally removing the controller Device from the Host USB jack.

My Device has two 384-byte Isochronous endpoints, for a total of 768 bytes per Full Speed USB frame. The TM4C1294 seems to have a hardware or firmware bug, but it may only affect Isochronous endpoints that are larger than 256 bytes. I've never seen bad data in the first 256 bytes of those Isochronous packets. Even though they "support" larger Isochronous packets, they may not have tested them very well. I haven't worked on this in a while, though.

I have not implemented USB-MIDI Host firmware on the TM4C129x, but I assume it would be easy and bug-free, given the nature of Bulk packets.

> Right now, literally in my other terminal tab as I type this, I'm busy
> writing up the documentation for my PIC24-based USB MIDI host synthesizer
> module, which I'm hoping to release in the new year.

That sounds very cool !

Given their prevalence, it seems like a very useful feature to be able to Host USB-MIDI controllers.

> It's been quite an
> adventure getting it to work, both because I had no previous USB
> experience and because the hardware is only sort-of documented.
> Microchip just tells people to use the C-language driver they provide.
> For various reasons, I wanted to write my own in assembly language.  I'm
> pleased to have that working now and that's why USB-MIDI is fresh in my
> mind.

I don't know the details of the PIC24, but if they handle enough of USB in hardware, including some DMA, then you might as well use the C Language driver. However, most processors have completely separate documentation teams for the hardware versus the firmware, so there should be enough documentation for you to provide all of the firmware ... I just wouldn't recommend working that hard.

I prefer to leverage the generic USB support - and I've never had to write it (just debug it) - and spend time on the unique aspects. Depending upon the maturity of the USB firmware stack, you can leverage the fixes for the mistakes of those who have been working with the chip longer.

>> after the Host was talking directly to the Device successfully, a lot
>> more code had to be added to support a Hub between the Host and Device.
> 
> It is a hardware erratum of the PIC24 that it cannot be host for a hub
> when there is a low-speed device on the bus.  Now, technically, that
> wouldn't preclude using a hub with a PIC24 and USB-MIDI devices (because
> USB-MIDI is never low-speed...); but it made a good excuse for me to say
> that at least in the initial version of my project, I just don't support
> hubs, period.  Made my life simpler, and not a huge imposition on the
> intended use case of my module.

I wasn't convinced that I *needed* to support a Hub, but I wanted to see how hard it would be. So, I allocated a finite amount of time to experiment. I started with examples that only worked with common USB Devices, via a Hub, and confirmed that those worked. Then I dug into the firmware source and figured out how to merge Hub support into my existing custom Device support. I turned out that this took only about a day of effort, so I added Hub support.

Of course, given that I am not running an operating system on this ARM chip, there will be no support for any Device attached to the Hub other than the two that I code for - the custom controller Device and a USB memory stick.

I think it would be fair for you to say that your product supports a Hub, but only with Devices attached that are already supported without the Hub. I guess that could be a problem if you support a mouse and/or keyboard, because then that support would go away with a Hub.

On the other hand, maybe there's a way to work around the hardware erratum in firmware? That would be some serious PIC24-fu.

> -- 
> Matthew Skala
> North Coast Synthesis Ltd.

By the way, the custom USB controller Device that I keep referring to is based on the Texas Instruments TMS320VC5506 16-bit fixed-point DSP. The two 384-byte Isochronous endpoints each send multi-touch pressure data for half of the surface, with an update rate of 1 kHz and a latency of 2 or 3 ms. This was a challenging USB Device firmware because it combines DSP assembly language to run 16,000 128-point FFT operations per second at 40-bit precision while maintaining a minimum Full Speed bandwidth of 768 KB/sec. There are additional USB endpoints for tracking pots and debugging information, but those aren't really used much. It's just amazing that everything can be squeezed into the out-of-date USB 1.0 spec with any problems. The normal Host is macOS, which doesn't have the Isochronous bugs on larger-than-256-byte Isochronous packets that the TM4C129x has.

One very interesting detail about this device is that it's native sampling rate is 125 kHz, and that works out to only a 976.5625 Hz packet rate. Although the Host USB clock is not locked to the Device sample rate, it's still true that the Host wants about 1,000 packets per second. This means that 1 out of every 42+2/3 frames has a 0-byte Isochronous packet instead of a 384-byte packet. Everything still manages to work out, because USB was designed for this.

Brian

p.s. To get back to the subject... any of these 8-bit or 16-bit or 32-bit processors should be able to handle incoming MIDI clock, lock in a Timer channel, and multiply the frequency for rock-solid output timing. Because the TM4C129x has 8 full UART peripherals, I've been tempted for several years to design an 8-input, 8-output MIDI merger than properly handles MIDI clock between everything (although that could be a mess if you don't have a way to turn off conflicting clocks).