Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.
homa Guru Joined: 05/11/2021 Location: GermanyPosts: 672
Posted: 09:51pm 21 Aug 2026
Copy link to clipboard
Print this post
Hi Peter, I tried to write a data stream from the serial port to a file, but I always lose a few bytes in the process—even with PRINT MEMORY. Then I had the idea to write it not to the flash disk, but to the SD card, where it now works perfectly.
Is this because of this explanation I tried to find using AI? I hope the translation is good enough... I would really appreciate some confirmation or an explanation from you. Thanks in advance!
Matthias
The UART hardware (typically a PL011-type UART on the RP2350 with a very small hardware FIFO, often only 16–32 bytes) must be read. This hardware FIFO represents a bottleneck: 32 bytes × 320 µs = 10,240 µs ≈ 10.2 ms.
This means that if more than ~10 ms elapses between two reads of the hardware FIFO (performed by the underlying MMBasic firmware, not directly by your Midiin subroutine), the tiny hardware FIFO overflows before your 4,096-byte software buffer even notices it. This happens silently, without an error message.
That would explain why:
a 4096-byte software buffer (with a 1.3-second reserve) is insufficient, since the bytes are already lost at the hardware level beforehand. The losses occur irregularly and are consistent with normal SD write latencies (a range from a few milliseconds to the low double-digit millisecond range is sufficient). Some SD/SPI drivers temporarily disable global interrupts during critical transfer phases (which is common during bit-banging or DMA setup phases). If MIDI bytes arrive precisely during this window, even a “real” hardware interrupt cannot respond in time, and the 10-ms hardware FIFO buffer is quickly exhausted.
That’s a very good question—and yes, that could indeed be the crucial point. I’ve looked at the source code for the PicoMite/MMBasic firmware (the basis for RP2350 MMBasic devices like PicoCalc) as well as the underlying Raspberry Pi Pico SDK documentation on flash access.
The fundamental difference: A: (Flash) vs. B: (SD card)
A: Internal flash memory
The RP2350’s internal flash chip operates in XIP (Execute-In-Place) mode. This means that the processor executes its own program code directly from the flash while it is running. This has a serious hardware consequence, which is explicitly mentioned in the official Pico SDK documentation:
“Flash cannot be erased or written to when in XIP mode. […] It is therefore critical that no code or data is being read from flash while flash is being written or erased.”
And further, specifically regarding interrupts:
“They are unsafe if you have interrupt handlers or an interrupt vector table in flash, so you must disable interrupts before calling [flash_range_program/erase]”
This means: Every write or erase operation on the internal flash forces the SDK to completely disable all interrupts on the writing core. This is not optional but physically necessary, since the flash chip cannot be read simultaneously while it is being written to or erased (not even for interrupt vectors or handlers that are themselves stored in flash!). A flash sector erase typically takes 10–100+ ms, while a pure program (without erase) tends to take 1–3 ms. However, as soon as an erase cycle is due (which happens regularly with sequential data appending), the interrupts are completely disabled for the entire duration.
Interestingly, there is even a note to this effect in the README of the PicoMite source code: It describes that the GPIO interrupt handler had to be deliberately placed in RAM rather than in flash—an indication that interrupt/flash conflicts are a known and actively addressed issue in this codebase.
B: (SD Card)
The SD card, on the other hand, is connected to an external SPI/SDIO bus. It is a standard peripheral device through which data transfer occurs not via the internal flash address space, but via a separate hardware component. No global interrupts need to be disabled for this access. Your Midiin interrupt can continue to fire during an SD write operation and retrieve bytes from the UART hardware.
Why this matches your symptoms exactly: A 10 ms hardware FIFO buffer (from my previous calculation) is more than enough to handle normal SD card latencies (which occur even with SPI without an interrupt disable), but not a flash erase cycle of, say, 50 ms, during which literally nothing responds. It is precisely in those moments when a flash write operation coincides with a dense burst of notes that bytes are lost.
matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 11776
Posted: 10:06pm 21 Aug 2026
Copy link to clipboard
Print this post
what s/w? what version? Options if applicable? Example code? Not a mind reader Edited 2026-08-22 08:07 by matherp
homa Guru Joined: 05/11/2021 Location: GermanyPosts: 672
Posted: 06:57pm 22 Aug 2026
Copy link to clipboard
Print this post
Why not? That would be impressive! I’ve been thinking about this problem and how to optimise it for the last two days, so I simply forgot all about it.
' Ring buffer setup (4096 entries for bytes and timestamps) Dim rb_byte%(4095) Dim rb_time%(4095) Dim rb_head% = 0 Dim rb_tail% = 0
' Open the serial interface using a 1-byte interrupt trigger! SetPin GP4, GP5, COM2 Open "COM2:31250, 4096, midiin, 1" As #2 Open "temp.bin" For RANDOM As #3
' Flash-Init dummy start Dim dummy%(7) For i% = 0 To 7: dummy%(i%) = 0: Next i% Memory PRINT #3, 8, dummy%()
' OPTIMISATION: Micro-burst buffer (only 32 events = 256 bytes) Dim buffer%(31) Dim idx% = 0
Dim st% = 0, rs% = 0 Dim dat1% = 0, exp% = 0 Dim t_evt% = 0 Dim run_flag% = 0
' Variables for loop reception Dim b% = 0 Dim t_incoming% = 0
' Check whether the ring buffer contains any new data' If rb_head% = rb_tail% Then Continue Do
' Retrieve a byte and a precise timestamp from the ring buffer b% = rb_byte%(rb_tail%) t_incoming% = rb_time%(rb_tail%) rb_tail% = (rb_tail% + 1) Mod 4096
If b% >= &HF8 Then Continue Do
If b% >= &HF0 And b% <= &HF7 Then rs% = 0 If b% = &HF0 Then st% = 3 If b% = &HF7 Then st% = 0 Continue Do EndIf If st% = 3 And b% < &H80 Then Continue Do
' Status-Byte If b% >= &H80 Then rs% = b% st% = 1 run_flag% = 0 t_evt% = t_incoming%
Select Case (b% And &HF0) Case &HC0, &HD0: exp% = 1 Case Else : exp% = 2 End Select Continue Do EndIf
If rs% = 0 Then Continue Do
If st% = 1 Then If run_flag% = 1 Then t_evt% = t_incoming% run_flag% = 0 EndIf
dat1% = b%
If exp% = 1 Then buffer%(idx%) = t_evt% Or (rs% << 32) Or (dat1% << 40) idx% = idx% + 1 st% = 1 run_flag% = 1 Else st% = 2 EndIf
ElseIf st% = 2 Then buffer%(idx%) = t_evt% Or (rs% << 32) Or (dat1% << 40) Or (b% << 48) idx% = idx% + 1 st% = 1 run_flag% = 1 EndIf
' MICRO-BURST WRITE: Write as few as 32 events (256 bytes)! If idx% = 32 Then Memory PRINT #3, 256, buffer%() idx% = 0 EndIf Loop
Close #2
' Run the remaining data in the ring buffer through the state machine in its entirety' Do While rb_head% <> rb_tail% b% = rb_byte%(rb_tail%) t_incoming% = rb_time%(rb_tail%) rb_tail% = (rb_tail% + 1) Mod 4096
If b% >= &HF8 Then Continue Do If b% >= &HF0 And b% <= &HF7 Then rs% = 0 If b% = &HF0 Then st% = 3 If b% = &HF7 Then st% = 0 Continue Do EndIf If st% = 3 And b% < &H80 Then Continue Do
If b% >= &H80 Then rs% = b% st% = 1 run_flag% = 0 t_evt% = t_incoming% Select Case (b% And &HF0) Case &HC0, &HD0: exp% = 1 Case Else : exp% = 2 End Select Continue Do EndIf
If rs% = 0 Then Continue Do
If st% = 1 Then If run_flag% = 1 Then t_evt% = t_incoming% run_flag% = 0 EndIf dat1% = b% If exp% = 1 Then buffer%(idx%) = t_evt% Or (rs% << 32) Or (dat1% << 40) idx% = idx% + 1 st% = 1 run_flag% = 1 Else st% = 2 EndIf ElseIf st% = 2 Then buffer%(idx%) = t_evt% Or (rs% << 32) Or (dat1% << 40) Or (b% << 48) idx% = idx% + 1 st% = 1 run_flag% = 1 EndIf
If idx% = 32 Then Memory PRINT #3, 256, buffer%() idx% = 0 EndIf Loop
' Write the remaining events to the event buffer If idx% > 0 Then Memory PRINT #3, idx% * 8, buffer%() EndIf
Close #3 Print "Saved temp.bin" End
SUB midiin Local t% Local b_in% Local next_head%
Do While Loc(#2) > 0 t% = Fix(Timer) And &HFFFFFFFF b_in% = Asc(Input$(1, #2)) next_head% = (rb_head% + 1) Mod 4096
If next_head% <> rb_tail% Then rb_byte%(rb_head%) = b_in% rb_time%(rb_head%) = t% rb_head% = next_head% EndIf Loop END SUB
Comparison between the transmitted data stream and the recording:
As soon as I switch the working directory from the flash drive (drive a:) to the SD card (b:), I no longer have any missing data!
Hence my question to the AI and my attempt to verify this explanation with you.
Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.
Print this post
That would be impressive! 
