TheBackShed.com

Forum Index : Microcontroller and PC projects : "Not enough heap memory"

Posted: 03:17am 02 Apr 2026 Copy link to clipboard	lizby Guru

I've run into an issue with heap memory on a WeAct RP2350B module with 8MB PSRAM.

This appeared running my PicoRR program using PicoDB, about 5000 lines of code. I don't know how to make the example truly minimal, but 273 lines does it. Only 19 lines are executable--the rest are variable definitions--including big arrays.

The key constant is MAX_RECS. The larger program runs fine if it is set at 500. In this instance, set at 920, the code reports "Heap: 6291712" and then calls an empty stub SUB ("SUB DummySub", which has no code, only a SUB END) and there's an immediate "Error : Not enough Heap memory".

Here's the code:

' =============================================================
'rr.bas - PicoDB + Model Railroad OS
' =============================================================
Option EXPLICIT
Option Default Float

' =============================================================
' PICODB ENGINE GLOBALS (Required for library.bas)
' =============================================================
option local variables 500
Dim Integer GLOBAL_MODE = 2 ' for option resolution 640x320, 320x240 16 colors
Dim Integer DBG_LVL = 3 ' 0=min; 1=min; 2=max
Dim Integer Is_LCD = 0
Const MAX_TBL = 5
Const MAX_FLD = 30
Const TYPE_STR = 1
Const TYPE_INT = 2
Const TYPE_FLOAT = 3
Const MAX_ENGINES = 5
Const MAX_RECS = 920 ' Tuned for HDMI Memory, no PSRAM (150 works)
Const MAX_CARS_PER_TRAIN = 15
Const MAX_PARKED_CARS = 20
Const MAX_GROUPS=5
Const MAX_SCRIPT_LINES=100
Const STOP_MARGIN = 24.0 ' pixels
Const SpriteSize = 22
Const HC_06 = 0

' =============================================================
' RGB121 (4bpp) HARDWARE SPRITE PALETTE
' Bit Layout: [Red] [Green_Hi] [Green_Lo] [Blue]
' =============================================================
Const HW_BLACK = 0 ' (0000) MMBasic 0
Const HW_BLUE = 1 ' (0001) MMBasic 1

Const HW_MYRTLE = 2 ' (0010) MMBasic 8 (Low Green)
Const HW_COBALT = 3 ' (0011) MMBasic 9
Const HW_MIDGREEN = 4 ' (0100) MMBasic 10 (High Green)
Const HW_CERULEAN = 5 ' (0101) MMBasic 11

Const HW_GREEN = 6 ' (0110) MMBasic 2 (Full Green)
Const HW_CYAN = 7 ' (0111) MMBasic 3

Const HW_RED = 8 ' (1000) MMBasic 4
Const HW_MAGENTA = 9 ' (1001) MMBasic 5

Const HW_BROWN = 10 ' (1010) MMBasic 14 (Red + Low Green)
Const HW_LILAC = 11 ' (1011) MMBasic 15
Const HW_RUST = 12 ' (1100) MMBasic 12 (Red + High Green)
Const HW_FUCHSIA = 13 ' (1101) MMBasic 13

Const HW_YELLOW = 14 ' (1110) MMBasic 6 (Red + Full Green)
Const HW_WHITE = 15 ' (1111) MMBasic 7

Const TransparentColor = HW_MAGENTA ' 9=magenta

Dim Float D_Scale
Dim Float D_CAR_GAP

' --- DISPLAY SETTINGS ---
Dim Float PARK_MARGIN
Dim Float CAR_WIDTH

Dim STRING dbname$ = "layout", db_command$
Dim STRING makedb_default$ = "make-db layout roster, segments index on location, sequence use"
Dim STRING cmd$, defLine$, token$, workLine$, tmp_f$
Dim INTEGER fNum, commaPos, tokIdx, startTime, tStart
Dim STRING temp_alias$, temp_file$, temp_owner$, temp_name$
Dim INTEGER temp_len, temp_type, temp_start, temp_width, spc

Dim STRING Q_Filter$, Q_OrderFld$, Q_ShowFlds$, Q_OutFile$
Dim INTEGER Q_Limit, Q_OrderDesc, Q_OutType, Q_OutHandle
Dim INTEGER Result_Count
Dim INTEGER Aggr_Mode, Aggr_Fld_ID, Aggr_Count, Action_Mode
Dim FLOAT Aggr_Val
Dim INTEGER SQL_Mode, CACHE_LOADED
Dim String DB_Callback$
Dim INTEGER DB_File_Ptr(MAX_TBL)
Dim INTEGER DB_Auto_Index = 1
Dim INTEGER DB_Dirty_Flag(MAX_TBL)
Dim STRING Upd_Target_Fld$, Upd_Target_Val$, Upd_Target_Tbl$
Dim STRING Group_Keys$(MAX_GROUPS) length 31
Dim FLOAT Group_Vals(MAX_GROUPS)
Dim INTEGER Group_Counts(MAX_GROUPS)
Dim INTEGER Group_Top, Group_Fld_ID
Dim INTEGER Join_Driver_Tbl, Link_Fld_ID
Dim STRING DB_Link_Fld$, DB_Link_File$

Dim STRING DB_Tbl_Name$(MAX_TBL) LENGTH 31
Dim STRING DB_Tbl_File$(MAX_TBL) LENGTH 31
Dim INTEGER DB_Tbl_Len(MAX_TBL)
Dim INTEGER Table_Count

Dim STRING DB_Fld_Owner$(MAX_FLD) LENGTH 31
Dim STRING DB_Fld_Name$(MAX_FLD) LENGTH 31
Dim INTEGER DB_Fld_Type(MAX_FLD), DB_Fld_Start(MAX_FLD), DB_Fld_Width(MAX_FLD)
Dim INTEGER Field_Count, db_Record_Count

Dim STRING DB_RowBuf$(MAX_TBL)

Type IndexItem
Key As STRING LENGTH 31
RecNo As INTEGER
End Type

Type IndexItemNum
Key As FLOAT
RecNo As INTEGER
End Type

Dim IndexItemNum DB_IndexNum(MAX_RECS)
Dim INTEGER Link_Rec_Count
Dim IndexItem DB_Index(MAX_RECS)
Dim IndexItem DB_Lookup(MAX_RECS)
Dim IndexItemNum DB_LookupNum(MAX_RECS)
Dim IndexItem Result_List(MAX_RECS)

Type joinRecsNum
RecNo1 As INTEGER
RecNo2 As INTEGER
Key As FLOAT
End Type

Type joinRecsAlpha
RecNo1 As INTEGER
RecNo2 As INTEGER
Key As STRING * 31
End Type

Dim joinRecsNum DB_JoinResNum(1)
Dim joinRecsAlpha DB_JoinResAlpha(1)

' =============================================================
' RAILROAD GRAPHICS GLOBALS
' =============================================================
Dim Float Scale
Dim Integer OffX, OffY
Dim Float GridW, GridH
Dim Float ind = 0.6
Dim Float x1_top, x1_bot, x2_top, x2_bot
'Dim Float SegLen(10)
'Dim Integer MacSeg(40), MacRev(40)
'Dim Float MacLen(40), MacOdo(40)
Dim Float TotalMacroLen
Dim Integer mIdx = 0

Dim Integer cTrack = RGB(255, 255, 255)
Dim Integer cSiding = RGB(255, 255, 255)
Dim Integer cSwitch = RGB(255, 255, 0)
Dim Integer cCross = RGB(255, 128, 0)

' --- GLOBAL ACTIVE LAYOUT ---
Dim String Active_Layout$ LENGTH 4 = "1"

' Database Active Roster Arrays (Loaded on RUN)
Const MAX_SWITCHES = 20
Dim Integer SwState(MAX_SWITCHES)
Dim String dCmd$ = ""
Dim Float V(MAX_ENGINES), TV(MAX_ENGINES), D(MAX_ENGINES), oldD(MAX_ENGINES)
Dim String E_Leg$(MAX_ENGINES) length 20, old_E_Leg$(MAX_ENGINES) length 20
Dim Integer wasMoving(MAX_ENGINES)

' Multi-Train Physics Arrays
Dim Integer TR_Count(MAX_ENGINES)
Dim Float TR_PixLen(MAX_ENGINES, MAX_CARS_PER_TRAIN)
Dim Integer TR_IsEng(MAX_ENGINES, MAX_CARS_PER_TRAIN)
Dim Integer TR_Color(MAX_ENGINES, MAX_CARS_PER_TRAIN)
Dim Integer TR_Dir(MAX_ENGINES)
Dim Integer Active_Engine
Dim String TR_ID$(MAX_ENGINES, MAX_CARS_PER_TRAIN) LENGTH 4

E_Leg$(1) = "L3" : old_E_Leg$(1) = "L3"
V(1) = 8.75
Dim String Found_Loc$

Dim Integer PK_Count = 0
Dim String PK_Leg$(MAX_PARKED_CARS) LENGTH 16
Dim Float PK_Dist(MAX_PARKED_CARS), PK_Len(MAX_PARKED_CARS), PK_Offset(MAX_PARKED_CARS)
Dim Integer PK_Color(MAX_PARKED_CARS), PK_IsEng(MAX_PARKED_CARS), PK_Dir(MAX_PARKED_CARS)
Dim String PK_ID$(MAX_PARKED_CARS) LENGTH 4

' --- SPRITE ENGINE GLOBALS ---
Dim Integer Use_Sprites = 0
Dim Integer TR_Sprite(MAX_ENGINES, MAX_CARS_PER_TRAIN)
Dim Integer PK_Sprite(MAX_PARKED_CARS)

' New dynamic cache globals
Dim String Cache_Seg_Name$ LENGTH 16
Dim Integer Cache_Seg_Idx, Cache_Dir
Dim Float Cache_Start_D

Dim Float LegLen(15), Cursor_X, Cursor_Y

Dim Integer Node(MAX_SWITCHES, 2)

' --- DYNAMIC GEOMETRY ARRAYS ---
Dim Integer Layout_Seg_Count, headSeq
Dim String Seg_ID$(MAX_RECS) LENGTH 16
Dim String Seg_Type$(MAX_RECS) LENGTH 2, Seg_CurveType$(MAX_RECS) LENGTH 2
Dim Float Seg_Len(MAX_RECS), Seg_Rad(MAX_RECS)
Dim Float Seg_X1(MAX_RECS), Seg_Y1(MAX_RECS)
Dim Float Seg_X2(MAX_RECS), Seg_Y2(MAX_RECS)
Dim Integer Seg_WestNorth(MAX_RECS) ' 0 if x1,y1 is West/North, 1 if x2,y2

' NEW: Friendly Names
Dim String Seg_Friendly$(MAX_RECS) LENGTH 20
Dim Integer Show_Friendly = 0 ' flag
Dim Integer Show_Names = 0 ' flag
Dim Integer Show_Cars = 0 ' flag
Dim Integer Show_Legend = 1 ' flag

' NEW: Switch and Crossing Storage
Dim Float Switch_X(MAX_SWITCHES), Switch_Y(MAX_SWITCHES)
Dim Float Cross_X, Cross_Y
Dim Integer Has_Cross = 0
Dim Integer Trace_Mode = 0
Dim Integer Force_Redraw = 1 ' flag

Dim String Seg_N1$(MAX_RECS) length 20
Dim String Seg_N2$(MAX_RECS) length 20
Dim String Seg_N3$(MAX_RECS) length 20

' --- DIRTY SEGMENT TRACKING ---
Dim Integer isErase_Global = 0 ' flag
Dim Integer Dirty_Seg(MAX_RECS)

' --- TURNTABLE TRACKING ---
Dim Float RH_CX, RH_CY, RH_Len, RH_BaseAng
Dim Float RH_Orig_X1, RH_Orig_Y1, RH_Orig_X2, RH_Orig_Y2
Dim Float RH_Angle = 0.0
Dim Float RH_Target = 0.0
Dim Integer RH_Idx = 0
Dim Integer Override_Color = -1
Dim Integer rh_moving = 0

' --- SCRIPT ENGINE GLOBALS ---
Dim String Script_Lines$(MAX_SCRIPT_LINES) length 39
Dim Integer Script_Line_Map(MAX_SCRIPT_LINES)
Dim Integer Script_Count = 0
Dim Integer Script_Idx = 0
Dim Float Script_Wait_Time = 0.0
Dim String Script_Wait_Event$ = ""
Dim Integer Is_Recording = 0 ' flag
Dim String Record_File$ = ""

' --- GRAPH CACHE ---
Dim String FastSeg_ID$(5)
Dim Integer FastSeg_Idx(5)
Dim Integer FastSeg_Ptr = 1

Dim Integer Sys_Paused = 0 ' flag

' =============================================================
' MAIN COMMAND LOOP
' =============================================================
If DBG_LVL>2 then Print "Diag0: MAX_RECS: ";MAX_RECS;"; Heap: ";mm.info(heap)
Print "Initializing Database Engine... " + MM.Info(CURRENT)
Option CONSOLE SERIAL ' in program only--not a console option which remais persistant
IF instr(mm.info$(lcdpanel),"MODE") then
mode GLOBAL_MODE ' set video mode for HDMI or VGA
Print mm.device$+" ";mm.info(version);" Video Mode ";GLOBAL_MODE
Else
Is_LCD=1 ' LCD--not HDMI/DVI or VGA
endif
If DBG_LVL>2 then Print "Diag0: MAX_RECS: ";MAX_RECS;"; Heap: ";mm.info(heap)
DummySub

Do
Loop
End

Sub DummySub
End Sub

Diag0: MAX_RECS: 920; Heap: 6291712
Initializing Database Engine... NONE
Diag0: MAX_RECS: 920; Heap: 6291712
[272] Sub DummySub
Error : Not enough Heap memory

No library is present

> memory
Program:
8K ( 2%) Program (273 lines)
320K (98%) Free

Saved Variables:
16K (100%) Free

RAM:
206K ( 3%) 221 Variables
135K ( 2%) General
6178K (95%) Free
> ?mm.info(heap)
6292224
>

I don't know if running this on a different device would produce a different error. If so, changing MAX_RECS might make it fail at this point. I don't have a non-WeAct Pico2 with PSRAM to test with.

PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS

Posted: 08:12am 02 Apr 2026 Copy link to clipboard	matherp Guru

When PSRAM is enabled there are two heap memory pools. One in main RAM and one in PSRAM. For performance reasons internal functions that use memory always only use normal RAM so in your example calling a subroutine needs memory and always gets it from the normal ram pool. Variable memory usage will first try to use normal RAM as it is faster and if not enough is available will then use PSRAM. It will automatically use PSRAM for a variable if the request for memory is more than half the size of the RAM pool (i.e. big arrays).
You have created an edge case. You have lots of relatively small variables that are filling the heap in normal RAM and not triggering the condition that places them in PSRAM. Then when the system needs RAM for internal use there isn't any hence the error.
I can't see an easy solution that is performant. The obvious answer would be to place variables in PSRAM if the heap in normal ram falls below a certain level. The problem with this is that calling the routine to calculate how much normal ram is left would massively slow down variable creation. This wouldn't matter too much for global variables but would be a disaster for local variables.

Posted: 12:38pm 02 Apr 2026 Copy link to clipboard	lizby Guru

Thanks for looking into it. Perhaps a solution will make itself known.

The problem as I see it is the array sizes set by MAX_RECS. The number of variables doesn't seem to be an issue if MAX_RECS is small enough.

These arrays of structures chew up a lot of heap.

Dim IndexItemNum DB_IndexNum(MAX_RECS) ' a structure, size 16
Dim IndexItem DB_Index(MAX_RECS) ' a structure, size 40
Dim IndexItem DB_Lookup(MAX_RECS) ' a structure, size 40
Dim IndexItemNum DB_LookupNum(MAX_RECS) ' a structure, size 16
Dim IndexItem Result_List(MAX_RECS) ' a structure, size 40

For MAX_RECS of 1000, that would be about 150K. I'd like for MAX_RECS to be a maximum of 2000.

If there were a way to make sure they went into PSRAM, it would probably solve the problem.

Something like Dim PSRAM IndexItem DB_Index(MAX_RECS) might do it.

~
Edited 2026-04-02 22:44 by lizby

Posted: 12:59pm 02 Apr 2026 Copy link to clipboard	matherp Guru

I assume you are running standard PicoMiteRP2350 firmware? What happens if you set MAX_RECS very big, say 20,000

Posted: 02:49pm 02 Apr 2026 Copy link to clipboard	lizby Guru

WeAct RP2350B 48-pin module with ILI9488 LCD:

Quote PicoMite MMBasic RP2350B V6.02.01RC5
OPTION SYSTEM SPI GP18,GP19,GP16
OPTION SYSTEM I2C GP10,GP11
OPTION FLASH SIZE 16777216
OPTION LIBRARY_FLASH_SIZE 52000
OPTION COLOURCODE ON
OPTION PICO OFF
OPTION CPUSPEED (KHz) 340000
OPTION DISPLAY 48, 133
OPTION LCDPANEL ILI9488, LANDSCAPE,GP24,GP23,GP21
OPTION TOUCH GP27,GP29
OPTION SDCARD GP22
OPTION PSRAM PIN GP0

MAX_RECS=5000 works for stub program; also works for PicoRR+PicoDB:

?MAX_RECS,MM.INFO(HEAP)
5000 5166336

So it looks like those big arrays of structures are in PSRAM now. And all with the change in the value of a single variable.

Quote > memory
Program:
88K (26%) Program (2489 lines)
240K (74%) Free

Library:
51K (15%) Library
277K (85%) Free

Saved Variables:
16K (100%) Free

RAM:
909K (13%) 279 Variables
455K ( 6%) General
5155K (81%) Free

Thanks so much for all your efforts.

Posted: 09:39pm 02 Apr 2026 Copy link to clipboard	bfwolf Senior Member

matherp said When PSRAM is enabled there are two heap memory pools. One in main RAM and one in PSRAM. For performance reasons internal functions that use memory always only use normal RAM so in your example calling a subroutine needs memory and always gets it from the normal ram pool. Variable memory usage will first try to use normal RAM as it is faster and if not enough is available will then use PSRAM. It will automatically use PSRAM for a variable if the request for memory is more than half the size of the RAM pool (i.e. big arrays).
You have created an edge case. You have lots of relatively small variables that are filling the heap in normal RAM and not triggering the condition that places them in PSRAM. Then when the system needs RAM for internal use there isn't any hence the error.
I can't see an easy solution that is performant. The obvious answer would be to place variables in PSRAM if the heap in normal ram falls below a certain level. The problem with this is that calling the routine to calculate how much normal ram is left would massively slow down variable creation. This wouldn't matter too much for global variables but would be a disaster for local variables.

I've been thinking about your dilemma regarding RAM usage, and the following idea came to me:

Why not consider a directive that allows you to control RAM priority from within the program? In other words, make the priority of using fast internal RAM and slow QSPI RAM controllable for subsequent DIM commands?

This could be done e.g. using an OPTION that would be valid for all DIM commands until the "opposite" OPTION is used.

For example:

OPTION RAMPRIORITY [FASTRAM | SLOWRAM]

Instead of controlling it via OPTION, you could also introduce true directives, similar to how it's done in Turbo Pascal/Delphi: through a "special syntax" of comments. This would have the advantage of being compatible with other BASIC dialects.

I did some research: The FreeBasic compiler also does something similar using "meta-commands" that begin with the 2-character-sequence '$ . I assume that FreeBasic additionally expects the sequence '$ to be at the beginning of a line (i.e. the first non-whitespace position).

https://www.freebasic.net/wiki/CatPgCompilerSwitches

There are then meta-commands like '$DYNAMIC, '$LANG, etc.

Following this scheme, one could, for example, introduce the meta-commands

'$RAMPRIORITY FASTRAM
and
'$RAMPRIORITY SLOWRAM

in MMBasic.

I wouldn't bring the term PSRAM into this context, because that would be too specifically tailored to microcontrollers with PSRAM. And slow external RAMs with different interfaces are also common in other controllers.

How do you think about it?

Regards, bfwolf.

Posted: 10:31pm 02 Apr 2026 Copy link to clipboard	lizby Guru

You were probably addressing Peter, but FYI, I haven't tested everything, but right now the programs appear to be working with just the 3 biggest arrays of structure given big enough dimensions that they are now going into PSRAM, as Peter suggested. I had to fiddle some to get it right.

It might have to be tunable by program. The database test program, us500, doesn't need it at all. PicoRR does.

I do fear it may be a continuing problem. Pico2s without PSRAM don't work for PicoRR--that was my first encounter with the heap error--but they do for the database application. I am hoping that growth of the application isn't too much restricted.

Posted: 07:16am 03 Apr 2026 Copy link to clipboard	Mixtel90 Guru

If you need vast amounts of RAM for a database then you and your AI friend are doing something wrong. ;)

CP/M could run some pretty big databases under dBASE II (65535 records IIRC) with less than 60k. It's been a long time, but I don't think you ever had more than the core routines, the current sort routine and the sorted index file in RAM at the same time normally. You don't even have the whole program in RAM at once. All the data and the overlaid commands were on disk (but you had a 32MB hardware limit!).

Ok, 65535 records is probably a bit puny for some now, but it was very big at the time. And a Z80 couldn't count past &hFFFF in a register pair. :)

Posted: 08:30am 03 Apr 2026 Copy link to clipboard	matherp Guru

An option to do with memory allocation would slow the PicoMite down. This code is heavily tuned and not going to change. lizby found an edge case for which we found a workaround so no need to do anything else.

Posted: 08:33am 03 Apr 2026 Copy link to clipboard	bfwolf Senior Member

Mixtel90 said If you need vast amounts of RAM for a database then you and your AI friend are doing something wrong. ;)

CP/M could run some pretty big databases under dBASE II (65535 records IIRC) with less than 60k.

I completely agree with you! AI is indeed "quite naive" in many respects.

To manage large datasets, it would certainly be better to keep only a portion of the data in RAM — for example, in a "window" buffering the current position of the table being processed. Or a more intelligent cache that can buffer multiple areas using an LRU strategy. Nevertheless, it's crucial to remember that the number of write operations on flash filesystems must be minimized. Unlike magnetic storage media or RAM, they don't survive an unlimited number of writes — not to mention that writes on flash are very slow. With magnetic storage media, writes are only "imperceptibly slower" and practically unlimited.

And of course, there are other conceivable use cases where a lot of information needs to be stored in RAM. Therefore, it would be fantastic to be able to control whether slow or fast RAM should be prioritized for specific variables. As an application programmer, you can assess whether certain variables are speed-critical or not.

Mixtel90 said It's been a long time, but I don't think you ever had more than the core routines, the current sort routine and the sorted index file in RAM at the same time normally. You don't even have the whole program in RAM at once. All the data and the overlaid commands were on disk (but you had a 32MB hardware limit!).

Ok, 65535 records is probably a bit puny for some now, but it was very big at the time. And a Z80 couldn't count past &hFFFF in a register pair. :)

Code overlays were one of the reasons why many programs were so terribly slow back then.

I'll just mention the C64 with its floppy drive and the sluggish IEC serial bus. Of course, you were often just glad it worked at all.

bfwolf

Posted: 12:49pm 03 Apr 2026 Copy link to clipboard	Mixtel90 Guru

We have the luxury of Library routines though, consuming virtually no user RAM. We don't need overlays. The data is best stored on SD card, not flash. It's far easier to back up then even if it's slower than A:. It's bigger too. :)

Posted: 02:19pm 03 Apr 2026 Copy link to clipboard	lizby Guru

Mixtel90 said CP/M could run some pretty big databases under dBASE II (65535 records IIRC)

Around about 1983 I went to a trade show with my CP/M Z80 database handler, C code. Someone else there had a dBase II database with similar data. They'd enter a query and there'd be wait ... wait ... wait ... and a screen of data would show. Mine with a like query would show a screen full as fast as you could hit the <Enter> key--with data on floppy disk.

If you want to put your indices on storage media, you have the wait ... problem--although flash access on the RP2350B is orders of magnitude faster than floppy disks.

There are trade-offs.

Posted: 02:46pm 03 Apr 2026 Copy link to clipboard	Mixtel90 Guru

Oh yeah, I realize that. The point I'm making is that database size isn't necessarily proportional to the amount of RAM available.

You can have blindingly fast databases if all the data is in RAM, but you have a delay while it is all read into RAM arrays or a block of RAM at the beginning and again after every write operation while the modified RAM is saved to disk or flash. The database size is severely restricted.

You can have almost as fast databases with the data stored in flash if you don't mind the wear and tear on the flash (and, preferably, force the user to do a backup after a predetermined number of write operations). The database size isn't quite so restricted as a RAM system. Using on-board flash for the database may not be a great idea due to the wear problem, even if it's fast.

It's all compromises depending on what the application is. :)

To reply to this topic, you need to log in.