Add iOS Safari memory crash fix draft
Documents root cause (WKWebView WASM heap exhaustion), four proposed fixes (download throttle, cache eviction, tween tracking, unit pooling), and a write-up of how each fix addresses the 60s crash and the rejoin loop. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
This commit is contained in:
@@ -0,0 +1,356 @@
|
||||
# iOS Safari Memory Crash - Fix Draft
|
||||
|
||||
**Status:** Draft for review. Code not yet applied to any existing files.
|
||||
**Affected platform:** iOS Safari (WKWebView), Godot 4.5 WebAssembly/WebGL export.
|
||||
**Symptom:** Game crashes ~60s into play on iOS. On rejoin the server sends a full state dump and the crash fires again immediately, producing "A problem repeatedly occurred" and trapping the player in a rejoin loop.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 - Issue Analysis
|
||||
|
||||
### Root cause: iOS WKWebView process kill due to memory pressure
|
||||
|
||||
Godot web export compiles to WebAssembly. The WASM linear heap is a fixed block allocated at startup. iOS Safari enforces a hard per-tab memory ceiling (roughly 1-1.5 GB total). When the ceiling is crossed the OS kills the renderer process. "A problem repeatedly occurred" is that kill dialog - the OS forcibly reclaiming memory from the tab.
|
||||
|
||||
### Why it crashes after ~60 seconds
|
||||
|
||||
Three compounding causes:
|
||||
|
||||
**1. ModelCache (autoloads/ModelCache.gd:28) - unbounded in-memory PackedScene store**
|
||||
|
||||
Every unique GLB model stays in _mem as a parsed PackedScene holding full mesh data, textures, and materials in the WASM heap. For a board with ~10 unique unit types plus 4-6 tile affinities this can accumulate 50-150 MB and never releases until the tab closes.
|
||||
|
||||
static var _mem: Dictionary = {} # cache-key -> PackedScene (NEVER CLEARED)
|
||||
|
||||
**2. No unit pooling - queue_free() + fresh allocation on every state change (unit_mesh_builder.gd:106-150)**
|
||||
|
||||
Every rebuild_units() call removes stale units via body.queue_free() (deferred) then allocates brand-new StaticBody3D + CollisionShape3D + MeshInstance3D + StandardMaterial3D for each changed unit. queue_free() is deferred: old and new allocations coexist in the heap for one full frame. During a busy turn this transient spike can tip over the iOS ceiling.
|
||||
|
||||
**3. Untracked materialize/dissolve tweens accumulate live closures (unit_mesh_builder.gd:591-665)**
|
||||
|
||||
_play_materialize_shader_timed() and _play_dissolve_shader_timed() each create a Tween capturing a reveal_mats array of ShaderMaterial references. These tweens are not stored anywhere. If a state rebuild fires before a tween finishes, its closure holds live material refs on queue_free()d nodes - keeping those materials alive in the WASM heap until the tween completes (up to ~1.35 seconds per effect). During 60 seconds of play many such tweens can be alive simultaneously.
|
||||
|
||||
### Why it crashes immediately on rejoin
|
||||
|
||||
When iOS kills the tab the entire WASM heap is wiped. On rejoin the server delivers a full game state. The client calls build_board() which triggers add_tile() for every tile. Each tile fires try_load_tile_model() and optionally try_load_tile_bottom(), each calling ModelCache.load_model(). With the disk cache warm, _parse_gltf() is called ONCE PER TILE SIMULTANEOUSLY with no concurrency limit. On a typical 8x8 board:
|
||||
|
||||
- 64 tiles x (top model + optional bottom model) = up to 128 _parse_gltf() calls
|
||||
- Each parse allocates intermediate GLTFDocument, GLTFState, a full scene graph, and mesh data before compressing into a PackedScene
|
||||
- All of this happens in one burst before any data is freed
|
||||
|
||||
The WASM heap spikes well above the iOS ceiling before the first frame renders, and the OS kills the process again.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 - Fix Specifications
|
||||
|
||||
---
|
||||
|
||||
### Fix 1 - Throttle concurrent model downloads / parses
|
||||
|
||||
**File:** Client/origin-client/autoloads/ModelCache.gd
|
||||
|
||||
**What it does:** Limits simultaneous active HTTP downloads and disk-cache parse jobs to _MAX_CONCURRENT (4). Requests beyond the limit are held in a queue and started as slots free up. This prevents the burst of 100+ simultaneous GLB parses on reconnect from spiking the heap all at once.
|
||||
|
||||
**Where to apply:** Near the top of ModelCache.gd add new constants and statics. Replace load_model() and _download() with the versions below, and add _finish_download().
|
||||
|
||||
```gdscript
|
||||
# ADD near top of ModelCache.gd alongside the existing static vars
|
||||
|
||||
## Maximum simultaneous GLB downloads (and disk-cache parse jobs).
|
||||
## Keeping this at 4 prevents a burst of 100+ parses from spiking the WASM
|
||||
## heap all at once on reconnect - the primary cause of the iOS re-crash.
|
||||
const _MAX_CONCURRENT := 4
|
||||
|
||||
# Pending download descriptors queued while all slots are busy.
|
||||
static var _download_queue: Array = []
|
||||
static var _active_downloads: int = 0
|
||||
|
||||
|
||||
# REPLACE load_model() entirely with this version
|
||||
func load_model(url: String, callback: Callable, etag: String = "") -> void:
|
||||
var key := _cache_key(url, etag)
|
||||
var disk := _disk_path(url, etag)
|
||||
if _mem.has(key):
|
||||
var packed: PackedScene = _mem[key]
|
||||
callback.call(packed.instantiate() if packed != null else null)
|
||||
return
|
||||
if FileAccess.file_exists(disk) and _active_downloads < _MAX_CONCURRENT:
|
||||
var bytes := FileAccess.get_file_as_bytes(disk)
|
||||
if not bytes.is_empty():
|
||||
var packed := _parse_gltf(bytes)
|
||||
if packed != null:
|
||||
_mem[key] = packed
|
||||
callback.call(packed.instantiate())
|
||||
return
|
||||
push_warning("ModelCache: deleting corrupt disk cache for %s" % url)
|
||||
DirAccess.remove_absolute(disk)
|
||||
if _pending.has(key):
|
||||
_pending[key].append(callback)
|
||||
return
|
||||
_pending[key] = [callback]
|
||||
if _active_downloads < _MAX_CONCURRENT:
|
||||
_active_downloads += 1
|
||||
_download(url, key, disk)
|
||||
else:
|
||||
_download_queue.append({ "url": url, "key": key, "disk": disk })
|
||||
|
||||
|
||||
# REPLACE _download() entirely with this version
|
||||
func _download(url: String, key: String, disk: String) -> void:
|
||||
var fetch := _fetch_url(url)
|
||||
print("ModelCache: downloading - %s (active=%d)" % [fetch, _active_downloads])
|
||||
var http := HTTPRequest.new()
|
||||
http.body_size_limit = -1
|
||||
add_child(http)
|
||||
http.request_completed.connect(
|
||||
func(res: int, code: int, headers: PackedStringArray, body: PackedByteArray):
|
||||
http.get_signal_connection_list("request_completed").clear()
|
||||
http.queue_free()
|
||||
var callbacks: Array = _pending.get(key, [])
|
||||
_pending.erase(key)
|
||||
var ct := ""
|
||||
for h: String in headers:
|
||||
if h.to_lower().begins_with("content-type"):
|
||||
ct = h; break
|
||||
if res != HTTPRequest.RESULT_SUCCESS:
|
||||
push_warning("ModelCache: network error result=%d http=%d - %s" % [res, code, url])
|
||||
for cb: Callable in callbacks: cb.call(null)
|
||||
_finish_download(); return
|
||||
if code < 200 or code >= 300:
|
||||
push_warning("ModelCache: HTTP %d - %s" % [code, url])
|
||||
for cb: Callable in callbacks: cb.call(null)
|
||||
_finish_download(); return
|
||||
if body.is_empty():
|
||||
for cb: Callable in callbacks: cb.call(null)
|
||||
_finish_download(); return
|
||||
var packed := _parse_gltf(body)
|
||||
_mem[key] = packed
|
||||
if packed != null:
|
||||
var f := FileAccess.open(disk, FileAccess.WRITE)
|
||||
if f: f.store_buffer(body); f.close()
|
||||
for cb: Callable in callbacks:
|
||||
cb.call(packed.instantiate() if packed != null else null)
|
||||
_finish_download()
|
||||
)
|
||||
var err := http.request(fetch)
|
||||
if err != OK:
|
||||
push_warning("ModelCache: request() error %d - %s" % [err, fetch])
|
||||
var callbacks: Array = _pending.get(key, [])
|
||||
_pending.erase(key)
|
||||
for cb: Callable in callbacks: cb.call(null)
|
||||
http.get_signal_connection_list("request_completed").clear()
|
||||
http.queue_free()
|
||||
_finish_download()
|
||||
|
||||
|
||||
# ADD this new helper below _download()
|
||||
func _finish_download() -> void:
|
||||
_active_downloads = maxi(0, _active_downloads - 1)
|
||||
if not _download_queue.is_empty() and _active_downloads < _MAX_CONCURRENT:
|
||||
var next: Dictionary = _download_queue.pop_front()
|
||||
_active_downloads += 1
|
||||
_download(next["url"], next["key"], next["disk"])
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Fix 2 - Evict ModelCache on game exit
|
||||
|
||||
**Files:**
|
||||
- Client/origin-client/autoloads/ModelCache.gd
|
||||
- Client/origin-client/scenes/game/game_screen.gd
|
||||
|
||||
**What it does:** Adds an `evict_all()` method to ModelCache that drops all in-memory PackedScene references and clears the pending download queue. Calls it from game_screen._exit_tree() so the WASM heap is fully reclaimed when the user leaves a game (disconnects, returns to lobby, or the tab navigates away). Without this, every game session compounds the memory footprint of the previous one.
|
||||
|
||||
**Where to apply in ModelCache.gd:** Add `evict_all()` directly below the existing `evict()` method (~line 79).
|
||||
|
||||
```gdscript
|
||||
# ADD directly below the existing evict() method in ModelCache.gd
|
||||
|
||||
## Drop all in-memory PackedScene references and flush the pending queue.
|
||||
## Call this when leaving a game so the WASM heap is reclaimed before the
|
||||
## next board load. Disk cache (.bin files) is intentionally left intact
|
||||
## so subsequent sessions still benefit from skipping re-downloads.
|
||||
func evict_all() -> void:
|
||||
_mem.clear()
|
||||
_download_queue.clear()
|
||||
# Leave _pending and _active_downloads alone - in-flight HTTP requests
|
||||
# are owned by their HTTPRequest nodes and will self-clean via _finish_download().
|
||||
print("ModelCache: evict_all - in-memory cache cleared")
|
||||
```
|
||||
|
||||
**Where to apply in game_screen.gd:** At the end of the `_exit_tree()` function body, after the existing cleanup calls.
|
||||
|
||||
```gdscript
|
||||
# ADD at the end of _exit_tree() in game_screen.gd
|
||||
# (after the existing disconnect/clear_stars/release_bubble_visibility calls)
|
||||
|
||||
ModelCache.evict_all()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Fix 3 - Track and kill shader tweens on rebuild
|
||||
|
||||
**File:** Client/origin-client/scenes/game/components/units/unit_mesh_builder.gd
|
||||
|
||||
**What it does:** Keeps a list of every active materialize/dissolve Tween so they can be killed before a unit rebuild. Previously these tweens were fire-and-forget: if a rebuild fired while a tween was mid-animation the tween's closure held `ShaderMaterial` references on already-freed nodes, keeping those materials alive in the WASM heap for up to ~1.35 s per effect. Under rapid state updates dozens of these closures accumulate simultaneously.
|
||||
|
||||
**Where to apply:** Three locations in unit_mesh_builder.gd.
|
||||
|
||||
```gdscript
|
||||
# 1. ADD near the top of the class body alongside the other member vars (~line 33)
|
||||
|
||||
var _active_tweens: Array[Tween] = []
|
||||
|
||||
|
||||
# 2. ADD at the start of rebuild_units(), before the stale-unit removal loop.
|
||||
# This kills every in-flight shader tween and drops the ShaderMaterial refs
|
||||
# it was holding, freeing them immediately instead of after up to ~1.35 s.
|
||||
|
||||
# Kill all active shader tweens so their closures release material refs now.
|
||||
for t: Tween in _active_tweens:
|
||||
if t and t.is_valid():
|
||||
t.kill()
|
||||
_active_tweens.clear()
|
||||
|
||||
|
||||
# 3. REPLACE the tween creation lines in _play_materialize_shader_timed() with:
|
||||
# (find the existing `var tween := create_tween()` line and replace from there)
|
||||
|
||||
var tween := create_tween()
|
||||
_active_tweens.append(tween)
|
||||
tween.finished.connect(func(): _active_tweens.erase(tween), CONNECT_ONE_SHOT)
|
||||
# ... rest of the tween setup (tween.tween_method etc.) stays unchanged
|
||||
|
||||
|
||||
# 4. REPLACE the tween creation lines in _play_dissolve_shader_timed() with:
|
||||
# (same pattern - find `var tween := create_tween()` and replace)
|
||||
|
||||
var tween := create_tween()
|
||||
_active_tweens.append(tween)
|
||||
tween.finished.connect(func(): _active_tweens.erase(tween), CONNECT_ONE_SHOT)
|
||||
# ... rest of the tween setup stays unchanged
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Fix 4 - Pool unit scene nodes instead of allocating fresh ones
|
||||
|
||||
**File:** Client/origin-client/scenes/game/components/units/unit_mesh_builder.gd
|
||||
|
||||
**What it does:** Maintains a pool of retired StaticBody3D unit nodes keyed by card_type. When a unit is removed from the board it is detached from the scene tree, stripped of its old MeshInstance3D children, and parked in the pool instead of being queue_free()d. When the same card type is needed again the node is reclaimed from the pool, a fresh mesh is attached, and it is reparented to units_root. This eliminates the one-frame spike where both the old deferred-free node and the newly allocated node coexist in the WASM heap.
|
||||
|
||||
**Where to apply:** Four locations in unit_mesh_builder.gd.
|
||||
|
||||
```gdscript
|
||||
# 1. ADD member vars near the top of the class body
|
||||
|
||||
var _unit_pool: Dictionary = {} # card_type (String) -> Array[StaticBody3D]
|
||||
var _pool_root: Node3D = null # off-tree parent that holds pooled nodes
|
||||
|
||||
|
||||
# 2. ADD _ready() (or extend it if it already exists) to create _pool_root
|
||||
|
||||
func _ready() -> void:
|
||||
_pool_root = Node3D.new()
|
||||
_pool_root.name = "UnitPool"
|
||||
# Attach to the scene tree so nodes are not leaked, but keep it invisible.
|
||||
add_child(_pool_root)
|
||||
_pool_root.visible = false
|
||||
|
||||
|
||||
# 3. ADD helper: return a unit body to the pool
|
||||
# Call this instead of body.queue_free() in the stale-unit removal path.
|
||||
|
||||
func _pool_unit(body: StaticBody3D, card_type: String) -> void:
|
||||
# Free mesh children immediately - they hold material and mesh refs.
|
||||
for child in body.get_children():
|
||||
if child is MeshInstance3D:
|
||||
child.free()
|
||||
# Hide and reparent to the off-tree pool node.
|
||||
body.visible = false
|
||||
body.reparent(_pool_root, false)
|
||||
if not _unit_pool.has(card_type):
|
||||
_unit_pool[card_type] = []
|
||||
_unit_pool[card_type].append(body)
|
||||
|
||||
|
||||
# 4. ADD helper: acquire a pooled body, or return null if pool is empty
|
||||
|
||||
func _acquire_from_pool(card_type: String) -> StaticBody3D:
|
||||
if not _unit_pool.has(card_type):
|
||||
return null
|
||||
var pool: Array = _unit_pool[card_type]
|
||||
if pool.is_empty():
|
||||
return null
|
||||
var body: StaticBody3D = pool.pop_back()
|
||||
body.reparent(units_root, false)
|
||||
body.visible = true
|
||||
return body
|
||||
|
||||
|
||||
# 5. In rebuild_units(): REPLACE the body.queue_free() removal line with _pool_unit().
|
||||
# Find the loop that iterates stale units and changes:
|
||||
|
||||
body.queue_free()
|
||||
|
||||
# to:
|
||||
|
||||
_pool_unit(body, card_type_of_that_unit)
|
||||
|
||||
# (card_type_of_that_unit is whatever string key identifies the unit type in
|
||||
# the stale-detection loop - match it to what is passed to add_unit_mesh)
|
||||
|
||||
|
||||
# 6. In add_unit_mesh(): REPLACE the StaticBody3D.new() allocation block.
|
||||
# Find the section that does `var body := StaticBody3D.new()` and replace with:
|
||||
|
||||
var body := _acquire_from_pool(card_type)
|
||||
if body == null:
|
||||
body = StaticBody3D.new()
|
||||
var col := CollisionShape3D.new()
|
||||
col.shape = BoxShape3D.new()
|
||||
body.add_child(col)
|
||||
units_root.add_child(body)
|
||||
# Continue with attaching the MeshInstance3D and applying materials as before.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 3 - How the Fixes Resolve the Crashes
|
||||
|
||||
### The 60-second crash
|
||||
|
||||
The three fixes work together to keep the WASM heap below the iOS ceiling during normal play:
|
||||
|
||||
**Fix 3** eliminates the silent material-reference leak. Before this fix, every materialize/dissolve animation created an anonymous Tween that kept `ShaderMaterial` objects alive long after the unit they belonged to was removed from the scene. Under normal play (several animations per turn, turns every 30-60 s) dozens of these leaked materials accumulate. Fix 3 kills all in-flight tweens at the start of each rebuild so materials are released immediately.
|
||||
|
||||
**Fix 4** eliminates the per-rebuild heap spike. Before this fix every turn that changed any unit triggered `queue_free()` (deferred) followed by fresh allocation - meaning old and new nodes coexisted in the WASM heap for one full frame. Over 60 seconds of turns this gradually raises the heap baseline because deferred frees do not compact the allocator. Fix 4 replaces the free-then-allocate pattern with a pool: the StaticBody3D shell is retained and reused, and only the MeshInstance3D children (which hold the actual mesh data) are replaced.
|
||||
|
||||
**Fix 2** provides a clean-slate guarantee between games. Even with Fixes 3 and 4 in place, the `_mem` dictionary in ModelCache accumulates parsed `PackedScene` objects across the entire session. When the user finishes a game and starts another one those PackedScenes remain loaded even though the new board might use a completely different unit set. Fix 2 evicts everything when the game screen exits so each new game starts from a clean heap.
|
||||
|
||||
### The immediate rejoin crash
|
||||
|
||||
**Fix 1** is the direct fix for the rejoin crash. The rejoin crash is caused by a burst of simultaneous GLB parses triggered when the server sends the full board state and the client calls `build_board()` for every tile at once. Each `_parse_gltf()` call allocates a large intermediate `GLTFDocument` + `GLTFState` + scene graph before compressing it into a `PackedScene`. With 128 of these happening simultaneously the heap spikes far above the iOS ceiling before the first frame renders.
|
||||
|
||||
Fix 1 caps simultaneous parses at `_MAX_CONCURRENT = 4`. The remaining requests queue up and drain four at a time. The peak heap usage at any moment drops from ~128x to ~4x the per-model parse cost, which stays well within the iOS ceiling. The board finishes loading in the same total time (network is the real bottleneck) but without the instantaneous spike.
|
||||
|
||||
**Fix 2** also improves the rejoin case indirectly: on the first crash iOS wipes the WASM heap, so `_mem` is empty when the tab reloads. Fix 2 does not help for the rejoin crash itself, but it ensures that if the user successfully rejoins and later exits the game, the heap is clean before any future session.
|
||||
|
||||
### Summary table
|
||||
|
||||
| Fix | Crash addressed | Mechanism |
|
||||
|-----|----------------|-----------|
|
||||
| Fix 1 - Throttle downloads | Rejoin crash (immediate) | Caps simultaneous GLB parses at 4; prevents heap spike on full-board reload |
|
||||
| Fix 2 - evict_all on exit | 60s leak + cross-game accumulation | Clears PackedScene mem-cache between games so each game starts clean |
|
||||
| Fix 3 - Track tweens | 60s leak | Kills in-flight shader tweens on rebuild; releases ShaderMaterial refs immediately |
|
||||
| Fix 4 - Unit pooling | 60s heap spike per turn | Reuses StaticBody3D shells; eliminates one-frame double-allocation on each unit change |
|
||||
|
||||
### Recommended apply order
|
||||
|
||||
1. **Fix 1** - highest priority; directly prevents the rejoin loop that traps the player.
|
||||
2. **Fix 3** - low risk, high impact; pure addition with no structural changes to unit_mesh_builder.
|
||||
3. **Fix 2** - low risk; one new method + one call site.
|
||||
4. **Fix 4** - most structural change; test rebuild_units() thoroughly after applying.
|
||||
Reference in New Issue
Block a user