Files
sor/ios_crash_fix_draft_v2.md
8325370069 Add iOS Safari memory crash fix draft v2
Patches all 8 gaps from code review: CONNECT_ONE_SHOT replaces no-op
signal clear, evict_all flushes pending callbacks, disk-cache hits queue
correctly without re-downloading, pool reset covers transform/meta/groups,
pool size is capped, instrumentation added, _exit_tree cleanup added to
unit_mesh_builder for both tweens and pool nodes.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
2026-05-09 09:54:54 -04:00

22 KiB

iOS Safari Memory Crash - Fix Draft v2

Status: Draft v2 — addresses 8 review gaps from v1. Code not yet applied to any existing files. Affected platform: iOS Safari (WKWebView), Godot 4.5 WebAssembly/WebGL export. Symptom: Game crashes ~60s into play on iOS. On rejoin the server sends a full state dump and the crash fires again immediately, producing "A problem repeatedly occurred" and trapping the player in a rejoin loop.


Changes from v1

# Review gap v1 status v2 fix
1 get_signal_connection_list().clear() is a no-op Bug carried from original source Replaced with CONNECT_ONE_SHOT
2 evict_all() leaves _pending alive — callbacks fire into freed scene Missing Flush _pending with null on evict
3 Disk-cache hits silently re-download when slots full Bug in v1 draft Separate parse-slot path from network path
4 Pool reuse missing transform/meta reset Missing Full reset in _acquire_from_pool()
5 Pool has no size cap Missing Per-type cap + overflow free()
6 No instrumentation plan Missing Debug-gated counters on all hot paths
7 Null fallback for parse storm Caller responsibility, unverified Confirmed + noted in patch
8 unit_mesh_builder._exit_tree() hard cleanup Missing Added with tween kill + pool clear

Part 1 - Issue Analysis

(unchanged from v1 — root cause and failure modes are correctly identified)

Root cause: iOS WKWebView process kill due to memory pressure

Godot web export compiles to WebAssembly. The WASM linear heap is a fixed block allocated at startup. iOS Safari enforces a hard per-tab memory ceiling (roughly 1-1.5 GB total). When the ceiling is crossed the OS kills the renderer process. "A problem repeatedly occurred" is that kill dialog - the OS forcibly reclaiming memory from the tab.

Why it crashes after ~60 seconds

1. ModelCache (autoloads/ModelCache.gd:28) — unbounded in-memory PackedScene store

Every unique GLB model stays in _mem as a parsed PackedScene holding full mesh data, textures, and materials in the WASM heap. Never released until the tab closes.

2. No unit pooling — queue_free() + fresh allocation on every state change (unit_mesh_builder.gd:178)

Every rebuild removes stale units via body.queue_free() (deferred) then allocates brand-new StaticBody3D + CollisionShape3D + MeshInstance3D + StandardMaterial3D. Old and new coexist in the heap for one full frame.

3. Untracked materialize/dissolve tweens accumulate live closures (unit_mesh_builder.gd:614, 657)

_play_materialize_shader_timed() and _play_dissolve_shader_timed() each create a Tween that is not stored. If a rebuild fires while a tween runs, its closure holds live ShaderMaterial refs for up to ~1.35 s.

Why it crashes immediately on rejoin

build_board() triggers add_tile() for every tile simultaneously. Each tile calls ModelCache.load_model(). With the disk cache warm, _parse_gltf() is called ONCE PER TILE with no concurrency limit — up to 128 simultaneous parses — spiking the WASM heap above the iOS ceiling before the first frame renders.


Part 2 - Fix Specifications (v2)


Fix 1 — Throttle concurrent parse/download jobs

File: Client/origin-client/autoloads/ModelCache.gd

What it does:

  • Caps simultaneous jobs (network downloads AND disk-cache parses) at _MAX_JOBS (4). Both are parse-budget consumers.
  • Names the counters _active_jobs / _job_queue to make the shared budget explicit and avoid future confusion.
  • Replaces get_signal_connection_list("request_completed").clear() — which is a no-op in Godot 4 (returns a copy) — with CONNECT_ONE_SHOT, which auto-disconnects after the first fire.
  • Disk-cache hits when all slots are busy now queue a parse-only descriptor ("parse_from_disk": true) so they drain correctly when a slot opens, instead of silently re-downloading.
  • Adds _finish_job() to drain the queue after each job completes.
  • Adds debug-gated logging via OS.is_debug_build().

Where to apply: Replace the top statics, load_model(), _download(), and add _finish_job() in ModelCache.gd.

# ── ADD near top of ModelCache.gd alongside existing static vars ──────────────

## Shared parse-budget cap. Both HTTP downloads and disk-cache parses count
## against this limit because both call _parse_gltf() and spike the WASM heap.
## Renamed from _active_downloads to make the scope clear.
const _MAX_JOBS := 4

# Jobs queued while all slots are busy.
# Each entry is a Dictionary with keys: url, key, disk, parse_from_disk (bool)
static var _job_queue: Array = []
static var _active_jobs: int = 0

# Set to true during evict_all(); pending callbacks are nulled instead of fired.
static var _shutting_down: bool = false


# ── REPLACE load_model() entirely ────────────────────────────────────────────

func load_model(url: String, callback: Callable, etag: String = "") -> void:
	if _shutting_down:
		callback.call(null)
		return
	var key  := _cache_key(url, etag)
	var disk := _disk_path(url, etag)

	# 1 — memory hit
	if _mem.has(key):
		var packed: PackedScene = _mem[key]
		callback.call(packed.instantiate() if packed != null else null)
		return

	# 2 — already in-flight
	if _pending.has(key):
		_pending[key].append(callback)
		return

	# 3 — disk hit; queue a parse-only job if all slots are busy
	if FileAccess.file_exists(disk):
		_pending[key] = [callback]
		if _active_jobs < _MAX_JOBS:
			_active_jobs += 1
			_parse_disk_job(key, disk, url)
		else:
			_job_queue.append({ "url": url, "key": key, "disk": disk, "parse_from_disk": true })
		return

	# 4 — start a network download
	_pending[key] = [callback]
	if _active_jobs < _MAX_JOBS:
		_active_jobs += 1
		_download(url, key, disk)
	else:
		_job_queue.append({ "url": url, "key": key, "disk": disk, "parse_from_disk": false })

	if OS.is_debug_build():
		print("ModelCache: queued %s  (jobs=%d  queued=%d)" % [url, _active_jobs, _job_queue.size()])


# ── ADD: parse from disk in a job slot ───────────────────────────────────────

func _parse_disk_job(key: String, disk: String, url: String) -> void:
	var bytes := FileAccess.get_file_as_bytes(disk)
	if not bytes.is_empty():
		var packed := _parse_gltf(bytes)
		if packed != null:
			_mem[key] = packed
			var callbacks: Array = _pending.get(key, [])
			_pending.erase(key)
			for cb: Callable in callbacks:
				if not _shutting_down:
					cb.call(packed.instantiate())
			_finish_job()
			return
		push_warning("ModelCache: corrupt disk cache for %s — re-downloading" % url)
	DirAccess.remove_absolute(disk)
	# Fall through to network download within the same job slot (no _finish_job yet)
	_download(url, key, disk)


# ── REPLACE _download() entirely ─────────────────────────────────────────────

func _download(url: String, key: String, disk: String) -> void:
	var fetch := _fetch_url(url)
	if OS.is_debug_build():
		print("ModelCache: downloading %s  (active_jobs=%d)" % [fetch, _active_jobs])

	var http := HTTPRequest.new()
	http.body_size_limit = -1
	add_child(http)

	# CONNECT_ONE_SHOT auto-disconnects after first fire; no manual clear needed.
	http.request_completed.connect(
		func(res: int, code: int, headers: PackedStringArray, body: PackedByteArray):
		http.queue_free()

		var callbacks: Array = _pending.get(key, [])
		_pending.erase(key)

		if res != HTTPRequest.RESULT_SUCCESS:
			push_warning("ModelCache: network error result=%d http=%d - %s" % [res, code, url])
			for cb: Callable in callbacks:
				if not _shutting_down: cb.call(null)
			_finish_job(); return

		if code < 200 or code >= 300:
			push_warning("ModelCache: HTTP %d - %s" % [code, url])
			for cb: Callable in callbacks:
				if not _shutting_down: cb.call(null)
			_finish_job(); return

		if body.is_empty():
			push_warning("ModelCache: empty response - %s" % url)
			for cb: Callable in callbacks:
				if not _shutting_down: cb.call(null)
			_finish_job(); return

		var packed := _parse_gltf(body)
		_mem[key] = packed
		if packed != null:
			var f := FileAccess.open(disk, FileAccess.WRITE)
			if f: f.store_buffer(body); f.close()

		if OS.is_debug_build():
			print("ModelCache: parsed ok=%s  mem_entries=%d  jobs=%d" % [packed != null, _mem.size(), _active_jobs])

		for cb: Callable in callbacks:
			if not _shutting_down:
				cb.call(packed.instantiate() if packed != null else null)
		_finish_job()
	, CONNECT_ONE_SHOT)

	var err := http.request(fetch)
	if err != OK:
		push_warning("ModelCache: request() error %d - %s" % [err, fetch])
		var callbacks: Array = _pending.get(key, [])
		_pending.erase(key)
		for cb: Callable in callbacks:
			if not _shutting_down: cb.call(null)
		http.queue_free()
		_finish_job()


# ── ADD _finish_job() below _download() ──────────────────────────────────────

func _finish_job() -> void:
	_active_jobs = maxi(0, _active_jobs - 1)
	if not _job_queue.is_empty() and _active_jobs < _MAX_JOBS:
		var next: Dictionary = _job_queue.pop_front()
		_active_jobs += 1
		if next.get("parse_from_disk", false):
			_parse_disk_job(next["key"], next["disk"], next["url"])
		else:
			_download(next["url"], next["key"], next["disk"])

Fix 2 — Evict ModelCache on game exit, flush pending callbacks

Files: Client/origin-client/autoloads/ModelCache.gd, Client/origin-client/scenes/game/game_screen.gd

What it does:

  • evict_all() sets _shutting_down = true first, then flushes all entries in _pending with null callbacks so in-flight HTTP requests that complete after the scene is freed do not fire into destroyed nodes.
  • Clears _mem and _job_queue.
  • Resets _shutting_down to false after flushing so the cache is usable again immediately (autoload persists between games).
  • Disk cache (.bin files) is intentionally left intact.

Where to apply in ModelCache.gd: Add evict_all() directly below the existing evict() method (~line 79).

# ADD directly below evict() in ModelCache.gd

func evict_all() -> void:
	# Signal in-flight callbacks to return null instead of instantiating into
	# a freed scene. Must be set before flushing _pending.
	_shutting_down = true

	# Flush all queued callbacks with null so callers don't hang.
	for key: String in _pending:
		for cb: Callable in _pending[key]:
			cb.call(null)
	_pending.clear()

	_mem.clear()
	_job_queue.clear()
	# _active_jobs is left alone; in-flight HTTPRequest nodes own those slots
	# and will call _finish_job() when complete (which will drain an empty queue).

	# Re-arm immediately so load_model() works for the next game.
	_shutting_down = false

	if OS.is_debug_build():
		print("ModelCache: evict_all complete")

Where to apply in game_screen.gd: At the end of _exit_tree(), after the existing cleanup calls (line 379).

# ADD at end of _exit_tree() in game_screen.gd (after Chat.release_bubble_visibility)

	ModelCache.evict_all()

Fix 3 - Track and kill shader tweens on rebuild + hard cleanup on exit

File: Client/origin-client/scenes/game/components/units/unit_mesh_builder.gd

What it does (v2 additions over v1):

  • Adds _active_tweens member var and kills all tracked tweens at the start of each rebuild_units() call.
  • Tracks both materialize (line 614) and dissolve (line 657) tweens identically.
  • Adds _exit_tree() which kills remaining tweens — prevents retained ShaderMaterial refs if the scene tears down mid-animation.
  • Debug-gated counter log on each rebuild.

Where to apply: Four locations.

# 1. ADD near line 34 alongside existing member var declarations

var _active_tweens: Array[Tween] = []


# 2. ADD at the very start of rebuild_units(), before the move_anims kill loop
#    (before line 43: "for iid: String in move_anims:")

	# Kill all in-flight shader tweens so ShaderMaterial refs are released now.
	for t: Tween in _active_tweens:
		if is_instance_valid(t):
			t.kill()
	_active_tweens.clear()
	if OS.is_debug_build():
		print("UnitMeshBuilder: rebuild shader tween kill done")


# 3. REPLACE the Tween creation line in _play_materialize_shader_timed()
#    Current line 614: var tween: Tween = create_tween()
#    Replace with:

	var tween: Tween = create_tween()
	_active_tweens.append(tween)
	tween.finished.connect(func() -> void: _active_tweens.erase(tween), CONNECT_ONE_SHOT)
	# Lines 615-629 (set_trans, set_ease, tween_property, tween_method, tween_callback) unchanged


# 4. REPLACE the Tween creation line in _play_dissolve_shader_timed()
#    Current line 657: var tween: Tween = create_tween()
#    Replace with:

	var tween: Tween = create_tween()
	_active_tweens.append(tween)
	tween.finished.connect(func() -> void: _active_tweens.erase(tween), CONNECT_ONE_SHOT)
	# Lines 658-664 (set_trans, set_ease, tween_method) unchanged

Fix 4 - Pool unit scene nodes with full reset, size cap, and hard cleanup

File: Client/origin-client/scenes/game/components/units/unit_mesh_builder.gd

What it does (v2 additions over v1):

  • Full reset in _acquire_from_pool(): resets position, rotation, scale, collision_layer, collision_mask, removes all groups and stale metadata. Prevents stale transform or collision state from the previous occupant.
  • Per-type pool cap (_MAX_POOL_PER_TYPE = 6): overflow bodies are free()d immediately (not queue_free()) to reclaim memory synchronously.
  • Debug counters: logs pool hit/miss and pool sizes on each unit add.
  • _exit_tree() pool teardown: combined with Fix 3 into a single exit handler.

Where to apply: Six locations.

# 1. ADD member vars near line 34

const _MAX_POOL_PER_TYPE := 6   # overflow beyond this is freed immediately

var _unit_pool: Dictionary = {}   # card_type (String) -> Array[StaticBody3D]
var _pool_root: Node3D = null


# 2. ADD _ready() -- unit_mesh_builder currently has no _ready()

func _ready() -> void:
	_pool_root = Node3D.new()
	_pool_root.name = "UnitPool"
	_pool_root.visible = false
	add_child(_pool_root)


# 3. ADD _pool_unit() helper
#    Call instead of body.queue_free() in the stale-unit removal path.

func _pool_unit(body: StaticBody3D, card_type: String) -> void:
	if not is_instance_valid(body):
		return
	# Free MeshInstance3D children synchronously - releases material/mesh refs now.
	for child: Node in body.get_children():
		if child is MeshInstance3D:
			child.free()
	body.visible = false
	body.reparent(_pool_root, false)
	if not _unit_pool.has(card_type):
		_unit_pool[card_type] = []
	var pool: Array = _unit_pool[card_type]
	if pool.size() >= _MAX_POOL_PER_TYPE:
		body.free()   # pool full -- free immediately
	else:
		pool.append(body)


# 4. ADD _acquire_from_pool() helper

func _acquire_from_pool(card_type: String) -> StaticBody3D:
	if not _unit_pool.has(card_type):
		return null
	var pool: Array = _unit_pool[card_type]
	if pool.is_empty():
		return null
	var body: StaticBody3D = pool.pop_back()
	if not is_instance_valid(body):
		return null
	# Full reset -- clears all state from the previous occupant.
	body.position        = Vector3.ZERO
	body.rotation        = Vector3.ZERO
	body.scale           = Vector3.ONE
	body.collision_layer = 1
	body.collision_mask  = 1
	for group: String in body.get_groups():
		body.remove_from_group(group)
	# Remove stale metadata; add_unit_mesh() re-sets these five keys immediately.
	for key: String in body.get_meta_list():
		if key not in ["unit_id", "owner_id", "card_type", "team_color", "above_tile"]:
			body.remove_meta(key)
	body.reparent(units_root, false)
	body.visible = true
	return body


# 5. In rebuild_units(): REPLACE the else branch at line 122
#
#    Current else branch:
#      else:
#          body.queue_free()
#
#    New else branch:

		else:
			_pool_unit(body, card_type)

#    The ascend-departure timer path keeps queue_free() because the body
#    must stay alive for the 1.55 s departure animation.


# 6. In add_unit_mesh(): REPLACE the StaticBody3D.new() block at lines 178-184
#
#    Current code:
#      var body: StaticBody3D = StaticBody3D.new()
#      body.set_meta("unit_id", iid)
#      body.set_meta("owner_id", owner_id)
#      body.set_meta("card_type", card_type)
#      body.set_meta("team_color", team_color)
#      if not owner_id.is_empty():
#          unit_owner_ids[iid] = owner_id
#
#    Replace with:

	var _pool_hit := false
	var body: StaticBody3D = _acquire_from_pool(card_type)
	if body == null:
		body = StaticBody3D.new()
		# CollisionShape3D added once on fresh alloc; pooled bodies already have one.
		var cshape: CollisionShape3D = CollisionShape3D.new()
		body.add_child(cshape)
		units_root.add_child(body)
	else:
		_pool_hit = true
	# Metadata re-set unconditionally (pool reset leaves stale values).
	body.set_meta("unit_id", iid)
	body.set_meta("owner_id", owner_id)
	body.set_meta("card_type", card_type)
	body.set_meta("team_color", team_color)
	if not owner_id.is_empty():
		unit_owner_ids[iid] = owner_id
	if OS.is_debug_build():
		var pool_sizes := {}
		for ct: String in _unit_pool:
			pool_sizes[ct] = _unit_pool[ct].size()
		print("UnitMeshBuilder: add_unit iid=%s pool_hit=%s pools=%s" % [iid, _pool_hit, pool_sizes])
	# NOTE: Remove the original "var shape: CollisionShape3D = CollisionShape3D.new()" line
	# at line 186 -- it is now inside the if-block above as "cshape". The var mi / var mat
	# lines (187-189) and all code below are unchanged.


# 7. ADD _exit_tree() -- combines Fix 3 and Fix 4 cleanup (single function)

func _exit_tree() -> void:
	# Kill all in-flight shader tweens (Fix 3)
	for t: Tween in _active_tweens:
		if is_instance_valid(t):
			t.kill()
	_active_tweens.clear()
	# Free all pooled unit nodes (Fix 4)
	# Freeing _pool_root frees all children (pooled StaticBody3D nodes) with it.
	if is_instance_valid(_pool_root):
		_pool_root.free()
	_unit_pool.clear()

Part 3 - How the Fixes Resolve the Crashes

The 60-second crash

Fix 3 eliminates the silent ShaderMaterial leak. Before this fix every materialize/dissolve animation created an anonymous Tween holding live ShaderMaterial refs on nodes that were already removed from the scene. Under normal play several animations fire per turn; over 60 seconds dozens of these leaked materials accumulate. Fix 3 kills all in-flight tweens at the start of each rebuild so materials are released immediately.

Fix 4 eliminates the per-rebuild heap spike. Before this fix every unit removal triggered queue_free() (deferred) followed by a fresh StaticBody3D allocation — old and new coexisted in the WASM heap for one full frame. Over many turns the heap baseline rises because deferred frees do not compact the WASM allocator. Fix 4 replaces the pattern with a pool: the StaticBody3D shell is retained and reused; only the MeshInstance3D children (which carry mesh + material data) are replaced per-turn.

Fix 2 provides a clean-slate guarantee between games. Even with Fixes 3 and 4 in place, _mem in ModelCache accumulates parsed PackedScene objects across the session. When the user finishes a game and starts another, those PackedScenes remain loaded even if the new board uses a completely different unit set. Fix 2 evicts everything when the game screen exits, and flushes pending callbacks with null so in-flight HTTP requests that complete after scene teardown do not fire into freed nodes.

The immediate rejoin crash

Fix 1 is the direct fix. The rejoin crash is caused by up to 128 simultaneous _parse_gltf() calls when the server delivers a full board state and build_board() fires add_tile() for every tile at once. Each parse allocates a large GLTFDocument + GLTFState + scene graph before compressing to a PackedScene. With 128 running in parallel the WASM heap spikes far above the iOS ceiling before the first frame renders.

Fix 1 caps simultaneous jobs (both network downloads and disk-cache parses) at _MAX_JOBS = 4 via a shared budget. Disk-cache hits that arrive when all slots are busy now queue as parse-only descriptors and drain correctly when a slot opens — v1 had a bug where these would silently trigger a full re-download instead. The board finishes loading in the same total time (network is the real bottleneck) but without the instantaneous spike.

Null handling for failed model parses (item 7)

ModelCache.load_model() already calls the callback with null on parse failure. Callers in board_controller.gd (try_load_tile_model, try_load_tile_bottom) should be verified to handle a null node gracefully (i.e. skip model attachment, leave placeholder mesh). This is not a change to ModelCache itself but a verification step before applying Fix 1.

Summary table

Fix Crash addressed Mechanism
Fix 1 - Throttle jobs Rejoin crash (immediate) Caps simultaneous GLB parses/downloads at 4; queues disk-cache hits correctly
Fix 2 - evict_all on exit 60s leak + cross-game accumulation Clears PackedScene mem-cache and flushes pending callbacks on game exit
Fix 3 - Track tweens 60s ShaderMaterial leak Kills in-flight shader tweens on rebuild; releases refs immediately
Fix 4 - Unit pooling 60s heap spike per turn Reuses StaticBody3D shells; full reset on acquire; capped pool size
  1. Fix 1 — highest priority; directly prevents the rejoin loop that traps the player.
  2. Fix 3 — low risk, high impact; pure additive change to unit_mesh_builder.
  3. Fix 2 — low risk; one new method + one call site.
  4. Fix 4 — most structural; verify rebuild_units() and add_unit_mesh() thoroughly after applying.