Week 10: Narration, Onboarding v2 & Final Demo

Overview

This week shifted from writing to building: recording personal narration for all 8 spaces, converting them into the UE experience, and debugging a persistent ISDK cursor drift issue. Also updated the SplatRenderer portfolio page for v1.1.0.

---

Narration System

The thesis direction from Week 7 (“I take you into my memories”) now has voice. Each space has a personal voiceover that plays after the viewer enters.

Recording & Script Process

1. Voice-recorded narration stories for all 8 spaces (raw, in English)
2. Transcribed with Whisper, then edited into narration scripts
3. Formatted for ElevenLabs: added pauses (<break time="0.8s"/>), oral connectors, natural phrasing
4. Generated ElevenLabs audio with custom voice (Dazai)
5. Converted to WAV (48kHz mono) for UE import

The 8 Narration Scripts

#	Space	Core Memory
01	Okinawa Beach	Standing in warm water at sunset, watching the sky change colors
02	National Concert Hall, Taipei	Growing up with the building, from childhood to lighting designer
03	NTU Campus, Taipei	First time as an outsider, walking through a city that felt foreign
04	Nankunshen Temple, Tainan	Annual family pilgrimages, the smell of incense, grandpa’s devotion
05	Sanheyuan, Tainan	Great-grandma’s house that no longer exists, the space lives only in memory
06	Brooklyn Apartment	First home alone in a new country, the strange comfort of an empty room
07	Brooklyn Rooftop	A private escape above the city, watching sunsets alone
08	Home, Taipei	Where the project began, the room where everything connects

UE Integration

• WAVs imported to /Game/Asset/Audio/Narration/
• NarrationMap in TS_WidgetCarousel maps level names to audio assets
• Playback: 1.5s delay after level load, then fade-in from volume 0.5 to 0.7
• Cleanup: fades out on carousel show (1s) or level switch (0.5s)
• 2D audio (SpawnSound2D), no spatialization. The narration is intimate, like a voice in your head.

---

ISDK Cursor Drift Fix

A persistent bug where the ISDK cursor would drift/fly off when reopening the carousel after entering a level.

Root Cause

ResetOrientationAndPosition was called on every level load to make the user face X+. This shifts the UE HMD coordinate system origin. But Meta’s hand tracking (ISDK) does NOT follow this reset. Result: the carousel position (based on HMD) and the hand ray (based on Meta tracking) end up in different coordinate systems. The more the user moves after the reset, the worse the drift.

Fix

Disabled ResetOrientationAndPosition. The coordinate system stays consistent, and ISDK cursor tracking works correctly.

Lessons Learned

• SetActorEnableCollision(false) does NOT prevent ISDK ray hits. Must use SetActive(false) on interactables directly.
• ISDK’s SetActive disable/enable cycle works fine as long as the coordinate system stays consistent.
• Never use ResetOrientationAndPosition with Meta ISDK hand tracking.

---

SplatRenderer v1.1.0

Updated the SplatRenderer portfolio page (EN/ZH) with:

• New demo reel video (v2)
• Level Sequencer feature section
• Updated YouTube links
• Fixed thesis route conflict (static page vs dynamic route)

---

Thesis Draft

• Expanded Ch4.1 curation paragraph with personal significance: Concert Hall (dual identity), Sanheyuan (space no longer exists), Brooklyn apartment (first home alone), Taipei home (project’s personal meaning)
• Created Week 9 doc: Professor Camila’s written feedback on full thesis draft

---

Onboarding v2

Rebuilt the onboarding from scratch. The old version had 4 separate steps with 3 tutorial videos (palm gesture, swipe left/right, swipe up/down). The new version is 2 steps with a single demo video.

New Structure

Step	Content	Audio
0 - Welcome	Text panel with dynamic captions	onboarding_seg1.mp3 (ElevenLabs, 13.87s)
1 - Demo Video	Single mp4 with embedded narration	Audio baked into thesis_onboarding_seg2.mp4 (37.38s)

Key Changes

• Single demo video replaces 3 tutorial clips. Shows the full interaction flow: hover, pinch, enter memory, open palm menu, tap album button.
• Text overlay synced to audio using Whisper word-level timestamps. Text appears slightly after audio (0.3s delay) so the voice leads.
• Progress bar at the bottom of the video widget (PB_Progress in WBP).
• Welcome captions match the spoken audio beat-by-beat, transitioning through 5 phrases across 13.87 seconds.
• Skip button enlarged and repositioned to top-right.
• MediaSoundComponent attached for video embedded audio playback.

Narration Script

Written in works/thesis/notes/onboarding-script.md. Two segments:

Segment 1 (Welcome): “Hello, welcome. I’m Dazai. So, I want to take you into some of the places that mattered to me. This is my spatial memory album. But before we step in, let me show you how you can interact with this world.”

Segment 2 (Demo + Closing): Narrates each interaction step, then closes with “Alright, take your time, look around. Each place here holds a little piece of my story. Hope you enjoy.”

---

Final Demo Recording

Recorded a full VR walkthrough of the experience and captured 18 stills.

Demo Video

Screenshots

Onboarding

Video Tutorial

Spatial Memories

---

Next Steps

• Thesis revisions based on Camila’s feedback (Ch3.1 Nankunshen, Ch4 UX/UI expansion)
• Fine-tune onboarding timing after more VR testing
• IDM Showcase uploads (poster + website image to Google Drive)