C4 / LITERACY / EVALUATED AGES 6–8
Draw a thought. Discover a word.
A learner’s sketch becomes an object-word connection. Spelling and pronunciation-oriented activities unlock its corresponding AR representation.
Sketch-Driven Multisensory Vocabulary Learning
Illustrative playback. This visual explains the concept, rather than running the research application.
THE LEARNING PROBLEM
What this component
helps learners explore.
Connecting visual objects, written words and spoken practice can be difficult. A learner-generated sketch provides a concrete starting point.
Technical approach
A CNN sketch classifier was developed with Python, TensorFlow/Keras, OpenCV, FastAPI, NumPy and Pillow. The development dataset contains 25,000 Quick, Draw!-derived images: 17,500 training, 3,750 validation and 3,750 held-out test images. Unity loads a corresponding representation rather than reconstructing an arbitrary custom object.
FROM INPUT TO REINFORCEMENT
The learning workflow.
- 01
Sketch Apple, Cat, Fish, Sun or Tree
- 02
Classify the sketch and load its corresponding 3D representation
- 03
Complete the spelling activity
- 04
Complete the pronunciation-oriented activity
- 05
Unlock AR only after both activities succeed
SUPPORTED SCOPE
Purposeful boundaries.
- Apple
- Cat
- Fish
- Sun
- Tree
Pilot evaluation
20/25 spelling completions (80%); 21/25 pronunciation-oriented acceptances (84%); 17/25 completed both tasks and unlocked AR (68%).
What the findings do not establish
Confidence is not classification accuracy: an independent intended-object label was not retained. Pronunciation-oriented acceptance is not independently measured speech-recognition accuracy. The evaluated task is vocabulary learning, not formal grapheme–phoneme instruction.
Source: Manuscript pp. 3–5. Read the manuscript ↗
Download individual proposal ↓LET’S EXPLORE WHAT’S NEXT
Bring a new dimension
to learning.
Start a conversation about a demonstration, school pilot or research partnership.