bring a question, stay still,
and receive its answer.

Live prototype · viewer_v4
↓ scroll to read

Zixin Yan  ·  2026

What the Body Knows
A digital membrane that fractures in response to your physiological stillness — drawing its crack vocabulary from 3,000-year-old Chinese divination practice.
Interactive installation prototype
Three.js · ml5.js FaceMesh · WebGL GPU compute
Research prototype

How to experience this work

The live artwork is at the top of this page. Follow these steps:

1
Activate the artwork
Scroll back to the top. Click the button labelled "Click to interact" — this unlocks the artwork so your mouse and keyboard work inside it.
2
Let the oracle read you with camera
Click "Connect Camera" in the bottom-right corner and allow access when your browser asks. Sit so your face is visible. Hold a question in your mind — something you genuinely want to know, the way you might ask a tarot card or flip to a page in a book of answers. You do not need to speak it aloud. Stay completely still while you hold it. A progress bar will fill over five seconds, and when it completes, the membrane will crack open with a reading.
Or ask without a camera
Click "Consult the Oracle" at the bottom of the screen for a reading without camera access. The crack type will be chosen for you.
4
Scroll down to continue reading
Click "Exit interaction" (the same button, now renamed) and your trackpad will scroll this page normally again.
View the static painting layers
Press the letter L on your keyboard to dissolve the particle membrane back into its original painted layers — a behind-the-scenes look at how the work is constructed.

Concept

In ancient Chinese divination, a diviner carved a question into a turtle plastron and applied heat from beneath. The shell cracked. The pattern of fractures — branching left or right, radiating outward — was read as a response from what could not be directly seen. The oracle bone was a membrane: interior pressure passed through biological material and became visible as legible form.

I keep returning to this image because it describes something I recognise from my own practice. For five years I have been painting on translucent fabrics — tulle, muslin, calico — surfaces that refuse opacity. Paint bleeds through them. Light passes through them. They are membranes: thresholds between interior and exterior, between what is felt and what is seen.

Membrane Oracle asks: what if the oracle bone were not a turtle shell but a digital surface built from one of those paintings? What if the heat applied beneath it were not fire but the viewer's own sustained attention — their physiological and emotional state, made readable?

"The oracle is not supernatural. It is the participant's own interiority, externalised and made legible."

This work is a prototype for that proposition. The viewer stands before a digital membrane and holds still. The surface reads them. The crack that appears is not chosen by me, nor by chance — it is selected by a matrix of the viewer's own arousal, affect, and stillness. The reading belongs to them.

Material Ground

For over five years I have painted on translucent fabrics precisely because they refuse the opacity of canvas. Paint bleeds through. Light passes through. The viewer does not stand before the work — they see through it, into it, around it. The fabric is a membrane: a threshold between what is felt and what is seen.

The digital membrane in this work is built from a 3D Gaussian splat reconstruction of one of my calico paintings. Fifteen million particles, each carrying the colour of a specific point in the original canvas. When the oracle fires, the particles part and the surface cracks — not as animation, but as physics: GPU-computed particle repulsion along the fracture line, soft and gradual, the way real fabric pulls away from a seam.

The crack patterns are drawn from the eight crack-type vocabulary of Shang dynasty plastromancy — the formal taxonomy of fracture shapes that Shang diviners used to read the shells. Each type carries a different meaning, mapped in this work onto the viewer's physiological state rather than the will of ancestors.

Current Prototype · How It Reads You

This version uses your laptop camera and ml5.js FaceMesh — 468 facial landmarks tracked at ~15fps — to derive three proxy axes from your face in real time:

Input Webcam → 468 facial landmarks (ml5 FaceMesh)
Arousal Eye aspect ratio (openness) — normalised 0→1
Valence Brow-to-eye distance (relaxation vs tension)
Stillness Rolling average of nose-tip velocity across 20 frames
Reading Hold still for 5 real seconds → crack fires
Crack type Selected from arousal × valence matrix (below)

The reading panel in the top-right corner shows these axes live. The READING bar fills in real time as you hold still. When it completes, the membrane cracks and an oracle reading appears — drawn from the eight crack-type vocabulary of Shang dynasty plastromancy.

直裂
zhí liè · The Upright
"Auspicious. The surface begins to open."
High arousal, high valence.
弧裂
hú liè · The Curving
"The answer approaches indirectly. Wait."
Low arousal, high valence.
枝裂
zhī liè · The Branching
"Many paths open. The question is still forming."
High arousal, low valence.
放裂
fàng liè · The Radiant
"The oracle speaks. You already know."
Sustained stillness, high arousal and valence.

Where This Is Going

The facial expression proxies in this prototype are a demonstration — they show that the oracle can be driven by the viewer's interior state. But the face is a performance. What I am working toward is a version that reads beneath the face: direct physiological and neural sensing that the viewer cannot consciously control or perform for.

Now Webcam · FaceMesh · expression proxies prototype
Next Heart rate variability (HRV) · galvanic skin response planned
Further fNIRS prefrontal cortex oxygenation planned
Further still Full neuro-physiological stream: fNIRS + HRV + movement planned

fNIRS (functional near-infrared spectroscopy) measures the oxygenation of blood in the prefrontal cortex — a non-invasive, wearable technology that gives a continuous signal of cognitive and affective engagement. Unlike EEG, it does not require a clinical setting or conductive gel. A participant wears a lightweight headband, stands before the installation, and holds a question.

The three proxy axes remain: arousal, valence, and stillness. The crack matrix does not change. The oracle reads the same vocabulary. Only the sensing layer becomes real.

The participant is not being measured. They are being read. The distinction matters: measurement assumes an objective ground truth. A reading proposes that the system's interpretation — emergent, culturally framed, poetic — is itself a form of knowledge, neither more nor less true than what the body already knows.

Research Context

This project sits at the intersection of three areas that rarely speak to each other directly: interoception research (the body's own sensing of its internal states), Chinese material culture and the oracle bone tradition, and the growing field of biofeedback art.

On interoception: Antonio Damasio's somatic marker hypothesis proposes that physiological signals carry information that precedes and sometimes exceeds conscious reasoning. People often know things before they can say them. Divination practices — oracle bones, tarot, I Ching — can be understood as frameworks that give permission to attend to what the body already knows. This work does the same thing, but shows its working.

On precedents in biofeedback art: Refik Anadol's Melting Memories (2018) translated EEG data into data sculptures; Marco Donnarumma's Nigredo fed bioacoustic signals back to the participant in a ritualised space; Dr Friendred Peng's Moving Photon (2022) streamed live EEG data to control a kinetic installation during dance performance. What distinguishes this work is the specific cultural framework — the oracle bone as interface between interior and exterior — and the proposition that the system does not merely visualise data but produces emergent, culturally situated readings.

My practice: BA First Class, Central Saint Martins (2023). MRes, Royal College of Art (2024), researching the visual history of the uterus in Western medical imaging. Selected exhibitions: Passing through her body (2024); The Body Entangled (East Bound Residency, 2024, curatorial); Exhibition Road Festival (2024, 200 participants).

Technical Note

The live prototype above runs entirely in your browser. No data is collected or transmitted. Camera access is optional and processed locally by ml5.js; no video frames leave your device. The 3D Gaussian splat is a point cloud rendered via WebGL GPU compute shaders in Three.js — approximately fifteen million particles simulated on GPU each frame.

The animated particle membrane loads by default. Press L to switch to the static painted layers. Click Connect Camera to enable facial expression reading. Click Consult the Oracle for a manual reading without camera.

References

Barad, K. (2007) Meeting the Universe Halfway. Duke University Press.
Damasio, A. (1994) Descartes' Error. Putnam.
Keightley, D. (1978) Sources of Shang History. University of California Press.
Merleau-Ponty, M. (1962) Phenomenology of Perception. Routledge.
Peng, Y. (2024) Composing Embodied Experience and Agency. PhD thesis, Goldsmiths.
Peng, Y., Tanaka, A. and Ward, J.A. (2020) 'The Light.' UbiComp/ISWC '20 Adjunct.