Can Five Dancers Move to a Song Just by Listening? — A Choreography Generator That Writes Point Moves, Formations and Paths for an Idol Track Made from Hwang Sun-won's "Sonagi"

When you think of an idol group's stage, the moves sometimes come to mind before the song. A few beats the five members dance in perfect sync in every chorus, the move that makes you want to copy it. In K-pop it is usually called the point move. Between those moments the members never stop changing places. Whoever is singing comes to the front center, the group fans out into a V for the chorus, and lines up in a row for the bridge.

Making a song with AI is pretty doable now. In an earlier A Page for Today I made three songs on a laptop GPU. So can a machine listen to that song and choreograph it too? Not one move, but the moves, formations and paths of five people at once. That is what I built this time.

What I set out to do

1. Make a new idol girl-group song that borrows only the motifs of Hwang Sun-won's short story "Sonagi" (The Shower). Write new lyrics, and generate the track on a laptop.
2. Build an ear for the song: beats, bars, sections (verse, pre-chorus, chorus, bridge), and the timing of every lyric line.
3. Build a generator that choreographs for five: pick moves, repeat the same point move in every chorus, split the vocal parts, and plan formations and paths. Members must not collide.
4. Put five simple figures with 20 joints each on a 3D stage and make them dance to the song.
5. Lay out ways to write dance down as numbers, and use those numbers to pick one choreography from many candidates.
6. Give an AI the same move dictionary, have it write the plan itself, and score that plan with the same judge.

Research on the history and the literature went to an AI agent. I told it to prefer original papers, court rulings and official announcements, and to mark anything it could not confirm as unconfirmed. The song generation, analysis, choreography generator, 3D view and experiments were also built and run by an AI agent.

A hundred years of writing dance down

For a long time dance was an art you could not write down. Music had scores; movement passed only from body to body. Attempts to put it on paper go back a long way. Jeongjae Mudo Holgi, a record of Korean court dances of the Joseon era, wrote out in words, beat by beat (bak), who does which move to which music. That is almost exactly how this generator places moves on a beat grid.

The person who built a system in the West was Rudolf von Laban. In 1928, in Schrifttanz, he set out a way to write dance in symbols. In a tall staff like a vertical score, the left and right sides of the body, arms and legs get their own columns; the shape of a symbol gives direction, its length gives duration, and its shading gives level. In English it is called Labanotation. In the mid-1950s the Beneshes created a separate notation for ballet.

Rudolf von Laban and his notation signs. Source: Wikimedia Commons, author unknown, public domain

Besides the symbols, Laban left a framework for the quality of movement: the four Effort factors of Laban Movement Analysis. Is it strong or light (Weight), sudden or sustained (Time), direct or indirect (Space), bound or free (Flow)? The same arm raise becomes a different move when these four change. They come back later as numbers.

Computers arrived around 1989. Merce Cunningham choreographed by moving figures on a screen in LifeForms, a program from Simon Fraser University. As motion capture spread, dance simply became numbers. Formats like BVH, which writes three rotation values per joint, one line per frame, became a de facto standard.

AI began learning dance once there was data. In 2019 Japan's National Institute of Advanced Industrial Science and Technology (AIST) released 13,939 videos of 40 professional dancers performing 10 genres, and in 2021 Google researchers turned 5.2 hours of it into 3D motion as AIST++, with a generative model called FACT. Since then models such as Bailando (2022), EDGE (2023) and Lodge (2024) take a song and produce one person's dance. In 2019, for his company's 25th anniversary, Wayne McGregor worked with Google on a tool trained on footage of past works and put "Living Archive" on stage. Show it a new phrase of movement and it offered several candidate phrases to follow.

But there is almost no K-pop in this data. AIST's 10 genres are street styles such as break, pop, lock, hip-hop, house, krump and jazz. The FineDance (2023) paper even writes that "Anime and K-pop do not count as specific dance genres." POPDG (2024), which does include KPOP, estimates 3D from cover videos uploaded by users of a Chinese video site. The group-dance dataset AIOZ-GDANCE (2023, 16.7 hours) is mostly Zumba and aerobics. I could not find public data that tackles K-pop group dance head on, with five people in razor-sharp sync trading places.

Nor could I find an official announcement of a K-pop agency making choreography with AI. The confirmed uses are in voices and virtual artists. In Korea a government R&D project on choreography-creation AI, led by DanceTract with KAIST participating (4.869 billion won in total), is under way.

Choreography also carries rights. In 2011 the Seoul Central District Court recognized the choreography of the girl group Secret's "Shy Boy" as a copyrighted work (2011Gahap23960), and in 2012 the Seoul High Court found that a dance-lesson business teaching it without permission had infringed the rights of performance, reproduction, transmission and attribution (2011Na104668). In 2024 the Korea Choreography Copyright Association launched. So the generator's move dictionary does not copy any song's choreography; it holds only common moves like bounces, step touches and arm waves, plus point moves written fresh for this song.

First, the song: "Sonagi" as an idol track

Hwang Sun-won's "Sonagi" first appeared in 1953 in the magazine Sinmunhak. The author died in 2000, and the work stays under copyright until the end of 2070. So I did not borrow a single sentence. I took only motifs, the stepping stones across the creek, the pebble, the sudden shower, the pink sweater, the jujubes, and wrote new lyrics. If the original is a boy's story, this song is something like a late-summer memory sung by five girls. The chorus went like this (ssoa is the sound of a sudden downpour):

Ssoa ssoa, sudden rain — you and me, no umbrella Ssoa ssoa, sudden rain — I don't mind getting soaked My heart falls drip, drop — here and gone like a shower

The track was made with ACE-Step 1.5, the open-source music model used in episode 17 of A Page for Today, on the laptop's RTX 3070 (8 GB). I set the tempo to 120 BPM, the length to 2 minutes 30 seconds and the genre to girl-group dance pop, and drew three songs from three seeds. Each took 102 to 114 seconds, and GPU memory peaked at 3.7 GB.

Choosing one of the three was also left to a machine. I had the speech model Whisper transcribe the lyrics and picked the song in which the most lines I wrote were heard in order. One song lost the first minute and a half entirely; in another, the first verse was not heard. The remaining one had every line in order except one in the bridge. "Ssoa ssoa" was transcribed as "saya saya" to the very end.

An ear for the song

To choreograph, you first need the skeleton of the song. When the beats fall, where each bar starts, when the chorus begins.

Fed to the beat tracker of librosa, a Python library common in music analysis, the song came out at 117.45 BPM. Odd, since I had asked the generator for 120. So I measured another way. I slid a beat grid across the song, changing the spacing from 100 to 140 BPM in steps of 0.05, and averaged the onset strength (how sharply sound starts) at the grid points. At 120.00 there was a peak more than three times taller than anywhere else. The generated song's beat was fixed like a machine's; laid at 117.45, the grid falls more than six beats behind over 2:30. By the end the dance would trail the song by a bar and a half.

The first beat of each bar was aligned to where the first lyric of verse 1 comes in. Sections were split by the transcript: "creek" (gaeulga) means verse 1, "silly, silly" (babo babo) means the pre-chorus, "umbrella" means the chorus. Snapping the time of each section's first line to the nearest bar start split the song into 75 bars and 10 sections: a 5-bar intro, a 7-bar verse 1, an 8-bar pre-chorus, a 9-bar chorus and so on. There are 30 lyric lines. One bridge line the transcript missed, "I wait, but you don't come," I inserted by hand between its neighbors.

For every bar I also extracted loudness and timbre and harmony features. Measuring how alike every pair of bars is gives a single board (the left panel of a figure further down). Bright lines parallel to the diagonal mean "this passage resembles one heard earlier." That board becomes the reference to compare the dance against.

How the choreography is put together

The generator mimics the order in which a human choreographer works: decide who sings where, choose moves, decide who stands where, and decide how they walk there.

① Splitting the parts. Each non-chorus lyric line goes to one of the five, keeping the counts even and never giving the same person two lines in a row. When the same lyrics return, as in the second pre-chorus, the same person sings them. Everyone sings the chorus, and one fixed member stands in the center.

② The move dictionary. I built 24 moves by hand: bounce, sway, step touch, shoulder pop, arm wave, point sweep, overhead clap, overhead heart, kick ball change, spin, body wave, raindrop fingers, umbrella, wave bye, crouch and rise and so on. Each move has a length (4 or 8 beats), an intensity (1–3) and a category (light groove, arms, feet, slow). A move is a few keyframes, each a pose at a beat position. A pose is a bundle of numbers: how high the arm is raised (0° down, 90° horizontal, 180° overhead), which way it points, how much the knees bend, how far the body leans. Every keyframe is either "arrive smoothly" or "arrive fast and stop." That is Laban's Time factor: sustained versus sudden.

③ The point moves. Two 8-beat moves were written separately for the chorus. One is "Ssoa ssoa sonagi." Both hands go up overhead, come down fluttering like rain, the knees bend as the rain hits the ground, one arm bends over the head as an umbrella, and the dancer points forward and steps out. The other is "Drip drop, come and go": tapping beside the cheek, then waving goodbye. Every time the chorus comes, all five dance these 8 beats identically.

④ Repetition and variation. When the music repeats, the dance repeats. The move sequence chosen for verse 1 is reused in verse 2, with roughly three in ten swapped for other moves. "Raindrop fingers" resembles the opening of the point move, so it sometimes shows up in the verses too, a small preview of what the chorus will show.

⑤ Group modes. Five people can dance the same move in several ways: unison (the sharp kal-gunmu), mirror (members on the left do it flipped), ripple (half a beat later each, from the left), solo + backing (only the singer does the move, the others sway), and split (alternating in pairs). The chorus is mostly unison, the intro a ripple, the bridge solo + backing or ripple.

⑥ Formations. Ten of them: line, V, inverted V, W, 3 front · 2 back, 2 front · 3 back, pyramid, column, diagonal, cross. Each has a "spot 0" at the front center. These are names people use on the floor, not terms with fixed definitions, so I defined them with a drawing. The intro hides in a column and then opens up; the chorus goes to formations where all five are visible, like V and W; the verses go to formations with the singer at the apex.

⑦ Paths. Who goes to which of the five spots in the next formation is decided by checking every possible pairing. With five people that is 5×4×3×2×1 = 120. The singer must take spot 0; under that condition the generator picks the pairing with the smallest sum of squared walking distances and no crossing paths. Then it checks whether any two people come within 0.7 m while all five walk at once. If they do, it bends their two paths away from each other and checks again. While walking, the feet step down and move one step every half beat.

⑧ The body. Each figure has 20 joints: pelvis, chest, neck, head, and on each side shoulder, elbow, wrist, fingertip, hip, knee, ankle and toe. She is about 1.63 m tall, with limb lengths in adult female proportions. Arms and torso turn the pose numbers straight into positions (forward kinematics); for the legs the generator first decides where the foot goes and then solves the knee angle back from the hip (inverse kinematics), so the feet stay on the floor when the knees bend. Each bone gets a cylinder and each joint a ball for the 3D view.

How to write dance as numbers

The moment the figure dances, the dance is already numbers. One person, one frame: 20 joints × 3 coordinates = 60 numbers. At 30 frames a second, 2:30, five people, that is 1.35 million. What Laban wrote in symbols is now written in tables. The problem is how to read a "good dance" out of that pile of numbers.

The metrics researchers use look roughly like this.

Beat alignment is measured like this. When the body dances to the beat, it pauses for an instant whenever the move changes. The moment the average speed of the 20 joints is smallest is called a kinematic beat, and for each one you measure the time to the nearest music beat. A difference of 0 earns 1 point, 0.05 seconds earns 0.61, and so on, and the points are averaged. Real AIST++ dance scored 0.292, the best model of the time 0.241.

This generator scored 0.46 for beat alignment, higher than real dancers. Nothing to brag about. The keyframes sit exactly on the beats, so it is a given. Human dancers sometimes lean a little ahead of the beat or lay a little behind it; these figures cling to it like machines. When I randomly jittered every keyframe by ±0.3 beat, beat alignment dropped only from 0.426 to 0.355. You can half-ignore the beat and barely lose points. It is the same criticism researchers make: this metric alone cannot pick a dance.

So I looked at several numbers together, ten in all.

• Moving together: how alike the five speed curves are, averaged over the ten pairs. 1 means perfect unison. Similar in spirit to AIOZ-GDANCE's GMC.
• Repetition structure: one board of bar-to-bar similarity for the music, one for the dance, and the correlation between the two. It rises when the dance repeats where the music repeats.
• Following intensity: whether the music's loudness and the body's kinetic energy rise and fall together per bar.
• Singer in the center: the share of time the member singing a line stands in the formation's spot 0.
• Closest gap: the smallest distance between any two members over the whole song. Under 0.5 m counts as a collision.
• Path crossings: how often two paths crossed during formation changes. Similar to AIOZ-GDANCE's TIF.
• Formation changes, distance walked, moves used, foot sliding.
• Approximations of Laban's four Effort factors: Weight as the peak kinetic energy, Time as the size of the acceleration, Flow as the size of the jerk (the change in acceleration), and Space as the directness of the path the hands trace (straight-line distance ÷ distance actually traveled). There is no settled academic definition; I followed common approximations. In the lower panel above, Weight, Time and Flow spike in each chorus (pink), because the point move is big, sudden and punctuated. In the bridge (green) all three sink and Space rises instead, because the slow, straight "reach out" comes in. From the numbers alone you can tell where the chorus and the bridge are.

I made 300 candidates and picked one

The generator produces a different choreography for each seed number. Who gets which part, which moves are chosen and which formations are used all change. I made 300, seeds 1 to 300, and scored them with the numbers above. The score is beat alignment + repetition structure × 0.8 + following intensity × 0.6 + singer in the center × 0.5, minus penalties for path crossings, collisions and excessive walking. The weights were set by a person. Making and scoring one took about 0.4 seconds on the laptop.

Scores spread widely, from -0.75 to 1.40. First place went to seed 114. Re-measured at 30 frames a second: beat alignment 0.46, moving together 0.71, repetition structure 0.19, singer in the center 73%, closest gap 0.69 m, 0 path crossings, 38 formation changes, 20 moves used. Over 2:30 each of the five walks 27 to 40 m. This is the choreography that appears first in the view below.

To see what each part of the generator does, I removed one at a time, ran the same test on 60 seeds and compared medians.

• Without repetition, verse 2 and every chorus are chosen afresh. Repetition structure falls from 0.147 to 0.019, while the other scores barely move. You can see in numbers that repetition is what makes the dance feel like the song.
• Without detours, the pairings are chosen the same way but everyone walks in straight lines. The closest gap goes from 0.691 m to 0.081 m, and in all 60 seeds two people come within 0.5 m somewhere. Whenever the formation changes, someone bumps into someone.
• Jittering moves by ±0.3 beat costs beat alignment only 0.07, but moving together falls from 0.727 to 0.613. A slightly missed beat is less visible than five people drifting apart from each other.

When an AI writes the plan

Next I replaced the generator's random choices with an AI. I gave it the same 24 moves, 10 formations and 5 group modes and had it choose the moves, mode, formation and center member for each of 40 phrases. I also wrote down K-pop conventions such as "repeat the same point move in every chorus, repeat the dance when the music repeats, put the singing member in the center." Footsteps and gap checks stayed with the generator. Claude Sonnet and Claude Opus each got three tries.

All six were formally flawless. No invented move names, no moves overflowing a phrase, and the point move in every chorus without exception. The best plan would have ranked 2nd among the 300 random ones. But the same request also produced a 164th. On average they beat the median of the random candidates (0.88) and fall short of the top of 300. That is roughly where "AI choreographs" stands now: ask once and you may be lucky or not; ask several times and pick with a judge, and you land near the top.

The first time, I did not tell it who sings which line. "Singer in the center" came out at 8.5%: it kept sending members who were not singing to the center. Not the AI's fault; I had left the information out. The table above is the rerun with the part split included.

Try it

Below is the stage I built. Press Play and the song starts and the five dance. The song is packed into the page, so there is nothing else to download; the first press takes a moment to decode it.

On a narrow screen, open it in a new window. "Jump to the point move" takes you straight to the first chorus. There are three cameras: front, top and orbit. Drag the stage to turn it and use the wheel to zoom. From the top you can clearly see the five stepping around each other as the formation changes.

"New choreography" makes eight candidates from a new seed, scores them quickly and shows the best. It took about 4 seconds on the laptop. Who takes which part and which formations are used change, but the chorus point move is always the same. The numbers in the "Numbers" panel update for the new choreography, and clicking a phrase in "Plan" jumps there.

Where it went wrong

• It heard the beat 3% wrong. The librosa beat tracker gave 117.45 BPM. Used as is, it would have drifted more than six beats over 2:30, and the chorus point move would have started in the middle of the chorus. Because the generated song has a fixed beat, laying a grid on it directly was more accurate. That trick will not work for a song played by people.
• The first beat of the bar was off by two beats. At first I aligned bars to "the first sung sound" and caught the intro chant "ssoa ssoa" at 7 seconds as the first lyric. The chant sits on the third beat of a bar, so every section shifted two beats. I switched the reference to the first line of verse 1.
• Every bar of the song looked alike. When I first measured how alike the bars' timbre and harmony features were, any two bars scored 0.88 or more. The sound common to the whole track had taken up all the similarity. After subtracting the song-wide mean, choruses matched each other at 0.91 and chorus against verse at 0.49.
• Two people collided head on. When the singer changed and two members swapped the center spot, they walked the same path in opposite directions and met exactly halfway. The gap was 0.002 m. Even after adding the detour step, there was one more collision: one person standing still while another walked straight through her. Someone standing still has no "direction of travel," so there was no way to compute which side to bend toward. I changed it to bend relative to the path of whoever walks farther. Even so, 11 of the 300 candidates bring two people within 0.5 m somewhere; scoring just penalizes them so they are not picked.
• A hand teleported. While walking to a new formation the arm moves were scaled to half size, and when the walk ended they snapped back to full size in a single frame. The wrist jumped at 24 m per second. A one-frame spike in the speed curve gave it away. I made the walking weight largest in the middle of the move and zero at both ends.
• After a spin she unwound a full turn backwards. The "spin" goes from a body turn of 0° to 360°. The next move starts at 0°, so bridging the two went back 360° in reverse, in a single beat. Angles now take the short way.
• Arms went over the back of the head. Scaling the moves up 1.15× in the chorus turned an arm raised 170° overhead into about 195°, tipping it backwards. I capped the arm raise at 178°.
• The parts are uneven. The counts are balanced when lines are handed out, but the balance breaks at the step where repeated lyrics are overwritten so the same person sings them. In the top choreography the five sing 3, 4, 2, 4 and 4 lines. In a real group, fans would have noticed first.

Why it works in one click

Composition, music analysis, 3D figures, scoring, AI-written plans: many pieces went into this episode. They hold together for a simple reason. Dance can be written as numbers, and those numbers can judge part of what makes a "good dance." Did anyone collide, did paths cross, is the singer in the center, is the same point move danced in every chorus? A program judges these without error. Once judging works, the making side can just make a lot and throw most away. It is the same structure as last time's sudoku generator.

Where it does not work in one click is the same place. There is no number for "is this dance cool?" Beat alignment rates a robot dance highly as long as it hits the beats, and FID scored real dance worse than generated dance. This scoring only picks "a choreography where nobody collides, things repeat and the singer is up front." The 8 beats that make people want to copy them were still written by a person. The two point moves were not chosen by the generator; they were made by hand.

If you want to try this

• Use a song with a fixed beat. Generated songs and electronic music have constant beat spacing that a grid catches exactly. For songs played by people, use a beat tracker and always check the result by ear.
• Split sections by the lyrics. Finding the first line of each section in a transcript was far more accurate here than splitting by audio features, because I wrote the lyrics and knew the answer.
• Solve the paths separately, and first. Do not decide moves and positions together. Decide formations and walking paths first, then lay moves on top. TCDiff, a 2025 group-dance study, also makes trajectories first and motion second.
• Do not check gaps with straight lines. When five people walk at once, straight paths are bound to meet somewhere. Measure the distance between each pair over time, and bend the paths when they get close.
• Look at several metrics together. Push beat alignment alone and you get a robot dance. Here, jittering by 0.3 beat left beat alignment almost unchanged while the unison fell apart.
• When you ask an AI, ask several times, and give it everything. The same request produced 2nd place and 164th. Leave out who sings and it puts the wrong person in the center.

What it can't do

• It is not a trained model. This generator is a hand-made move dictionary, rules and random search, not a model that learned from real dance data like EDGE or Lodge. A move that is not in the dictionary will never appear. Had 3D data of K-pop group dance been public, I would have gone another way.
• It was not checked by human eyes. I do not know whether the top-scoring choreography is also the best one to people. Studies run separate human evaluations; I did not.
• No physics. The figures do not balance. They do not fall when the center of mass leaves the feet, and a jump just goes up and comes down. About 8.4 cm/s of foot sliding remains.
• No fingers or facial expressions. In K-pop choreography, fingertips and faces matter a lot. These hands are a single rod; even the "drip drop" tap uses the whole hand rather than an index finger.
• It clings to the beat. Beat alignment of 0.46 is higher than human dancers. There is no groove of pushing or pulling the beat a little.
• The song is not perfect. The generated track blurs the pronunciation of the lyrics in places. The transcript wrote "ssoa" as "saya" to the end and missed one bridge line entirely.
• Do not put this choreography on a real stage as is. Real people would struggle to pass each other 0.7 m apart while swinging their arms, and 38 formation changes make for a very busy stage. Treat it as a draft for a choreographer to look at and fix.

In "Sonagi," the boy and the girl meet a few times by the creek and part. The five girls of this song change places thirty-eight times in two and a half minutes. Working out each of those paths, the machine did. Which 8 beats make people want to dance along is still chosen by a person.

If you enjoyed this, please leave a comment.

Sources

• History of notation: Labanotation (Britannica), Laban movement analysis (secondary), Benesh and other notations (JMU Libraries), BVH format (University of Wisconsin), Cunningham and LifeForms: SFU News, Nov 2019, Jeongjae Mudo Holgi: Encyclopedia of Korean Culture
• Data and models: AIST Dance DB ISMIR 2019, AIST++ · FACT arXiv 2101.08779 (ICCV 2021, Tables 1–2), Bailando arXiv 2203.13055, EDGE arXiv 2211.10658 (Sec. 4.3 on FID, PFC), FineDance arXiv 2212.03741, Lodge arXiv 2403.10518, POPDG arXiv 2405.03178, AIOZ-GDANCE arXiv 2303.12337, TCDiff AAAI 2025
• Choreographers and AI: Wayne McGregor, Living Archive, Google Experiments
• Korea: DanceTract's government R&D project Venturesquare, 2025-05-07, "Shy Boy" ruling, first instance Aju News, 2011-11-08, appeal Seoul High Court 2011Na104668, Korea Choreography Copyright Association Dailian, 2024-09-16
• "Sonagi": Encyclopedia of Korean Culture, "Sonagi", Hwang Sun-won. The lyrics were newly written without using sentences from the original
• Song generation: ACE-Step 1.5 (MIT), transcription: faster-whisper (medium), analysis: librosa, 3D: three.js
• The song, choreography generator, figures, stage view, scoring of 300 candidates, removal experiments, AI-plan experiment and charts were all made for this article

Image credits

• Photo of Rudolf von Laban: Wikimedia Commons, File:Labanotation1.jpg, author unknown, public domain
• All other figures and screenshots were made for this article