JeVJ
Audiovisual research, 2026. By Rivian Pratama. Making an LLM understand emotion in audiovisual media.
Music is full of emotion but that emotion is hard to put into words. This project asks whether a machine can sense a song the way we do and show that emotion as a picture.
The app downloads a track with yt-dlp and sweeps it with a DSP pass extracting band energies, onset envelope, beat grid, and Krumhansl-Kessler key estimate. The Jev model labels each passage with a mood vector and each moment with a transition kind. The system writes the answers to a cue timeline in track seconds, and the renderer converts the timeline to smoke that moves with the music.
I wanted to see if a machine could learn the emotion of a song and show it as something you can watch.