A first in Spanish cinema. Foley-VAE was used to create the sound effects of the first Spanish short film with AI-assisted Foley — a concrete demonstration of how generative audio can open new creative possibilities for film sound.
Watch
How it works
The system extends RAVE, a real-time variational autoencoder, trained here on a large library of natural sounds. Because the model is generative and operates on a latent space, you can:
- Reconstruct a recorded effect, transferring its character through the model.
- Blend two materials by mixing their latent representations, producing textures that don’t exist in the source library.
Listen
The first grid pairs original footstep recordings with their reconstructions. The second presents new effects generated by blending the latent characteristics of two materials.
Reconstructions
Footstep Foley recordings on different surfaces, each passed through the VAE and reconstructed.
| Example | Original | Reconstructed |
|---|---|---|
| Wood 1 | ||
| Wood 2 | ||
| Metal 1 | ||
| Metal 2 | ||
| Stone 1 | ||
| Stone 2 | ||
| Fabric 1 | ||
| Fabric 2 | ||
| Earth 1 | ||
| Earth 2 | ||
| Other 1 | ||
| Other 2 |
Generated material mixes
New Foley textures created by blending the latent characteristics of two materials.
| Example | Generated mix |
|---|---|
| Asphalt + wood | |
| Asphalt + mud | |
| Asphalt + wood | |
| Carpet + grass | |
| Carpet + wood | |
| Carpet + water | |
| Mud + rocks | |
| Mud + wood | |
| Grass + gravel | |
| Grass + wood | |
| Wood + snow | |
| Wood + puddle | |
| Wood + linoleum | |
| Marble + wood | |
| Metal + concrete | |
| Metal + wood | |
| Metal + puddle | |
| Wood + rocks |