Mateo Cámara

Comparison of Phase Audio Processing Methods for Variational Autoencoders

Authors: Mateo Cámara and José Luis Blanco

Abstract: This paper analyzes the impact of signal phase handling} in one of the most popular architectures for generative synthesis of audio effects: Variational Autoencoders (VAEs). Until quite recently, autoencoders based on the Fast Fourier Transform routinely avoided the phase of the signal. They store the phase information and retrieve it at the output or rely on signal phase regenerators such as Griffin-Lim. We evaluate different Variational Autoencoder networks capable of generating a latent space with intrinsic information from signal amplitude and phase. The Modulated Complex Lapped Transform has been evaluated as an alternative to the Fourier Transform. A novel database on beats has been designed for testing the architectures. Results were compared in objective, and subjective-equivalent terms with autoencoders on FFT and MCLT representations, using Griffin-Lim phase regeneration, multi-channel networks, as well as the Complex VAE. The autoencoders successfully learned to represent the phase information and handle it in a holistic approach. State-of-the-art quality standards were reached for audio effects. The autoencoders show a remarkable ability to generalize and deliver new sounds, while overall quality depends on the reconstruction of phase and amplitude.

Results using STFT

Original	Drumstick with wood	Drumstick with metal	Drumstick with terrasso	Drumstick with rubber	mallet with wood	mallet with metal	mallet with terrasso	mallet with rubber
Original
2D-VAE with STFT
3D-VAE with STFT
STFT + Griffin-Lim
CVAE with STFT

Results using MCLT

Original	Drumstick with wood	Drumstick with metal	Drumstick with terrasso	Drumstick with rubber	mallet with wood	mallet with metal	mallet with terrasso	mallet with rubber
Original
2D-VAE with MCLT
3D-VAE with MCLT
MCLT + Griffin-Lim
CVAE with MCLT