Mateo Cámara

Perceptual Test Questionnaire

Appendix: Test

In this appendix, we provide detailed information about the questionnaire given to participants for the subjective evaluation. The questionnaire was accessible online here. Originally written in Spanish, we include an English translation of each section of the questionnaire in the following pages. This translation aims to present the content accurately to English-speaking audiences and ensure a broader understanding of the subjective assessment process used in our study. The questionnaire’s structure and questions are meticulously replicated in English, retaining the essence and purpose of the original Spanish version. The listening questions are available online here.

Introduction & Control Questions

Introduction

Welcome to our subjective test on intelligibility and the voice synthesizer’s imitation capability compared to the human voice. Thank you for taking the time to participate.

The aim of this evaluation is to assess the automatic control of an articulatory synthesizer called Pink Trombone. The Pink Trombone is a tool that can mimic human speech by automatically controlling the movement of the tongue, lips, vocal cords, and other articulators.

Before you begin, take a moment to familiarize yourself with the Pink Trombone by visiting this link: https://www.dood.al/pinktrombone/

In this test, we seek to verify the accuracy with which the Pink Trombone can reproduce human vocalic sounds that can be correctly interpreted, as well as imitate a certain human speech recording. Please note the following details:

• In some questions, you will be asked to evaluate the interpretability of certain vowels or vowel sequences. Assess the vowels according to Spanish phonology (5 different vowels). Do not evaluate the quality of the synthesizer (if it sounds robotic, it is normal and acceptable).
• In other questions, you will be presented with a human recording as a reference and asked to evaluate recordings synthesized by the Pink Trombone, so we ask you to evaluate the ability to imitate (or to act, as if it were an actor) and not the resemblance as such (we understand coherence as if a person were trying to imitate another, but without being able to change their own vocal cords/throat/tongue etc…).
• Rate each attempt on a scale from 0 (not interpretable/very poor imitation) to 100 (completely interpretable/very good imitation) to indicate to what extent the reproduction is coherent with the reference.
• There will be a total of 21 tests. The complete test will last approximately 10 to 15 minutes. Make sure you are in a comfortable auditory environment, preferably with headphones, and click “Next” to continue.

If you have any questions, you can contact us at mateo.camara@upm.es

Questions

• Question 1: What age group are you in? 18 - 24; 25 - 34; 35 - 44; 45 - 54; 55 - 64.
• Question 2: What acoustic environment are you in at the moment? Quiet room; Noisy room; Public space.
• Question 3: How would you describe your hearing ability? I have normal capacity; I suspect that I have an impairment in one or both ears; I have a diagnosed hearing impairment.
• How would you describe your knowledge in the field of audio engineering? I am a professional working in this field; I am an audio enthusiast even though I do not work in this field; I do not pay special attention to this field.
• Training example. This is an example of what you will encounter during the test. Check that you can hear the sample sounds. You can take as much time as you need to evaluate what is asked in each question. Please read carefully what is being asked. In the slider bar you can leave your rating, where 100 points is the best rating and 0 the worst.

First Set of Questions: Interpret a Static Vowel

Introduction

Evaluate the ability to interpret a vowel.

For the following sounds you should rate the resemblance to the vowel indicated.

Rate from 0 (I do not interpret that vowel at all) to 100 (it is perfectly interpreted).

Questions

Available online.

Second Set of Questions: Interpret a Sequence of Vowels

Introduction

For the following sounds, evaluate the resemblance to a set of sounds indicated.

For the following sounds you will be asked how well you appreciate a sequence of vowels.

Rate from 0 (I do not interpret that sequence at all) to 100 (it is perfectly interpreted).

Questions

Available online.

Third Set of Questions: Interpret Imitations

Introduction

In the following questions you will be asked to evaluate the imitation ability of synthetic sounds with respect to a human reference sound. Note that the sounds can NOT be the same. You are asked to assess coherence, defined as if a person is trying to imitate another person, keeping in mind that you cannot change your own vocal cords/throat/tongue etc….

Evaluate the ability to imitate the sound with respect to a human reference. Listen to the reference first.

Rate from 0 (imitation is very poor) to 100 (imitation is very credible).

Questions

Available online.