Research, Innovation
- Type: Software, MaxMSP external
- Author: IMTR R&D team, IRCAM
- Collaboration: Conception, Interface Design and Experimentations
- Creation: First Released, Forum IRCAM, November 2012
- more info
- On the first use of the VoiceFollower in theater
The VoiceFollower is the first of a series of Max objects, we are currently developing, dedicated to gesture and sound interaction .
TheVoiceFollower indicates in real-time the time progression of live performance in comparison to a pre-recorded soundfile. This can be used for example to synchronize media processes to the live performance. It was successfully tested for theater with spoken or sung voice.
The VoiceFollower requires reading first a reference sound file, that appears on the object window. Markers can be then added to this reference recording. The synchronization will only work if the text used in the performance is identical with the one used in the reference recording. Thus, this object is not a voice recognition system, but rather a synchronization tool to work with a set of predefined texts. The message follow is used to start the real-time alignment: the audio signal received by the object is analyzed and appeared as superimposed to the reference recording. The object outputs continuously the time position and the markers labels.
Among possible applications of the VoiceFollower:
- Automatic triggering of synchronous events:
- Automatic state control: (Sound, lights, video, scenography…)
- Faked speech recognition, faked word-spotting.
- Automatic “sur-titrage” synchronisation.
- Augmented Lectures.
- Continuous and instantaneous following of the voice:
- Perfect stream synchronization to the performance (Sound, lights, video, scenography…)
- Faked Voice Conversion: Speaking with the voice of other.
- Reverse Karaoke: Accompaniment is following you.
- Continuous and instantaneous comparison of the voice:
- Real-time prosodic variations Estimation: (Intonation, intensity, speech rate, articulation degree, vocal quality…)
- Real-time estimation of the performance insights.
- Real-time decision making regards to the performer’s variations
So far, only the first line has been experimented and we are looking forward to test the others…
