Seminar "Speech and Gaze in Virtual Agents" (SS 2011)

Seminar conducted by Alexis Heloir, Maria Staudte, Michael Kipp and Matthew Crocker

Number of participants is limited to 12, first come, first served. In order to register, please contact Alexis Heloir.

The Kick off will be held on Thursday April, 14th between 2 and 4pm in the room U.15 of the Computer Linguistics building (C7.1). Following slots are scheduled every Thursday between 2 and 4pm (see table below).


In this project seminar we will discuss recent technological advances in the control of virtual agent behaviors, and psychological experiments aimed at determining natural and communicatively effective agent behaviors. Apart from designing interactive gaze controllers, a particular emphasis will be placed on the combination of speech and gaze and on the principles that guide the integration of these modalities.

Course participants will work in small teams of 4-5 students mixing computer scientists and computer linguists. Each team will be in charge of designing, implementing and conducting empirical study. This study will aim at validating one particular hypothesis drawn from one precise aspect of multi-modal communication in the context of dialogue between a human user and an animated virtual character.

In the frame of this seminar, the following resources will be made available to the students:

  • The EMBR Character animation engine
  • A dedicated state chart based behavior authoring framework
  • A TOBI eye tracking device
  • Tutors from the psycholinguistics group
  • Tutors form the EMBOTS groups

The main goal of this seminar is to raise student's awareness about four key aspects of scientific research which are:

  • the ability to work efficiently in a multidisciplinary team,
  • the importance of defining clear and sound research hypothesis,
  • the scientific rigour necessary to conduct an empirical study,
  • the technical soundness and clarity of both implementation and exposÚs (in oral or written form).
Student's work will be assessed according to the aspects listed in the Requirements Section (below).

Seminar Organisation

14/04Kick off meeting CoLi building, room U.15 (Computer Linguistics Basement)--
21/04Tools and software presentationHeloir et Kipp, CS building, room 119slides, EMBOTS Framework, example package, EMBR animation language windows installer, mac version coming soon.
28/04Theoretical backgroundStaudte et Crocker, CoLi building, room U.15slides
05/05Technical assessmentHeloir et Kipp, CS building, room 119slides + homework + groups
12/05Project workplan presentation CS building, room 119slides + project workplan (PDPC) contact Daniel or Anand to set up a Tobii meeting. A short tutorial on the FSM capabilities of the EMBOTS framwork written by Daniel is available here.
19/05Implementation kickoff Heloir et Kipp, CS building, room 119slides | New version of the EMBR animation language
26/05Individual assistance (check office availabilities)--
02/06Individual assistance (check office availabilities)--
09/061st milestoneHeloir et Kipp, CS building, room 119slides + code
16/061st milestone reloadedCS building, room 119slides + demo
23/06Technical checkupCS building, room 119code
30/06Technical checkupCS building, room 119code and presentation
07/07User study starts--
14/07Seminar assignmentroom U.15slides

Requirements (deliverables)

  • project workplan (5 pages exposÚ) + presentation (8 min slides)
  • first milestone presentation (8 min slides)
  • implementation (code + documentation)
  • user study results (to be defined)
  • final exposÚ (to be defined)