COMP7712 Embodied Conversational Agents

2018

4.5

1 x 100-minute lecture weekly
1 x 100-minute computer lab fortnightly
1 x 200-minute project work weekly

COMP4712 has been successfully completed

Reasonable programming skills and a broad background in Computer Science, Information Technology, Software Engineering or other areas of Cognitive Science is sufficient. A background in Artificial Intelligence, Text Mining, Information Retrieval, Human Computer Interfaces, Human Factors, Linguistics, Psychology or the Creative Arts would be useful but is NOT required as appropriate background will be studied.

This topic examines animated anthropomorphic interface agents that are capable of engaging the user in real-time, multimodal dialogue. We commence with a review of both the theoretical underpinnings and practical applications of this technology. Subsequently, we develop the knowledge and skills that will be necessary to build such an agent. An early focus is on chatbot programming and dialogue management, with a strong basis in natural language processing. The notions of embodiment and autonomous agents are reviewed, and multimodal perception and synthesis are discussed as a means of enriching the interaction. This includes introductions to several key technologies, including computer vision, real-time character animation, and audiovisual speech processing. We furthermore address the roles of affect and nonverbal signals in the broader context of social computing and give students the opportunity to create their own embodied conversational agent in association with the (ARC/NHMRC) Thinking Head project.

Students of this topic will obtain an understanding of the theoretical and practical foundation, including interdisciplinary work from Psychology, Linguistics and the Creative Arts, that underlies current development and research in the field of embodied conversational agents. The topic will endow the student with the necessary skills to deploy systems that can work in an audio-visual or textual verbal mode, communicating freely with users in natural language. The means by which this is accomplished will also establish a relevant proficiency in system design, specification, implementation, evaluation, and communication.

At the completion of the topic, students are expected to be able to:

1. Understand the basic foundations to deal with the broad range of embodied conversation agents (ECAs), applications and literature, including producing and critiquing proposals and reports appropriate to the field
2. Develop an appreciation for the different components that make up an ECA and how to combine components from different systems and incorporate them into an information system
3. Work in a team towards constructing a human-computer interface agent that requires a large and diverse set of software skills and components
4. Understand a practical knowledge of character animation systems and understand the differences between realism and user acceptability
5. Have a deeper insight into the technologies underlying computer speech recognition and synthesis as well as a working knowledge of open-source projects
6. Script simple conversational agents and also recognize the value of more complex dialogue management systems that involve some form of reasoning
7. Understand the need for an ECA to have perceptual system (e.g. computer vision) and be aware of different techniques for realizing such systems
8. Understand what a knowledge base is and how ECAs can obtain new information or learn from their environment