Assistive and Educational Technologies

Teaching Heads

Many of our Talking Thinking Teaching Head application are educational in nature. The original concept of the Teaching Head was to turn around our language learning research and the robot world simulations we developed for simulated grounding of our computational natural language learners.  The same scenes that are useful for teaching a computer English are also useful for teaching student English – but the task is actually a lot easier.  You don't need a complete native speaker competence in the language to teach basic vocabulary and grammar. Moreover, if you don't understand what the student says to you in response to a question, it doesn't matter, because you know what the answer should have been according to the goals of the current lesson and exercise.  This fits very nicely with the template approach that underlies most conversational agents, including the default version of our Thinking Heads.

An obvious development from teaching grammar is to teach phonology, or microphonology – the Teaching Head has the ability to show you exactly what you should have said, and how you should have said it, in any arbitrary face and voice – including yours!  The concept of mirroring your actual speech as you give an answer with your poor learner's accent, and suspect grammar, followed by repeating back to you the correct answer, with a perfect accent, in your own face and voice – this is a key focus of our current work on the Teaching Head, and should provide a huge improvement in the ability of a student to learn the correct pronuncation and accent, as well as learning the syntax and semantics of the language.  Of course, English is not the only language we could teach, and collaborators at TUB, Berlin and BJUT, Beijing are working on German and Chinese speaking versions of the head, as well as being interested in English Teaching Head for German and Chinese students.

However, another twist on the language Teaching Head is to teach English literacy and numeracy to English Speakers, Australians in our case, or indigenous Australians, or new immigrants to Australia. In fact, it was the indigenous Australian communities that got excited about this project: VALIANT.

A further twist on the language Teaching Head is to teach children with disabilities – our AudioVisual Speech Recognition work has shown that we have the potential to distinguish sounds by lipreading, that are traditionally thought to be indistinguishable, to be part of a singe viseme, so teaching lip reading seemed to be a natural extension.  However, what the deaf community and their health teams really though was a key need which the Teaching Head could help with, was learning social skills.  This very quickly lead to other communities and conditions where teaching social skills is important, and children with Autism Spectrum Disorders are the current focus of this project: AVAST.

This takes us into the area of Assistive Technologies, which is also a focal area within the Medical Device Research Institute. A new teaching head application in the health-oriented assistive and educational technology area is in Motivational Interviewing – at this stage training students in the health sciences through the provision of a simulated patient to interview. Physical and haptic patient simulation is an important aspect of the MDRI Medical Simulation group.


Educational Games

Games are an important tool for learning, both independently and in conjunction with the Thinking Head, and most of the games being developed by our Games, Haptics and Simulation researchers are either assistive or educational in nature - and many both have an assistive character and a training character.


Educational and Emotional Monitoring

Another new technology we are exploring is Audio-Visual Brain Computer Interface technology that can help us understanding where a student is in their learning curve for both physical and mental learning activities, or skill acquisition. This connects also research using the same technologies to monitor the cognitive, motivational and emotional states of a student, including task-related workload and situation awareness. 


For More Information...

For more information on this area, including research projects, courses and topics, or scholarships, please contact Professor David Powers.



Relevant Courses and Topics

The following awards and topics link directly to research within the AI Lab and final year projects are supervised for the courses shown (for further degree combinations see the individual course descriptions):

Bachelor of Computer Science (Honours)
Bachelor of Behavioural Science (Psychology)/Bachelor of Computer Science
Bachelor of Engineering (Biomedical)
Bachelor of Engineering (Electronics)
Bachelor of Engineering (Robotics)
Bachelor of Engineering (Software)
Bachelor of Information Technology (Honours)
Bachelor of Information Technology (Digital Media) (Honours)
Bachelor of Science (Honours) – Computer Science & Information Systems
Master of Engineering (Biomedical)
Master of Engineering (Electronics)
COMP3722 Theory and Practice of Computation
COMP3742 Intelligent Systems
COMP3751 Interactive Computer Systems
COMP3752 Computer Game Development
COMP4712 Embodied Conversational Agents
COMP4715 Computational Intelligence
COMP4716 Information Retrieval and Text Processing
COMP4717 Mobile Application Development
ENGR3721 Signal Processing
ENGR3741 Physiological Measurement
ENGR3771 Robotic Systems
ENGR4722 Haptic Enabled Systems
ENGR4761 Image Processing

Featured Product Range

  • Head X – Free Customizable Virtual Head developed under the ARC Thinking Systems SRI, "From Talking Heads to Thinking Heads". Free download for Windows systems (also tested on Mac virtual machines).

  • Head X Whiteboard – Prototype XML-based tool to allow lesson development by non-programmers

  • AVAST Social Tutor – Pilot version, with two lessons, and current version, with a growing number of curriculum-based lessons

Recent Successes

  • Reynolds, K., Owen, H. and Williamson, P. (2007-2009). ISim – a realistic intubation simulator using Virtual Reality technology. ARC Linkage: $195,000.
  • Burnham, D. K., Dale, R., Stevens, C. J., Powers, D. M., Davis, C. W., Buchholz, J. M., Kuratate, T., Kim, J., Paine, G. C., Kitamura, C. M., Wagner, M., Moeller, S., Black, A. W., Schultz, T. and Bothe, H. H. (2006-2010). From Talking Heads to Thinking Heads: A Research Platform for Human Communication. ARC Thinking Systems: $3,400,000.
  • Burnham, D., Cox, F., Butcher, A., Fletcher, J., Wagner, M., Epps, J., Ingram, J., Arciuli, J., Togneri, R., Rose, P., Kemp, N., Cutler, A., Dale, R., Kuratate, T., Powers, D., Cassidy, S., Grayden, D., Loakes, D., Bennamoun, M., Lewis, T., Goecke, R., Best, C., Bird, S., Ambikairajah, E., Hajek, J., Ishihara, S., Kinoshita, Y., Tran, D., Chetty, G. and Onslow, M. (2010). The Big Australian Speech Corpus: An audio-visual speech corpus of Australian English. ARC LIEF: $650,000.