Language Research and Technology
Flinders language research technology is largely focussed around two areas, getting computers to learn language, and getting computers to teach language, but there are many more aspects than these.
Computational Psycholinguistics and NLL
Our research in language learning seeks to balance two different goals:
- to model theories of human language acquisition (Psycholinguistics);
- to develop better Natural Language Processing Systems (NLL).
Prof. David Powers is one of the pioneers in the area of Natural Language Learning and sought to establish this balance as Founding President of the peak body and conference in this area, SIGNLL and CoNLL. Currently, NLL is dominated by statistical models and a Computer Science orientation, but SIGNLL and CoNLL continue to make efforts to connect with the Psycholinguistic community and encourage computational modeling of psychological theories of language acquisition, and our research remains strongly focussed on psychological and biological plausibility, not just for their own sake but as heuristics that help us to bias our learning algorithms in ways that will lead to human like language and social behaviour.
Our orientation towards approaches that address both of these goals lead to us having a particular focus and expertise in the application of unsupervised learning techniques. This endeavour faces major challenges, including a movement within Linguistics that claims that language is not learned but innate, on the basis of "Poverty of the Stimulus". Two of the factors that are missed by these opponents of the possibility of language learning are Grounding, Idiolect, and Anticipation.
Grounding refers to the impossibility of really understanding language without understanding the world in which we are embedded, and which we talk about. For this reason we work with Intelligent Robots , Simulated Robots or Enheaded Embodied Conversational Agents, including in particular the concept of a Hybrid World that is also an essential part of our language teaching system.
Idiolect refers to the fact that no two people actually learn exactly the same language. In fact, children negotiate a language that works for them across all contexts that they are in. While they are at home with their mother, they start to learn their mother tongue, their mother's language, but as soon as they are in contact with other people, including other children, and in particular siblings, and most especially twins, they adapt their language and invent new language as part of integrating into each of those subcommunities.
Anticipated Learning was a mechanism proposed independently by David Powers and Chris Turk and is discussed in detail in their joint monograph (1989). Powers proposed a requirement to have separate Recognition and Production Models to allow self-correction to operate, whilst Turk proposed the term "Anticipated Learning" for a theory in which previously heard utterances can trigger a kind of negative feedback that is missing according to "Poverty of the Stimulus". If a sentence is only slightly beyond the limits of our current language competence, it is optimal for learning and we seem to be able to start to hook in the new components. If we have made a slight grammatical or lexical error and produce an incorrect sentence, most of us will be aware that we become aware of it and self-correct. In fact the sentence need not even be grammatically wrong, it may just be semantically ambiguous. Our hearing/understanding model recognizes this on hearing the sentence, if not before, and either amends it on the fly, or adds a rider to disambiguate – or makes a joke of it (pun not intended).
Our research group on NLL/Computational Psycholinguistics thus has a focus not just on AI/NLP applications but specifically on psychologically plausible models and unsupervised learning, as well as Computational Cognitive Linguistics models that depend on linguistic mechanisms and learning algorithms based on the idea of similarity, or metaphor and paradigm. However, the focus on unsupervised learning also is important because supervised learning techniques cannot meaningfully be applied when there is no agreed gold standard theory that can be used to create data to train on – not to mention that if we had an agreed believed theory there would be no need for the unsupervised psycholinguistic modelling. This lack of gold standard theory and data also highlights the difficulty of evaluating the results of unsupervised learning – thus evaluation becomes another major focus of our group.
One of the main application areas for language technology is web search or information retrieval. Currently most research and all commercial search engines are based purely on statistics and have no real understanding of the text or its grammar: no syntax, no semantics, no ontology. Our research group in Information Retrieval and Visualization, also addresses an other issue – that currently there has been little success in replacing the list of hits with more visual graphical representations.
Interestingly current approaches to evaluation of supervised or hand crafted natural language processing systems are based on evaluation techniques drawn from Information Retrieval, in particular Precision, Recall and their derivatives (in particular F-Measure and Rand Accuracy, which are different kinds of averages of Precision and Recall). These techniques are even problematic in Information Retrieval, when there is no known set relevant documents to use for calculating Recall, and thus the only statistic that can be calculated is Precision. Even for fixed datasets for which a set of relevant documents has been determined by a set of human annotators, the set of relevant documents is something of a fuzzy concept dependent on individual preferences and judgement, but at least an estimate of the true negatives (correctly eliminated irrelevant documents) and false negatives (incorrectly eliminated relevant documents) can be estimated.
An important aspect of Information Retrieval is understanding the semantic distinctions that language conveys, and determining similarity of documents, words and phrases in terms of meaning, not just words. This thus connects our work on learning syntax (Leibbrandt and Powers, 2010-12) to our work on learning semantic relationships and understanding word similarity (Yang and Powers, 2005-10).
Entwisle and Powers (1998) showed there are severe problems with the use of Precision, Recall and their means when distributions are skewed, as they often are in Information Retrieval, or in semantic or syntactic labelling of words. In fact, it is possible to bias even a system operating at chance level to get very high values of all of these statistics. In particular, if 90% of the examples are positive, then guessing everything is positive gives you 90% Precision, 100% Recall with all possible averages lying in the range 90 to 100%.
Powers (2003) formally derived an Informedness measure that accurately estimates the probability (or the proportion of the time) that a system is making an informed decision rather than a chance level guess, and generalized it to the multiclass case under the sobriquet Bookmaker due to it "handicapping" the value of a decision the same way as in racing. In a series of subsequent papers, other techniques have been critiqued, in attempt to determine what value if any they have. Some equivalent measures have been found for the two-class case, but no existing measure delivers such a meaningful probablity for the general multiclass case: although ROC is a useful technique which is closely related to Informedness in the dichotomous case but AUC has very special properties that measure something other than the performance of the specific application (Powers, 2012bc); Kappa is also not a bad technique, but can be shown to be misleading when the learner bias differs significantly from the prevalences (Powers, 2012a), as in the case detailed above (Entwisle and Powers, 1998).
For unsupervised evaluation, none of the above techniques are directly usable athough Powers (2003) does discuss a heuristic method for comparing supervised classes with unsupervised clusters, and Pfitzner, Leibbrandt and Powers (2009) undertake a comprehensive review of techniques for comparison of clusterings, or of unsupervised classifications with a gold standard classification.
Our Talking Thinking Teaching Heads build on our expertise in Speech and Language technology generally, and applications also extend into the areas of companions for older people, or helpers for students new to the campus. The actual conversational agent technology currently built into these applications is currently very primitive and template based, and the applications have been chosen because of their amenability to this simplistic technology.
Many of our Talking Thinking Teaching Head application are educational in nature. The original concept of the Teaching Head was to turn around our language learning research and the robot world simulations we developed for simulated grounding of our computational natural language learners. The same scenes that are useful for teaching a computer English are also useful for teaching student English – but the task is actually a lot easier. You don't need a complete native speaker competence in the language to teach basic vocabulary and grammar. Moreover, if you don't understand what the student says to you in response to a question, it doesn't matter, because you know what the answer should have been according to the goals of the current lesson and exercise. This fits very nicely with the template approach that underlies most conversational agents, including the default version of our Thinking Heads.
An obvious development from teaching grammar is to teach phonology, or microphonology – the Teaching Head has the ability to show you exactly what you should have said, and how you should have said it, in any arbitrary face and voice – including yours! The concept of mirroring your actual speech as you give an answer with your poor learner's accent, and suspect grammar, followed by repeating back to you the correct answer, with a perfect accent, in your own face and voice – this is a key focus of our current work on the Teaching Head, and should provide a huge improvement in the ability of a student to learn the correct pronuncation and accent, as well as learning the syntax and semantics of the language. Of course, English is not the only language we could teach, and collaborators at TUB, Berlin and BJUT, Beijing are working on German and Chinese speaking versions of the head, as well as being interested in English Teaching Head for German and Chinese students.
However, another twist on the language Teaching Head is to teach English literacy and numeracy to English Speakers, Australians in our case, or indigenous Australians, or new immigrants to Australia. In fact, it was the indigenous Australian communities that got excited about this project, and the aim of the VALIANT project is to help both the community directly with literacy, numeracy and health issues, as well as to assist with the training of indigenous health workers.
A further twist on the language Teaching Head is to teach children with disabilities – our AudioVisual Speech Recognition work has shown that we have the potential to distinguish sounds by lipreading, that are traditionally thought to be indistinguishable, to be part of a singe viseme, so teaching lip reading seemed to be a natural extension. However, what the deaf community and their health teams really though was a key need which the Teaching Head could help with, was learning social skills. This very quickly lead to other communities and conditions where teaching social skills is important, and children with Autism Spectrum Disorders are the current focus of the AVAST project.
A related variant of the Teaching Head is a new project in the area of Motivational Interviewing. Here the aim is to help health professionals learn how to assist a patient in recognizing their needs, and developing an achievable plan to overcome them. Conversely, we can also exemplify the Motivational Interviewing technique, achieving a system that may actually be useful as an adjunct to some of our other Teaching Head systems, for motivating and encouraging people - in particular our Open Day Buddy and Study Buddy are aimed at helping students, whilst other systems including Clevertar Connect and MANA are aimed at helping older people live more independent, meaningful and fulfilling life.
For More Information...