An update on my gaze project… This diagram shows pre-formalization work (in AI) on a mapping of gaze actions, mental attitudes, and causal chains among them. The set of actions, and their conditions and effects, are being fleshed out by referring to standard textbooks on nonverbal communication. The mental attitudes, and their conditions and effects, are being fleshed out by referring to BDI models in AI. Eventually, this knowledge will be formalized as rules (probably in Prolog). This effort is made somewhat harder by having to write the rules in a way that they can be used either to “run” the mind of an agent or be used by the agent to explain/predict the observed actions of other agents. This AI microtheory should be useful in a variety of domains, but I’m especially interested in the problem of detecting when an elderly person in one’s care needs help. Eventually, I’ll want to add recognition of facial expressions, gestures, etc but one can tell a lot just from gaze: level of alertness, goals, level of satisfaction with one’s work toward a goal, emotions of fear and surprise, attempts to get one’s attention, and more. To give a sense of how this theory of mind could be applied to multi-agent interactions, here is a sample analysis of an imaginary interaction between Randy, Betty, and Greta. Randy admires Betty as she passes by, but she notices and has a negative reaction. Randy realizes he’s been caught out and tries to repair the situation, but doesn’t act quickly enough before Betty looks away. Greta is looking on (without being noticed by the others) and can predict what they each think and feel by the end.One could adjust the scenario in several ways:
- Betty is shy and avoids meeting Randy’s eyes before even realizing he’s leering.
- Randy anticipates a bad outcome from leering before doing so and is able to suppress it.
- Randy and Betty had been introduced a few days before, and Randy recognizes her only after starting to leer. This incident sours their cordial relationship.
- Before this encounter, Greta had been under the impression that Betty had a romantic interest in Randy, but this encounter changes her mind.
- At the end, Randy realizes Greta has been watching the whole time and gets mad (explain why!), or they share a laugh.