Gaze (aka “looking at something”) can be automatic (scanning the environment), strategic (getting a closer look at something interesting), and even communicative (indicating interest, anger, or eagerness to cooperate). I’m aiming for rules that would allow not just simulating an agent, but allow an agent to predict or explain another agent.

Following my evaluation of potential inference tools last week, I’ve tentatively settled on using either JIProlog or AmziProlog because Prolog is the only rule language that allows me to be expressive enough, and these two tools allow Java and Prolog to invoke each other (while Prolog is embedded in a Java Runtime (JVM)).

It’s important that the tool run in a JVM because I want outside users not to be restricted to using the same platform as I do, and because I want to use industry-standard libraries for graphics, etc.

The high-level architecture of my system (named ATOM = automated theory of mind) is:

• A set of Prolog facts for each agent representing that agent’s initial mental attitudes.
• A set of Prolog rules for each agent representing what mental attitudes cause others under what conditions.
• A Java object that simulates a physical environment. When the Prolog module infers an “attempt” attitude, this is converted into a call into the Java module to see what follows from the attempt. When the Java module determines a causal change that should be observable to an agent, then agent-specific “percept” facts are injected into the Prolog module and may trigger new inference there.
• A set of JUnit tests that swap out files for different initial states in Prolog (and different simulated environments in Java) to verify that the outputs match expected values. There may also be unit tests that swap out rules to test predictions about agents that veer from the “norm”.

Because a fair amount of research goes into the formulation of each rule, each rule will be in its own Prolog file accompanied by comments describing the scenarios that inspired the rule (also present as unit tests) and the particular pages in articles that inspired the rules or any changes to them. (This may seem fussy, but a career in reading related research tells me we all need to be much better at providing such “provenance”.)

As mentioned in the summary above, gaze is a good human behavior to start with. But read here for even better motivation: Vertegaal et al, 2001: “Eye Gaze Patterns in Conversations: There is More to Conversational Agents Than Meets the Eyes”.

Some initial generalizations I wanted to capture:

• The blurriness of peripheral percepts is high.
• Moving the eyes in the dir of a peripheral percept will change it to a focal percept, and whatever was focal will become peripheral.
• The blurriness of a focal percept is usually less than if it were peripheral.
• For intriguing resemblances with high blurriness, one would want less blurriness.
• For intriguing resemblances in focus, one usually wants to ignore peripheral resemblances that are less intriguing.
• For surprising peripheral items, one automatically looks even if intending not to.

I want to write rules for these in such a way that, not only could the rules drive the behavior of one agent, but they could also be used by that agent to explain the gaze behavior of another agent that one is focused on. That is, it’s the start of an automated “theory of mind”.