Purpose
Our goal this year has been to explore reinforcement-learning methods to enable a robotic camera system to learn to make eye movements to locations of interest in its visual field. As part of this goal, we have sought to explore the use of a predictive world model, as implemented by a neural network, to help the vision system predict the consequences of its actions. Predicting the consequences of its actions can help the vision system speed up its learning process and improve its object-tracking abilities.

Experiments
A program was written to move the camera system such that an object of interest, in this case a circle, remained somewhere in the camera's visual field. The camera was kept 6.5 inches from a black wall on which a white circle was placed. Each camera displacement (pan, tilt) and corresponding visual circle displacement (deltaX, deltaY) was written to a data file. After this data was collected, plots were made between the visual circle displacements and their corresponding camera displacements (deltaX vs. pan and deltaY vs. tilt) to visualize these relationships.

From these plots, approximations of the relationships between the visual circle displacements and camera movements were obtained in the form of a mathematical function. We had no prior knowledge of the relationship between motor commands and visual displacements - this mapping was learned solely from the data we collected. This function was then used to simulate an internal forward model in the camera system. The camera was moved randomly such that a circle was kept in its image frame, and the forward model was used to predict the resultant visual displacement of the circle.

Results
The relationships between the visual circle displacements and their corresponding camera displacements were modeled with a single linear equation. The forward model seemed to do fairly well in predicting the displacements of the circle for a given camera movement. Average absolute error for deltaX was 2.4 pixels, and absolute error for deltaY was 3.3 pixels.

Current and Future Work
The inverse of the function described above can be used by the camera system to simulate an internal inverse model that predicts, for a given visual displacement, the action to realize that displacement. We are currently testing this inverse model, and using radial basis function (rbf) networks to learn the forward and inverse models from the data we collected.

We plan to use reinforcement-learning methods to get the camera to learn to move to an area of interest in its visual field in one movement. The forward model described above can be used to simulate the camera system and speed up reinforcement learning.

Lastly, we would like to explore a probabilistic model of the relationship between camera movement and visual displacement. We would like to use rbf networks to learn forward and inverse models that predict, respectively, the distribution of visual circle displacements for a given camera movement, and the distribution of movements to achieve a particular visual displacement.