GOALS AND PURPOSE:
The main goal in the creation of StonyBot, the Stony Brook robot, is to design an autonomous walking machine that uses sensors and computer vision to sense its environment. StonyBot consists of a little walker used to walk forward and backward quickly, which is connected to two large legs that lift the little walker and rotate it with a turntable. A new camera vision system detects numeric digits and red objects, while a digital compass provides information about the robot.s orientation. StonyBot.s mechanics and electronics work together reliably to provide precise movement, maximum versatility, and impressive speed.

PROCESS:
The following is an overview of some of the computer science related tasks worked on by CREW funded student researchers.

Machine Vision System - The Machine Vision system is basically an implementation of a "simple" OCR (Optical Character Recognition) system. The purpose of such a system is that when the walking machine navigates through the world (the grid in the competition) it has a predetermined pattern of characters that lay in front of it as it walks. Currently, still images will be taken in 5-second intervals using an off the shelf Logitech Quickcam 3000 Pro for processing in the machine vision system. These images will determine the course of action for the robot to maneuver through its allowed world.

The reason it is designated as "simple" is because the character set was limited to only the following eleven characters: 1,2,3,4,5,6,7,8,9,S, and F.

General Overview of Machine Vision System

1. Assumption made
2. Acquire dirty image
3. Gaussian Smoothing
4. Binarize Image
5. Determine number of objects in image
6. Get rid of any artifacts, left only with the number that is wanted
7. Get bounding box of character
8. Image Rotation to a 90 degree position
9. Determine if 6, 8, or 9 first
10. If not 6, 8, or 9 then perform Center Line Method to determine if the character is a 1, 2, 3, 4, 5, 7, S, or F.
11. Finished!

More Detailed Description

1. Assumptions Made
a. Character Set is made from the following 11 characters; 1, 2, 3, 4, 5, 6, 7, 8, 9 , S ,and F.
b. All images are boxy looking (no rounding of corners)
c. Image size can vary
d. Image thickness can vary
e. Image rotation can vary
f. Image brightness and contrast can vary

2. Acquire "Dirty" Image
a. StonyBot's vision system was initially tested by using doctored images made in Photoshop of the characters it must recognize.
b. The next test was to test actual images of numbers taken with the Logitech QuickCam 3000 Pro.

3. Gaussian Smoothing
a. A 5*5 kernel (matrix) is used with a sigma value of 2.
b. The kernel is passed over the whole image.
c. This operation's purpose is to get rid of as much dirt as possible, like dust particles and the like, from the picture.

4. Binarize Image
a. The purpose is to separate objects of interest from the background.
b. The problem is that grayscale images being taken in are consistently changing.
c. The solution to this problem is variable thresholding.
d. In variable thresholding, an initial guess of a threshold value is made, and then the algorithm iterates over the image trying to get the average of the two regions (black and white). If the average agrees with the initial guess, then that was the proper threshold value, if not then the threshold value determined is now the new chosen one and the algorithm begins again. The algorithm finally stops when the threshold value does not change for a consecutive pass through the algorithm.

5. Determine the Number of Objects
a. The purpose is to find all the pertinent objects in the image, and then to distinguish which is the character of interest and get rid of all the other objects.
b. First all of the black objects that are separated by white borders are found.
c. This is implemented with a sequential algorithm that scans through the image and labels all the unique components in its path by looking at its neighbor.s labels as well.

6. Get rid of any artifacts so the picture is left only with the picture of a desired numeral.
a. In the end of the last step the objects all have distinct labels.
b. There may be several objects in the image, either the actual character desired, a line from the grid, or dirt.
c. So if anything is touching the sides of the image, it is immediately eliminated, since it is claimed to be a line, dirt, or a number that is too far past the border of the image to be recognizable.
d. Next all objects smaller than the largest object in the image are discarded, since it is initially assumed that the largest object that is clearly visible will be the desired character.
e. Now the picture should either contain nothing or just the character desired.

7. Determine Bounding Box
a. The purpose of doing this is to not waste computational time running calculations on the entire image when the character clearly might take up much less real estate on the image plane. Thus all computations would be performed on the bounding box of the character.
b. This is done by scanning from left to right (and getting the x1 coordinate), scanning right to left (and getting the x2 coordinate), scanning top to bottom (and getting the y1 coordinate), and lastly scanning bottom to up (and getting the y2 coordinate).

8. Image Rotation to a 90o position
a. The reason for rotating it to a 90o position is because the centerline method depends on a straight character as a human being does when looking at a number. The angle of rotation can be determined using the Vector digital compass. The initial heading will be stored at the start of an event. This value and the current heading can be used to determine the needed rotation angle.

9. Determine 6, 8, or 9
a. The purpose of this step is to find the number of holes in the image.
b. If there are two holes, then it must be an eight, if there is one hole than it must be either a 6 or 9.
c. The recognition of whether the unknown character is a 6 or a 9 is dependent on which section the hole is located in. For a 9, the hole would be in the top section, and for a 6, the hole would be in the bottom section.

10. Determine 1, 2, 3, 4, 5, 7, S, or F
a. The purpose of this method is to distinguish between the rest of the characters, those being 1, 2, 3, 4, 5, 7, S, or F.
b. To do this a method called the centerline technique is used where the algorithm simply counts the number of lines (or holes) going down the middle of the bounding box.
c. No Holes: 1, 4, or 7
One Hole: F
Two Holes: 2, 3, or 5
d. For No Holes
i. Scan center line and count number of black pixels and white pixels
ii. If black pixels > white pixels, it.s a character 1, else it.s either a 4 or 7
iii. If black line within top 20% or bottom 10% it.s a 7, else it.s a 4
e. or One Holes
i. F is the only character with 1 hole left
ii. Its an F!
f. For Two Holes
i. Scan from center of each hole to the right to determine which character it is.
ii. If black pixels encountered for both holes, then it.s a 3.
iii. If black pixels encountered to the right on top hole only, then it.s a 2.
iv. If black pixels encountered to the right for the bottom hole only, it.s a 5.

The system is fully functional as of this writing. However, there have been cases of failure or inaccuracy noted. The team is currently working vigorously to try to clean out the occasional inaccurate results. However, test runs with sample images doctored in Photoshop, and real images taken with a Logitech web cam were very positive and the characters were recognizable by the system nearly always.

The only real problem in the vision system itself is that the character S and 5 look alike to the system. But, on the grid where the images are taken, plus the fact that the artificial intelligence program keeps track of its current position, would solve this problem. Thus this would not be an issue for the robot navigation system. Currently a new method using statistical moments is being developed such that the character recognition is more robust and can be applied to a wider variety of characters.

AUTONOMOUS PROGRAMMING - The chief difficulties that need to be overcome to achieve autonomy are navigation and interaction with the world. Achieving this allows the robot to reliably move around the world, account for changes in the world, and respond to objects placed in it.

The World - The robot considers the 9 by 9 grid its world. When the robot is first initialized, it will note the direction it is pointing and will consider that direction to be the perceived North.

The robot navigates the world similarly to the way one might read a map. Up and down (forward/backwards) is considered North and South. Any distance traveled in the North or South direction will be noted in terms of Latitude. In a similar vein, the right and left direction is considered East and West, and any distance traveled in the East or West direction will be noted in terms of Longitude.

Navigating Using Sensors - The robot relies on sensory input to orientate itself and interact with the world. The robot also relies on sensory input to correct for possible errors resulting from inconsistencies in the walking surface, which may result in alterations in the desired velocity and direction traveled.

Sensor Overview:
1. Orientation is determined with two fluxgate compasses, one on the lower section and one of the upper sections.
2. Velocity is calculated using input from infrared line sensors.

Using Machine Vision To Determine Location In The World - Other than using the compass and the infrared line sensors, the robot is able to approximate its orientation and position using machine vision.

When the robot crosses a longitudinal or latitudinal line it can - using the onboard camera - take a picture of the square in front of it. Depending on the character recognized by the machine vision, the robot can obtain various amounts of information.

If the character is a 1, 3, 4, 8 or F the robot can estimate its current direction, by approximating the distance to the character and the angle at which the character is being observed. If the character observed is a 2, 5, 6, 9 or S the robot must in most cases rely on its knowledge of its current position to determine what exact character it is observing. This does not cause a great loss of functionality, since the robot must be significantly off with its estimated position to confuse a 6 with a 9 or either of the other characters. After the robot has differentiated between the possible characters, it can again obtain an approximate position by using the angle at which the character is observed and its distance.

Note that the capability of identifying the location of the cube for the object-seeking event has recently been added to the vision system. Due to this accomplishment, the distance sensor will no longer be used to locate the object.

The front of the robot.s bottom section is equipped with an infrared distance sensor. This sensor is used to identify low obstacles, such as a tire field.

Finally, the robot is equipped with an array of infrared line sensors under the side feet in the upper robot. These sensors can detect changes in the distance from the robots foot to the walking surface. Using these measurements, the robot can determine if the foot is placed over a small object, such as a cube.

Tracking The Internal State - The robot continually keeps track of its current state through the use of state variables contained in a simple data structure. Using these state variables, the robot is able to determine if it needs to raise or lower itself prior to turning, if it is about to reach the limit of a particular movement, and how each component of the robot it is situated compared to the home position. Whenever the robot moves one or more of its parts, the relevant state variables are updated to reflect these movements.

Determining Movement Of Robot Parts - To keep track of its state, the robot relies on the accuracy of the stepping motors. Since the motors are being driven by a constant frequency, the displacement of the individual robot parts relates directly to the time interval during which the motor is operating. Hence timed intervals are directly translated into displacements and corresponding state variables are kept up to date.

Tracking Body Direction - The direction of the body is defined as the direction of the lower section. This direction continually tracks relative to perceived North. This value is of key importance, since the direction of the body is the direction the robot will walk.

Tracking The Leg Height - The robot continually track the height of the side legs. This is necessary for the robot to determine if it is in a position where it can walk or if it is in a position to turn itself. Before each movement involving rotation of the upper or lower section, sliding the rack and pinion system, or walking the bottom robot the current leg position will be checked. If the current leg position is not the required position, the robot will first move the legs and then proceed to execute the original command.

Furthermore, prior to moving the side legs, the robot always checks the current state and the expected end state, to ensure that the legs will not be lower so far that they exit the corner brackets.

Tracking The Rack Position - The robot needs to keep track of the rack position relative to the extremes defined by the left and right side legs. This is done so the robot can center the rack before walking and so it can stop the rack when it reaches the extremities.

Tracking the rack position is also important to facilitate frame-walking capabilities. By knowing the current position and the maximum left/right positions the robot can rapidly and consistently move the frame from extreme to extreme and hence act as a frame walker.

Tracking the Rack Direction - The rack direction is always tracked compared to perceived North. The rack direction and the direction of the bottom section are tracked and controlled independently. Determining the rack direction relative to the direction of the bottom section is simply a matter of comparing the rack and body direction.

CONCLUSIONS AND RESULTS ACHIEVED:
Having redesigned the walking machine for the 2003 SAE Walking Machine Challenge, there is confidence among the team that StonyBot is a significant improvement from previous designs. By using the advantages gained by two basic robot designs and coupling those two independent designs together, combined with an advanced sensor system, a walking machine has been produced that is easy to control and has a great machine vision system. This lends itself to having applications outside the competition for which it was primarily designed. While some of the systems described in the preceding document will not be fully implemented for this year.s event, they will be used in future instances. Systems such as the computer based vision system and rugged .double-robot. design allow StonyBot great latitude so that it can be applied and reused in other future endeavors. We placed fourth among American Schools and ninth over in the 2003 SAE Walking Machine Challenge.

Much time was spent in getting StonyBot to the point of reliable movement, thus there are many future goals the team has. StonyBot has a microphone, a speech recognition system and a sound platform for testing them out. A future goal is to make StonyBot fully controllable through a human voice command system. We also have begun work on a new robot for the AAAI (American Association for Artificial Intelligence) conference this August in Mexico. Some goals for the AAAI robot will be special navigation techniques involving a camera, infra-red sensors and map-building algorithms, vision and recognition algorithms for motion-detection and determining where a large number of humans are located, and information processing algorithms to provide people with answers to various questions. For example, the robot may determine where a group of people are located, and go up to them and ask if they need help getting somewhere within the conference. Then a user may click on a graphical map displayed on the robot.s laptop, and the robot will guide the person to the desired location. The robot is considered a host and is to fulfill duties that a human host would be expected to fulfill.

PUBLICATIONS:
Team Website: http://www.ic.sunysb.edu/Clubs/sae/robot/index.html

Conference: Attended the Society of Automotive Engineers (SAE) Walking Machine Challenge in Mexico City, Mexico during the Spring semester.