Iris.4 Mobile Robot Group: Overview

Introduction

Welcome to the Mobile Robot Group's homepage; this webpage serves as an overview of the project - who we are, what we want to achieve, and the areas we need new researchers to help us with!

Virtual Robotics Lab: The Iris.4 Mobile Robot Project is building an honest-to-goodness physical, working robot. However, associated with this project we also have a Virtual Robotics Lab where students can build a virtual version of the Iris.4 robot. Here students learn about the hardware and the software elements that constitute Iris.4.

Robot Body, V2

Overview

This is a student research project where students do 95% of all of the work. Faculty and researchers serve in a primarily advisory capacity. These research projects are designed to develop products (in this case a working robot) that help students of all ages to explore the mysteries of 'Minds, Computers, Brains and Robots' -- among other things. We are partially funded by the NSF and NIH our various projects (interactive online experiments, multimedia-rich curriculum, the mobile robot, the virtual robot lab, etc.) are available online to educators, researchers, and anyone else out in cyber-space with an interest!

Iris.4 is the fourth incarnation of this effort. Our goal is for Iris.4 to be an autonomous mobile agent capable of understanding English via ProtoThinker, recognizing human faces, learning how to win at games (e.g. tic tac toe), interacting with her environment using a robotic arm, and much, much more. Below is a Flash animation of a robot that will have some of the same functionality as our Iris.4 robot.

The value of such a robot to the Mind Project stems from the role Iris.4 will play: Iris.4 will be a teaching tool used by instructors subscribing to the Mind Project's Curriculum services, built (replicated) and used in classrooms, to introduce students from kindergarten through college to robotics, AI, the philosophy of the mind, and many other interesting topics. Thus the long-term plan is to create and distribute Iris as a stimulating instructional tool, and it is this role that helps define the direction we are taking our mobile robot project.

Our Current Project

As of the Spring semester, 2006, the Robot Group is working on the tic tac toe(TTT) Project. The TTT project is an effort to tie together various existing hardware and software components, as well as to create new ones, to enable the robot to play a game of tic tac toe with a human. Of course, there's a catch: we want to interact with the robot exclusively in 'human' ways! Consider the following scenario:

You sit down at a table in the Mind Project lab and ask Iris to 'Come here.' Iris approaches the table. You speak, 'Iris, let's play tic tac toe.' Iris responds, 'Okay. Would you like to start, or should I?' You reply, 'I will start. I want to play against your neural net learning algorithm.' Iris launches the tic tac toe GUI and initializes her learning agent. Iris then picks up her tic tac toe board (she likes to carry one around wherever she goes :-) and sets it on the table, along with a black marker. You draw an X in the center square. Iris' vision system queries the Logitech Quickcam for a snapshot and processes it to determine the move you just made. The vision system then alerts the tic tac toe game controller about your move. The tic tac toe game controller informs the neural network-based learning agent, which then decides how Iris will respond. The response is returned to the tic tac toe game controller, which in turn triggers the arm script that tells Iris's Robix arm to draw an O in the upper left corner of the board. Play continues until you draw, at which point Iris thanks you for playing and asks you if you would like to play again. Declining, you get up and walk away. Iris picks up her board and marker and puts them away. The game is over.

By nature, the computer scientist is a problem solver. Reading the previous paragraph probably evoked an analytical response as you tried to figure out what it would take to build a robot capable of such feats. A few of the core components of such a robot include a message passing system for the different processes to communicate, a vision system that can perceive the state of a tic tac toe board given just a digital image of the board mid-game, a speech recognition system to capture commands and responses, as well as a natural language processor and interpreter to 'understand' what your English words mean, a text-to-speech module is necessary to speak the robots responses, a game logic engine must manage the whole process, and to decide what moves to play next, a learning algorithm must use its previous experiences to decide how best to play the current game.

At this stage, many of the pieces required to complete the tic tac toe task have been developed including the Central Control Program (a message-passing system and centralized point of control for a 'Mind Module' to plug into and manage the whole robot), the natural language processor and interpreter, a machine vision system (written by Luis Pimentel of the Technical Institute of Lisbon, Portugal), a learning algorithm, and a few other modules. However, there is still much to do! We need people working on the GUI/game logic controller, the arm controller, and several other projects. Aside from working on the tic tac toe project, we are also interested in adding a few other cool abilities to Iris' repertoire (e.g. playing with a set of colored blocks using English commands and deductive logic). In the next section, you can read about some of the areas/jobs we need your help with.

New Jobs list

  • Learning Algorithms

    Tic Tac Toe

    Create new algorithms, and/or re-implement an existing algorithm (our first tic tac toe learning algorithm, PRANO, was written many years ago and needs to be re-coded to work with the new Iris Software Environment).

    Other Games

    Supporting Software

  • Vision System

    Lettered Blocks Manipulation Game

    A partial order planner will be used to solve problems whose solutions require organizing a set of wooden blocks with letters on them.

    Playing with Toy Blocks

    We would like a relative chain-code based vision system that can identify the type and position of different colored and shaped toy blocks.

  • ProtoThinker

    We need a prolog programmer with a knowledge (or desire to learn about) Natural Language Processing to convert our existing WinProlog NLP engine to a new version of WinProlog. This job also requires stripping off GUI code and creating a network capable client-server model.

  • Neural Network Expert

    There are a number of projects that we need a Neural Network Experts help on.

  • Virtual Face

    Artists are needed to design the face and programmers are needed to animate it using visemes as input (i.e. implement face morphing).

Notes:

All software modules running on Iris.4 must be CCP-clients. The CCP allows for standardized communication and data sharing between processes running on the same or different computers, platforms, and operating systems. Any programming language supporting sockets over TCP/IP can be used. For C++ and Java programmers, many of the details for creating CCP-clients have been abstracted by a set of CCP classes. The CCP is a novel system and new researchers are not expected to know how to use it; we will teach you.

The following list is not complete - there are many other areas of robotics we would like to be involved in, including bottom-up robotics. Contact us with your ideas and interests!

Jobs Details

  • Learning Algorithms - Agent Development

    AI and Learning Algorithms are focal points of the mobile robot group. We would like to develop a variety of learning algorithms for both the tic tac toe project, and games in general. Any person with a serious interest in AI or learning algorithms should contact us. We are always open to giving Iris new abilities. Note: as with all Iris software, the AI/LA must be implemented as a CCP-client or should be 'wrappable' by a CCP-client. Also, we would like a common interface for all AI/LA modules performing the same task so that as GUIs or logic engines are developed, only one protocol needs to be developed between GUI/game logic and AI/LA modules.

    PRANO

    PRANO is our first tic tac toe learning algorithm. The software was written many years ago for a previous robot, but we would like to use it again with Iris.4. Unfortunately, the PRANO source code is mixed in with a colosal mess of other programs and we need someone to get it out! Another option would be to learn how the algorithm works and to simply re-implement it (this is probably the easier and better option of the two). We are looking for a C++ or Java programmer to study PRANO and extract it or re-implement it, and give it a CCP-client wrapping.

  • Vision System - Lettered Blocks Manipulation Game

    We are devising a simple game that can be solved by Iris using a Partial Order Planner to develop a solution, and a vision system to analyze images of the game environment. A set of toy blocks about one cubic inch in size with a stenciled letter drawn on the faces of each block will be used as the playing pieces. The blocks will be allowed in one of seven or eight locals, stacked five high, for a total of 35 to 40 positions. One possible game might be as follows. Fifteen blocks will be randomly placed in the available positions. Iris will be given a verbal command to spell a word (e.g. 'Iris, please spell your name). Iris's vision system will grab an image from her camera, process it, and determine the letters and positions of each playing piece. This will be fed into a partial order planner which will determine a solution (if one exists). Output from the partial order planner will be sent to a plan linearizer which will convert the partial plan into a sequence of move commands, which will then be fed to the script manager, which controls the arm. When a solution exists, the arm will carry it out and spell the word.

    An example of the layout, Robix arm, and blocks, ares shown below.

    (mostly) Lettered Blocks

    Blocks

    Layout

    Layout

    Top Down View of the Arm

    Arm

  • Vision System - Playing with Blocks

    One of the more 'flashy' abilities we would like to give Iris involves the robot playing with a set of toy blocks. In this system, a module could be issued a command from a human such as, 'pick up your red block, and stack it on your yellow block,' interpret the command, and break it apart into logical steps. With the aid of pre-scripted arm movements and a vision system, she could then carry out each of the logical steps to satisfy the request. The first thing we need to do to get this system up and running is develop a means of identifying objects and their positions. Their positions may be any of a set of predetermined positions, and their composition any of a predetermined set of blocks (different colors, shapes, sizes). Therefore any vision system that can analyze bitmap images of the playing field and report back block types and positions would be acceptable to us.

  • ProtoThinker - Prolog GURU

    ProtoThinker (PT) is a natural language processor and interpreter that can have dialog with human users, hold beliefs about the world and itself, use deductive reasoning to infer conclusions, and much, much more. Both PT's program logic and GUI were written in Prolog. Recently, our PT/Prolog expert left ISU to start a new career in the quad-cities. We desperately need a replacement for him. Ideally, a person who could fill this role will have a good understanding of sentence composition and grammar rules of the English language, some knowledge of Prolog, and a strong motivation to learn more about each. We are going to be using PT as the centerpiece of communication with Iris. We have Speech to Text and Text to Speech modules that hook up to PT to provide Iris the abilities of both speech and hearing. Anyone who has experience with Prolog (or a desire to gain some) and wants to learn more should contact us immediately about this position.

    Once we have a PT expert (someone who knows Prolog and has worked with the PT code), we would like to have that person split PT into pieces so that the GUI is stripped off and the processing modules are broken up. We would then like to implement a client-server module for PT. In the short run, we will be using PT mainly as a language module for Iris. In the long run, we would like develop an online version of PT so that users around the world can connect to PT over the Internet and have conversations with it.

  • Neural Network Expert

    Much of our future research will depend upon the correct use of Neural Networks, particularly in areas such as Vision Systems and Learning Algorithms. The Robot Group would like to have an in-house neural network expert to help out with these projects, as well as write up documentation, tutorials, and discussions about neural nets. While we would prefer a person who is already familiar with neural networks, we would also be happy to work with a Freshman or Sophomore who may not know much about NNs right now, but wants to learn more.

  • Virtual Face - Visemes => Face Morphs

    The output of the text-to-speech module includes the output of visemes. Visemes can be associated with facial expressions in such a way that the visemes derived from a sentence can be used to create a series of expressions/facial positions that imitate the way a human face would look when speaking that particular sentence. We already have a very simple (and rather ugly) face that can take visemes as input and shift through facial expressions. We would like to replace this face with a much more attractive face and create an effective morphing function so that the face looks natural as it slowly interpolates between the before and after images of each viseme. Thus we need either a person who is both 'artistically inclined' and knows how to program, or a pair of people to accomplish this goal. Faces can be created using 3D-Studio Max or any of a number of software packages. Anyone interested in helping us create the face/head, or working on other aspects of this project should contact us immediately.

Old Stuff

This section contains old, outdated materials that have appeared elsewhere on the Mobile Robot Homepage.

  • You can read the State of the Robot address to see get a snapshot of our work as of July 2004.