Incremental-learning for robot control

I-Jen Chiang, Jane Yung jen Hsu

Research output: Chapter in Book/Report/Conference proceedingConference contribution


A robot can learn to act by trial and error in the world. A robot continues to obtain information about the environment from its sensors and to choose a suitable action to take. Having executed an action, the robot receives a reinforcement signal from the world indicating how well the action performed in that situation. The evaluation is used to adjust the robot's action selection policy for the given state. The process of learning the state-action function has been addressed by Watkins' Q-learning, Sutton's temporal-difference method, and Kaelbling's interval estimation method. One common problem with these reinforcement learning methods is that the convergence can be very slow due to the large state space. State clustering by least-square-error or Hamming distance, hierarchical learning architecture, and prioritized swapping can reduce the number of states, but a large portion of the space still has to be considered. This paper presents a new solution to this problem. A state is taken to be a combination of the robot's sensor status. Each sensor is viewed as an independent component. The importance of each sensor status relative to each action is computed based on the frequency of its occurrences. Not all sensors are needed for every action. For example, the forward sensors play the most important roles when the robot is moving forward.

Original languageEnglish
Title of host publicationProceedings of the IEEE International Conference on Systems, Man and Cybernetics
Editors Anon
Number of pages6
Publication statusPublished - 1995
Externally publishedYes
EventProceedings of the 1995 IEEE International Conference on Systems, Man and Cybernetics. Part 2 (of 5) - Vancouver, BC, Can
Duration: Oct 22 1995Oct 25 1995


OtherProceedings of the 1995 IEEE International Conference on Systems, Man and Cybernetics. Part 2 (of 5)
CityVancouver, BC, Can

ASJC Scopus subject areas

  • Hardware and Architecture
  • Control and Systems Engineering


Dive into the research topics of 'Incremental-learning for robot control'. Together they form a unique fingerprint.

Cite this