Organization responsible: UvA

People involved:

• Leon Kester (project coordinator)
• Bram Bakker (researcher)

Project description:

The “Hierarchical awareness in distributed agents”  project investigated artificial intelligence techniques that provide approximate or near-optimal solutions to complex situation awareness and associated control problems. The most interesting tasks concern sequential decision making, in which many actions are required to accomplish long-term goals.  Such tasks require the system to adapt to changing circumstances and cope with uncertainty in sensors and actions.

This project’s main researcher, Bram Bakker, reduced the time effort invested in this project in the course of 2008, and the project was finalized on November 1, 2008. There are a few remaining papers in preparation or under submission, and the work for this will be completed.

We focused in particular on the framework of (Partially Observable) Markov Decision Processes ([PO]MDPs) and on the distributed or multi-agent setting. Within these, we investigatde issues such as state estimation based on incomplete and noisy sensors, coordination of multiple agents (such as robots) to achieve common goals, and hierarchical approaches. We  used learning and planning algorithms (such as reinforcement learning and dynamic programming) to find approximate solutions to these complex problems. Theoretically sound probabilistic (Bayesian) methods play an important role in our approaches.

We were interested in real-world applications with direct relevance to society. Examples include crisis management, wherein intelligent systems must analyze crises and help human managers make decisions in real time, and traffic management, which can be optimized via intelligent agents.  With our partners in the DECIS lab, we worked on collaborative decision making for teams of agents that are part of a real world application. For agents embodied in the real world, collaborative decision making is a challenging task because the world is often stochastic and only partially observable, for example because of sensor noise or perceptual aliasing. Although optimal solutions are important, the focus of our research is on robust and scalable practical solutions.  Application areas within the DECIS lab include crisis control applications and intelligent traffic management, in each of which sensor noise, state uncertainty, and collaborative decision making play a role.

In particular, distributed and hierarchical approaches were investigated, and machine learning methods such as reinforcement learning were used. The experimental work was mostly concerned with traffic optimization, and in this way connects to many related ICIS projects, with which exchanges, cooperations, and interactions were realized. There was extensive collaboration with other researchers and students in individual projects focusing on individual subproblems of the overall problem.

Publications: