AI Learns Faster by Drawing a Map of its World

A key challenge in artificial intelligence is the 'exploration-exploitation' trade-off: should an agent stick with what it knows or explore for better options? A new framework, called GNN-IRL, offers an elegant solution by giving the agent a curiosity-driven bonus.

This approach uses a Graph Neural Network (GNN)—a type of AI specialised in understanding relationships—to build a map-like model of all the possible states in its environment. The system then generates 'intrinsic rewards' for the agent whenever it visits new or rarely-seen parts of this map. This internal reward system encourages the agent to behave like an inquisitive explorer rather than just repeating safe actions.

In tests across four benchmark virtual environments, this graph-based exploration strategy outperformed state-of-the-art methods. The GNN-IRL agent learned faster and achieved better results, demonstrating a more effective balance between exploring its world and exploiting its knowledge. While the current modelling requires discrete states, it marks a significant step in creating more sample-efficient AI.