Robot learning under occlusion

AREPO: Uncertainty-Aware Robot Ensemble Learning Under Extreme Partial Observability

Yurui Du1, Louis Hanut2, Herman Bruyninckx2, and Renaud Detry1,2

1 KU Leuven, Dept. Electrical Engineering, Research unit Processing Speech and Images, Leuven, Belgium
2 KU Leuven, Dept. Mechanical Engineering, Research unit Robotics, Automation and Mechatronics, Leuven, Belgium

AREPO revisits VAE-based visual reinforcement learning through uncertainty quantification and uses an uncertainty-aware ensemble to improve robot learning under severe visual occlusion, noisy observations, and sim-to-real shift.

Robot learningUncertainty-aware RLSim-to-real
PaperPDFCode

Method highlights

AREPO
Uncertainty-quantifying variational autoencoder

Uncertainty-quantifying VAE

Visual observations are compressed into denoised latent states while reconstruction uncertainty estimates occlusion-induced ambiguity.

Uncertainty-guided reinforcement learning exploration

Uncertainty-guided RL

Uncertainty is used to steer exploration and prioritize actions that reduce partial-observability-induced uncertainty.

Uncertainty-aware ensemble learning for sim-to-real transfer

Uncertainty-aware ensemble

The ensemble combines policies according to uncertainty, emphasizing agents expected to generalize better under target-domain occlusions.

Proxy task

Setup

Experimental setup

A proxy task reproduces extreme partial observability with a sand-sprinkling robot, a target region, and a vision system that must recover target-surface geometry under occlusion.

Why the task is hard

The target surface is only partially visible during operation. Dust and plume occlusions create corrupted heightmaps and can lead to poor control performance without uncertainty awareness.

Results in simulation

Figure
AREPO simulation results figure
Training stability and sampling efficiency are improved in simulation.

Results in the sim-to-real experiment

Figure
AREPO sim-to-real comparison figure
Real-world comparison: grayscale observations and heightmaps across deposition progress.

Zero-shot sim-to-real experiment

Final demo
Visual comparison of AREPO and baseline methods in the zero-shot sim-to-real experiment.