Overview
Autonomous robotic systems are increasingly deployed in unstructured, open-world, and safety-critical environments, where sensing and perception are inherently imperfect. Classical modular autonomy pipelines often assume that perception provides a sufficiently accurate state estimate for planning and control. In practice, however, robots must operate under partial observability, uncertain semantic information, limited fields of view, noisy localization, and perception modules that may degrade under distribution shift.
At the same time, modern perception systems are shifting from geometric state estimation toward richer, semantic and language-conditioned representations. Advances in vision-language models (VLMs), vision-language-action (VLA) systems, and large-scale multimodal learning enable robots not only to recognize objects, but to reason about context, relationships, and the implications of their actions. These developments introduce new opportunities for more general and context-aware autonomy, but also raise fundamental challenges: how should such high-dimensional, semantic, and often uncertain representations interface with planning and control? How can semantic reasoning be translated into actionable decisions with reliability and safety guarantees? These developments blur the boundary between perception and decision-making, making the design of their interface a central challenge.
This workshop examines emerging challenges in tightly coupled perception, planning, and control under these new conditions. In particular, we focus on settings where decision-making must operate over uncertain, high-dimensional, and semantically structured representations, rather than well-defined state estimates. We are interested in both modular and end-to-end approaches, and the trade-offs between explicit modeling of uncertainty and implicit reasoning in learned systems. The workshop will bring together researchers from robotics, machine learning, controls, formal methods, and field robotics to foster interdisciplinary discussion on perception-aware autonomy. Our goal is to identify key open problems at the interface of perception and decision-making, evaluate emerging paradigms enabled by modern learned perception, and outline principled directions for building reliable autonomous systems in the real world.
Discussion Questions
-
How should uncertainty from perception—especially from learned and semantic models—be represented and incorporated into planning and control?
-
How can high-dimensional perception outputs be translated into representations, constraints, and objectives for decision-making?
-
What representations best support both reasoning and control (e.g., belief states, scene graphs, latent/world models), and how should they be constructed?
-
How do we reason about context, interactions, and temporal dynamics in perception-aware planning?
-
How do learned planning and control methods interact with learned perception, and what new challenges arise when both perception and decision-making are data-driven?
-
What new failure modes arise from modern learned perception systems (e.g., hallucination, distribution shift), and how should decision-making and control systems account for them?
-
What are the trade-offs between modular pipelines and end-to-end learning approaches in perception, planning, and control?
-
What is the role of formal guarantees and verification when perception is uncertain, learned, and semantically rich?
-
How should we evaluate perception-aware autonomy? What benchmarks, datasets, and metrics are needed to measure reliability and safety in real-world deployment?
Call for Papers
We invite submissions of extended abstracts to share novel ideas on topics relevant to the workshop themes, which include but are not limited to:
- Planning and control under sensing uncertainty
- Vision-based and learned perception (including VLM/VLAs)
- Active perception and information gathering
- Context-aware and semantic planning and control
- Classification, object, and semantic uncertainty
- Partially Observable Markov Decision Processes
- Sim-to-real transfer and uncertainty quantification
- Multi-agent interaction with semantic information
- Learned planning and control
We welcome both ongoing work and recently published results. Accepted contributions will be presented as posters during the workshop, with selected submissions invited for spotlight talks. All accepted abstracts and posters will be made publicly available on the workshop website. This is a non-archival venue, and submissions may be published elsewhere. Abstracts should be a maximum of 2 pages long (excluding references) in RSS paper format (LaTeX).
Important Dates
- Abstract submission: June 5, 2026 (11:59pm AoE)
- Notification:
June 12, 2026June 14, 2026 - Workshop: July 17, 2026
Invited Speakers and Panelists
Invited Talks
POMDP Planning: Where are we now?
Abstract
Partially Observable Markov Decision Processes (POMDPs) is the principled framework for sequential decision-making under uncertainty. It is also a powerful general framework for the decision-making side of robotics. In this talk, I will present our lab’s recent work in developing fully vectorised POMDP planner, which can be run fully in CPU or fully in GPU. This work enables POMDP planning to utilise massive parallelisation without complicated scheduling. I will briefly present how such a POMDP planner can be integrated with component-wise model learning to significantly reduce data requirements. If time permits, I will also present our work on using POMDPs to provide user-friendly assessment of robot’s safety.Assured Autonomy in Unknown Environments with Uncertainty-aware Perception
Abstract
Designing robots that can navigate unfamiliar environments while accomplishing complex tasks is a fundamental challenge in embodied intelligence. Although recent advances in AI-enabled perception have led to impressive empirical capabilities, these systems often provide limited reasoning about perceptual uncertainty, resulting in overconfident decisions and limited performance or safety assurances in unknown environments.In this talk, I will present a new autonomy architecture that enables robots equipped with AI-enabled perception systems to complete semantic tasks in unknown environments with a user-specified success probability. The proposed architecture integrates AI-enabled perception, conformal prediction-based uncertainty quantification, and planning within a unified framework that propagates calibrated estimates of perceptual uncertainty from perception to decision-making. This introspective reasoning capability over perceptual uncertainty enables robots to determine when to safely act versus when to actively gather additional information to reduce uncertainty. In turn, this yields closed-loop mission completion guarantees that are agnostic to specific sensor models and noise characteristics. The talk will conclude with simulation and hardware case studies that empirically validate these assurance guarantees and discuss open challenges toward achieving assured perception-enabled autonomy in complex real-world environments.
Hierarchical Representations for Robot Perception
Abstract
Hierarchical 3D Scene Graphs (3DSG) have emerged as an actionable and scalable representation for long-term autonomy incorporating metric, semantic, and topological information in the scene. However, the question of scene representation / scene abstraction is more fundamental -- what are the right scene representations / abstractions for a robot and how do we build them? I will discuss recent progress we have made in hierarchical scene representation (Hickory to Hydra++), and conclude with some thoughts on how abstraction / scene representation can help ameliorate imperfect sensing.Planning Through the Partner: Joint World Models of Physics and Intent under Imperfect Observation
Abstract
When a perception-planning-control loop includes a human partner, planning with the human is the paramount challenge. The partner can be characterized as a system with partial observability, uncertain dynamics arising from goal-directed behavior, and time-delayed closed-loop reactions -- all of which we can only probe only partially through quantitative and qualitative measures. Typical planning and control systems assume well-behaved dynamics and passive sensing; this assumption breaks for partner-aware planning at multiple levels. I will discuss how my group approaches this class of problems by learning the right mix of representations from data, folding reactivity into planning, and constructing safety filters that structurally respect human input. I will also discuss how we assess reliability using real and simulated human participants. I close with a set of open challenges we face in perceiving human partners across driving and robotics.Organizers
For inquiries, please contact: rss2026wpcis@gmail.com
Program
All times are local to the conference venue (July 17, 2026).
| Time | Session |
|---|---|
| 2:00 – 2:10 PM | Opening remarks |
| 2:10 – 2:40 PM | Invited Talk: Hanna Kurniawati -- POMDP Planning: Where are we now? |
| 2:40 – 3:10 PM | Invited Talk: Yiannis Kantaros -- Assured Autonomy in Unknown Environments with Uncertainty-aware Perception |
| 3:10 – 3:30 PM | Lightning Talks |
| 3:30 – 4:00 PM | Coffee break & poster session |
| 4:00 – 4:30 PM | Invited Talk: Rajat Talak -- Hierarchical Representations for Robot Perception |
| 4:30 – 5:00 PM | Invited Talk: Jonathan DeCastro -- Planning Through the Partner: Joint World Models of Physics and Intent under Imperfect Observation |
| 5:00 – 5:50 PM | Panel discussion & debate |
| 5:50 – 6:00 PM | Closing remarks |
Accepted Posters
- NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception.
- Belief-Guided Interactive Perception for Manipulation in Clutter under Noisy Proprioception.
- Uncertainty-resilient Formation Control in Maritime Buoy Swarms.
- Perception and Planning Framework for Agile UAV Landing on Marine Vessels in Rough Seas.
- Attention-Based Multi-MAV Planning and Control under Imperfect Sensing.
- A Unified AI-Driven Edge Framework for Autonomous Search and Rescue Drone Mission Planning.
- DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action Model.
- Task-Relevant Depth Quality Metrics for Suction Grasping.
- Inference on a Budget: Planning Over Imperfect Observations in Heterogeneous Multi-Robot Teams.
- Subterra: An Open-Source Validated Benchmark for GNSS-Denied Tunnel Navigation.
- SENTINEL: Quantifying Sensing Uncertainty on Range-Only LiDAR without Training Data.
- SensorPerch: Sense Wherever and Whenever it Matters.