From natural-language task descriptions to grounded, self-correcting robot behaviors — LLM symbolic planning · code-generated primitives · closed-loop effect verification.
Given a task set, LLMs heuristically propose promising action models, while a sound & complete BT planner verifies their completeness. Planning contexts guide the search — lifting GPT-4o consistency from 50% → 90%.
For each action model, VLMs sample control policies and hyper-parameters guided by environment feedback. A hierarchical Molmo + cuRobo + APIs policy reaches 62% success — well above end-to-end and rule-based baselines.
When no policy grounds a model, CABTO fuses planning & execution contexts so the VLM repairs missing preconditions and stale effects. Defect-repair success rises 12% → 74% with feedback.
Behavior Trees (BTs) offer a powerful paradigm for designing modular and reactive robot controllers. BT planning, an emerging field, provides theoretical guarantees for the automated generation of reliable BTs. However, BT planning typically assumes that a well-designed BT system is already grounded — comprising high-level action models and low-level control policies — which often requires extensive expert knowledge and manual effort. In this paper, we formalize the BT Grounding problem: the automated construction of a complete and consistent BT system. We analyze its complexity and introduce CABTO (Context-Aware Behavior Tree grOunding), the first framework to efficiently solve this challenge. CABTO leverages pre-trained Large Models to heuristically search the space of action models and control policies, guided by contextual feedback from BT planners and environmental observations. Experiments spanning seven task sets across three distinct robotic manipulation scenarios demonstrate CABTO's effectiveness and efficiency in generating complete and consistent behavior tree systems.
The CABTO framework comprises three phases. (1) High-level model proposal uses planning contexts for LLMs to heuristically explore the space of action models. (2) Low-level policy sampling uses execution contexts for VLMs to explore the space of control policies. (3) Cross-level refinement combines both planning and execution contexts to refine inconsistent action models — closing the loop between symbolic planning and physical grounding.
Averaged over 10 trials (max FC=3). ASR=action success rate, CSR=consistency success rate, FC=feedback cycles. w/o→w: without → with planning contexts.
| Robot | Task Set | GPT-3.5-Turbo | GPT-4o | |||
|---|---|---|---|---|---|---|
| ASR | CSR | ASR | CSR | FC | ||
| Franka | Cover | 60.0→66.7 | 40→50 | 100→100 | 100→100 | 0.0 |
| Blocks | 70.0→70.0 | 30→50 | 60.0→80.0 | 50→80 | 1.1 | |
| Dual-Franka | Pour | 80.0→96.7 | 70→90 | 66.7→100 | 60→100 | 0.6 |
| Handover | 80.0→90.0 | 70→90 | 56.7→90.0 | 30→90 | 1.3 | |
| Storage | 56.7→73.3 | 0→60 | 53.3→76.7 | 20→70 | 1.7 | |
| Fetch | Tidy Home | 53.3→56.7 | 40→50 | 53.3→90.0 | 30→90 | 1.3 |
| Cook Meal | 70.0→70.0 | 50→60 | 73.3→100 | 60→100 | 0.4 | |
| Total | 67.1→74.8 | 42.9→64.3 | 66.2→91.0 | 50→90.0 | 0.9 | |
Success (out of 10) for 5 typical action models across policy families. Molmo+cuRobo+APIs with execution contexts is CABTO's grounded policy.
| Action | OpenVLA | VoxPoser | ReKep | Molmo+cuRobo | APIs | w/o Ctx | with Ctx |
|---|---|---|---|---|---|---|---|
| Pick(obj) | 4/10 | 4/10 | 6/10 | 5/10 | 6/10 | 6/10 | 7/10 |
| Place(obj,loc) | 5/10 | 3/10 | 7/10 | 5/10 | 6/10 | 6/10 | 8/10 |
| Open(container) | 1/10 | 1/10 | 1/10 | 3/10 | 1/10 | 2/10 | 4/10 |
| Close(container) | 2/10 | 2/10 | 3/10 | 4/10 | 2/10 | 3/10 | 5/10 |
| Toggle(switch) | 2/10 | 1/10 | 4/10 | 6/10 | 5/10 | 5/10 | 7/10 |
| Total | 28% | 22% | 42% | 46% | 40% | 44% | 62% |
VLM defect-repair success (SR%), averaged over 10 trials ($N_{FC}\le3$). Pre/Add/Del = precondition / add effect / delete effect defects.
| Action & Defect | Textual | w/o Feedback | with Feedback | Avg. FC |
|---|---|---|---|---|
| PutIn · Pre: missing IsOpen(container) | 10% | 40% | 80% | 1.1 |
| Stack · Pre: missing Clear(obj_b) | 20% | 30% | 70% | 2.1 |
| Lift · Pre: missing Holding(r2,box) [dual-arm] | 10% | 80% | 90% | 0.3 |
| Pick · Add: unverified InReach(robot,obj) | 20% | 50% | 90% | 0.8 |
| Put · Del: stale At(obj,loc_old) | 0% | 20% | 40% | 2.4 |
| Total | 12% | 44% | 74% | 1.3 |
Single-arm and dual-arm Franka manipulation tasks in Isaac Sim — covering Cover, Blocks, and the cooperative dual-arm Pour, Handover, and Storage task sets.
Deployment of CABTO in OmniGibson. Given a task set, CABTO generates a complete and consistent BT system; for a specific task, BT planning produces the solution BT, which is then executed so the Fetch robot successfully achieves the goal.
@inproceedings{cai2026cabto,
title = {CABTO: Context-Aware Behavior Tree Grounding for Robot Manipulation},
author = {Cai, Yishuai and Chen, Xinglin and Mao, Yunxin and Hu, Kun and
Yang, Yaodong and Chen, Yuanpei and Yang, Wenjing and Wang, Ji and Li, Minglong},
booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026}
}