Robowaiter

Behavior Tree Generation · LLM × Optimal Planning

Integrating Intent Understanding and
Optimal Behavior Planning for
Behavior Tree Generation from Human Instructions

Xinglin Chen*, Yishuai Cai*, Yunxin Mao, Minglong Li↑ Wenjing Yang, Weixia Xu, Ji Wang

College of Computer Science and Technology · National University of Defense Technology

* Equal contribution  ·  ↑ Corresponding author

View on GitHub arXiv:2406.00965
Overview of the Robowaiter framework: from human instruction to executable behavior tree.

Abstract

Robots executing tasks following human instructions in domestic or industrial environments essentially require both adaptability and reliability. Behavior Trees (BTs) emerge as an appropriate control architecture for these scenarios due to their modularity and reactivity. Existing BT generation methods, however, either do not involve interpreting natural language or cannot theoretically guarantee the BTs' success.

This paper proposes a two-stage framework for BT generation: first employing large language models (LLMs) to interpret goals from high-level instructions, then constructing an efficient goal-specific BT through the Optimal Behavior Tree Expansion Algorithm (OBTEA). We represent goals as well-formed formulas in first-order logic, effectively bridging intent understanding and optimal behavior planning. Experiments in hospitality industry settings validate the proficiency of LLMs in producing grammatically correct and accurately interpreted goals, demonstrate OBTEA's superiority over the baseline BT Expansion algorithm in various metrics, and finally confirm the practical deployability of our framework.

Pipeline

A two-stage framework: LLM-driven intent understanding meets optimal behavior planning.

Robowaiter pipeline diagram showing instruction parsing and behavior tree expansion.

Demo

Three tiers of instructions — from a single intent to compositional logical goals.

Easy Level

"It's quite toasty inside, could you please lower the air conditioning temperature?"

Goal Low(ACTemperature)
Behavior tree for the easy-level instruction.

A Day at the Café

Live demonstration exhibited at ChinaSoft 2023.

RoboWaiter on GitHub

Citation

If you find this work helpful for your research, please consider citing:

@inproceedings{chen2024robowaiter,
  title     = {Integrating Intent Understanding and Optimal Behavior Planning
               for Behavior Tree Generation from Human Instructions},
  author    = {Chen, Xinglin and Cai, Yishuai and Mao, Yunxin and Li, Minglong
               and Yang, Wenjing and Xu, Weixia and Wang, Ji},
  booktitle = {Proceedings},
  year      = {2024}
}