Specifically, in this expansion VLBiMan++, we have added several new components to enhance its functionality and comprehensiveness. (1) State-Aware Skill Abstraction: We introduce an explicit distinction between state-invariant and state-conditioned manipulation skills, allowing the framework to reason about how manipulation behaviors depend on object states rather than treating all demonstrated motions uniformly. (2) Generalized Object-State Adaptation: The original rigid-object adaptation is substantially extended to accommodate unseen object categories, geometric variations, articulated configurations, and representative deformable objects, providing a more general vision-language anchored adaptation mechanism. (3) Closed-Loop Trajectory Composition: We further develop progressive IK refinement, dynamic collision compensation, and closed-loop re-observation and re-composition, enabling the system to continuously recover from object displacement and execution-time disturbances without retraining. (4) Broader Generalization Evaluation: We systematically investigate the generalization boundary of one-shot bimanual manipulation across task composition, unseen objects and object states, cluttered scenes, heterogeneous dual-arm embodiments, dynamic disturbances, and prolonged deployment, substantially extending the evaluation scope beyond the conference version. (5) Extensive Real-World Validation and Analysis: We add a substantially larger set of real-robot experiments, detailed ablation studies, robustness evaluations, failure analysis, and long-duration deployment trails to provide a more comprehensive characterization of the proposed framework.

Abstract

Generalizable bimanual robotic manipulation requires a reusable task prior that can persist across increasingly diverse tasks, objects, scenes, embodiments, and execution conditions, while avoiding the prohibitive cost of large-scale robot demonstrations and policy retraining. In this work, we present VLBiMan++, an extended framework that expands the generalization boundary of vision-language anchored one-shot bimanual manipulation. Starting from a single human demonstration, VLBiMan++ performs task-aware decomposition to identify reusable and adaptable skill components, and employs vision-language grounded geometric adaptation to transfer these skills to novel configurations without retraining. Building on this foundation, we systematically extend generalization along five dimensions: task generalization through diverse and long-horizon skill compositions; object generalization across unseen categories, varying geometries, and more complex articulated or deformable objects; scene generalization under clutter, occlusion, and dynamic interference; embodiment generalization across heterogeneous dual-arm robotic platforms; and deployment generalization through prolonged closed-loop execution under repeated external perturbations. To support this broader scope, we further introduce object-state-aware adaptation and lightweight trajectory optimization mechanisms that accommodate changes beyond simple rigid-body pose variations while preserving reliable bimanual coordination. Extensive real-world experiments demonstrate that VLBiMan++ maintains strong task success and adaptation capability across these increasingly challenging settings. Overall, VLBiMan++ advances one-shot bimanual manipulation from demonstrating isolated transferability toward a more systematic and scalable framework for generalization across tasks, objects, scenes, embodiments, and long-term deployment conditions.

▶ Overview and Framework of VLBiMan++

Our contributions in this extended journal research are fourfold: (1) We introduce VLBiMan++, a substantial extension of our previous work VLBiMan, and formulate a Generalization Pyramid for one-shot bimanual manipulation that systematically studies generalization across tasks, objects, scenes, embodiments, and long-term deployment conditions. (2) We extend vision-language anchored adaptation beyond rigid-object pose transfer toward more complex object-state variations, including articulated and deformable manipulation, enabling the reusable prior to remain effective under substantially broader geometric and physical changes. (3) We establish a substantially expanded real-world evaluation suite covering unseen object categories, cluttered and dynamically perturbed scenes, new heterogeneous dual-arm embodiments, and prolonged closed-loop execution, providing comprehensive evidence of VLBiMan++'s robustness, versatility, and deployment stability. (4) We provide extensive analyses of the expanded generalization boundary, including failure modes, robustness under persistent disturbances, and cross-condition transferability, revealing both the practical strengths and remaining limitations of one-shot bimanual manipulation.

The expanded Vision-Language Anchored Bimanual Manipulation (VLBiMan++). Left: Taking pouring water as an example, we sketch the entire process of VLBiMan++ based on the one-shot demonstration. Right: Diagram illustration of the proposed Generalization Pyramid: Task ➝ Object ➝ Scene ➝ Embodiment ➝ Deployment. VLBiMan++ aims to investigate how far a single human demonstration can serve as a reusable manipulation prior across increasingly diverse tasks, objects, scenes, embodiments, and execution conditions.

Detailed framework of the expanded VLBiMan++. Taking the pouring water as an example, the paradigm consists of three stages (e.g., decomposition, adaptation, and composition) based on a given demonstration. VLBiMan++ can achieve generalization of unseen spatial placements, category-level new instances, non-rigid object state variations, and heterogeneous novel embodiments under the same task.

Representative Anchor Points. (Left Part) Illustrations of representative points for target objects in four tasks including pouring, reorient+unscrew, tool-use:spoon, and zip penbag. The first three tasks involve only rigid objects, while the last task involves an articulated object. These anchor points will be used to calculate the change in object position and orientation (not always required). (Right Part) Illustrations of extracted various anchor points for deformable objects in two tasks including coil cable and fold towel. We present the computational results for four test cases of each task. In addition, we also present intermediate results for handling linear or rectangular deformable objects.

▶ Hardwares, Tasks and Implementation Details

Left Part: The contralateral dual-arm platform (P1), and all manipulated object assets involved in ten previously defined bimanual tasks. Based on this platform, we have defined 10 representative bimanual tasks, including 6 primary tasks plugpen, inserting, unscrew, pouring, pressing and handover, 4 long-horizon multi-stage tasks reorient+unscrew, unscrew+pouring, tool-use spoon and tool-use funnel. Each object has been scaled down proportionally.
Right Part: The humanoid platform (P2), and all manipulated object assets involved in six previous and nine newly defined bimanual tasks. Based on this platform, we have defined 9 newly added bimanual tasks, including 6 rearrangement tasks flatting, reorient, flipping, upright place bottle_mug and place fork_spoon, 3 non-rigid targets related tasks zip penbag, coil cable, and fold towel. Each object has been scaled down proportionally.

Left Part (Algorithm 1): It is used to estimate the orientation of a rigid body with a distinct directional nature, and the input is the 2D mask of the manipulated rigid body. Right Part (Algorithm 2): It is used to calculate anchor points for two representative types of deformable objects. For rectangular objects, four ordered corner points on the boundary are extracted. For linear objects, the continuous skeleton and arbitrary quantile points are extracted.

▶ Visualization and Video Records of Real Robot Rollouts

⚠︎ Here, we present only the tasks and experiments that are new compared to the conference version VLBiMan. All of these new additions were completed on the platform P2. Other original experiments (including those in the main text and appendices of the conference paper) are not repeated here, and readers may refer to the website of the previous conference paper.

New Diverse Bimanual Tasks. Visualizations of newly added nine tasks executed on the humanoid-style dual-arm platform P2. They are defined to further confirm the universality of VLBiMan++, including (a) four left-arm or right-arm rearrangement skills, (b) combination of new basic skills for two long-horizon rearrangement tasks, (c) a complex task involving articulated bodies, (d) two dual-arm tasks involving linear or rectangular deformable objects.

Cross-Embodiment Transferability. Visualizations of two cross-embodiment transferred primary bimaual tasks (plugpen and handover) executed on the new humanoid-style robot (P1 ➝ P2). In addition, there are four other successfully transferred dual-arm tasks (inserting, unscrew, pouring and unscrew+pouring). Please refer to the project webpage associated with the conference version paper.

Super Long-Duration Execution. Video snapshots recorded from a third-person perspective for experiments of reorient on platform P2. It contains keyframes from 20 trials in which the left arm (cyan circle) or right arm (yellow circle) was used to upright a bottle lying down in an arbitrary orientation.

Citation

Acknowledgements

We acknowledge the providers of various hardware used in this project, including the Aubo-i5 robotic arm, Rokae xMate CR7 robotic arm, DH gripper PGI-80-80, Jodell Robotics RG75-300, and kingfisher binocular camera.

The website template was borrowed from Jon Barron and Zip-NeRF.