Towards Versatile Human-Machine Collaboration: Multi-Dimensional Preference Learning via Gaussian Bandits

Human-machine collaboration relies on the complementary strengths of humans and AI machines in real-world applications such as semi-autonomous driving and industrial automation. Achieving effective collaboration requires machines to not only align with human preferences but also to assess when such preferences are essential. In this work, we propose a Bayesian learning framework for developing versatile and adaptable human-machine collaboration. A crucial aspect of the framework is to guarantee robustness in changing environments across different system objectives, for which we formalize the sequential decision-making problem as a new constrained Gaussian best-arm identification problem, where multi-dimensional human preferences are modeled as implicit constraints. Our Dynamic Multi-Constraints Learning (DMCL) algorithm incorporates a novel kernel function to capture the latent temporal correlation and a Dynamic Constraint Selection (DCS) strategy to integrate essential preferences with minimal human involvement. We establish theoretical guarantees on regret and constraint violation and validate our approach through experiments in robotic control and autonomous driving, demonstrating robust and efficient preference learning for scalable human - machine collaboration.