MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction
Modeling interaction dynamics to generate robot trajectories that enable a robot to adapt and react to a human's actions and intentions is critical for efficient and effective collaborative Human-Robot Interactions (HRI). Learning from Demonstration (LfD) methods from Human-Human Interactions (HHI)...
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Articolo |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2210.12418 http://arxiv.org/abs/2210.12418 |
| Tags: |
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1879174315450040320 |
|---|---|
| author | Prasad, Vignesh Koert, Dorothea Stock-Homburg, Ruth Peters, Jan Chalvatzaki, Georgia |
| author_browse | Chalvatzaki, Georgia Koert, Dorothea Peters, Jan Prasad, Vignesh Stock-Homburg, Ruth |
| author_facet | Prasad, Vignesh Koert, Dorothea Stock-Homburg, Ruth Peters, Jan Chalvatzaki, Georgia |
| author_sort | Prasad, Vignesh |
| building | arXiv |
| collection | Acceso abierto |
| description | Modeling interaction dynamics to generate robot trajectories that enable a robot to adapt and react to a human's actions and intentions is critical for efficient and effective collaborative Human-Robot Interactions (HRI). Learning from Demonstration (LfD) methods from Human-Human Interactions (HHI) have shown promising results, especially when coupled with representation learning techniques. However, such methods for learning HRI either do not scale well to high dimensional data or cannot accurately adapt to changing via-poses of the interacting partner. We propose Multimodal Interactive Latent Dynamics (MILD), a method that couples deep representation learning and probabilistic machine learning to address the problem of two-party physical HRIs. We learn the interaction dynamics from demonstrations, using Hidden Semi-Markov Models (HSMMs) to model the joint distribution of the interacting agents in the latent space of a Variational Autoencoder (VAE). Our experimental evaluations for learning HRI from HHI demonstrations show that MILD effectively captures the multimodality in the latent representations of HRI tasks, allowing us to decode the varying dynamics occurring in such tasks. Compared to related work, MILD generates more accurate trajectories for the controlled agent (robot) when conditioned on the observed agent's (human) trajectory. Notably, MILD can learn directly from camera-based pose estimations to generate trajectories, which we then map to a humanoid robot without the need for any additional training. |
| doi_str_mv | 10.1109/Humanoids53995.2022.10000239 |
| first_indexed | 2026-10-16T00:11:15Z |
| format | Article |
| fullrecord | <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ https://www.openarchives.org/OAI/2.0/oai_dc.xsd"><identifier>arxiv-2210.12418</identifier><datestamp>2023-01-24</datestamp>
<dc:title>MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction</dc:title>
<dc:creator>Prasad, Vignesh</dc:creator>
<dc:creator>Koert, Dorothea</dc:creator>
<dc:creator>Stock-Homburg, Ruth</dc:creator>
<dc:creator>Peters, Jan</dc:creator>
<dc:creator>Chalvatzaki, Georgia</dc:creator>
<dc:subject>Robotics</dc:subject>
<dc:subject>Artificial Intelligence</dc:subject>
<dc:subject>Human-Computer Interaction</dc:subject>
<dc:subject>Machine Learning</dc:subject>
<dc:description>Modeling interaction dynamics to generate robot trajectories that enable a robot to adapt and react to a human's actions and intentions is critical for efficient and effective collaborative Human-Robot Interactions (HRI). Learning from Demonstration (LfD) methods from Human-Human Interactions (HHI) have shown promising results, especially when coupled with representation learning techniques. However, such methods for learning HRI either do not scale well to high dimensional data or cannot accurately adapt to changing via-poses of the interacting partner. We propose Multimodal Interactive Latent Dynamics (MILD), a method that couples deep representation learning and probabilistic machine learning to address the problem of two-party physical HRIs. We learn the interaction dynamics from demonstrations, using Hidden Semi-Markov Models (HSMMs) to model the joint distribution of the interacting agents in the latent space of a Variational Autoencoder (VAE). Our experimental evaluations for learning HRI from HHI demonstrations show that MILD effectively captures the multimodality in the latent representations of HRI tasks, allowing us to decode the varying dynamics occurring in such tasks. Compared to related work, MILD generates more accurate trajectories for the controlled agent (robot) when conditioned on the observed agent's (human) trajectory. Notably, MILD can learn directly from camera-based pose estimations to generate trajectories, which we then map to a humanoid robot without the need for any additional training. </dc:description>
<dc:description>Accepted at the IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2022</dc:description>
<dc:date>2022-10-22</dc:date>
<dc:type>text</dc:type>
<dc:identifier>https://arxiv.org/abs/2210.12418</dc:identifier>
<dc:identifier>http://arxiv.org/abs/2210.12418</dc:identifier>
<dc:identifier>doi:10.1109/Humanoids53995.2022.10000239</dc:identifier>
</oai_dc:dc> |
| id | arxiv-2210.12418 |
| institution | arXiv |
| last_indexed | 2026-10-16T00:11:15Z |
| publishDate | 2022 |
| publishDateRange | 2022 |
| publishDateSort | 2022 |
| record_format | oai_dc |
| spelling | arxiv-2210.124182023-01-24 MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction Prasad, Vignesh Koert, Dorothea Stock-Homburg, Ruth Peters, Jan Chalvatzaki, Georgia Robotics Artificial Intelligence Human-Computer Interaction Machine Learning Modeling interaction dynamics to generate robot trajectories that enable a robot to adapt and react to a human's actions and intentions is critical for efficient and effective collaborative Human-Robot Interactions (HRI). Learning from Demonstration (LfD) methods from Human-Human Interactions (HHI) have shown promising results, especially when coupled with representation learning techniques. However, such methods for learning HRI either do not scale well to high dimensional data or cannot accurately adapt to changing via-poses of the interacting partner. We propose Multimodal Interactive Latent Dynamics (MILD), a method that couples deep representation learning and probabilistic machine learning to address the problem of two-party physical HRIs. We learn the interaction dynamics from demonstrations, using Hidden Semi-Markov Models (HSMMs) to model the joint distribution of the interacting agents in the latent space of a Variational Autoencoder (VAE). Our experimental evaluations for learning HRI from HHI demonstrations show that MILD effectively captures the multimodality in the latent representations of HRI tasks, allowing us to decode the varying dynamics occurring in such tasks. Compared to related work, MILD generates more accurate trajectories for the controlled agent (robot) when conditioned on the observed agent's (human) trajectory. Notably, MILD can learn directly from camera-based pose estimations to generate trajectories, which we then map to a humanoid robot without the need for any additional training. Accepted at the IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2022 2022-10-22 text https://arxiv.org/abs/2210.12418 http://arxiv.org/abs/2210.12418 doi:10.1109/Humanoids53995.2022.10000239 |
| spellingShingle | Robotics Artificial Intelligence Human-Computer Interaction Machine Learning Prasad, Vignesh Koert, Dorothea Stock-Homburg, Ruth Peters, Jan Chalvatzaki, Georgia MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction |
| title | MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction |
| title_full | MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction |
| title_fullStr | MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction |
| title_full_unstemmed | MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction |
| title_short | MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction |
| title_sort | mild multimodal interactive latent dynamics for learning human robot interaction |
| topic | Robotics Artificial Intelligence Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2210.12418 http://arxiv.org/abs/2210.12418 |
| work_keys_str_mv | AT prasadvignesh mildmultimodalinteractivelatentdynamicsforlearninghumanrobotinteraction AT koertdorothea mildmultimodalinteractivelatentdynamicsforlearninghumanrobotinteraction AT stockhomburgruth mildmultimodalinteractivelatentdynamicsforlearninghumanrobotinteraction AT petersjan mildmultimodalinteractivelatentdynamicsforlearninghumanrobotinteraction AT chalvatzakigeorgia mildmultimodalinteractivelatentdynamicsforlearninghumanrobotinteraction |