Jiahang
Liu刘佳杭

WELCOME TO MY HOME PAGE!欢迎来到我的主页！

I am an M.S. candidate in Mechanical Engineering at Dalian University of Technology and a former Special Research Student at The University of Tokyo. My research explores multimodal fusion of vision and sEMG for hand-motion understanding, human-intent perception, and dexterous teleoperation.

我是大连理工大学机械工程学院硕士研究生，曾赴东京大学交流。我的研究聚焦视觉与表面肌电的多模态融合，用于手部运动理解、人体意图感知与灵巧遥操作。

I believe human-robot interaction is shaped by more than technology. Aesthetics is also fundamental to how interaction is perceived and experienced. This belief connects my engineering research with my work on industrial design projects.我认为，人机交互不仅由技术塑造，美学同样影响交互被感知和体验的方式。因此，我也参与了多个工业设计项目，并将工程研究与产品美学贯穿于同一套实践之中。

Email邮箱jiahangliupro@gmail.com Phone电话+86 135 5589 5355

Based in所在地Dalian, China大连，中国

ORCID0009-0005-8706-3198

Scroll向下

Foundation学术基础

Education教育经历

2025.10
— 2026.04

The University of Tokyo东京大学

Special Research Student · Graduate School of Engineering工学系研究科 · 特别研究学生

Tokyo, Japan东京，日本

2024.09
— 2027.07

Dalian University of Technology大连理工大学

M.S. Student · Mechanical Engineering机械工程 · 硕士研究生

Dalian, China大连，中国

2020.09
— 2024.07

Dalian University of Technology大连理工大学

B.Eng. · Mechanical Design, Manufacturing and Automation机械设计制造及其自动化 · 工学学士

GPA 3.35 / 4.00 · Social Practice ScholarshipGPA 3.35 / 4.00 · 社会实践奖学金

Peer-reviewed work同行评议成果

Publications发表论文

IEEE Robotics and Automation Letters · 2026 · First author

Precise Hand-Arm Teleoperation of a Dexterous Robotic Manipulator via Markerless Vision and High-Density sEMG Fusion

Jiahang Liu, Zhipeng Li, Renzhen Le, Xiao Zhang, Naohiko Sugita, Zhenzhi Ying, Liming Shu

IEEE/ASME Transactions on Mechatronics · 2026 · Accepted

VEMG-TeleGrasp: A Stiffness-Aware Teleoperation Framework for Dexterous Manipulation via Multimodal Fusion

Zhenzhi Ying, Wenbo Zhang, Jiahang Liu, Jiaze Sun, et al.

Project Experience项目经历

Research科研经历

Current research · Under Review当前研究 · 审稿中

Multimodal Hand Joint-Angle Regression from Egocentric Vision and Forearm High-Density Surface Electromyography基于第一视角视觉与前臂高密度表面肌电的多模态手部关节角回归

Dalian University of Technology / University of Tokyo大连理工大学 / 东京大学

2025.09 — PRESENT至今

DoF-Query multimodal fusion architecture

Egocentric RGB and motion-capture experimental environment

Reflective markers and forearm high-density sEMG acquisition hardware

Introduction研究简介

Continuous and accurate estimation of hand joint angles is important for intuitive human-machine interaction, yet a single sensing modality may not remain reliable across changing operating conditions. Egocentric RGB images provide hand appearance and spatial configuration, whereas forearm HD-sEMG captures muscle activation even when visual observations are degraded.连续、准确的手部关节角估计是实现直观人机交互的重要基础，但单一传感模态难以在不断变化的工作条件下始终保持可靠。第一视角 RGB 图像提供手部外观与空间构型信息，前臂高密度表面肌电则能在视觉信息受损时继续反映肌肉激活。

Method研究方法

We combine a Vision Transformer visual encoder, a Transformer-based EMG encoder, and a DoF-query fusion decoder to regress 20 hand joint angles from egocentric RGB images and 128-channel forearm HD-sEMG. An independent learnable query is assigned to each hand DoF, enabling joint-specific and context-adaptive integration of visual and EMG representations.我们结合视觉 Transformer 编码器、基于 Transformer 的肌电编码器与 DoF-query 融合解码器，从第一视角 RGB 图像和 128 通道前臂高密度表面肌电中回归 20 个手部关节角。每个手部自由度对应一个独立的可学习 query，从而实现面向关节、上下文自适应的视觉与肌电特征融合。

Results实验结果

R²

0.68

sEMG

0.82

Vision

0.88

Fusion

MAE

5.13°

sEMG

3.21°

Vision

2.79°

Fusion

More accurate, stable estimation. Fusion reached an overall R² of 0.88 and an MAE of 2.79°, outperforming vision only (0.82 / 3.21°) and sEMG only (0.68 / 5.13°), while showing the smallest cross-subject range.

Interpretable, context-adaptive fusion. Separate DoF queries learned different modality-attention patterns for individual joints and motion phases. Vision preserved spatial hand configuration, while sEMG supplied complementary dynamic cues when occlusion, finger overlap, or motion blur weakened visual evidence.

更准确、更稳定的估计。 融合模型的整体 R² 达到 0.88，MAE 为 2.79°，优于仅视觉模型（0.82 / 3.21°）和仅肌电模型（0.68 / 5.13°），同时具有最小的跨受试者变化范围。

可解释、上下文自适应的融合。 独立的 DoF query 针对不同关节与运动阶段学习了不同的模态注意模式。视觉保留手部的空间构型；当遮挡、手指重叠或运动模糊削弱视觉证据时，肌电提供互补的动态线索。

Egocentric image第一视角图像

Ground Truth真实值

Fusion (Ours)融合模型（本文）

Vision only仅视觉模型

sEMG only仅肌电模型

Precise Hand–Arm Teleoperation System via Markerless Vision and High-Density sEMG Fusion基于无标记视觉与高密度表面肌电融合的精准手-臂遥操作系统

Dalian University of Technology大连理工大学

2024.09 — 2025.07

Teleoperation is crucial for robots in human–robot interaction and collaboration. It requires natural and precise mapping of human intent across multiple scales of motion to complete tasks. However, current human–robot interfaces based solely on vision or physiological signals still face the challenge of incomplete information.遥操作是机器人实现人机交互与协作的重要方式。为完成复杂任务，系统需要以自然、精准的方式映射不同尺度下的人体运动意图。然而，当前仅依赖视觉或生理信号的人机接口仍面临信息不完整的挑战。

Estimated global hand-arm pose from RGB-D and validated markerless tracking against Vicon ground truth.基于 RGB-D 估计全局手臂位姿，并使用 Vicon 真值验证无标记跟踪精度。
Fused visual keypoints and HD-sEMG for gesture recognition and finger-force regression.融合视觉关键点与高密度肌电，实现手势识别和手指力回归。
Built a wireless ROS2 system mapping posture, gesture, and grasp force to a dexterous manipulator.构建无线 ROS2 系统，将姿态、手势与抓握力实时映射至灵巧机械臂。

View publication查看论文

A Distributed High-Density sEMG-driven Soft Glove System for Stroke Rehabilitation面向脑卒中康复的分布式高密度表面肌电驱动软体手套系统

Bachelor graduation thesis · Dalian University of Technology本科毕业设计 · 大连理工大学

2023.10 — 2024.07

Upper-limb dysfunction is one of the most common motor impairments following stroke. Conventional therapist-assisted passive rehabilitation is constrained by limited clinical resources and variable outcomes, whereas mirror therapy has demonstrated considerable rehabilitative benefits. As a bioelectrical modality for brain–computer interfaces, surface electromyography (sEMG) shows substantial potential in clinical rehabilitation and intelligent medical devices. These needs motivate the development of a new generation of intelligent, brain–computer-interface-based systems for post-stroke upper-limb rehabilitation. Accordingly, this study developed an intelligent upper-limb rehabilitation system that integrates mirror therapy with sEMG sensing.上肢功能障碍是脑卒中患者发病后多见的一种运动功能障碍。传统的被动人工临床上肢康复训练疗法存在着人工资源紧缺与康复效果不稳定等不足，而镜像康复疗法的康复效果十分显著。表面肌电信号（sEMG）作为脑-机接口生物信息的一种，在临床康复和智能化医疗器械中展现出了极大潜力。综上，新一代基于脑-机接口的智能卒中上肢康复系统亟待开发。因此，本文基于镜像康复疗法与表面肌电信号，搭建一套智能卒中上肢康复系统。

Developed a pneumatic glove supporting independent finger flexion and abduction.开发气动软体手套，实现手指独立屈曲与外展。
Designed an HD-sEMG sleeve and optimized electrodes through energy mapping and impedance analysis.设计高密度肌电采集袖套，并通过能量分布映射和阻抗分析优化电极布局。
Evaluated machine-learning models and adopted a PSO-optimized network for five-gesture classification.评估多种机器学习模型，最终采用 PSO 优化神经网络完成五类手势识别。

Form, structure, and product thinking形态、结构与产品思维

Design设计作品

Aesthetics shapes how interaction is perceived.美学，决定交互如何被感知。

As Design Director at Venture Electronics, I work across product form, enclosure architecture, and visual communication for personal-audio hardware. I also design research hardware at Dalian University of Technology, centered on high-density sEMG acquisition systems.

作为微翼音频设计总监，我参与个人音频硬件的产品形态、外壳结构与视觉表达。同时，我也在大连理工大学参与高密度表面肌电采集系统相关的科研硬件设计。

Design Director设计总监