Jiahang
Liu刘佳杭

WELCOME TO MY HOME PAGE!欢迎来到我的主页！

I am an M.S. candidate in Mechanical Engineering at Dalian University of Technology and a former Special Research Student at The University of Tokyo. My research explores multimodal fusion of vision and sEMG for hand-motion understanding, human-intent perception, and dexterous teleoperation.

我是大连理工大学机械工程硕士研究生，曾赴东京大学担任特别研究学生。我的研究聚焦视觉与表面肌电的多模态融合，用于手部运动理解、人体意图感知与灵巧遥操作。

I believe human-robot interaction is shaped by more than technology: aesthetics is also fundamental to how interaction is perceived and experienced. This belief connects my robotics research with my work across industrial design projects.我认为，人机交互不仅由技术塑造，美学同样影响交互被感知和体验的方式。因此，我也参与了多个工业设计项目，并将工程研究与产品美学贯穿于同一套实践之中。

Emailjiahangliupro@gmail.com Phone电话+86 135 5589 5355

Based in所在地Dalian, China

ORCID0009-0005-8706-3198

Scroll向下

Foundation学术基础

Education教育经历

2025.10
— 2026.04

The University of Tokyo东京大学

Special Research Student · Graduate School of Engineering工学系研究科 · 特别研究学生

Tokyo, Japan

2024.09
— 2027.07

Dalian University of Technology大连理工大学

M.S. Student · Mechanical Engineering机械工程 · 硕士研究生

Dalian, China

2020.09
— 2024.07

Dalian University of Technology大连理工大学

B.Eng. · Mechanical Design, Manufacturing and Automation机械设计制造及其自动化 · 工学学士

GPA 3.35 / 4.00 · Social Practice ScholarshipGPA 3.35 / 4.00 · 社会实践奖学金

Peer-reviewed work同行评议成果

Publications发表论文

IEEE Robotics and Automation Letters · 2026 · First author

Precise Hand-Arm Teleoperation of a Dexterous Robotic Manipulator via Markerless Vision and High-Density sEMG Fusion

Jiahang Liu, Zhipeng Li, Renzhen Le, Xiao Zhang, Naohiko Sugita, Zhenzhi Ying, Liming Shu

IEEE/ASME Transactions on Mechatronics · 2026 · Accepted

VEMG-TeleGrasp: A Stiffness-Aware Teleoperation Framework for Dexterous Manipulation via Multimodal Fusion

Zhenzhi Ying, Wenbo Zhang, Jiahang Liu, Jiaze Sun, et al.

Project Experience项目经历

Research科研经历

Current research · Under Review当前研究 · 审稿中

Multimodal Hand Joint-Angle Regression from Egocentric Vision and Forearm High-Density Surface Electromyography

Dalian University of Technology / University of Tokyo

2025.09 — PRESENT至今

DoF-Query multimodal fusion architecture

Egocentric RGB and motion-capture experimental environment

Reflective markers and forearm high-density sEMG acquisition hardware

Introduction研究简介

Continuous and accurate estimation of hand joint angles is important for intuitive human-machine interaction, yet a single sensing modality may not remain reliable across changing operating conditions. Egocentric RGB images provide hand appearance and spatial configuration, whereas forearm HD-sEMG captures muscle activation even when visual observations are degraded.连续、准确的手部关节角估计是实现直观人机交互的重要基础，但单一传感模态难以在不断变化的工作条件下始终保持可靠。第一视角 RGB 图像提供手部外观与空间构型信息，前臂高密度表面肌电则能在视觉信息受损时继续反映肌肉激活。

Method研究方法

We combine a Vision Transformer visual encoder, a Transformer-based EMG encoder, and a DoF-query fusion decoder to regress 20 hand joint angles from egocentric RGB images and 128-channel forearm HD-sEMG. An independent learnable query is assigned to each hand DoF, enabling joint-specific and context-adaptive integration of visual and EMG representations.我们结合视觉 Transformer 编码器、基于 Transformer 的肌电编码器与 DoF-query 融合解码器，从第一视角 RGB 图像和 128 通道前臂高密度表面肌电中回归 20 个手部关节角。每个手部自由度对应一个独立的可学习查询，从而实现面向关节、随情境自适应的视觉与肌电特征融合。

Results实验结果

R²

0.68

sEMG

0.82

Vision

0.88

Fusion

MAE

5.13°

sEMG

3.21°

Vision

2.79°

Fusion

More accurate, stable reconstruction. Fusion reached an overall R² of 0.88 and an MAE of 2.79°, outperforming vision only (0.82 / 3.21°) and sEMG only (0.68 / 5.13°), while showing the smallest cross-subject range.

Interpretable, context-adaptive fusion. Separate DoF queries learned different modality-attention patterns for individual joints and motion phases. Vision preserved spatial hand configuration, while sEMG supplied complementary dynamic cues when occlusion, finger overlap, or motion blur weakened visual evidence. The attention indicates selective internal feature use, rather than explicit detection of modality failure.

更准确、更稳定的手部重建。 融合模型的整体 R² 达到 0.88，MAE 为 2.79°，优于仅视觉模型（0.82 / 3.21°）和仅肌电模型（0.68 / 5.13°），同时具有最小的跨受试者变化范围。

可解释、随情境自适应的融合。 独立的 DoF 查询针对不同关节与运动阶段学习了不同的模态注意模式。视觉维持手部空间构型；当遮挡、手指重叠或运动模糊削弱视觉证据时，肌电提供互补的动态线索。这里的注意模式表示模型内部对特征的选择性使用，而不是对模态失效的显式检测。

Egocentric image第一视角图像

Ground Truth真实值

Fusion (Ours)融合模型（本文）

Vision only仅视觉模型

sEMG only仅肌电模型

Precise Hand–Arm Teleoperation System via Markerless Vision and High-Density sEMG Fusion

Dalian University of Technology

2024.09 — 2025.07

Estimated global hand-arm pose from RGB-D and validated markerless tracking against Vicon ground truth.基于 RGB-D 估计全局手臂位姿，并使用 Vicon 真值验证无标记跟踪精度。
Fused visual keypoints and HD-sEMG for gesture recognition and finger-force regression.融合视觉关键点与高密度肌电，实现手势识别和手指力回归。
Built a wireless ROS2 system mapping posture, gesture, and grasp force to a dexterous manipulator.构建无线 ROS2 系统，将姿态、手势与抓握力实时映射至灵巧机械臂。

A Distributed High-Density sEMG-driven Soft Glove System for Stroke Rehabilitation

Bachelor graduation thesis · Dalian University of Technology本科毕业设计 · 大连理工大学

2023.10 — 2024.07

Developed a pneumatic glove supporting independent finger flexion and abduction.开发气动软体手套，实现手指独立屈曲与外展。
Designed an HD-sEMG sleeve and optimized electrodes through energy mapping and impedance analysis.设计高密度肌电采集袖套，并通过能量分布映射和阻抗分析优化电极布局。
Evaluated machine-learning models and adopted a PSO-optimized network for five-gesture classification.评估多种机器学习模型，最终采用 PSO 优化神经网络完成五类手势识别。

Form, structure, and product thinking形态、结构与产品思维

Design设计作品

Aesthetics is an integral part of interaction.美学，决定交互如何被感知。

As Design Director at Venture Electronics, I work across product form, enclosure architecture, and visual communication for personal-audio hardware. I also design research hardware at Dalian University of Technology, centered on high-density sEMG acquisition systems.

作为 Venture Electronics 设计总监，我参与个人音频硬件的产品形态、外壳结构与视觉表达。同时，我也在大连理工大学参与高密度表面肌电采集系统相关的科研硬件设计。

Design Director设计总监