穿墙感知 ‘’Through-wall human pose estimation using radio signals‘’

发表在2018年CVPR会议上,MIT CSAIL 团队。
主要研究内容如下:
In this paper, we introduce RF-Pose, a neural network system that parses wireless signals and extracts accurate 2D human poses, even when the people are occluded or behind a wall.

优势:
Note how our pose estimator tracks the person even when he is fully occluded behind a wall. While this example shows a single person, RF-Pose works with multiple people in the scene just as a state-of-art vision system would.


Cross-model supervision:teacher-student model如图所示:
穿墙感知 ‘’Through-wall human pose estimation using radio signals‘’
Cross-modal teacher-student network that transfers the visual knowledge of human pose using synchronized images and RF signals as a bridge(利用同步的图片和无线信号作为桥梁来变换人体姿势的视觉知识)

(I,R): I 表示无线信号的热力图,R表示对应的图片。
老师网络T(-):I作为该函数的输入,并预测关键节点信用地图(T(I));
学生网络S(-): 学习预测无线信号的关键节点信用。
根据骨架结构,本文针对14个骨架节点进行预测和侦测。

在训练阶段所使用的评价量:
穿墙感知 ‘’Through-wall human pose estimation using radio signals‘’
上述公式表示学生网络预测S(R)与老师网络预测T(I)之间差异的最小。(学生网络训练对象)

损失函数是信用地图中每个像素的二进制交叉信息熵的之和
穿墙感知 ‘’Through-wall human pose estimation using radio signals‘’

各种情况分析和对比:

穿墙感知 ‘’Through-wall human pose estimation using radio signals‘’


待续。。。。。。