Learning to Sample

2019 CVPR

摘要

原文	译文
Processing large point clouds is a challenging task. Therefore, the data is often sampled to a size that can be processed more easily.	处理大规模点云是一项具有挑战性的任务，因此，一般把点云下采样到数量较少的size，方便处理
The question is how to sample the data? A popular sampling technique is Farthest Point Sampling (FPS). However, FPS is agnostic to a downstream ap-plication (classification, retrieval, etc.).	问题是如何对点云进行采样？目前最常用的方法是最远点采样(FPS)，但是FPS对于应用的任务是不可知的(FPS是通用的一个采样技术，没有争对点云处理的不同任务做不同的适应)
The underlying assumption seems to be that minimizing the farthest point distance, as done by FPS, is a good proxy to other objective functions.	FPS潜在的最远点假设似乎对于点云处理的目标函数指一种不错的proxy。
We show that it is better to learn how to sample. To do that, we propose a deep network to simplify 3D point clouds.	本文提出学习如何采样，提出用网络来采样\简化三维点云
The network, termed S-NET, takes a point cloud and produces a smaller point cloud that is optimized for a particular task.	我们称他S-Net,S-Net的输入时一个点云，输出一个更小的对特定任务优化后的点云
he simplified point cloud is not guaranteed to be a subset of the original point cloud. Therefore, we match it to a subset of the original points in a post-processing step.	输出的点云可能不是输入点云的子集，所以我们提出一种匹配方法。
We contrast our approach with FPS by experimenting on two standard data sets and show significantly better results for a variety of applications.	我们在2个数据集上，将本文的采样方法和FPS进行对比实验。

创新点

1. A task-specific data-driven sampling approach for point clouds
2. A Progressive sampling method that orders points according to their relevance for the task

S-Net

按照文中的描述，先把task network预训练好，用的是PointNet。然后用S-Net输入原始点云，输入指定点数的点云，把这个点云经过匹配后送入Task Network里，然后训练S-Net。
训练S-Net用到的loss包括：
（1）$L_f(G,P)=\frac{1}{|G|}\sum _{g\in G}min||g-p||_{2}^{2}$Lf​(G,P)=∣G∣1​∑g∈G​min∣∣g−p∣∣22​
对所有生成的点$g$g要和原始点云$P$P相似。
（2）$L_m(G,P)=max_{g \in G}min_{p \in P}||g-p||_{2}^{2}$Lm​(G,P)=maxg∈G​minp∈P​∣∣g−p∣∣22​
对于最不相似的点要尽量相似。
（3）$L_b(G,P)=\frac{1}{|P|}\sum _{p\in P}min_{g \in G}||g-p||_{2}^{2}$Lb​(G,P)=∣P∣1​∑p∈P​ming∈G​∣∣g−p∣∣22​
希望生成的点不冲突，尽量分散在输入点云中。

这张图最能反应这篇文章的核心内容，S-Net的优势体现在当点数少的时候，S-Net可以将那些关键点取出，而FPS不行。

Matching

由于不能保证生成的点集$G$G 是原始点集$P$P的子集，所以需要加入一个匹配的步骤，作者用邻近点搜索方法。
这里要提一下原文中一句话

The matching process is only applied at inference time, as the final step of inference. During training, the generated points are processed by the task network as-is, since the matching is not differentiable and cannot propagate the task loss back to S-NET .

因为取点这一操作是不可逆不可导的，所以task loss是不能回传到S-Net的。之前考虑过从采样这一环节入手改进PointNet++，看到文章题目的时候以为作者也是做了这样的工作，其实不然。最大的问题和困难就在不能BP，如何设计可以学习的采样策略还是没有解决，sad。

ProgressiveNet

这部分是对S-Net的一个扩展，可以产生任意size的点云，作者页做了相关工作。

文献

从这篇文章里我发现了几篇不错有关点云的文章，待读

1. [2018 ACCV] Flex-Convolution
2. PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
3. Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds