《ODE-inspired Network Design for Single Image Super-Resolution》(CVPR 2019)
————————————————
本篇博客为我写的第一篇原创博客,以此开始我的超分辨学习之旅。。
(该博客仅为个人对于此篇论文的理解,如有不当之处希望和大家多多交流)
本篇论文以常微分方程(ordinary differential equation ,ODE)的思想来设计残差网络,改变了以往残差网络设计中固定的Conv和ReLU的连接方式。
基本思想如下:
将欧拉系统描述为ODE的形式,如下:
通过该公式可以得到如下的映射:
其中表示SISR中的映射关系,也对应公式1中的解。
When the system is nonlinear, in many cases, there is no simple formula de-scribing the map, we have to turn to numerical methods.比如,利用欧拉方法来近似它:(可以直接从该公式开始看)
(4)
它可以看做一个ODE,也是(first-order method),which使用y′在宽度h的区间上的积分的近似值:
而对于残差模块,有:
因此,我们可以建立:
(6)
进而实现了欧拉方法和残差块之间的映射。
在本文中,利用欧拉方法设计两个类型的网络结构:LF-block and RK-block,which correspond to the Leapfrog method and Runge-Kutta method in numeri-cal ordinary differential equations。
LF-block:(second-order linear 2-step method)
rewrite the approximation of y′ in the form of
then derive the following equation
(也就是说,LF不仅仅设计残差块G,也改变了前边的y,使得n+1次的y不仅仅与n次的y有联系,还与n-1次的y有联系)which can be directly interpreted into CNN diagram using the definition in formula (6)。
In order to retain flexibility and obtain a block architecture, every three formulas above are grouped into a block as
RK2-Block: (two-stage second-order)
and replace ˜yn+1 with its first-order approximation (4), we will obtain the following equations:
Compared with ResNet, there are multiple branches in RK2-block, which is commonly used in recent popular networks.
(该RF仅仅是对残差块G进行设计,没有改变前边的y)
RK3-Block:
The knowledge of numerical ODEs suggest that higher-order methods (e.g., with order p) obtain a smaller local truncation error (e.g., O(hp+1)). This fact inspires us to explore higher stages Runge-Kutta methods.
Formally, explicit iterative Runge-Kutta methods can be extended to arbitrary n stages by using the following formulas
In particular, 3-stage Lunge-Kutta with third order can be described as:
不同的残差块的设计如下图所示:
The overall architecture is a Convolution-PixelShuffle framework:
(也就是说,将图1中的模块来带入到图2中的OISR-Blocks,就形成了本文所提出的OISR网络)
根据公式(4) 可知,通过自适应的调整 (in this case, learned parameter α in ParametricReLU ),可以同时提高算法的效率和稳定性。此外,对于α的初始化对应于数值常微分方程的精度与步长 有关,如果不是有限精度算法,截断误差会随着 变为0而变为0。(可以先忽略这一段)
G(·) : There is a large searching space to search G. Here, we only choose three different forms to illustrate the general effectiveness of ODE-inspired schemes. Each of these designs keeps at least one activation function and one convolutional layer, thus promising the nonlinearity.
依据上述理论,本文设计了三种不同规模的网络:
(1) small-scale networks: using LF-block and RK2-block for each G. Since different forms of G vary in computation overhead, these networks are developed using different numbers of building blocks but maintaining comparable computation and parameters. This ensures us to have a fair comparison with other lightweight models.
(2)middle-scale networks: using LF-block and RK2-block with a similar size and computing cost as the state-of-the-art MSRN . This also enables us to verify that the performance will not degrade as the networks become larger.
(3)deep network: using RK3-blocks.
实验部分:
将 OISR-RK2表现最差的 G-v2 使用到OISR-RK3中.部分仿真结果如下:
个人理解:
1.本篇论文中公式(4)的ODE方式实际上就是Resnet的基本形式。因此,利用该方程来设计网络,实际上主要是对于残差块的设计。LF在设计残差块的同时对前一部分进行相应的改动,使得网络的n+1次不仅与n次有联系,还与n-1次有联系。RF只对残差块进行设计,改变为二阶和三阶的方式(据说高阶的误差更小)。
2.虽然该网络是基于ODE的超分辨网络,但是论文设计重点实际上是网络的前半部分,而不是重点设计网络的超分辨部分。
问题: 该论文将残差块用G表示,但是ODE中的G主要是对于**h的选择和微分的计算,这与其设计的G模块(PReLU+Conv)有何联系?**论文中说PReLU是用来选择h,但是微分呢?该论文程序没有看懂。
程序:https://github.com/HolmesShuan/OISR-PyTorch