语义分割是什么？

语义分割是典型的CV问题，输入为原始数据（如平面图像），输出为具有突出显示的感兴趣区域的掩膜。过程是通过查找每一个像素并根据其所属感兴趣的对象分配相应ID。

语义分割评价

• IOU=TP/(FP+FN+TP) 即交并比
• mIOU=mean(IOU) 即均交并比，在每个类上度量再求平均
• PA=TP/ALL 即像素精度
• mPA=mean(PA) 即均像素精度，是PA的简单提升，计算每个类内的正确分类像素比例再求平均。

语义分割相关概念可见论文：Zhou, Bolei, Zhao, Hang, Puig, Xavier,etal. Semantic Understanding of Scenes through the ADE20K Dataset[J]. International Journal of Computer Vision, 2016, 127(3):302-321.

语义分割数据集

• CityScapes 城市车道
• ADE20K 通用
• MSCOCO 通用
• CityScapes 3D 城市车道
• Pascal VOC 通用
• Mapillary Vistas 城市街道

经典算法介绍

以下是我整理的在理解各个算法时看到比较好的博客讲解（仅供参考），大部分博客中有论文原地址。

FCN

相关论文：Long J , Shelhamer E , Darrell T . Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4):640-651.
博客讲解：FCN的理解

Pyramid Scene Parsing Network

相关论文：Zhao H , Shi J , Qi X , et al. Pyramid Scene Parsing Network[J]. 2016.
博客讲解：论文笔记：《Pyramid Scene Parsing Network》

Deeplab V1 V2 V3 V3plus

相关论文：Chen L C , Papandreou G , Kokkinos I , et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
博客讲解：DeepLab系列之V1（点进去还有其他三个）

UperNet

相关论文：Xiao T , Liu Y , Zhou B , et al. Unified Perceptual Parsing for Scene Understanding[J]. 2018.
博客讲解：ECCV 2018 | 旷视科技提出统一感知解析网络 UPerNet，优化场景理解

Squeeze-and-Excitation Networks

相关论文：Jie, Shen, Samuel,etal. Squeeze-and-Excitation Networks.[J]. IEEE transactions on pattern analysis and machine intelligence, 2019.
博客讲解：Momenta详解ImageNet 2017夺冠架构SENet

Selective Kernel Networks

相关论文：Li X , Wang W , Hu X , et al. Selective Kernel Networks[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020.
博客讲解：选择核心网络——SKNet（解读）

Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks

相关论文：Hu J , Shen L , Albanie S , et al. Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks[J]. 2018.
博客讲解：NIPS2018深度学习(26)–点进去第二个

CBAM

相关论文：Woo S , Park J , Lee J Y , et al. CBAM: Convolutional Block Attention Module[J]. 2018.
博客讲解：【论文阅读】CBAM:Convolutional Block Attention Module

ENCnet

相关论文：Zhang H, Dana K J, Shi J, et al. Context Encoding for Semantic Segmentation[C]. computer vision and pattern recognition, 2018: 7151-7160.
博客讲解：语义分割–(EncNet)Context Encoding for Semantic Segmentation

PSANet

相关论文：Jia J . PSANet: Point-wise Spatial Attention Network for Scene Parsing[J]. 2018.
博客讲解：Spatial Attention Network for Scene Parsing

Attention is All you Need

相关论文：Vaswani A, Shazeer N, Parmar N, et al. Attention is All you Need[C]. neural information processing systems, 2017: 5998-6008.
博客讲解：《attention is all you need》解读

Non-local Neural Networks

相关论文：Wang X , Girshick R , Gupta A , et al. Non-local Neural Networks[J]. 2017.
博客讲解：Non-local neural networks

Asymmetric Non-Local Neural Networks

相关论文：Zhu Z , Xu M , Bai S , et al. Asymmetric Non-Local Neural Networks for Semantic Segmentation[J]. 2019.
博客讲解：ANN:Asymmetric Non-local Neural Networks for Semantic Segmentation 文章解读

DANET

相关论文：Fu J , Liu J , Tian H , et al. Dual Attention Network for Scene Segmentation[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020.
博客讲解：DANet论文及代码阅读笔记

GCNet

相关论文：Cao Y, Xu J, Lin S, et al. GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond[J]. arXiv: Computer Vision and Pattern Recognition, 2019.
博客讲解：CVPR2019之GCNet解读

CCNet

相关论文：Huang Z , Wang X , Wei Y , et al. CCNet: Criss-Cross Attention for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, PP(99):1-1.
博客讲解：CCNet

DNLnet

相关论文：Disentangled Non-Local Neural Networks ECCV 2020
博客讲解：解耦Non-Local模块，比传统Non-Local提升1.5-3个点 | Disentangled Non-Local Neural Networks

SFNet

相关论文：Xiangtai Li, Ansheng You et al. Semantic Flow for Fast and Accurate Scene Parsing. ECCV, 2020.
博客讲解：2020语义分割网络语义流：Semantic Flow for Fast and Accurate Scene Parsing

AlignSeg

相关论文：Zilong Huang et al. AlignSeg: Feature-Aligned Segmentation Networks. arXiv: 2003.00872.
博客讲解：AlignSeg 语义分割ADE20K数据集SOTA | Feature-Aligned Segmentation

OCRnet

相关论文：Object-Contextual Representations for Semantic Segmentation ECCV 2020
博客讲解：OCRNet: 目标区域上下文信息的特征表达 | ECCV 2020

HIERARCHICAL MULTI-SCALE ATTENTION

相关论文：HIERARCHICAL MULTI-SCALE ATTENTION FOR SEMANTIC SEGMENTATION ECCV 2020
博客讲解：【语义分割】分层多尺度注意力 Hierarchical multi-scale attention for semantic segmentation