复现 ResNet

ResNet 算得上是超经典的backbone了，其网络提出了残差结构，可有效缓解随网络层数的加深而导致的梯度消失和梯度爆炸现象。结构和设计在这里有讨论。这里主要尝试一下复现。ResNet的常见形式有：

Name	Top-1 error	Top-5 error
ResNet-18	30.43	10.76
ResNet-34	26.73	8.74
ResNet-50	24.01	7.02
ResNet-101	22.44	6.21
ResNet-152	22.16	6.16

除了上述常见形式，也有将channel数减半的half形式，以及使用膨胀卷积的形式等。

note

本文的代码部分主要使用paddlepaddle实现。可能使用的包有：

import paddle
import paddle.nn as nn
from paddle.nn import *
import paddle.nn.functional as F

ResNet主要由两种基础结构组成：

• Basic Block（resnet18，resnet34）

class BasicBlock(nn.Layer):
    def __init__(self,
                 in_channels,
                 out_channels,
                 stride,
                 dilation=1,
                 shortcut=True,
                 if_first=False,
                 data_format='NCHW'):
        super(BasicBlock, self).__init__()
        self.conv0 = ConvBNLayer(
            in_channels=in_channels,
            out_channels=out_channels,
            kernel_size=3,
            stride=stride,
            dilation=dilation,
            act='relu',
            data_format=data_format)
        self.conv1 = ConvBNLayer(
            in_channels=out_channels,
            out_channels=out_channels,
            kernel_size=3,
            dilation=dilation,
            act=None,
            data_format=data_format)

        if not shortcut:
            self.short = ConvBNLayer(
                in_channels=in_channels,
                out_channels=out_channels,
                kernel_size=1,
                stride=1,
                is_vd_mode=False if if_first or stride == 1 else True,
                data_format=data_format)

        self.shortcut = shortcut
        self.dilation = dilation
        self.data_format = data_format
        self.add = layers.Add()
        self.relu = layers.Activation(act="relu")

    def forward(self, inputs):
        y = self.conv0(inputs)
        conv1 = self.conv1(y)

        if self.shortcut:
            short = inputs
        else:
            short = self.short(inputs)
        y = self.add(short, conv1)
        y = self.relu(y)

        return y

• Bottleneck Block（resnet50，resnet101，resnet152）