模型的参数量与FLOPS计算

• 模型的参数量
  卷积层计算
  $$\begin{gathered} 模型的参数量=[卷积核的长\ast 卷积核的宽\ast 卷积核的高（即通道，由上一层的输出通道决定）]\ast \\ 卷积核的数量+偏置参数（其等于卷积核的数量） \end{gathered}$$
  全连接层
  由于不存在权值共享，它的FLOPs数目即是该层参数数目：$$N_{in}\ast N_{out}+N_{out}$$​ 。

• 模型FLOPs
  卷积层计算
  $$\begin{gathered} FLOPs数量=参数量\ast 该层输出特征图的大小 \\ 该层输出特征图的大小:h\ast w \end{gathered}$$
  
  
  全连接层 由于不存在权值共享，它的FLOPs数目即是该层参数数目：
  $$N_{in}\ast N_{out}+N_{out}$$ ​

深度学习中parameters个数和FLOPS计算（以CNN中经典的AlexNet网络结构为例） - Never-Giveup的博客 - CSDN博客
https://blog.csdn.net/qq_36653505/article/details/86700885


大话CNN经典模型：AlexNet - 雪饼的个人空间 - OSCHINA
https://my.oschina.net/u/876354/blog/1633143

• 矩阵乘法次数的计算过程
  矩阵乘法次数的计算过程 - Kellbook的博客 - CSDN博客 https://blog.csdn.net/qq_30622831/article/details/82730986

在计算FLOPs中，有

        kernel_ops = self.kernel_size[0] * self.kernel_size[1] * (self.in_channels / self.groups) 

见

rethinking-network-pruning/compute_flops.py at master · Eric-mingjie/rethinking-network-pruning https://github.com/Eric-mingjie/rethinking-network-pruning/blob/master/cifar/l1-norm-pruning/compute_flops.py

其中，self.group
pytorch的函数中的group参数的作用 - 慢行厚积 - 博客园 https://www.cnblogs.com/wanghui-garcia/p/10775851.html

例子1(VGG16 on imagenet)

图中计算，忽略了偏置参数（其等于卷积核的数量）

• LayerID1卷积核计算
  参数量
  $$\begin{gathered} Conv1\_1= \\ [(LayerID1,PatchSize(Height))\ast \\ (LayerID1,PatchSize(Width))\ast \\ (LayerID0,OutputSize(Channel))]\ast \\ (LayerID1,OutputSize(Channel)) \\ =(3\ast 3\ast 3）\ast 64 \\ =1728 \end{gathered}$$
  
  
  FLOPs计算
  $$\begin{gathered} Conv1\_1= \\ [(LayerID1,PatchSize(Height))\ast \\ (LayerID1,PatchSize(Width))\ast \\ (LayerID0,OutputSize(Channel))]\ast \\ (LayerID1,OutputSize(Height))\ast \\ (LayerID1,OutputSize(Height)) \\ =(3\ast 3\ast 3）\ast 64\ast 224\ast 224 \\ =1728\ast 224\ast 224 \\ =86,704,128 \end{gathered}$$

• LayerID12卷积核计算
  参数量
  $$\begin{gathered} Conv4\_1= \\ [(LayerID12,PatchSize(Height))\ast \\ (LayerID12,PatchSize(Width))\ast \\ (LayerID11,OutputSize(Channel))]\ast \\ (LayerID12,OutputSize(Channel)) \\ =(3\ast 3\ast 256）\ast 512 \\ =1,179,648‬ \end{gathered}$$
  
  
  FLOPs计算
  $$\begin{gathered} Conv4\_1= \\ [(LayerID12,PatchSize(Height))\ast \\ (LayerID12,PatchSize(Width))\ast \\ (LayerID11,OutputSize(Channel))]\ast \\ (LayerID12,OutputSize(Height))\ast \\ (LayerID12,OutputSize(Height)) \\ =(3\ast 3\ast 256）\ast 512\ast 28\ast 28 \\ =1,179,648‬\ast 28\ast 28 \\ =‭924,844,032‬ \end{gathered}$$

• LayerID22全连接层计算
  参数量
  $$\begin{gathered} FC1= \\ [(LayerID21,OutputSize(Height))\ast \\ (LayerID21,OutputSize(Width))\ast \\ (LayerID21,OutputSize(Channel))]\ast \\ [(LayerID22,OutputSize(Height))\ast \\ (LayerID22,OutputSize(Width))\ast \\ (LayerID22,OutputSize(Channel))] \\ =[(7\ast 7\ast 512)]\ast [1\ast 1\ast 4096] \\ =‭102,760,448‬ \end{gathered}$$
  
  
  FLOPs计算
  $$\begin{gathered} FC1= \\ [(LayerID21,OutputSize(Height))\ast \\ (LayerID21,OutputSize(Width))\ast \\ (LayerID21,OutputSize(Channel))]\ast \\ [(LayerID22,OutputSize(Height))\ast \\ (LayerID22,OutputSize(Width))\ast \\ (LayerID22,OutputSize(Channel))] \\ =[(7\ast 7\ast 512)]\ast [1\ast 1\ast 4096] \\ =‭102,760,448‬ \end{gathered}$$

• LayerID23全连接层计算
  参数量
  $$\begin{gathered} FC1= \\ [(LayerID22,OutputSize(Height))\ast \\ (LayerID22,OutputSize(Width))\ast \\ (LayerID22,OutputSize(Channel))]\ast \\ [(LayerID23,OutputSize(Height))\ast \\ (LayerID23,OutputSize(Width))\ast \\ (LayerID23,OutputSize(Channel))] \\ =[(1\ast 1\ast 4096)]\ast [1\ast 1\ast 4096] \\ =‭‭16,777,216‬ \end{gathered}$$
  
  
  FLOPs计算
  $$\begin{gathered} FC1= \\ [(LayerID22,OutputSize(Height))\ast \\ (LayerID22,OutputSize(Width))\ast \\ (LayerID22,OutputSize(Channel))]\ast \\ [(LayerID23,OutputSize(Height))\ast \\ (LayerID23,OutputSize(Width))\ast \\ (LayerID23,OutputSize(Channel))] \\ =[(1\ast 1\ast 4096)]\ast [1\ast 1\ast 4096] \\ =‭‭16,777,216‬ \end{gathered}$$