{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第2章 卷积"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 简介"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在如今计算机视觉领域，卷积（convolution）可能是出现得最频繁的术语。当然这是因为卷积神经网络被广泛地用于解决多种计算机视觉任务，已经成为计算机视觉领域的一种默认的工具。但是卷积这个术语并不来自于计算机科学领域，而是来自于信号处理领域。卷积是信号处理中的一个重要操作，用于计算线性系统对输入信号的输出响应。什么是卷积呢？顾名思义，就是先“卷”，再“积”。用专业术语来说，卷积的输入是两个函数，一个为输入信号函数，另一个为系统函数，卷积操作先将一个函数翻转，然后将其对另一个函数进行滑动叠加，最后得到输出结果（如图 2-1）。在函数连续的情况下，叠加指的是对两个函数的乘积求积分；而在离散情况下是对函数加权求和，为简单起见我们统一称之为叠加。\n",
    "\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://storage.googleapis.com/kagglesdsdata/datasets/2171976/3643430/pic1.png?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=databundle-worker-v2%40kaggle-161607.iam.gserviceaccount.com%2F20220522%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20220522T053420Z&X-Goog-Expires=345599&X-Goog-SignedHeaders=host&X-Goog-Signature=99a1f64fa0a5a4bebca2570a97c4f8bdbbca635f793026360fb565b13db12f469400347f0ff89639a4283507a6abb1fcf0577d3b3c5eca9e74b3ffb5d73282f03c6eb5a67010a571ea3c95e59c87335479ce122ae25e29db198e6fb80d6dcb2e1fe6a6e51db47f6f382fd1ca533700e814a5e4a27c3a797cc7cde718f64274c4ce3226afe1ef343e91bacd61b4997303ae6c6596c7e9726570cef86777288313e88e592f0e646e629c6193b7535dc7c0e14febef554cbf1cbb858816398881efa142060cbe3e97c9c0d6568c000cd6e2d7500161a474eb357fe252de505c1edd4fc9c48fcf9d5af2162d3d3d8cf136fb73f2e0bd2aec0ac40d2870e1cd5a74d8\" width=600>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图2-1 输入信号函数和系统函数的卷积</div>\n",
    "</center>\n",
    "\n",
    "\n",
    "那么卷积这样一个操作有什么意义呢？为什么要先对输入的函数进行翻转呢？信号处理的教材通常会这样解释：卷积的结果不仅仅和当前输入的信号响应值有关，也和该信号在过去所有时刻的输入响应值有关系。这表明卷积不仅仅考虑当下的情况，也考虑过去时刻的情况，它是一种对相应效果的累积，所以有一个先“卷”再“积”的过程。这么描述仍然很抽象，我们举一个实际生活中的例子。试想你站在一个池塘边，往池塘里扔石子。扔一颗石子，池塘里就会产生水波，然后慢慢衰退，水面重新平静。这里扔石子就是输入信号，产生的水波就是对输入的响应。那么如果按一定速度不停地往池塘中扔石子呢？这种情况下，在水面上某处某一时刻的水波的大小如何计算？根据常识，我们知道当前时刻某处的水波大小不但和当前时刻扔的石子产生的水波有关，还和之前所有扔的石子产生的水波有关，是这些水波的累积结果。这便是先“卷”再“积”的过程，“卷”考虑了与当前输入相关的所有输入，而“积”则把这些输入的效果进行叠加。\n",
    "\n",
    "类似的，在图像处理中，卷积的计算是把每个像素一个邻域内的像素值进行加权求和。这里空间上的邻域就对应于时间上的前后时刻，图像就是输入信号函数，加权的权重就是系统函数，卷积的结果就是系统函数对输入图像的响应，也就是图像特征。由此可见，对图像进行卷积就是提取图像特征的过程，这也是图像识别等计算机视觉任务的基础。\n",
    "\n",
    "在这一章节中，我们以一维卷积为引子，介绍卷积的数学定义与代码实现，然后推广到二维卷积，介绍对图像的卷积操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 一维卷积"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从数学上来说，卷积和加减乘除一样是一种运算，其运算的根本操作是将两个函数($f$和$k$)的其中一个先平移，然后再与另一个函数相称后的累加和。这个运算过程中，因为涉及到积分或者级数的操作，所以看起来很复杂。对于定义在连续域的函数，根据卷积的定义，一维卷积的公式如下：\n",
    "\n",
    "$$g(x) = f(x) * k(x) = \\int f(\\tau)k(x-\\tau)d\\tau$$\n",
    "\n",
    "在上式中，一维卷积的输入是一个一维信号$f(x)$和一个一维卷积核$k(x)$，在信号与系统中，$f(x)$是输入信号函数，$k(x)$是系统函数。我们可以很容易的看出，卷积是在（$-\\infty, \\infty$）上对$\\tau$求积分，积分对象为$f(\\tau)$和$k(x-\\tau)$的乘积。需要注意的是，等式右边只有$k(x-\\tau)$中提及了$x$，其余部分都在关注$\\tau$。\n",
    "\n",
    "\n",
    "根据连续函数一维卷积的公式，我们可以得出卷积操作的计算步骤如下：\n",
    "\n",
    "1. 变换：改变输入函数的横坐标，自变量由$x$变成$\\tau$；\n",
    "2. 反转：将其中一个函数反转；\n",
    "3. 平移：反转后的函数随变量$\\tau$平移，得到$k(x-\\tau)$。若$\\tau>0$，则向右平移，反之向左平移；\n",
    "4. 相乘：将$f(\\tau)$和$k(x-\\tau)$相乘；\n",
    "5. 积分：$f(\\tau)$和$k(x-\\tau)$乘积曲线下的面积即为在$x$位置的卷积值。\n",
    "\n",
    "同加减乘除一样，卷积作为一种计算操作，也有如下几种性质：\n",
    "1. 交换律：$f(x) * k(x) = k(x) * f(x)$；\n",
    "2. 结合律：$[f(x) * g(x)] * k(x) = f(x) * [g(x) * k(x)]$；\n",
    "3. 分配律：$f(x) * [g(x) + k(x)] = f(x) * g(x) + f(x) * k(x)$。\n",
    "\n",
    "<!-- 4. 微分:$\\frac{d}{dx}[f(x) * k(x)] = \\frac{df(x)}{dx} * k(x) = f(x) * \\frac{dk(x)}{dx}$ -->\n",
    "\n",
    "连续函数的一维卷积在音频处理等方面具有举足轻重的地位。但是，通常计算机所能处理的信号并不是连续的而是离散的，因此，接下来我们介绍离散型函数的一维卷积。\n",
    "\n",
    "根据卷积的定义，离散函数的一维卷积公式如下：\n",
    "\n",
    "$$\\mathbf{g}[i] = \\mathbf{f}[i] * \\mathbf{k}[i] = \\sum_{n} \\mathbf{f}[n]\\mathbf{k}[i-n]$$\n",
    "\n",
    "其中，与连续函数一维卷积不同的是，输入信号和卷积核是离散函数$\\mathbf{f}[i]$和$\\mathbf{k}[i]$（通常可用数组或者向量表示），而积分变成了求和。卷积核的长度一般是奇数，这是为了对称而设计的。通常情况下，输入信号$\\mathbf{f}[i]$的长度远远大于卷积核$\\mathbf{k}[i]$的长度。图 2-2 所示是离散函数一维卷积的计算过程。\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://storage.googleapis.com/kagglesdsdata/datasets/2171976/3676249/example.png?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=databundle-worker-v2%40kaggle-161607.iam.gserviceaccount.com%2F20220522%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20220522T054201Z&X-Goog-Expires=345599&X-Goog-SignedHeaders=host&X-Goog-Signature=2c68b0ae326f898675acd031ccd1f167cd14243663c004d279da7e4482fb0c79cfe20f8326a90ff91668b67fd152078695d47df0af16c0dede5faea7bca1197d3e7b429e52bf86ff39567b421b3f1e4a009f3591335ca30643640431010c42d582a039e705f848744bceb374d6767ecc0d1cb40b3add0fc7ec1952c90bbf03f0e5a1ccb70e77148b75f725de816bb8c2de7a1f39e54a38a3e54b956fb66e4caae6ad4ff1d1478304cdccafe3dd2fe6ee60cdcc2d566f017a056c329bc64e5506d701b874417b073c21a8fb074ecfab586058cbd7d0474a13e978a66793a4c35f6766b4f4f5de50eba3ea38e1eb9749975c4cdd89a0134a10afc1574b8e0a9da2\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图2-2 一维离散函数的卷积计算 </div>\n",
    "</center>\n",
    "\n",
    "\n",
    "由上图不难得出，对于离散变量，卷积可以理解为“卷和”，即相乘之后累加求和。除此之外，不知道大家是否注意到输出信号的长度和输入信号、卷积核之间的关系？\n",
    "\n",
    "假设输入信号长度为$m$，卷积核长度为$n$，按照图示方法，我们可以很容易的得到输出信号的长度为$m-n+1$。但这是不是意味着对一个输入信号反复卷积，它的输出长度将会越来越小？\n",
    "\n",
    "实际上，这种情况是确实可能存在的。因此，为了防止这种情况出现，我们通常会对输入信号进行“补零”（zero-padding）操作。对于图示输入，我们会在“1”的左边和“7”的右边各添加个数字“0”。如图 2-3 所示，如此一来输入信号的长度便增加了2，因此对应的输出信号长度也随之增加。补零操作让输出信号和输入信号的长度能够保持一致，更有甚者，如果我们对图示中输入信号的左右两边各添加两个“0”，输出信号的长度则大于输入信号。一般来说，在输入信号一边补零的个数要小于卷积核的长度。这是因为当长度超过或等于卷积核时，卷积所得的结果为0，相当于给输出信号也进行了补零操作，没有意义。更为直观的补零操作可以见二维卷积。\n",
    "\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://storage.googleapis.com/kagglesdsdata/datasets/2171976/3643430/zero.png?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=databundle-worker-v2%40kaggle-161607.iam.gserviceaccount.com%2F20220522%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20220522T053555Z&X-Goog-Expires=345599&X-Goog-SignedHeaders=host&X-Goog-Signature=6a23e63cb6aac90bb6c79f8b0b853beafb9619e90df5675ba6fae660876b4dead356b47cf348c4ad16172fb061c32bd78e738e3d87e9845c2e0ad101d8c27940de5f12666d6a537140ab458b2112d172b9ed9c9b87b1ce30a023994b14c4037e0fcbade16718a38f7bc513922e40ee95489929cdc47b2f2e84fc7646d40d124553d74252520dbfac4fbaf47eb30b72503af5d14e484970cf66481e690e488231a5fac8c6c71e89c143004418204cf0a82967d63b5dc574761d25b3b76d18e4f2f4d16c16f8f34eb068ae655586b138f590baac7a9a451249f605c444821029a7448cf2f8aa267cb880fc8ec66a368068efe8674c3421d5fd99018e4e5c5c44b5\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图2-3 输入信号的补零 </div>\n",
    "</center>\n",
    "\n",
    "\n",
    "\n",
    "根据以上定义，我们可以得出离散函数一维卷积的计算步骤：\n",
    "\n",
    "1. 变换：改变输入函数的横坐标，自变量由$i$变成$n$；\n",
    "2. 反转：将其中一个函数反转；\n",
    "3. 平移：反转后的函数随变量$n$平移，得到$\\mathbf{k}[i-n]$。若$n>0$，则向右平移，反之向左平移，且$n$为整数；\n",
    "4. 相乘：将$\\mathbf{f}[n]$和$\\mathbf{k}[i-n]$相乘；\n",
    "5. 累加：$\\mathbf{f}[n]$和$\\mathbf{k}[i-n]$乘积和的累加值即为在$n$位置的卷积值。\n",
    "\n",
    "\n",
    "接下来，我们将动手编写一维卷积函数。我们用数组表示离散函数，并对输入信号补零，补零的个数为卷积核长度减一，输出信号的长度为$m+2(n-1) - n + 1 = m+n-1$。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-24T11:34:38.646143Z",
     "iopub.status.busy": "2022-05-24T11:34:38.645730Z",
     "iopub.status.idle": "2022-05-24T11:34:38.679790Z",
     "shell.execute_reply": "2022-05-24T11:34:38.679354Z",
     "shell.execute_reply.started": "2022-05-24T11:34:38.646049Z"
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "# 一维卷积\n",
    "class conv_1d():\n",
    "    def __init__(self, a, b):\n",
    "        self.a = a  # 输入信号\n",
    "        self.b = b  # 卷积核\n",
    "        self.ax = [i for i in range(len(a))]  # 输入信号的坐标，默认从0开始\n",
    "        self.bx = [i for i in range(len(b))]  # 卷积核的坐标，默认从0开始\n",
    "\n",
    "    def conv(self):\n",
    "        lst1 = self.a\n",
    "        lst2 = self.b\n",
    "        l1 = len(lst1)\n",
    "        l2 = len(lst2)\n",
    "        lst1 = [0] * (l2 - 1) + lst1 + [0] * (l2 - 1)\n",
    "        lst2.reverse()\n",
    "        c = [0 for x in range(0, l1 + l2 - 1)]\n",
    "        for i in range(l1 + l2 - 1):\n",
    "            for j in range(l2):\n",
    "                c[i] += lst1[i + j] * lst2[j]\n",
    "        return c\n",
    "\n",
    "    def plot(self):\n",
    "        a = self.a\n",
    "        b = self.b\n",
    "        ax = self.ax\n",
    "        bx = self.bx\n",
    "\n",
    "        # 为了更直观的查看结果，我们分别绘制a、b与它们卷积得到的信号\n",
    "        plt.figure(1)\n",
    "        plt.subplot(1, 2, 1)  # 图一包含1行2列子图，当前画在第一行第一列图上\n",
    "        plt.title('Input')\n",
    "        plt.bar(ax, a, color='lightcoral', width=0.2)\n",
    "        plt.plot(ax, a, color='lightcoral')\n",
    "\n",
    "        plt.figure(1)\n",
    "        plt.subplot(1, 2, 2)  # 当前画在第一行第2列图上\n",
    "        plt.title('Kernel')\n",
    "        plt.bar(bx, b, color='lightgreen', width=0.2)\n",
    "        #plt.plot(bx, b, color='lightgreen')\n",
    "\n",
    "\n",
    "        # 计算输出信号以及其坐标，并画图\n",
    "        c = self.conv()\n",
    "        length = len(c)\n",
    "        cx = [i for i in range(length)]\n",
    "        plt.figure()\n",
    "        plt.title('Output')\n",
    "        plt.bar(cx, c, color='lightseagreen', width=0.2)\n",
    "        plt.plot(cx, c, color='lightseagreen')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们举例来显示上述卷积实现的效果。我们用不同的卷积核去对一个三角波信号进行卷积，然后观察卷积的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.1 冲激信号"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-24T11:34:38.681075Z",
     "iopub.status.busy": "2022-05-24T11:34:38.680868Z",
     "iopub.status.idle": "2022-05-24T11:34:39.126887Z",
     "shell.execute_reply": "2022-05-24T11:34:39.126056Z",
     "shell.execute_reply.started": "2022-05-24T11:34:38.681048Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 12888 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 13048 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 定义输入信号与卷积核\n",
    "a = [0,1,2,3,2,1,0]\n",
    "\n",
    "# 冲激函数\n",
    "k = [0,0,1,0,0]\n",
    "\n",
    "conv = conv_1d(a, k)\n",
    "conv.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当我们所用的卷积核是一个单位冲激信号（即“面积”等于1只在一个位置出现的窄脉冲）时，输入信号在卷积之后并没有发生变化。这是卷积的一个重要性质，请大家记住。下面我们更换一下卷积核，看看输出会有什么不同的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.2 方波信号"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-24T11:34:39.128268Z",
     "iopub.status.busy": "2022-05-24T11:34:39.128027Z",
     "iopub.status.idle": "2022-05-24T11:34:39.668668Z",
     "shell.execute_reply": "2022-05-24T11:34:39.667651Z",
     "shell.execute_reply.started": "2022-05-24T11:34:39.128242Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 12900 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 13604 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 定义输入信号与卷积核\n",
    "a = [0,1,2,3,2,1,0]\n",
    "# 方波信号\n",
    "k = [1,1,1,1,1]\n",
    "\n",
    "conv = conv_1d(a, k)\n",
    "conv.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "大家是否注意到，用一个方波信号对一个三角波信号进行卷积，其效果是将三角波变得平滑了。请大家记住这个现象，后续我们会再次提到。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 二维卷积"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在计算机视觉领域，我们常常需要对图像进行处理，而卷积则是一种常用的处理操作。一张数字图像可以看作一个二维空间的离散函数，表示为$\\mathbf{I}[i, j]$。与一维离散函数对应，我们可用矩阵表示图像这种二维离散函数。假设图像高为$H$，宽为$W$，设置卷积核$\\mathbf{K}[i,j]$，可以得到二维卷积的公式：\n",
    "$$\\mathbf{G}[i,j] = \\mathbf{I}[i,j] * \\mathbf{K}[i,j] = \\sum_{m=1}^{H}\\sum_{n=1}^{W}\\mathbf{I}[m,n]\\mathbf{K}[i-m,j-n]$$\n",
    "\n",
    "我们将卷积的概念从一维扩展到二维，其结果非常类似，只不过变成了矩阵的“卷和”操作。卷和就是离散的卷积，卷积是积分，而卷和是点与点的乘法和加法。一个二维图像 (矩阵) 同一个二维卷积核卷积的过程是这样的：\n",
    "1. 将二维卷积核对应的矩阵水平翻转，竖直翻转。\n",
    "2. 将这个卷积核矩阵与输入图像左上角对齐。\n",
    "3. 将卷积核矩阵的每一个元素，同输入图像中每一个对齐的元素相乘，再把所有的乘积加起来，并让这个卷积核矩阵在输入图像上滑动，重复上述计算。\n",
    "\n",
    "图 2-4 描述了二维卷积的计算过程。通过在输入图像上滑动，逐步得到整张图像的卷积输出：\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://storage.googleapis.com/kagglesdsdata/datasets/2171976/3643430/2w4.png?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=databundle-worker-v2%40kaggle-161607.iam.gserviceaccount.com%2F20220522%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20220522T053418Z&X-Goog-Expires=345599&X-Goog-SignedHeaders=host&X-Goog-Signature=9f2bfbf894a523d9ce4845f08f2d2fe57dd5296a6ebadd9ece3ea7b6a06f8b00fc7bc4462387336a5eed04b53125060eb7a739c277b9ce09287572859df897864df9c79ee1a31784ede5a27a308b18f3ae43bec9c8c602b59da0645d6263199d39cd6b6c162515c130c2ee7e5f2adf1db0983785c2a8b4350d810538a785aab9340cffcc3135770353773db73aa3c329bd3c12ee2c820937dba0aecbee0ef81ab11ad1eae918bdabde62416e5be9027055494506b88d3952e2ffaeb9e5bdde123785c1fbd496f8fb4f79c9fdb9b6850d8671ae72847c9216f1f9e938945f62c20c729a400cb5fb22c22b8dfc3886ed9ed84ad8f7960df0a17a30ccbbb3fa9694\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图2-4 二维卷积计算过程 </div>\n",
    "</center>\n",
    "\n",
    "不难发现，由于卷积核的大小限制，我们没法直接对左上角的像素点“1”进行卷积操作。因此，在计算最边缘的一行和一列的输出结果时，我们一般对原矩阵外围补零，再进行矩阵的卷和操作。\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://storage.googleapis.com/kagglesdsdata/datasets/2171976/3686018/2w3.png?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=databundle-worker-v2%40kaggle-161607.iam.gserviceaccount.com%2F20220524%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20220524T030513Z&X-Goog-Expires=345599&X-Goog-SignedHeaders=host&X-Goog-Signature=2703926ae985b3c105bb1c649eb4fe5c04818c34fef97aca8f962e10eb014ca7c7f918dab660da4dd3400f24375508d9e964660685cfc418a767bd14262c31468ba26c18512c0f17ed8057781891904cff9c3063b26a82566d1b31e79dcce1f2c49e16cb61e866b8601cb69f1c86f1229cf938dc0345db5f77fdfa678b1b1a587c4ec7e6090dd67dc55f508875ef07a0ac6d90c6a6d372774ed1737d15f40a1c26f604e7aec03d9a69165886be8d9abc713deff3618dda56d03ff011902ed2eef5096bfa51fc9f22616fc520215d1678b39725c8b099a8559217f0971ebbc88070564eae5cf316c41624293ece3ed6b7385e298528ea24f4134c1004ca76cd3d\" width=300>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图2-5 图像补零后进行二维卷积 </div>\n",
    "</center>\n",
    "\n",
    "\n",
    "\n",
    "接下来，我们开始编写二维卷积。我们将对一张图像进行处理，并直观的展示卷积前后图像的变化。和一维卷积类似，我们也使用冲激信号和方波信号作为卷积核。大家注意代码中二维冲激信号和二维方波信号的实现。这里我们直接调用Python中的库函数（cv2.filter2D）进行二维卷积操作。我们把二维卷积的代码实现作为留给大家的习题，请大家自己完成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-24T11:34:39.670572Z",
     "iopub.status.busy": "2022-05-24T11:34:39.670331Z",
     "iopub.status.idle": "2022-05-24T11:34:40.876823Z",
     "shell.execute_reply": "2022-05-24T11:34:40.875494Z",
     "shell.execute_reply.started": "2022-05-24T11:34:39.670539Z"
    }
   },
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import cv2\n",
    "import seaborn as sns\n",
    "\n",
    "class conv_2d():\n",
    "    def __init__(self, image, kernel):\n",
    "        self.img = image\n",
    "        self.k = kernel\n",
    "        \n",
    "    def plot(self):\n",
    "        # 展示原图\n",
    "        plt.figure()\n",
    "        plt.imshow(self.img[:, :, ::-1])\n",
    "        plt.axis('off')\n",
    "        plt.title('Input')\n",
    "        plt.show()   \n",
    "    \n",
    "    def plot_kernel(self):\n",
    "        sns.heatmap(self.k)\n",
    "        plt.axis('off')\n",
    "        plt.title('Kernel')\n",
    "        plt.show()   \n",
    "\n",
    "\n",
    "    # 定义二维卷积\n",
    "    def convolution(self, data, k):\n",
    "        # 直接调用库函数进行卷积操作\n",
    "        return cv2.filter2D(data, -1, k)\n",
    "\n",
    "    \n",
    "    # 展示二维卷积结果\n",
    "    def plot_conv(self):\n",
    "        img_new = self.convolution(self.img, self.k)  # 卷积过程\n",
    "        # 卷积结果可视化            \n",
    "        plt.figure()\n",
    "        plt.imshow(img_new[:, :, ::-1])\n",
    "        plt.title('Output')\n",
    "        plt.axis('off')\n",
    "        plt.show()\n",
    "        return\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了后续章节方便使用，我们将把上述代码导入utils.py文件中。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.1 冲激信号"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-24T11:34:40.879227Z",
     "iopub.status.busy": "2022-05-24T11:34:40.878950Z",
     "iopub.status.idle": "2022-05-24T11:34:41.551923Z",
     "shell.execute_reply": "2022-05-24T11:34:41.551024Z",
     "shell.execute_reply.started": "2022-05-24T11:34:40.879195Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 159316 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 4668 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 159468 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "img =cv2.imread('lena.jpeg')\n",
    "\n",
    "\n",
    "# 二维冲激卷积核\n",
    "size = 15\n",
    "k1 = np.zeros((size, size))\n",
    "mid = int((size-1) / 2)\n",
    "k1[mid][mid] = 1\n",
    "\n",
    "# 展示原图\n",
    "conv = conv_2d(img, k1)\n",
    "conv.plot()\n",
    "conv.plot_kernel()\n",
    "\n",
    "# 展示卷积结果\n",
    "conv.plot_conv()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由结果可见，用二维冲激函数对图像进行卷积，得到的结果和原图是一样的。这与我们在一维卷积中得到的结果一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.2 方波信号"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-24T11:34:41.553330Z",
     "iopub.status.busy": "2022-05-24T11:34:41.553115Z",
     "iopub.status.idle": "2022-05-24T11:34:42.363693Z",
     "shell.execute_reply": "2022-05-24T11:34:42.363291Z",
     "shell.execute_reply.started": "2022-05-24T11:34:41.553304Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 159316 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 8316 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 98400 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 二维方波卷积核\n",
    "size = 15\n",
    "k2 = 1/size/size * np.ones((size, size))\n",
    "\n",
    "# 展示原图\n",
    "conv = conv_2d(img, k2)\n",
    "conv.plot()\n",
    "conv.plot_kernel()\n",
    "\n",
    "# 展示卷积结果\n",
    "conv.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由结果可见，用方波信号作为卷积核，对图像做卷积，得到的结果是平滑模糊的图像，这也与我们在一维卷积中的实验结果一致。这是不是有一些图像处理的味道了？请大家记住这个结果，我们将在之后的章节中进一步阐述其原理。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4 总结 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "卷积是信号处理领域的重要操作，也是计算机视觉中的必备技术。在图像处理中，卷积不仅考虑某一个像素点的影响，还考虑周围像素点对其的作用与影响，这也使得卷积操作为滤波技术奠定了基础。卷积除了一维和二维，同样也有高维的处理方式，感兴趣的同学可以自行查阅相关资料。\n",
    "\n",
    "在接下来的章节中，我们会进一步介绍卷积在图像处理中的作用。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.5 习题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 习题 2.1：参照一维卷积的实现，编程实现二维卷积，基于该实现，完成以下任务：\n",
    "    \n",
    "    1. 计算图 2-4 中输入图像与卷积核卷积的结果（以矩阵表示）；\n",
    "    \n",
    "    2. 计算图 2-5 中补零后的输入图像与卷积核卷积的结果（以矩阵表示）；\n",
    "    \n",
    "    3. 重现2.3.1和2.3.2中对图像卷积的实验，并对比和用库函数得到的效果是否一致；\n",
    "    \n",
    "    4. 设计卷积核，通过卷积将一张图像往右下移动十个像素。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
