{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第3章 图像滤波"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 简介"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "还记得在上一章使用二维方波信号对图像进行卷积之后的结果吗？没错，图像变平滑了，也变模糊了。这便是所谓的图像滤波。滤波是图像处理的一个重要技术，在实际生活中有着广泛的应用。大家可能都使用过图像处理软件，如Photoshop和美图秀秀，去给自己的照片美颜，进行磨皮、祛痘、祛斑等处理，如图3-1 所示。这些美颜处理都离不开图像滤波技术的支持。\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://pic2.zhimg.com/80/v2-650c728d4911cc33d986784be93f40e9_1440w.jpg\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图3-1 软件修图</div>\n",
    "</center>\n",
    "\n",
    "\n",
    "滤波这个术语同样来自于信号处理领域。所谓“波”是指的图像的频谱，即图像信号在频域上的表示。大家可以把图像的频谱简单地理解为：图像中像素值变化剧烈的区域对应于高频信息，像素值变化不大的区域对应于低频信息。滤波就是过滤掉某些频率的信息，从而改变图像，使其达到想要的图像效果。用二维方波信号对图像进行卷积，使图像变得平滑，即像素值变化小，这样就滤掉了图像中高频信息。这个过程就是图像去噪（image denoising）的过程。当然，也可以反其道而行之，通过滤波去增强图像中的高频信息，从而实现图像锐化（image sharpening），也就是图像去模糊。通常把实现特定滤波操作的函数称为滤波器。\n",
    "\n",
    "在这一章中，我们将介绍一些典型的滤波器。我们先从图像去噪的角度，引出不同的滤波器，介绍它们的性质和去噪效果，然后再介绍如何通过滤波实现图像锐化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 图像噪声"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了从图像去噪的角度介绍滤波，我们首先介绍必备的背景知识：图像噪声。这里介绍两种典型的图像噪声：椒盐噪声和高斯噪声。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.1 椒盐噪声"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "椒盐噪声（salt & pepper noise）是数字图像中的一种常见噪声。所谓椒盐，椒就是像素值为0的黑点，盐就是像素值为255的白点。椒盐噪声就是在图像上随机出现的黑色和白色的像素，其产生的原因可能是图像传感器突然受到强烈的干扰，使传感器中模数转换器发生了错误。产生该噪声的算法比较简单，只需要在图像中随机的加入一些全黑或全白的像素点即可。接下来将编程实现椒盐噪声。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:50.212391Z",
     "iopub.status.busy": "2022-09-20T09:46:50.212002Z",
     "iopub.status.idle": "2022-09-20T09:46:51.821402Z",
     "shell.execute_reply": "2022-09-20T09:46:51.820134Z",
     "shell.execute_reply.started": "2022-09-20T09:46:50.212278Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([], [])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 331984 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# utils 为之前章节已经撰写完成的函数包，目前只有卷积的相关函数，之后我们会把写过的函数加入utils中\n",
    "from utils import *\n",
    "\n",
    "# 添加椒盐噪声\n",
    "def add_Salt(img, pro):\n",
    "    # img为输入图像\n",
    "    # pro为椒盐噪声的比例\n",
    "    \n",
    "    # 添加黑色像素点\n",
    "    noise = np.random.uniform(0, 255, img[:, :, 0].shape)\n",
    "    mask = noise < pro * 255\n",
    "    # 扩展mask的维度\n",
    "    mask = np.expand_dims(mask, axis=2)\n",
    "    mask = np.repeat(mask, 3, axis=2)    \n",
    "    img = img * (1 - mask)\n",
    "     \n",
    "    # 添加白色像素点\n",
    "    mask = noise > 255 - pro * 255    \n",
    "    mask = np.expand_dims(mask, axis=2)\n",
    "    mask = np.repeat(mask, 3, axis=2)\n",
    "    noise_img = 255 * mask + img * (1 - mask)\n",
    "    \n",
    "    return noise_img\n",
    "\n",
    "\n",
    "img = cv2.imread('lenaface.jpg')\n",
    "# 添加椒盐噪声的图像：\n",
    "img_salt = add_Salt(img, 0.05)\n",
    "plt.imshow(img_salt[:,:,::-1])\n",
    "plt.xticks([])\n",
    "plt.yticks([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.2 高斯噪声"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所谓高斯噪声是指它的概率密度函数服从高斯分布（即正态分布）的一类噪声。晚上拍照的时候，常常会发现拍摄出的图像和白天光照条件良好时相比有点模糊，这种模糊的感觉就是高斯噪声造成的。高斯噪声产生的原因一般有：图像在拍摄时亮度不够；图像传感器中电路各元器件自身噪声和相互影响；图像传感器长期工作温度过高等。下面，我们将编程实现高斯噪声，首先按照二为高斯分布随机生成噪声样本，并将其累加到图像中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:51.825384Z",
     "iopub.status.busy": "2022-09-20T09:46:51.824539Z",
     "iopub.status.idle": "2022-09-20T09:46:52.178453Z",
     "shell.execute_reply": "2022-09-20T09:46:52.177125Z",
     "shell.execute_reply.started": "2022-09-20T09:46:51.825319Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([], [])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 343068 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def add_Gaussian(img, sigma=20, mean=0):\n",
    "    lab = cv2.cvtColor(img, cv2.COLOR_BGR2Lab)\n",
    "    # 生成高斯噪声\n",
    "    noise = np.random.normal(mean, sigma, lab[:, :, 0].shape)\n",
    "    lab = lab.astype(float)\n",
    "    # 添加高斯噪声\n",
    "    lab[:, :, 0] = lab[:, :, 0] + noise\n",
    "    lab[:, :, 0] = np.clip(lab[:, :, 0], 0, 255)\n",
    "    lab = lab.astype(np.uint8)\n",
    "    noise_img = cv2.cvtColor(lab, cv2.COLOR_Lab2BGR)\n",
    "    return noise_img\n",
    "\n",
    "# 添加高斯噪声的图像\n",
    "img_gaussian = add_Gaussian(img)\n",
    "plt.imshow(img_gaussian[:,:,::-1])\n",
    "plt.xticks([])\n",
    "plt.yticks([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "实际上这两种噪声随时随地可能会出现在我们的生活中，人脸磨皮、人脸祛痘等其实就是通过滤波去除这两种噪声。接下来，我们将从实际的例子出发，介绍几种不同的滤波器。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 均值滤波"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "大家拍艺术照通常会要求对照片进行精修，这其中对人脸进行“磨皮”是精修中常用的操作。那么怎样可以实现磨皮呢？磨皮就是希望人脸变得平滑，回顾这章的简介，用方波信号对图像做卷积不就可以让图像变得平滑吗？所以，没错，用方波信号作为卷积核对人脸图像进行卷积就是实现磨皮的一种最简单的方法。那么这背后的原理又是什么呢？\n",
    "\n",
    "方波信号卷积核的数学形式大家都很熟悉了，如下式所示：\n",
    "$$ \\mathbf{K} = \\frac{1}{k^2}\\begin{bmatrix}\n",
    "   1 & 1 &... & 1 \\\\\n",
    "   1 & 1 &... & 1 \\\\\n",
    "   ...\\\\\n",
    "   1 & 1 &... & 1\n",
    "  \\end{bmatrix}_{k\\times k} $$\n",
    "  \n",
    "其中，$k$为卷积核的大小。设输入图像为$\\mathbf{I}$，对其用上述方波信号卷积核做卷积的输出图像为$\\hat{\\mathbf{I}}$，根据上一章卷积的计算公式，我们得到$\\mathbf{I}$和$\\hat{\\mathbf{I}}$间的关系：\n",
    "\n",
    "$$\\hat{\\mathbf{I}}[i,j] = \\mathbf{I}[i,j] * \\mathbf{K}[i,j] = \\sum_{m=i-k}^{i-1}\\sum_{n=j-k}^{j-1}\\mathbf{I}[m,n]\\mathbf{K}[i-m,j-n]=\\frac{1}{k^2}\\sum_{m=i-k}^{i-1}\\sum_{n=j-k}^{j-1}\\mathbf{I}[m,n]$$\n",
    "\n",
    "由此可见，用方波信号对图像做卷积，其实就是把图像中每个$k{\\times}k$邻域内的像素进行平均，所以方波卷积核也称为均值滤波器（mean filter），$k$被称为滤波器的大小。大家知道求均值可以降噪，而图像上的噪声一般是像素值发生突变的地方，比如人脸上的褶皱。通过使用均值滤波对图像处理，就可以去除这些褶皱，实现磨皮了。由于噪声是图像中的高频信息，而均值滤波器能去掉噪声，只保留图像的低频信息，所以是一种低通滤波器（即允许低频信息通过的滤波器）。接下来，我们对上一节（3.2.2）生成的有高斯噪声的人脸图像进行均值滤波，并观察它的效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:52.180703Z",
     "iopub.status.busy": "2022-09-20T09:46:52.179992Z",
     "iopub.status.idle": "2022-09-20T09:46:52.511347Z",
     "shell.execute_reply": "2022-09-20T09:46:52.510504Z",
     "shell.execute_reply.started": "2022-09-20T09:46:52.180656Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 345872 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 4868 characters]",
      "text/plain": [
       "<Figure size 200x150 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 220376 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值滤波器 size=3\n",
    "size = 3\n",
    "k3 = 1/size/size * np.ones((size, size))\n",
    "\n",
    "# 展示输入图像与滤波核\n",
    "conv_gaussian = conv_2d(img_gaussian, k3)\n",
    "conv_gaussian.plot()\n",
    "\n",
    "# 展示卷积结果\n",
    "conv_gaussian.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出图像变的平滑了，但噪声依旧存在。我们增加均值滤波器的大小，并再次测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:52.514034Z",
     "iopub.status.busy": "2022-09-20T09:46:52.513278Z",
     "iopub.status.idle": "2022-09-20T09:46:53.097319Z",
     "shell.execute_reply": "2022-09-20T09:46:53.096653Z",
     "shell.execute_reply.started": "2022-09-20T09:46:52.513996Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 179580 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 143140 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值滤波器 size=5\n",
    "size = 5\n",
    "k5 = 1/size/size *np.ones((size,size))\n",
    "\n",
    "conv = conv_2d(img_gaussian, k5)\n",
    "\n",
    "# 展示卷积结果\n",
    "conv.plot_conv()\n",
    "\n",
    "\n",
    "# 均值滤波器 size=9\n",
    "size = 9\n",
    "k9 = 1/size/size * np.ones((size, size))\n",
    "\n",
    "conv = conv_2d(img_gaussian, k9)\n",
    "# 展示卷积结果\n",
    "conv.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这时，可以很明显的看出，当均值滤波器的大小为$5\\times5$时，噪声基本消失不见；而当滤波器大小为$9\\times9$时，不仅噪声消除了，图像也变得模糊了。因此在实际滤波过程中，需要为滤波器选择合适的大小，尽可能在保留图像特征的情况下消除噪声。\n",
    "\n",
    "但不知道大家发现了没有，使用均值滤波器对图像进行去噪时，输出的图像中有很明显的“块状效应”（即图像中有很明显的成块状的区域），有没有什么好的办法能解决这一问题呢？这时候，高斯滤波器就派上用场了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 高斯滤波"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "高斯滤波（Gaussian filter）广泛用于消除高斯噪声。和均值滤波器不同的是，高速滤波对邻域内的每一个像素点进行加权平均而非直接平均，权重由由高斯分布来确定。直观的看，高斯滤波器形状如下图所示：\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://pic4.zhimg.com/80/v2-2a05528790e5913be0925bcc76681497_1440w.jpg\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图3-2 高斯滤波器</div>\n",
    "</center>\n",
    "\n",
    "高斯滤波器的高斯核同样是一个$k \\times k$大小的矩阵$\\mathbf{G}$，该矩阵每个位置$[i,j]$的值具体定义如下：\n",
    "$$\\mathbf{G}[i,j] = \\frac{1}{2\\pi\\sigma^2}e^{-\\frac{i^2+j^2}{2\\sigma^2}}$$\n",
    "\n",
    "其中，$\\mathbf{G}[i,j]$表示距离滤波器中心点位移为$[i,j]$处的高斯核权重大小，$\\sigma$是高斯核的标准差。标准差越大，高斯核开口越大，远处像素对中心像素的影响程度越大；标准差越小，高斯核开口越小，远处像素对中心像素的影响程度越小。用上述高斯核对输入图像$\\mathbf{I}$进行高斯滤波得到的输出图像$\\hat{\\mathbf{I}}$为\n",
    "$$\n",
    "\\hat{\\mathbf{I}}[i,j] = \\mathbf{I}[i,j] * \\mathbf{G}[i,j] = \\sum_{m=i-k}^{i-1}\\sum_{n=j-k}^{j-1}\\mathbf{I}[m,n]\\mathbf{G}[i-m,j-n]\n",
    "$$\n",
    "\n",
    "根据高斯函数的性质，有$\\sum_i\\sum_j\\mathbf{G}[i,j] \\approx 1$。由此可见，对输入图像用高斯滤波器进行处理的输出是以高斯核为权重的输入图像像素的加权平均，同时离中心越近的像素权重越高。因此，相对于均值滤波它的平滑效果更柔和，而且不会产生块状效应，整个图像也会变得更平滑。\n",
    "\n",
    "接下来，将编程实现高斯滤波器，并测试其效果。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:53.099123Z",
     "iopub.status.busy": "2022-09-20T09:46:53.098342Z",
     "iopub.status.idle": "2022-09-20T09:46:53.459493Z",
     "shell.execute_reply": "2022-09-20T09:46:53.458835Z",
     "shell.execute_reply.started": "2022-09-20T09:46:53.099072Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 345872 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 5100 characters]",
      "text/plain": [
       "<Figure size 200x150 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 149996 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 定义高斯核\n",
    "\n",
    "def gauss(kernel_size, sigma):\n",
    "    \n",
    "    kernel = np.zeros((kernel_size, kernel_size))\n",
    "    # 定义中心点坐标\n",
    "    center = kernel_size // 2\n",
    "    s = sigma ** 2\n",
    "    sum_val =  0\n",
    "    # 计算每个位置的高斯核权重\n",
    "    for i in range(kernel_size):\n",
    "        for j in range(kernel_size):\n",
    "            x, y = i-center, j-center\n",
    "            \n",
    "            kernel[i, j] = np.exp(-(x**2 + y**2)/ 2 * s)\n",
    "            sum_val += kernel[i, j]\n",
    "    \n",
    "    kernel = kernel/sum_val\n",
    "    \n",
    "    return kernel\n",
    "\n",
    "\n",
    "k_gaussian = gauss(9, 0.5)\n",
    "# 展示输入图像与滤波核\n",
    "conv = conv_2d(img_gaussian, k_gaussian)\n",
    "conv.plot()\n",
    "\n",
    "# 展示卷积结果\n",
    "conv.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们通过对比输入图像和输出图像，可以很明显的看出高斯噪声被有效的抑制，看来高斯噪声还得是高斯滤波器才能有效地去除。我们改变高斯核的大小，并再次测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:53.461538Z",
     "iopub.status.busy": "2022-09-20T09:46:53.460622Z",
     "iopub.status.idle": "2022-09-20T09:46:53.968281Z",
     "shell.execute_reply": "2022-09-20T09:46:53.967357Z",
     "shell.execute_reply.started": "2022-09-20T09:46:53.461497Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 177008 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 157196 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 147564 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 高斯核大小为5\n",
    "conv1 = conv_2d(img_gaussian, gauss(5, 0.5))\n",
    "conv1.plot_conv()\n",
    "\n",
    "# 高斯核大小为7\n",
    "conv2 = conv_2d(img_gaussian, gauss(7, 0.5))\n",
    "conv2.plot_conv()\n",
    "\n",
    "# 高斯核大小为11\n",
    "conv3 = conv_2d(img_gaussian, gauss(11, 0.5))\n",
    "conv3.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出，高斯滤波器在有效处理高斯噪声的同时，也避免了使用均值滤波时出现的“块状效应”。在之后的代码中，我们可以使用cv2.GaussianBlur()对图像进行高斯滤波。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.5 双边滤波"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "高斯滤波器虽然能得到很好的去噪效果，但是由上一节3.4的结果可知，图像的内容也被模糊了，比如人脸的五官变得模糊了，眼球和眼底之间也出现了灰色的部分。这是因为高斯滤波器在滤波的过程中，将白色区域与黑色区域进行加权叠加导致。那么有没有什么办法能够解决这个问题？这时候，就需要引入一个新的滤波器：双边滤波器，在有效的处理高斯噪声的同时，保持图像内容的清晰。\n",
    "\n",
    "设对一张图像$\\mathbf{I}$作用双边滤波器后的输出图像为$\\hat{\\mathbf{I}}$，则它们之间的关系由下式计算：\n",
    "\n",
    "$$\\hat{\\mathbf{I}}[i,j] = \\frac{1}{W_{sb}}\\sum_{m=i-k}^{i-1}\\sum_{n=j-k}^{j-1} \\mathbf{I}[m,n] \\mathbf{G}_{s}[i-m,j-n]\\mathbf{G}_{b}[\\mathbf{I}[m,n]-\\mathbf{I}[i,j]]$$\n",
    "\n",
    "其中，\n",
    "\n",
    "$$\\mathbf{G}_{s}[m,n] =\\frac{1}{2\\pi\\sigma_s^2} e^{-\\frac{m^2+n^2}{2\\sigma_s^2}},  \\mathbf{G}_{b}[d]=\\frac{1}{2\\pi\\sigma_b^2}e^{-\\frac{d^2}{2\\sigma_b^2}}$$\n",
    "\n",
    "$$W_{sb} = \\sum_{m=i-k}^{i-1}\\sum_{n=j-k}^{j-1} \\mathbf{G}_{s}[i-m,j-n]\\mathbf{G}_{b}[|\\mathbf{I}[m,n] - \\mathbf{I}[i,j]|]$$\n",
    "\n",
    "$\\mathbf{G}_{s}[m,n]$即是上一节3.4中的高斯卷积核。所以，双边滤波器只是在原有的高斯滤波器的基础上增加了一项高斯核$\\mathbf{G}_{b}[d]$，其中，$d=\\mathbf{I}[m,n]-\\mathbf{I}[i,j]$是位置$[m,n]$和$[i,j]$处图像像素值（亮度或者颜色）的差值。那么只有当$d$很小时，也就是位置$[m,n]$处像素值和位置$[i,j]$处像素值差别很小时，$[m,n]$处像素值才会以很大的权重去影响输出图像在位置$[i,j]$处的像素值$\\hat{\\mathbf{I}}[i,j]$的计算；反之，当$d$很大时，也就是位置$[m,n]$处像素值和位置$[i,j]$处像素值差别很大时，$[m,n]$处像素值不会对输出图像在位置$[i,j]$处的像素值$\\hat{\\mathbf{I}}[i,j]$的计算产生很大影响。图 3-3 可以帮助我们更加清楚地理解双边滤波器的工作原理：\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://pic1.zhimg.com/80/v2-9d7b27f95e5fbc569601632b9619b5e4_1440w.jpg\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图3-3 双边滤波器滤波示意</div>\n",
    "</center>\n",
    "\n",
    "在图 3-3 中，我们将一张输入图像（最左图）输入进双边滤波器中进行滤波，这里，图像中红色的点表示$[i,j]$ 所处的位置，而绿色和橙色代表不同的$[m,n]$所处的位置，图像的高低表示像素值的大小，这里可以很明显的看出红色与橙色位置的像素大小接近，而与绿色相差较多。我们知道双边滤波器是由两个高斯核（左上图和右上图）组成。当输入图像经过双边滤波器时，由于绿色和红色的亮度（像素值）差别较大，因此绿色会被抑制。换言之，在高斯滤波的基础上增加了一项之后，便可让亮度或颜色差别小的像素影响到高斯核的加权计算，而差别大的像素则会对高斯核的加权叠加影响较小。这就避免了图像的内容被模糊，例如对实验的人脸做双边滤波，黑色的眼球上的像素只会和黑色区域上的像素做加权平均，而不会和白色眼底上的像素做加权平均，这就保证了滤波后眼球和眼底的界线。总体而言，在像素图像强度或颜色变化不大的区域，双边滤波有类似于高斯滤波的效果，而在强度或颜色变化剧烈的区域，高斯滤波的效果被新增的一项减弱了。\n",
    "\n",
    "$W_{sb}$是一个归一化因子：$\\sum_{m=i-k}^{i-1}\\sum_{n=j-k}^{j-1} \\frac{\\mathbf{G}_{s}[i-m,j-n]\\mathbf{G}_{b}[\\mathbf{I}[m,n]-\\mathbf{I}[i,j]]}{W_{sb}}=1$，这是为了防止经过滤波器之后，图像像素值的大小超过其值域（0-255）。注意到，均值滤波器的归一化因子是均值滤波器的卷积核大小$k^2$，而高斯滤波器，由于其满足$\\sum_i\\sum_j\\mathbf{G}[i,j] \\approx 1$，无需专门做归一化。\n",
    "\n",
    "我们把双边滤波器的实现留给大家作为作业，我们将调用库函数（cv2.bilateralFilter()）对图像进行双边滤波。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:53.974022Z",
     "iopub.status.busy": "2022-09-20T09:46:53.969448Z",
     "iopub.status.idle": "2022-09-20T09:46:54.524983Z",
     "shell.execute_reply": "2022-09-20T09:46:54.523854Z",
     "shell.execute_reply.started": "2022-09-20T09:46:53.973965Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 292008 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 为了之后展示图片更加方便，我们将该功能封装成一个函数\n",
    "def plot_image(img, name, camp='gray'):\n",
    "    plt.imshow(img, cmap=camp)\n",
    "    plt.title(name)\n",
    "    plt.axis('off')\n",
    "    plt.show()\n",
    "\n",
    "# 展示双边滤波器过滤高斯噪声的效果\n",
    "plot_image(cv2.bilateralFilter(np.uint8(img_gaussian),30,55,45)[:, :, ::-1], 'output')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如图所示，双边滤波器在去噪的同时也很好的保持图像内容的清晰度，例如眼睛部位灰色的色块明显减少了。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6 中值滤波"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "祛痘也是美颜修图时的一个常用操作。人脸上的“痘”其实就是一种椒盐噪声。我们先试试高斯滤波器对椒盐噪声的效果，观察高斯滤波器能否有效“祛痘”。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:54.526918Z",
     "iopub.status.busy": "2022-09-20T09:46:54.526482Z",
     "iopub.status.idle": "2022-09-20T09:46:54.800413Z",
     "shell.execute_reply": "2022-09-20T09:46:54.799469Z",
     "shell.execute_reply.started": "2022-09-20T09:46:54.526863Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 243576 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 使用高斯滤波器过滤椒盐噪声\n",
    "conv_salt = conv_2d(np.uint8(img_salt), gauss(5, 1))\n",
    "conv_salt.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看来高斯滤波器无法有效处理椒盐噪声，因为高斯滤波器本质上是将部分像素点加权求和，并不能够抵抗椒盐噪声的影响。那么接下来，我们将介绍一种有效处理椒盐噪声的滤波器：中值滤波器。\n",
    "\n",
    "中值滤波器（median filter）是一种统计排序滤波器，即图像像素等于周围像素排序后的中值。对于图像中一点$(i,j)$，中值滤波以该点为中心，将邻域内所有像素的统计排序中值作为此点的响应。椒盐噪声点在图像中是少数，大部分像素点的像素值是正常的，所以中值通常是正常像素值。通过中值赋值，可以去除椒盐噪声点。这便是中值滤波器去除椒盐噪声的原理。\n",
    "\n",
    "下图是应用中值滤波器对图像滤波的一个例子。可以看到，图 3-4 中分布了像素值为0和1的椒盐噪声。当中值滤波器滑动到图中绿色区域时，先对区域内的像素进行排序；之后选取排序的中值作为滤波器中心点滤波的结果，即用145替换0。从图 3-4 也可以看出，当图像中出现椒盐噪声时，由于正常像素的多数性，中值通常是正常像素的像素值。\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://pic1.zhimg.com/80/v2-474295ca60a881234e7f6a9291e4ce6c_1440w.jpg\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图3-4 中值滤波器过滤椒盐噪声</div>\n",
    "</center>\n",
    "\n",
    "\n",
    "\n",
    "我们把中值滤波器的实现留给大家作为作业，我们将调用库函数（cv2.medianBlur()）对图像进行中值滤波，观察其对椒盐噪声的抑制效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:54.803017Z",
     "iopub.status.busy": "2022-09-20T09:46:54.802760Z",
     "iopub.status.idle": "2022-09-20T09:46:55.085711Z",
     "shell.execute_reply": "2022-09-20T09:46:55.084826Z",
     "shell.execute_reply.started": "2022-09-20T09:46:54.802989Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 162744 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "conv_salt = conv_2d(img_salt, k3)\n",
    "plot_image(cv2.medianBlur(np.uint8(img_salt),5)[:, :, ::-1], 'output')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "图中的人脸是不是看上去非常清晰？经过中值滤波器滤波之后，图像中连一点椒盐噪声的影子都看不见了。看来中值滤波器对椒盐噪声确实非常有效。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.7 图像锐化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "既然可以通过滤波的方式让图像变得平滑，那么也可以让图像变得清晰，这一过程被称作图像锐化。如前所述，图像平滑是滤掉图像中高频信息，保留低频信息，那么图像锐化是不是反其道而行之，去掉低频信息，保留高频信息就可以实现了？试想一下，当一张图像经过高斯滤波之后，它的高频信息被滤波器所去除，只保留了低频信息。如果我们用原图像减去滤波后的图像，是不是就可以得到图像中更多的高频信息？那我们先这样试验一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:55.094112Z",
     "iopub.status.busy": "2022-09-20T09:46:55.091471Z",
     "iopub.status.idle": "2022-09-20T09:46:55.323782Z",
     "shell.execute_reply": "2022-09-20T09:46:55.322749Z",
     "shell.execute_reply.started": "2022-09-20T09:46:55.094054Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 10180 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 由于卷积具有分配律，我们可以直接计算叠加后的卷积核\n",
    "size = 3\n",
    "k0 = np.zeros((size, size))\n",
    "k0[int((size-1)/2)][int((size-1)/2)] = 1\n",
    "\n",
    "kg = gauss(size, 1)\n",
    "k_shapen = 1 * (k0 - kg)\n",
    "\n",
    "# 得到高斯滤波的图像\n",
    "img_gau = cv2.GaussianBlur(img, (9,9), 5)\n",
    "shapen = conv_2d(img_gau, k_shapen)\n",
    "# 展示锐化效果\n",
    "shapen.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "好像和预计的结果不相符，大家知道是哪里出了问题吗？图像中的高频信息是很少的，低频信息才是主体，所以仅保留高频信息得到的就是这样一张“黑图”。修正我们的想法：图像锐化是增强高频信息，而不是完全去除低频信息。\n",
    "\n",
    "所以再进行以下尝试：再把原图加回到上面这张“黑图”，那就是在保留原来图像低频信息的同时，将高频信息增加了一倍。那如果把上面这张“黑图”多次叠加到原图上，图像是不是会越来越清晰？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:55.326940Z",
     "iopub.status.busy": "2022-09-20T09:46:55.326577Z",
     "iopub.status.idle": "2022-09-20T09:46:55.596555Z",
     "shell.execute_reply": "2022-09-20T09:46:55.595718Z",
     "shell.execute_reply.started": "2022-09-20T09:46:55.326895Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 198632 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 构建滤波核\n",
    "k_shapen = k0 + 1 * (k0 - kg)\n",
    "img_gau = cv2.GaussianBlur(img, (3,3), 1)\n",
    "shapen = conv_2d(img_gau, k_shapen)\n",
    "shapen.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "图像明显变得清晰了。将高频信息与原图像多次叠加，并测试其效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-20T09:46:55.598513Z",
     "iopub.status.busy": "2022-09-20T09:46:55.598074Z",
     "iopub.status.idle": "2022-09-20T09:46:55.855701Z",
     "shell.execute_reply": "2022-09-20T09:46:55.854897Z",
     "shell.execute_reply.started": "2022-09-20T09:46:55.598473Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 263000 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "k_shapen = k0 + 5 * (k0 - kg)\n",
    "img_gau = cv2.GaussianBlur(img, (3,3), 1)\n",
    "shapen = conv_2d(img_gau, k_shapen)\n",
    "shapen.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由此可见将原图减去高斯滤波后的图像，即图像的高频信息，反复叠加在原图上，可以不断增强锐化的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.8 小结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在本章中，我们学习了滤波的基本概念以及几种典型的滤波器，介绍了这些滤波器在图像去噪中的作用以及对不同类型噪声的抑制效果。最后，通过举一反三，学习了可以实现图像锐化的滤波器。对于计算机视觉的任务，滤波通常用于图像的预处理或者后处理，比如对图像进行去噪或者去模糊的预处理，可以帮助我们更准确地进行图像识别；又比如，也可以对识别检测的结果进行滤波，去除一些错误的识别和检测结果。在后续的学习中，会时常用到滤波的技术。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.9 习题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 习题3.1：请简述为何高斯滤波器比均值滤波器更适合平滑图像。\n",
    "#### 习题3.2：编程实现双边滤波器，使用提供的lenaface图像，重现章节3.5中的去噪效果，与库函数的结果进行对比。\n",
    "#### 习题3.3：编程实现中值滤波器，使用提供的lenaface图像，重现章节3.6中的去噪效果，与库函数的结果进行对比。\n",
    "#### 习题3.4：基于中值滤波器的实现，尝试用中值滤波器对加了高斯噪声的lenaface图像实现去噪效果。\n",
    "#### 习题3.5：参照本章中给出的图像锐化方法，请用一个卷积核来对lenaface图像实现锐化的功能。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
