{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第3章 图像滤波"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 简介"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "还记得在上一章我们使用二维方波信号对图像进行卷积之后的结果吗？没错，图像变平滑了，也变模糊了。这便是所谓的图像滤波。滤波是图像处理的一个重要技术，在实际生活中有着广泛的应用。大家可能都使用过图像处理软件，如Photoshop和美图秀秀，去给自己的照片美颜，进行磨皮、祛痘、祛斑等处理。这些美颜处理都离不开图像滤波技术的支持。\n",
    "\n",
    "滤波这个术语同样来自于信号处理领域，所谓“波”是指的图像的频谱，滤波就是过滤掉某些频段，从而改变图像，使其达到我们想要的图像效果。图像的频谱，大家可以简单地理解为，图像中像素值变化剧烈的区域对应于高频信息，像素值变化小的区域对应于低频信息。那么我们使用二维方波信号对图像进行卷积，使图像变得平滑（像素值变化小），也就是滤掉了图像中高频信息，这也是图像去噪（image denoising）的过程。当然，我们也可以反其道而行之，通过滤波去增强图像中的高频信息，从而实现图像锐化（image sharpening），也就是图像去模糊。\n",
    "\n",
    "在这一章中，我们将介绍一些典型的滤波操作。我们先从图像去噪的角度，引出不同的滤波操作，介绍它们的性质和去噪效果，然后再介绍如何通过滤波实现图像锐化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:28.096687Z",
     "iopub.status.busy": "2022-05-26T10:48:28.096396Z",
     "iopub.status.idle": "2022-05-26T10:48:28.100990Z",
     "shell.execute_reply": "2022-05-26T10:48:28.100105Z",
     "shell.execute_reply.started": "2022-05-26T10:48:28.096658Z"
    }
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "#os.chdir('/kaggle')\n",
    "#print(os.getcwd())\n",
    "from utils import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 图像噪声"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了从图像去噪的角度介绍滤波，我们首先介绍必备的背景知识，图像噪声。这里我们介绍两种典型的图像噪声：高斯噪声和椒盐噪声。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.1 椒盐噪声"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "椒盐噪声（salt & pepper noise）是数字图像的一个常见噪声。所谓椒盐，椒就是黑，盐就是白，椒盐噪声就是在图像上随机出现黑色白色的像素。椒盐噪声可以理解为斑点，随机出现在图像中的黑点或白点，这是一种因为信号脉冲强度引起的噪声，产生该噪声的算法也比较简单。接下来我们将编程实现椒盐噪声。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:28.106886Z",
     "iopub.status.busy": "2022-05-26T10:48:28.106655Z",
     "iopub.status.idle": "2022-05-26T10:48:28.233467Z",
     "shell.execute_reply": "2022-05-26T10:48:28.232537Z",
     "shell.execute_reply.started": "2022-05-26T10:48:28.106861Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([], <a list of 0 Text yticklabel objects>)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 85456 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 添加椒盐噪声\n",
    "def add_Salt(img, pro):\n",
    "    # img为输入图像\n",
    "    # pro为椒盐噪声的比例\n",
    "    noise = np.random.uniform(0, 255, img[:, :, 0].shape)\n",
    "    mask = noise < pro * 255\n",
    "    mask = np.expand_dims(mask, axis=2)\n",
    "    mask = np.repeat(mask, 3, axis=2)    \n",
    "    img = img * (1 - mask)\n",
    " \n",
    "    mask = noise > 255 - pro * 255    \n",
    "    mask = np.expand_dims(mask, axis=2)\n",
    "    mask = np.repeat(mask, 3, axis=2)\n",
    "    noise_img = 255 * mask + img * (1 - mask)\n",
    "    \n",
    "    return noise_img\n",
    "\n",
    "\n",
    "\n",
    "img = cv2.imread('lenaface.jpg')\n",
    "# 添加椒盐噪声的图像：\n",
    "img_salt = add_Salt(img, 0.05)\n",
    "plt.imshow(img_salt[:,:,::-1])\n",
    "plt.xticks([])\n",
    "plt.yticks([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.2 高斯噪声"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所谓高斯噪声是指它的概率密度函数服从高斯分布（即正态分布）的一类噪声，高斯噪声可以理解为拍摄图像时由于光照等原因造成的噪声。我们将编程实现高斯噪声。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:28.235922Z",
     "iopub.status.busy": "2022-05-26T10:48:28.235363Z",
     "iopub.status.idle": "2022-05-26T10:48:28.364680Z",
     "shell.execute_reply": "2022-05-26T10:48:28.363870Z",
     "shell.execute_reply.started": "2022-05-26T10:48:28.235875Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([], <a list of 0 Text yticklabel objects>)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 99292 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def add_Gaussian(img, sigma=20, mean=0):\n",
    "    lab = cv2.cvtColor(img, cv2.COLOR_BGR2Lab)\n",
    "    noise = np.random.normal(mean, sigma, lab[:, :, 0].shape)\n",
    "    lab = lab.astype(float)\n",
    "    lab[:, :, 0] = lab[:, :, 0] + noise\n",
    "    lab[:, :, 0] = np.clip(lab[:, :, 0], 0, 255)\n",
    "    lab = lab.astype(np.uint8)\n",
    "    noise_img = cv2.cvtColor(lab, cv2.COLOR_Lab2BGR)\n",
    "    return noise_img\n",
    "\n",
    "\n",
    "\n",
    "img_gaussian = add_Gaussian(img)\n",
    "plt.imshow(img_gaussian[:,:,::-1])\n",
    "plt.xticks([])\n",
    "plt.yticks([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "实际上这两种噪声随时随地可能会出现在我们的生活中，人脸磨皮、人脸祛痘等其实就是在处理这两种噪声。接下来，我们将从实际的例子出发，介绍几种不同的滤波器。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 均值滤波器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "大家拍艺术照通常会要求对照片进行精修，这其中对人脸进行“磨皮”是精修中常用的操作。那么怎样可以实现磨皮呢？磨皮就是希望人脸变得平滑，回顾这章的简介，用方波信号对图像做卷积不就可以让图像变得平滑吗？所以，没错，用方波信号作为卷积核对人脸图像进行卷积就是实现磨皮的一种最简单的方法。那么这背后的原理又是什么呢？\n",
    "\n",
    "方波信号卷积核的数学形式大家都很熟悉了，如下式所示：\n",
    "$$ k = \\frac{1}{n^2}\\begin{bmatrix}\n",
    "   1 & 1 &... & 1 \\\\\n",
    "   1 & 1 &... & 1 \\\\\n",
    "   ...\\\\\n",
    "   1 & 1 &... & 1\n",
    "  \\end{bmatrix}_{n\\times n} $$\n",
    "  \n",
    "其中，$n$为卷积核的大小。那么用方波信号对图像做卷积，其实就是把图像中每个$n{\\times}n$邻域内的像素进行平均，所以方波卷积核也称为均值滤波器（Mean filter），$n$也称为滤波器的大小。大家知道求均值可以降噪，图像上的噪声一般是像素值发生突变的地方，比如人脸上的褶皱。通过均值滤波，就可以去除这些褶皱，实现磨皮了。由于噪声是图像中的高频信息，而均值滤波器能去掉噪声，只保留图像的低频信息，所以是一种低通滤波器。又因为均值滤波器可通过对图像做卷积实现，所以它也是一种线性滤波器。接下来，我们对上一节（3.2.2）生成的有高斯噪声的人脸图像进行均值滤波，并观察它的效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:28.366481Z",
     "iopub.status.busy": "2022-05-26T10:48:28.366042Z",
     "iopub.status.idle": "2022-05-26T10:48:28.565375Z",
     "shell.execute_reply": "2022-05-26T10:48:28.564532Z",
     "shell.execute_reply.started": "2022-05-26T10:48:28.366441Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 100112 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 69012 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值滤波器 size=3\n",
    "size = 3\n",
    "k3 = 1/size/size * np.ones((size, size))\n",
    "\n",
    "# 展示原图\n",
    "conv_gaussian = conv_2d(img_gaussian, k3)\n",
    "conv_gaussian.plot()\n",
    "\n",
    "# 展示卷积结果\n",
    "conv_gaussian.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出图像变的平滑了，但噪声依旧存在。我们增加均值滤波器的大小，并再次测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:28.568146Z",
     "iopub.status.busy": "2022-05-26T10:48:28.567649Z",
     "iopub.status.idle": "2022-05-26T10:48:28.960734Z",
     "shell.execute_reply": "2022-05-26T10:48:28.959861Z",
     "shell.execute_reply.started": "2022-05-26T10:48:28.568101Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 100112 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 57748 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 46600 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值滤波器 size=5\n",
    "size = 5\n",
    "k5 = 1/size/size *np.ones((size,size))\n",
    "\n",
    "# 展示原图\n",
    "conv = conv_2d(img_gaussian, k5)\n",
    "conv.plot()\n",
    "\n",
    "# 展示卷积结果\n",
    "conv.plot_conv()\n",
    "\n",
    "\n",
    "# 均值滤波器 size=9\n",
    "size = 9\n",
    "k9 = 1/size/size * np.ones((size, size))\n",
    "\n",
    "conv = conv_2d(img_gaussian, k9)\n",
    "# 展示卷积结果\n",
    "conv.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这时，我们可以很明显的看出，当均值滤波器的大小为$5\\times5$时，噪声基本消失不见；而当滤波器大小为$9\\times9$时，不仅噪声消除了，图像也变得模糊了。因此在实际滤波过程中，我们需要为滤波器选择合适的大小，尽可能在保留图像特征的情况下消除噪声。\n",
    "\n",
    "但不知道你发现了没有，我们使用均值滤波器对图像进行去噪时，输出的图像中有很明显的“块状效应”，有没有什么好的办法能解决这一问题呢？这时候，高斯滤波器就派上用场了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 高斯滤波器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "高斯滤波（Gaussian filter）广泛用于消除高斯噪声。和均值滤波器不同的是，高速滤波对邻域内的每一个像素点进行加权平均而非直接平均，权重由由高斯分布来确定。直观的看，高斯滤波器形状如下图所示：\n",
    "\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://storage.googleapis.com/kagglesdsdata/datasets/2172833/3688792/gaussian.png?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=databundle-worker-v2%40kaggle-161607.iam.gserviceaccount.com%2F20220524%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20220524T125624Z&X-Goog-Expires=345599&X-Goog-SignedHeaders=host&X-Goog-Signature=5a460bbc66124dc0c67a605625c97c2b6405d4eb0a44f3ae0c9ce7982cd072cc4b159b7ce509367eae520813fbddc910cf036420009fd9056eb71d0803eb8491e066b7da417c70cc75f3853d4c48d91cf9e41a7eaee21964080519f9558521f9ccc0f5ee38fff1d3eaf555878b09934e3687725cddc81190e4b9faa450602ec0b75149621ba3e9a8f2c52bb3334b02407c94c7da97a85c07ba979f6e923a17097820f4ecf1aee31bbc9f24fa57fde88a4bb9f635d2909b6216d33b99bb548d153633cc1ad221da53a01bbfec5c492419a0863d380a738e4f044367f38d5dcac0816adda047f71c566bff11222976f38731e943b209a25e21b64ba1795770f1dd\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图3-1 高斯滤波器</div>\n",
    "</center>\n",
    "\n",
    "高斯滤波器的高斯核具体定义如下：\n",
    "$$\\mathbf{G}[i,j] = \\frac{1}{2\\pi\\sigma^2}e^{-\\frac{i^2+j^2}{2\\sigma^2}}$$\n",
    "\n",
    "其中，$(i,j)$表示像素点到滤波器中心点的位移，$\\mathbf{G}[i,j]$表示该位移处的高斯核权重大小，$\\sigma$是高斯核的标准差。标准差越大，高斯核开口越大，标准差越小，高斯核开口越小。高斯滤波器的输出是邻域内像素的加权平均，同时离中心越近的像素权重越高。因此，相对于均值滤波它的平滑效果更柔和，而且不会产生块状效应，整个图像也会变得更平滑。高斯滤波器和均值滤波器一样，也是低通滤波器，也可以由卷积实现，所以也是一种线性滤波器。接下来，我们将测试高斯函数的效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:28.963160Z",
     "iopub.status.busy": "2022-05-26T10:48:28.962523Z",
     "iopub.status.idle": "2022-05-26T10:48:29.164278Z",
     "shell.execute_reply": "2022-05-26T10:48:29.163344Z",
     "shell.execute_reply.started": "2022-05-26T10:48:28.963113Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 100112 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 63820 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 定义高斯核\n",
    "\n",
    "def gauss(kernel_size, sigma):\n",
    "    \n",
    "    kernel = np.zeros((kernel_size, kernel_size))\n",
    "    center = kernel_size//2\n",
    "    if sigma<=0:\n",
    "        sigma = ((kernel_size-1)*0.5-1)*0.3+0.8\n",
    "    \n",
    "    s = sigma**2\n",
    "    sum_val =  0\n",
    "    for i in range(kernel_size):\n",
    "        for j in range(kernel_size):\n",
    "            x, y = i-center, j-center\n",
    "            \n",
    "            kernel[i, j] = np.exp(-(x**2+y**2)/2*s)\n",
    "            sum_val += kernel[i, j]\n",
    "    \n",
    "    kernel = kernel/sum_val\n",
    "    \n",
    "    return kernel\n",
    "\n",
    "\n",
    "k_gaussian = gauss(5, 1)\n",
    "# 展示原图\n",
    "conv = conv_2d(img_gaussian, k_gaussian)\n",
    "conv.plot()\n",
    "\n",
    "# 展示卷积结果\n",
    "conv.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看来高斯噪声还得是高斯滤波器才能有效地去除。我们改变高斯核的大小，并再次测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:29.166763Z",
     "iopub.status.busy": "2022-05-26T10:48:29.166094Z",
     "iopub.status.idle": "2022-05-26T10:48:29.645527Z",
     "shell.execute_reply": "2022-05-26T10:48:29.644581Z",
     "shell.execute_reply.started": "2022-05-26T10:48:29.166714Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 63492 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 63460 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 63464 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "conv1 = conv_2d(img_gaussian, gauss(7, 1))\n",
    "conv1.plot_conv()\n",
    "\n",
    "conv2 = conv_2d(img_gaussian, gauss(9, 1))\n",
    "conv2.plot_conv()\n",
    "\n",
    "conv3 = conv_2d(img_gaussian, gauss(11, 1))\n",
    "conv3.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看出，高斯滤波器在有效处理高斯噪声的同时，也避免了使用均值滤波时出现的“块状效应”。在之后的代码中，我们可以使用cv2.GaussianBlur()对图像进行高斯滤波。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.5 双边滤波"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "高斯滤波器虽然能得到很好的去噪效果，但是由上一节（3.4）的结果可知，图像的内容也被模糊了，比如人脸的五官变得模糊了，眼球和眼底之间也出现了灰色的部分。这是因为高斯滤波器在滤波的过程中，将白色区域与黑色区域进行加权叠加导致。那么有没有什么办法能够解决这个问题？这时候，我们就需要引入一个新的滤波器：双边滤波器，在有效的处理高斯噪声的同时，保持图像内容的清晰。\n",
    "\n",
    "设对一张图像$\\mathbf{I}$作用双边滤波器后的输出图像为$\\hat{\\mathbf{I}}$，则它们之间的关系由下式计算：\n",
    "\n",
    "$$\\hat{\\mathbf{I}}[i,j] = \\frac{1}{W_{sb}}\\sum_m \\sum_n \\mathbf{I}[m,n] n_{\\sigma_s}[i-m][j-n]n_{\\sigma_b}[\\mathbf{I}[m,n]-\\mathbf{I}[i,j]]$$\n",
    "\n",
    "其中，\n",
    "\n",
    "$$n_{\\sigma_s}[m,n] =\\frac{1}{2\\pi\\sigma_s^2} e^{-\\frac{m^2+n^2}{2\\sigma_s^2}},  n_{\\sigma_b}[k]=\\frac{1}{2\\pi\\sigma_b^2}e^{-\\frac{k^2}{2\\sigma_b^2}}$$\n",
    "\n",
    "$$W_{sb} = \\sum_m\\sum_n n_{\\sigma_s}[i-m,j-n]n_{\\sigma_b}[|\\mathbf{I}[m,n] - \\mathbf{I}[i,j]|]$$\n",
    "\n",
    "可以很直观的看出，双边滤波器只是在原有高斯函数的基础上加了一项。其中，$k$是像素在$(m,n)$和$(i,j)$处图像强度或颜色的差值。也就是说增加这一项，使得只有强度或颜色差别小的像素才会参与到高斯核的加权计算，而差别大的像素不会参与高斯核的相互加权叠加。这就避免了图像的内容被模糊，例如对上图人脸做双边滤波，黑色的眼球上的像素只会和黑色区域上的像素做加权平均，而不会和白色眼底上的像素做加权平均，保证了滤波后眼球和眼底的界线。总体而言，在像素图像强度或颜色变化不大的区域，双边滤波有类似于高斯滤波的效果，而在强度或颜色变化剧烈的区域，高斯滤波的效果被新增的一项减弱了。\n",
    "\n",
    "我们把双边滤波器的实现留给大家作为作业，我们将调用库函数（cv2.bilateralFilter()）对图像进行双边滤波。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:29.647488Z",
     "iopub.status.busy": "2022-05-26T10:48:29.646890Z",
     "iopub.status.idle": "2022-05-26T10:48:29.890856Z",
     "shell.execute_reply": "2022-05-26T10:48:29.890092Z",
     "shell.execute_reply.started": "2022-05-26T10:48:29.647446Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 100112 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 89504 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "conv_gaussian.plot()\n",
    "plt.show()\n",
    "\n",
    "plt.imshow(cv2.bilateralFilter(np.uint8(img_gaussian),30,55,45)[:, :, ::-1])\n",
    "plt.title('output')\n",
    "plt.axis('off')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如图所示，双边滤波器在去噪的同时也很好的保持图像内容的清晰度，例如眼睛部位灰色的色块明显减少了。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6 中值滤波器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "祛痘也是美颜修图时的一个常用操作。人脸上的“痘”其实就是一种椒盐噪声。我们先试试高斯滤波器对椒盐噪声的效果，观察高斯滤波器能否有效“祛痘”。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:29.892904Z",
     "iopub.status.busy": "2022-05-26T10:48:29.892079Z",
     "iopub.status.idle": "2022-05-26T10:48:30.087194Z",
     "shell.execute_reply": "2022-05-26T10:48:30.086214Z",
     "shell.execute_reply.started": "2022-05-26T10:48:29.892858Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 86172 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 74012 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "conv_salt = conv_2d(np.uint8(img_salt), gauss(5, 1))\n",
    "conv_salt.plot()\n",
    "conv_salt.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看来高斯滤波器无法有效处理椒盐噪声，那么接下来，我们将介绍一种有效处理椒盐噪声的滤波器：中值滤波器。\n",
    "\n",
    "中值滤波器（Median filter）是一种统计排序滤波器，即图像像素等于周围像素排序后的中值。对于图像中一点$(i,j)$，中值滤波以该点为中心，将邻域内所有像素的统计排序中值作为此点的响应。椒盐噪声点在图像中是少数，大部分像素点的像素值是正常的，所以中值通常是正常像素值。通过中值赋值，可以去除椒盐噪声点。这便是中值滤波器去除椒盐噪声的原理。\n",
    "\n",
    "下图是应用中值滤波器对图像滤波的一个例子。可以看到，图 3-2 中分布了像素值为0和1的椒盐噪声。当中值滤波器滑动到图中绿色区域时，我们先对区域内的像素进行排序；之后我们选取排序的中值作为滤波器中心点滤波的结果，即用145替换0。从图 3-2 也可以看出，当图像中出现椒盐噪声时，由于正常像素的多数性，中值通常是正常像素的像素值。\n",
    "<center>\n",
    "    <img style=\"border-radius: 0.3125em;\" \n",
    "    src=\"https://storage.googleapis.com/kagglesdsdata/datasets/2172833/3699348/median.png?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=databundle-worker-v2%40kaggle-161607.iam.gserviceaccount.com%2F20220526%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20220526T104027Z&X-Goog-Expires=345599&X-Goog-SignedHeaders=host&X-Goog-Signature=a9ed886bf1a5a63fea4261ca3a06bd547350be81478f1d5a9207580365fddd2e6e16a1e7432e8bf53e8c6fa110ced9eb6951854fa0fc96299b8df01fc399ae968062098d45976fc31f6337fcb319e73d02772b146c2b82d526754f3cb90f96310013dde4a6d41a6578ff65a53c420783d4b8078523cedb63da4defe13d1242550b34c603c4a83f5380c1d4e8d98801ea0c9c191e739cd8af21245e6bcea3973e172adc1b1dc964dce15c9d71241130b7261bd4488972b1528f61a0f8934eb7790010b90ab7adf27e6ed8c2c8e01433d82d9ecf167dd046b1c8b872f576b01f50233b79cfe7e99d1a49505ccab1573e249417184389dd6080f3ad47402befc919\" width=500>\n",
    "    <br>\n",
    "    <div style=\"color:orange; \n",
    "    display: inline-block;\n",
    "    color: #999;\n",
    "    padding: 2px;\">图3-2 中值滤波器过滤椒盐噪声</div>\n",
    "</center>\n",
    "\n",
    "\n",
    "\n",
    "我们把中值滤波器的实现留给大家作为作业，我们将调用库函数（cv2.medianBlur()）对图像进行中值滤波，观察其对椒盐噪声的抑制效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:30.089718Z",
     "iopub.status.busy": "2022-05-26T10:48:30.089093Z",
     "iopub.status.idle": "2022-05-26T10:48:30.279553Z",
     "shell.execute_reply": "2022-05-26T10:48:30.278633Z",
     "shell.execute_reply.started": "2022-05-26T10:48:30.089669Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 86172 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 54248 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "conv_salt = conv_2d(img_salt, k3)\n",
    "\n",
    "conv_salt.plot()\n",
    "plt.imshow(cv2.medianBlur(np.uint8(img_salt),5)[:, :, ::-1])\n",
    "plt.title('output')\n",
    "plt.axis('off')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "图中的人脸是不是看上去非常清晰？经过中值滤波器滤波之后，图像中连一点椒盐噪声的影子都看不见了。看来中值滤波器对椒盐噪声确实非常有效。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.7 图像锐化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "既然可以通过滤波的方式让图像变得平滑，那么也可以让图像变得清晰，这一过程被称作图像锐化。如前所述，图像平滑是滤掉图像中高频信息，保留低频信息，那么图像锐化是不是反其道而行之，去掉低频信息，保留高频信息就可以实现了？试想一下，当一张图像经过高斯滤波之后，它的高频信息被滤波器所去除，只保留了低频信息。如果我们用原图像减去滤波后的图像，是不是就可以得到图像中更多的高频信息？那我们先这样试验一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:30.282326Z",
     "iopub.status.busy": "2022-05-26T10:48:30.281965Z",
     "iopub.status.idle": "2022-05-26T10:48:30.439764Z",
     "shell.execute_reply": "2022-05-26T10:48:30.438855Z",
     "shell.execute_reply.started": "2022-05-26T10:48:30.282286Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 41404 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 15512 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "size = 3\n",
    "k0 = np.zeros((size, size))\n",
    "k0[int((size-1)/2)][int((size-1)/2)] = 1\n",
    "\n",
    "kg = gauss(size, 1)\n",
    "k_shapen = 1 * (k0 - kg)\n",
    "img_gau = cv2.GaussianBlur(img, (9,9), 5)\n",
    "shapen = conv_2d(img_gau, k_shapen)\n",
    "shapen.plot()\n",
    "shapen.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "好像和我们预计的结果不相符，大家知道是哪里出了问题吗？图像中的高频信息是很少的，低频信息才是主体，所以仅保留高频信息得到的就是这样一张“黑图”。我们修正我们的想法：图像锐化是增强高频信息，而不是完全去除低频信息。\n",
    "\n",
    "所以我们再进行以下尝试：我们再把原图加回到上面这张“黑图”，那就是在保留原来图像低频信息的同时，将高频信息增加了一倍。那如果把上面这张“黑图”多次叠加到原图上，图像是不是会越来越清晰？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:30.441592Z",
     "iopub.status.busy": "2022-05-26T10:48:30.441302Z",
     "iopub.status.idle": "2022-05-26T10:48:30.642473Z",
     "shell.execute_reply": "2022-05-26T10:48:30.641547Z",
     "shell.execute_reply.started": "2022-05-26T10:48:30.441552Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 56548 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 64664 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "k_shapen = k0 + 1 * (k0 - kg)\n",
    "img_gau = cv2.GaussianBlur(img, (3,3), 1)\n",
    "shapen = conv_2d(img_gau, k_shapen)\n",
    "shapen.plot()\n",
    "shapen.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "图像明显变得清晰了。我们将高频信息与原图像多次叠加，并测试其效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-26T10:48:30.644315Z",
     "iopub.status.busy": "2022-05-26T10:48:30.644052Z",
     "iopub.status.idle": "2022-05-26T10:48:30.848243Z",
     "shell.execute_reply": "2022-05-26T10:48:30.847429Z",
     "shell.execute_reply.started": "2022-05-26T10:48:30.644279Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 56548 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 80272 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "k_shapen = k0 + 5 * (k0 - kg)\n",
    "img_gau = cv2.GaussianBlur(img, (3,3), 1)\n",
    "shapen = conv_2d(img_gau, k_shapen)\n",
    "shapen.plot()\n",
    "shapen.plot_conv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由此可见将原图减去高斯滤波后的图像，即图像的高频信息，反复叠加在原图上，可以不断增强锐化的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.8 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在本章中，我们学习了滤波的基本概念以及几种典型的滤波器，介绍了这些滤波器在图像去噪中的作用以及对不同类型噪声的抑制效果。最后，我们通过举一反三，学习了可以实现图像锐化的滤波器。对于计算机视觉的任务而言，滤波通常用于图像的预处理或者后处理，虽然看着不起眼，但是却扮演着重要作用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.9 习题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 习题 3.1：双边滤波器和中值滤波器是线性滤波器吗？请阐述理由。\n",
    "#### 习题 3.2：编程实现双边滤波器，重现章节3.5中的去噪效果，与库函数的结果进行对比。\n",
    "#### 习题 3.3：编程实现中值滤波器，重现章节3.6中的去噪效果，与库函数的结果进行对比。\n",
    "#### 习题 3.4：基于中值滤波器的实现，实验中值滤波器对加了高斯噪声的图像去噪效果。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
