机器学习 (Machine Learning)

The most interesting thing you would first come across when starting out with machine learning is the optimization algorithm and to be specific, it is the gradient descent, which is a first-order iterative optimization algorithm used to minimize the cost function.

从机器学习开始，您首先会遇到的最有趣的事情是优化算法，具体地说，它是梯度下降，它是用于最小化成本函数的一阶迭代优化算法。

The intuition behind gradient descent is converging to a solution, which could be a local minimum in the neighborhood or in best-case, the global minimum.

梯度下降背后的直觉正在收敛到一个解决方案，该解决方案可以是附近的局部最小值，或者在最佳情况下可以是全局最小值。

Everything seems fine and okay until you start questioning yourself about the convergence problem. Having a good understanding of convexity helps you prove the intuition behind the idea of gradient descent. So let us discuss the same.

一切似乎都很好，直到您开始对收敛问题提出疑问。 很好地了解凸度有助于您证明梯度下降概念背后的直觉。 因此，让我们讨论同样的问题。

I hope you have a good understanding of gradient descent. Check out this article for a recap.

希望您对梯度下降有所了解。 查看这篇文章以进行回顾。

凸集 (Convex Sets)

To simply things, think of convex sets as shapes where any line joining 2 points in this set is never outside the set. This is called a convex set.

简单来说，将凸集视为形状，其中连接该点中2个点的任何线都永远不会超出集合。 这称为凸集。

Take a look at the examples below.

看下面的例子。

It is evident that any line joining 2 points on a circle or say a square (the shapes on extreme left and middle), will have all the line segments within the shape. These are examples of convex sets.

显然，任何在圆上连接两个点或说一个正方形(最左端和中间的形状)的线都将具有该形状内的所有线段。 这些是凸集的示例。

On the other hand, the shape on the extreme right in the figure above has part of a line outside the shape. Thus, this is not a convex set.

另一方面，上图中最右端的形状在形状外部有一部分线。 因此，这不是凸集。

A convex set C can be represented as follows.

凸集C可以表示如下。

题词 (Epigraph)

Consider the graph of a function f. An epigraph is a set of points lying on or above the function’s graph.

考虑函数f的图。 题词是位于函数图上或函数图上方的一组点。

凸函数 (Convex Function)

Okay, now that you understand what are convex sets and epigraphs, we can talk about convex functions.

好的，既然您了解了凸集和题词集，我们就可以讨论凸函数。

A function f is said to be a convex function if its epigraph is a convex set (as seen in the green figure below on the left). This means that every line segment drawn on this graph is always equal to or above the function graph. Pause a minute and check for yourself.

函数f被称为凸函数 如果其题词是凸集(如左下方的绿色图所示) 。 这意味着在此图上绘制的每个线段 始终 等于或高于功能图。 稍等片刻，检查一下自己。

This means that a function f is not convex if there exist two points x, y such that the line segment joining f(x) and f(y), is below the curve of the function f. This causes the loss of convexity of the epigraph (as seen in the red-figure above on the right ). This means that every line segment drawn on this graph is not always equal to or above the function graph. The same can be proven by taking points on the bends.

这意味着，如果存在两个点x，y，使得连接f(x)和f(y)的线段位于函数f的曲线下方，则函数f不是凸的 。 这会导致碑文凸度的损失(如右上方的红色数字所示)。 这意味着此图 上 绘制的每个线段 都是 并不总是 等于或高于功能图。 可以通过在弯道上取点来证明这一点。

测试凸度 (Testing for convexity)

Most of the cost functions in the case of neural networks would be non-convex. Thus you must test a function for convexity.

在神经网络的情况下，大多数成本函数是非凸的。 因此，您必须测试一个函数的凸度。

A function f is said to be a convex function if the seconder-order derivative of that function is greater than or equal to 0.

如果函数f的二阶导数大于或等于0，则称该函数f为凸函数。

Examples of convex functions: y=eˣ, y=x². Both of these functions are differentiable twice.

凸函数的示例： y =eˣ，y =x²。 这两个功能都有两次微分。

If -f(x) (minus f(x)) is a convex function, then the function is called a concave function.

如果-f(x)(减去f(x))是凸函数，则该函数称为凹函数。

Examples of concave functions: y=-eˣ. The function is differentiable twice.

凹函数的示例： y =-eˣ。 该功能可微分两次。

Let’s check for convexity by plotting the exponential function eˣ.

让我们通过绘制指数函数eˣ来检查凸度。

The code output:

代码输出：

梯度下降优化中的凸性 (Convexity in gradient descent optimization)

As said earlier, gradient descent is a first-order iterative optimization algorithm that is used to minimize a cost function.

如前所述，梯度下降是一阶迭代优化算法，用于最小化成本函数。

To understand how convexity plays a crucial role in gradient descent, let us take the example of convex and non-convex cost functions.

要了解凸度在梯度下降中如何发挥关键作用，让我们以凸和非凸成本函数为例。

For a Linear Regression model, we define the cost function Mean Square Error(MSE), which measures the average squared difference between actual and predicted values. Our goal is to minimize this cost function in order to improve the accuracy of the model. MSE is a convex function (it is differentiable twice). This means there is no local minimum, but only the global minimum. Thus gradient descent would converge to the global minimum.

对于线性回归模型，我们定义成本函数均方误差(MSE)，该函数测量实际值和预测值之间的均方差。 我们的目标是最小化此成本函数，以提高模型的准确性。 MSE是一个 凸函数(可微分两次)。 这意味着没有局部最小值，只有全局最小值。 因此，梯度下降将收敛到全局最小值。

Now let us consider a non-convex cost function. In this case, take an arbitrary non-convex function as plotted below.

现在让我们考虑一个非凸成本函数。 在这种情况下，请采用如下所示的任意非凸函数。

Instead of converging to the global minimum, you can see that gradient descent would stop at the local minimum because the gradient at that point is zero (slope is 0) and minimum in the neighborhood. One way to solve this issue is by using the concept momentum.

您可以看到梯度下降将停止在局部最小值上，而不是收敛到全局最小值，因为该点处的梯度为零(坡度为0)并且在邻域内为最小值。 解决这一问题的一种方法是使用概念动量 。

结论 (Conclusion)

Convex functions play a huge role in optimization. Optimization is the core of a machine learning model. Understanding convexity is really important for the same, which I believe you did from this article.

凸函数在优化中起着重要作用。 优化是机器学习模型的核心。 理解凸度对于实现同样的意义确实很重要，我相信您是从本文中得出的。

谢谢。 下一个见。 (Thank you. See you at the next one.)