Total Differential and Gradient

Let $f(x,y)$ be a function of two variables. Suppose the first variable $x$ is changed by an amount $dx$ and the second by an amount $dy$. The value of the function at the new point is then $f(x+dx,y+dy)$. The amount that the value of the function has changed is

$$f(x+dx,y+dy)-f(x,y)\cdot$$

This expression is a function of the four variables $x,y,dx,dy$. It is called the total differential of $f(x,y)$, and is written simply as $df$:

$$df=f(x+dx,y+dy)-f(x,y)\cdot$$

Often the total differential is discussed in terms of "infinitesimals", this term referring to $dx$ and $dy$. However, no adequate definition of an infinitesimal is ever given. The only way we can work with $dx$ and $dy$ is to let these quantities be numbers, which is what we do. It is also commonly written that

$$df=f_{x}dx+f_{y}dy,$$

where $f_x$ and $f_y$ refer to $f_x(x,y)$ and $f_y(x,y)$. We shall see that this is really only an approximate formula, and we shall obtain some idea of the error in its application. There is no objection to the use of the formula, provided one interprets it in this sense.

Now let us reconsider the total differential. To begin with, we write $df=f(x+dx,y+dy)-f(x,y)$ as

$$f(x+dx,y+dy)-f(x+dx,y)+f(x+dx,y)-f(x,y)\cdot$$

Then we apply the mean value theorem to get

$$df=f_y(x+dx,\eta )dy+f_x(\xi ,y)dx,$$

where $\xi$ is between $x$ and $x+dx$ and $\eta$ is between $y$ and $y+dy$. We assume that $f_x$ and $f_y$ are continuous. Then

$$f_y(x+dx,\eta )-f_y(x,y)=\alpha (dx,dy),$$

where $\alpha (dx,dy)$ goes to zero as $\sqrt{d{x}^2+d{y}^2}$ goes to zero (since $(x+dx,\eta )$ is in the rectangle with vertices $(x,y),(x+dx,y),(x+dx,y+dy),(x,y+dy)$, and so approaches $(x,y)$ as $\sqrt{d{x}^2+d{y}^2}$, the diagonal of the rectangle, approaches $0$). Likewise,

$$f_x(\xi ,y)-f_x(x,y)=\beta (dx,dy)$$

approaches $0$ as $\sqrt{d{x}^2+d{y}^2}$ approaches $0$. Our formula now becomes

$$df=f_x(x,y)dx+f_y(x,y)dy+\beta (dx,dy)dx+\alpha (dx,dy)dy\cdot$$

Now if is any small positive number, we can choose $dx$ and $dy$ so that $\sqrt{d{x}^2+d{y}^2}$ is so small that and , for this is just the meaning of the statement that these quantities go to zero as $\sqrt{d{x}^2+d{y}^2}$ goes to zero. Then we have

\begin{aligned} |\alpha(dx, dy) dy + \beta(dx, dy) dx| &\le |\alpha(dx, dy)| |dy| + |\beta(dx, dy)| |dx| \\ &< \epsilon |dy| + \epsilon |dx| = \epsilon (|dx| + |dy|) \cdot \end{aligned}

Now $|dx|+|dy|\le 2\sqrt{d{x}^2+d{y}^2}$, so that when $dx$ and $dy$ are chosen small enough, the error in using $f_x(x,y)dx+f_y(x,y)dy$ to approximate the total differential is such that

In other words,

$$\frac{\text{error}}{\sqrt{d{x}^2+d{y}^2}}\to 0\text{as }\sqrt{d{x}^2+d{y}^2}\to 0\cdot$$

This is exactly what we shall mean when we say, "The error is small when compared to $\sqrt{d{x}^2+d{y}^2}$". Thus our result is the following:

$$df=f_x(x,y)dx+f_y(x,y)dy,$$

with an error which is small when compared to $\sqrt{d{x}^2+d{y}^2}$. This is then the sense in which the formula is correctly interpreted.

The expression $f_{x}dx+f_{y}dy$ which occurs in the above formula can be expressed in vector notation as the scalar product

$$(f_x,f_y)\cdot (dx,dy)\cdot$$

There is little reason to do this in two dimensions. However, we can define the total differential of a function $f(x_1,x_2,\dots ,x_n)$ of $n$ variables as the quantity

$$f(x_1+d{x}_1,x_2+d{x}_2,\dots ,x_n+d{x}_n)-f(x_1,x_2,\dots ,x_n),$$

a function of $2n$ variables, and prove as above that

$$df=(f_{x_1},f_{x_2},\dots ,f_{x_n})\cdot (d{x}_1,d{x}_2,\dots ,d{x}_n)+\text{error}$$

where the error is small compared to

$$\sqrt{d{x}_1^2+d{x}_2^2+\cdots +d{x}_n^2}\cdot$$

We denote the vector $(d{x}_1,d{x}_2,\dots ,d{x}_n)$ by $dX$, and we denote the vector $(f_{x_1},f_{x_2},\dots ,f_{x_n})$ by grad f (called "the gradient of $𝒇$"). In this notation, we have

$$df=(\text{grad }f)\cdot dX+\text{error},$$

where the error is small compared to $|dX|$.

Now, for a change, let $f(x,y,z)$ be a function of three variables, and let each of these be a function $x(t),y(t),z(t)$ of a single variable $t$. Then $g(t)=f(x(t),y(t),z(t))$ is a function of $t$, and we can investigate g'(t). To do so, we must consider

$$\frac{g(t+h)-g(t)}{h}=\frac{f(x(t+h),y(t+h),z(t+h))-f(x(t),y(t),z(t))}{h}\cdot$$

Now let $dx=x(t+h)-x(t),dy=y(t+h)-y(t),dz=z(t+h)-z(t)$; then we have

$$\frac{g(t+h)-g(t)}{h}=\frac{df}{h}=\frac{(\text{grad }f)\cdot dX}{h}+\frac{\text{error}}{h}.$$

Now

$$\frac{1}{h}dX=(\frac{x(t+h)-x(t)}{h},\frac{y(t+h)-y(t)}{h},\frac{z(t+h)-z(t)}{h}),$$

so that

\lim_{h \to 0} (\text{grad } f) \cdot \frac{dX}{h} = (\text{grad } f) \cdot X'(t) \cdot

The error term is of the form $\alpha dx+\beta dy+\gamma dz$, where $\alpha ,\beta$ and $\gamma$ go to zero as $|dX|$ goes to zero. As $h\to 0,|dX|\to 0$, since $x(t),y(t),z(t)$ are assumed continuous. Thus

\begin{array}{r cl c cl c cl} \dfrac{\text{error}}{h} = & \alpha & \dfrac{dx}{h} & + & \beta & \dfrac{dy}{h} & + & \gamma & \dfrac{dz}{h} \\ & \big\downarrow &\ \big\downarrow & & \big \downarrow &\ \big\downarrow & & \big\downarrow &\ \big\downarrow \\ & 0 & x'(t) & & 0 & y'(t) & & 0 & z'(t) \end{array}

as indicated by the vertical arrows.

Substituting for $g(t)$, we have the formula

\frac{d}{dt} (f(x(t), y(t), z(t))) = (\text{grad } f) \cdot X'(t) ,

which is the 3-variable case of the chain rule for functions of several variables. (In the case of $f(x(t))$, it becomes simply f'(x) \cdot x'(t), as we have seen before.)

EXERCISES