# MandysNotes

Wednesday, 02 July 2014 00:00

## Lagrange Multipliers

By  Gideon
Rate this item

Consider a particle that moves in the plane in a circle of radius $$a$$ described by the equation:

$g(x, y) = x^{2} + y^{2} - a^{2} = 0.$

A possible parametrization of this equation is:

$x(t) = a \cos{(t)},$

$y(t) = a \sin{(t)}$

where $$t$$ is time.

Then the particle revolves counterclockwise about the origin on a circle of radius $$a,$$ with a velocity vector:

$v = ( \frac{dx}{dt} , \frac{dy}{dt} ) = ( - a \sin{(t)}, a \cos{(t)} ) = ( - y, x).$

Note that the particle's speed is constant.

$\text{speed } := |v| := \sqrt{ v_{x}^{2} + v_{y}^{2}} = \sqrt{ a^{2} \sin^{2}{(t)} + a^{2} \cos^{2}{(t)} } = \sqrt{ a^{2} } = a.$

Now suppose that there is a temperature gradient in the plane described by the equation:

$T( x, y) = c_{0} + cy.$

Where $$c_{0}$$ and $$c$$ are constants.

We want to know where on the circle the particle will experience the most extreme (greatest, least) temperatures.

We cannot use $$dT=0,$$ because

$dT = \frac{ \partial T}{\partial x} dx + \frac{\partial T}{\partial y}dy = \frac{\partial T}{\partial y}dy = c dy$

and $$\frac{\partial T}{\partial y} = c$$ is a constant.

The technique that allows us to find the extremal values of $$T$$ subject to the constraint $$g = 0,$$ is the use of Lagrange multipliers.

By inspection we see that the two points on the circle $$g= 0$$ where the temperature has an extremal value are the top'' of the circle, $$(0,a),$$ and the bottom'' of the circle, $$(0, -a).$$

According to our parametrization, the particle's tangent vector at the point $$(0, a)$$ is $$(- a ,0.)$$

At the point $$(0, -a),$$ the particle's tangent vector is $$(a, 0).$$

The level surfaces of temperature are horizontal lines:

$y = k.$

That is, if $$y$$ is equal to a constant $$k,$$ then
$T( x, y ) = T (y) = T(k) = c_{0} + ck.$

Which is constant because $$c_{0}, c,$$ and $$k$$ are all constants.

Note therefore that the particle's tangent vector is contained completely within a line of constant $$T$$ only at the two points, $$(0,a)$$ and $$(0, -a)$$ where the temperature has an extremal value.

The tangent vector at the point $$(x, y,)$$ is a linear approximation to the motion of the particle at that point. When the particle experiences an extremal value of $$T,$$ the linear approximation to its motion is within a line of constant $$T,$$ so the temperature is neither increasing nor decreasing.

At all other points, the tangent vector cuts across several lines of constant temperature, and so at all other points, the linear approximation to the particles motion brings it from one temperature to another temperature either higher or lower as it moves.

Intuitively, we see that no point where the tangent vector to the particle fails to be parallel to a line of constant temperature can be an extremal point, because at such a point the particle is moving to a region of higher or lower temperature.

Because the particle transverses a path that corresponds to a constant value of the function $$g(x,y),$$ the tangent vector at each point of the particle's path must be normal to the one-form

$dg := \frac{\partial g}{\partial x} dx + \frac{ \partial g}{\partial y}dy.$

Similarly, a line of constant $$T$$ must be normal to the one-form:

$dT := \frac{\partial T }{\partial x} dx + \frac{\partial T }{\partial y} dy.$

But the particle on the curve $$g(x,y)=0$$ will experience an extremal temperature when its tangent vector is proportional to a line of constant $$T.$$

That is, the tangent vector to the particle and the line of constant $$T$$ are parallel at an extremal point, and both are normal to the one forms:

$dT,$

and

$dg.$

Therefore the two one forms must also be proportional at an extremal point and we can write:

$dT - \lambda dg = 0.$

Where $$\lambda$$ is a constant of proportionality to be determined.

$$\lambda$$ is called a Lagrange multiplier, and it allows us to express a constrained extremal problem as an unconstrained extremal problem with an extra variable.

This situation generalizes to higher dimensions and in general if we want to find the extremal values of a function $$f(x_{1}, ... , x_{n})$$ of $$n$$ variables, subject to the one constraint:

$\phi( x_{1}, ... , x_{n}) = 0,$

we can solve the unconstrained equation:

$df - \lambda d \phi = 0.$

For multiple constraints, say $$l$$ constraints

$\phi_{1} =0,$

$\dots$

$\phi_{l} = 0,$

where $$l \leq n$$ this becomes:

$df - \sum_{k=1}^{l} \lambda_{k}d\phi_{k} = 0.$

Let us confirm that the Lagrange multiplier equation gives the correct answer for our particle on a circle.

We have:

$dT = \frac{\partial T}{\partial x} dx + \frac{\partial T}{\partial y} dy$

$= 0 dx + cdy,$

$h(x, y) = x^{2} + y^{2} - a^{2} = 0,$

$d h = \frac{\partial h}{\partial x} dx + \frac{\partial h}{\partial y} dy$

$= 2x dx + 2y dy,$

so that the Lagrange multiplier equation gives:

$dT - \lambda dh = - \lambda 2x dx + \left( c - \lambda 2y \right)dy = 0.$

We can write this as two separate equations:

$2x = 0,$

and

$c - \lambda 2y = 0.$

Therefore:

$x = 0,$

$y = \frac{c}{2 \lambda }.$

Plugging these values of $$x$$ and$$y$$ back into the constraint equation we find:

$0^{2} + \frac{c^{2}}{4 \lambda^{2}} - a^{2} = 0,$

so that:

$\lambda^{2} = \frac{c^{2}}{4a^{2}},$

$\lambda = \pm \frac{c}{2a},$

and finally

$y = \pm a.$

Therefore the two extremal points are:

$(0, a),$

and

$(0, -a).$

As another example, consider the following problem from R. W. Hamming's Numerical Methods for Scientists and Engineers.

Find the minimal distance from the plane

$Ax + By + Cz - D = 0$

to the origin.

We will solve this problem in two ways: first without, and then with, Lagrange multipliers.

To solve the problem without multipliers, first find three points in the plane that are not also on same line (i.e., three noncolinear points).

First take $$y = z= 0,$$

then

$x = \frac{D}{A},$

so that the point:

$\alpha := ( \frac{D}{A} , 0 , 0 )$

is in the plane.

Similarly we find

$\beta := ( 0, \frac{D}{B} , 0),$

and

$\gamma := ( 0, 0 , \frac{D}{C}),$

are all points in the plane.

Therefore the two vectors:

$\delta := \beta - \alpha = \left( - \frac{D}{A} , \frac{D}{B}, 0 \right)$

and

$\epsilon := \gamma - \alpha = \left( - \frac{D}{A}, 0 , \frac{D}{C} \right)$

will be parallel to the plane.

Finally, the vector

$\zeta := \delta \times \epsilon,$

(where $$\times$$ is the vector cross product), will be normal to the plane.

Then a vector from the origin to the plane will be a multiple of $$\zeta,$$ and by making such a vector satisfy the defining equation for the plane (i.e., $$Ax + By + Cx -D = 0$$) we can find the minimal distance from the plane to the origin.

We write $$\zeta$$ as:

$= ( \frac{D^{2}}{BC}, \frac{D^{2}}{AC}, \frac{D^{2}}{AB}).$

So if $$\eta := \mu \zeta,$$ where $$\mu$$ is some constant, is a vector that satisfies:

$A\eta_{x} + B \eta_{y} + C \eta_{z} - D = 0,$

then $$\eta$$ will be normal to the plane and will have its tip'' in the plane, and so its length:

$\rho \left( \eta \right) = \sqrt{ (\eta_{x})^{2} + (\eta_{y})^{2} + (\eta_{z})^{2}}$

will give us the minimal distance to the plane.

We can find $$\eta$$ by solving the equation:

$\mu \left( ( \frac{D^{2}}{BC}, \frac{D^{2}}{AC}, \frac{D^{2}}{AB}) \bullet ( A, B, C) \right) = D,$

or

$\mu \left( \frac{AD^{2}}{BC} + \frac{BD^{2}}{AC} + \frac{CD^{2}}{AB}\right) = D.$

That is,

$\mu D \left( \frac{ A^{2} + B^{2} + C^{2} }{ABC}\right) = 1,$

or

$\mu = \frac{ ABC}{D( A^{2} + B^{2} + C^{2}) } .$

Then

$\eta = \mu \zeta$

$= ( \frac{AD}{ A^{2} + B^{2} + C^{2}} , \frac{bD}{ A^{2} + B^{2} + C^{2}} , \frac{cD}{ A^{2} + B^{2} + C^{2}} ),$

and finally

$\rho \left( \eta\right) = \frac{|D|}{\sqrt{ A^{2} + B^{2} + C^{2}}}.$

Now let's solve the same problem with Lagrange multipliers.

We want to minimize the function:

$\rho ( x, y , z) := \sqrt{ x^{2} + y^{2} + z^{2}},$

subject to the constraint:

$\phi(x, y, z) := Ax + By + Cz -D = 0.$

We can set up the Lagrange multiplier equation by writing:

$d \rho - \lambda d \phi = 0,$

however, because a minimum of $$\rho$$ will also be a minimum of $$\rho^{2}$$ we can set up the even simpler equation:

$d \rho^{2} - \lambda d \phi = 0.$

That is:

$d\left( x^{2} + y^{2} + z^{2}\right) - d \left(Ax + By + Cz -D\right) = 0,$

or

$\left( 2x - \lambda A\right)dx + \left( 2y - \lambda B\right)dy + \left( 2z - \lambda C\right)dz = 0.$

Because this is an unconstrained equation, all the basis one forms are independent, and we can write:

$2x = \lambda A,$

$2y = \lambda B,$

and

$2z = \lambda C.$

Therefore

$x = \frac{\lambda A}{2},$

$y = \frac{\lambda B}{2},$

and

$z = \frac{\lambda C}{2}.$

Plugging these values into the constrain equation gives:

$Ax + By + Cz = \frac{\lambda}{2} \left[ A^{2} + B^{2} + C^{2} \right] = D,$

or

$\lambda = \frac{ 2 D}{ \left[ A^{2} + B^{2} + C^{2} \right] }.$

Then

$\rho = \sqrt{ x^{2} + y^{2} + z^{2}} = \sqrt{ \left(\frac{\lambda A}{2}\right)^{2} + \left(\frac{\lambda B}{2}\right)^{2} + \left(\frac{\lambda C}{2}\right)^{2}}$

$= \frac{\lambda}{2} \sqrt{ A^{2} + B^{2} + C^{2} }$

$= \frac { |D|}{ \left[ A^{2} + B^{2} + C^{2} \right] } \sqrt{ A^{2} + B^{2} + C^{2}}$

$= \frac{|D|} { \sqrt{ A^{2} + B^{2} + C^{2}} }.$

Which is the same answer we found before.