In many advanced optimization field, you need to compute the 1st or 2nd differential of a functional. The rigor analysis of functional calculus is complex and difficult. But luckily, in most application, we just need to get the 1st order differential of a functional to compute the stable “point” (in fact it’s a function in the functional case). And further, to compute the 2nd order differential functional to check if the stable is the optimized point that we’re searching. From this perspective, what we need to do is making the process of computing 1st and 2nd order differential as convenient as possible.
Lots of books introduce the calculus of functional by variational method. But that’s a great amount of work. One efficient computation way is through the Taylor expansion in appendix of G.H. Fredrickson’s book The equilibrium theory of inhomogeneous polymers. I’d like to give some details of this great method.
Introduction to Dirac function
One useful method in the functional Taylor expansion is the Dirac function, which is defined by
and also constrained to satisfy the identity .
From Lebesgue integral with respect to the measure , Dirac function satisfies
This function is used widely in engineering field. But its rigorous explanation involves massive knowledge about generalized function and advanced functional analysis, which is beyond the scope of this post. Here, we only treat it as some useful mid tool to help us compute the functional calculus.
Let’s begin with the normal one dimension formula:
Then, let’s treat as , as and as , we can get
Then considering the characteristics of Dirac function, the functional can be transformed like the book format:
where the and corresponding to 1st and 2nd order functional differentials. Also I guess, the general term may be like (it’s welcome to discuss if anyone disagree with this):
Now, we only need to compare the expressions with our own functional form, we can get the 1st and 2nd order differential functional.
Example of computing functional
Let’s first try to compute functional .
For the 1st order differential, we just need to scrutinize the equation . It’s obvious that
For the 2nd order differential, we need to consider the equation:
as the left side can be
the equation can be reduced as
Use the measure characteristics of Dirac function, and treat the as ’s function, the right side can be
Now, it’s easy to see the 2nd order differential of our functional:
where we used the symmetry of Dirac function.
Another useful example is to compute the entropy functional.