Complex optimization
Optimization of functions defined on complex inputs ( ) is supported by simply passing a complex as input. The algorithms supported are all those which can naturally be extended to work with complex numbers: simulated annealing and all the first-order methods.
The gradient of a complex-to-real function is defined as the only vector such that
This is sometimes written
The gradient of a function is a map. Even if it is differentiable when seen as a function of to , it might not be complex-differentiable. For instance, take . Then , which is not complex-differentiable (holomorphic). Therefore, the Hessian of a function is in general not well-defined as a complex matrix (only as a real matrix), and therefore second-order optimization algorithms are not applicable directly. To use second-order optimization, convert to real variables.
Examples
We show how to minimize a quadratic plus quartic function with the LBFGS optimization algorithm.
using Random
Random.seed!(0) # Set the seed for reproducibility
# μ is the strength of the quartic. μ = 0 is just a quadratic problem
n = 4
A = randn(n,n) + im*randn(n,n)
A = A'A + I
b = randn(n) + im*randn(n)
μ = 1.0
fcomplex(x) = real(dot(x,A*x)/2 - dot(b,x)) + μ*sum(abs.(x).^4)
gcomplex(x) = A*x-b + 4μ*(abs.(x).^2).*x
gcomplex!(stor,x) = copyto!(stor,gcomplex(x))
x0 = randn(n)+im*randn(n)
res = optimize(fcomplex, gcomplex!, x0, LBFGS())The output of the optimization is
Results of Optimization Algorithm
* Algorithm: L-BFGS
* Starting Point: [0.48155603952425174 - 1.477880724921868im,-0.3219431528959694 - 0.18542418173298963im, ...]
* Minimizer: [0.14163543901272568 - 0.034929496785515886im,-0.1208600058040362 - 0.6125620908171383im, ...]
* Minimum: -1.568997e+00
* Iterations: 16
* Convergence: true
* |x - x'| ≤ 0.0e+00: false
|x - x'| = 3.28e-09
* |f(x) - f(x')| ≤ 0.0e+00 |f(x)|: false
|f(x) - f(x')| = -4.25e-16 |f(x)|
* |g(x)| ≤ 1.0e-08: true
|g(x)| = 6.33e-11
* Stopped by an increasing objective: false
* Reached Maximum Number of Iterations: false
* Objective Calls: 48
* Gradient Calls: 48
Similarly, with ConjugateGradient.
res = optimize(fcomplex, gcomplex!, x0, ConjugateGradient())Results of Optimization Algorithm
* Algorithm: Conjugate Gradient
* Starting Point: [0.48155603952425174 - 1.477880724921868im,-0.3219431528959694 - 0.18542418173298963im, ...]
* Minimizer: [0.1416354378490425 - 0.034929499492595516im,-0.12086000949769983 - 0.6125620892675705im, ...]
* Minimum: -1.568997e+00
* Iterations: 23
* Convergence: false
* |x - x'| ≤ 0.0e+00: false
|x - x'| = 8.54e-10
* |f(x) - f(x')| ≤ 0.0e+00 |f(x)|: false
|f(x) - f(x')| = -4.25e-16 |f(x)|
* |g(x)| ≤ 1.0e-08: false
|g(x)| = 3.72e-08
* Stopped by an increasing objective: true
* Reached Maximum Number of Iterations: false
* Objective Calls: 51
* Gradient Calls: 29Differentation
The finite difference methods used by Optim support real functions with complex inputs.
res = optimize(fcomplex, x0, LBFGS())Results of Optimization Algorithm
* Algorithm: L-BFGS
* Starting Point: [0.48155603952425174 - 1.477880724921868im,-0.3219431528959694 - 0.18542418173298963im, ...]
* Minimizer: [0.1416354390108624 - 0.034929496786122484im,-0.12086000580073922 - 0.6125620908025359im, ...]
* Minimum: -1.568997e+00
* Iterations: 16
* Convergence: true
* |x - x'| ≤ 0.0e+00: false
|x - x'| = 3.28e-09
* |f(x) - f(x')| ≤ 0.0e+00 |f(x)|: true
|f(x) - f(x')| = 0.00e+00 |f(x)|
* |g(x)| ≤ 1.0e-08: true
|g(x)| = 1.04e-10
* Stopped by an increasing objective: false
* Reached Maximum Number of Iterations: false
* Objective Calls: 48
* Gradient Calls: 48Automatic differentiation support for complex inputs may come when Cassete.jl is ready.