Why are nonlinear fits so challenging?
Authors: M. K. Transtrum, B. B. Machta, J. P. Sethna
Abstract: Fitting model parameters to experimental data is a common yet often challenging task, especially if the model contains many parameters. Typically, algorithms get lost in regions of parameter space in which the model is unresponsive to changes in parameters, and one is left to make adjustments by hand. We explain this difficulty by interpreting the fitting process as a generalized interpretation procedure. By considering the manifold of all model predictions in data space, we find that cross sections have a hierarchy of widths and are typically very narrow. Algorithms become stuck as they move near the boundaries. We observe that the model manifold, in addition to being tightly bounded, has low extrinsic curvature, leading to the use of geodesics in the fitting process. We improve the convergence of the Levenberg-Marquardt algorithm by adding the geodesic acceleration to the usual Levenberg-Marquardt step.
http://arxiv.org/abs/0909.3884
Subscribe to:
Post Comments (Atom)
Disclaimer: Haven't read thus one yet. This seems to be a flat local minima problem. Is it the global minima, and how can I tell where a better place to look is?
ReplyDeleteI don't see a way to address the global vs local minimum problem in any algorithm where properties are local. Generally, I have to evaluate the function any other place to know it is worse/better there. If I know I am near a minima and can model it as quatradic I can get to the bottom of it. How can I tell there might be another deeper minima?
I guess this is an example of an NP complete problem. I would only know it is the global minimum if all the other local minima were higher.
Next step, read the paper.