-
A minimalist approach to 3D photoemission orbital tomography: algorithms and data requirements
Authors:
Thi Lan Dinh,
G. S. Matthijs Jansen,
D. Russell Luke,
Wiebke Bennecke,
Stefan Mathias
Abstract:
Photoemission orbital tomography provides direct access from laboratory measurements to the real-space molecular orbitals of well-ordered organic semiconductor layers. Specifically, the application of phase retrieval algorithms to photon-energy- and angle-resolved photoemission data enables the direct reconstruction of full 3D molecular orbitals without the need for simulations using density funct…
▽ More
Photoemission orbital tomography provides direct access from laboratory measurements to the real-space molecular orbitals of well-ordered organic semiconductor layers. Specifically, the application of phase retrieval algorithms to photon-energy- and angle-resolved photoemission data enables the direct reconstruction of full 3D molecular orbitals without the need for simulations using density functional theory or the like. A major limitation for the direct approach has been the need for densely-sampled, well-calibrated 3D photoemission patterns. Here, we present an iterative projection algorithm that completely eliminates this challenge: for the benchmark case of the pentacene frontier orbitals, we demonstrate the reconstruction of the full orbital based on a dataset containing only four simulated photoemission momentum measurements. We discuss the algorithm performance, sampling requirements with respect to the photon energy, optimal measurement strategies, and the accuracy of orbital images that can be achieved.
△ Less
Submitted 31 January, 2024;
originally announced February 2024.
-
Stochastic Algorithms for Large-Scale Composite Optimization: the Case of Single-Shot X-FEL Imaging
Authors:
D. Russell Luke,
Steffen Schultze,
Helmut Grubmüller
Abstract:
We apply a recently developed framework for analyzing the convergence of stochastic algorithms to the general problem of large-scale nonconvex composite optimization more generally, and nonconvex likelihood maximization in particular. Our theory is demonstrated on a stochastic gradient descent algorithm for determining the electron density of a molecule from random samples of its scattering amplit…
▽ More
We apply a recently developed framework for analyzing the convergence of stochastic algorithms to the general problem of large-scale nonconvex composite optimization more generally, and nonconvex likelihood maximization in particular. Our theory is demonstrated on a stochastic gradient descent algorithm for determining the electron density of a molecule from random samples of its scattering amplitude. Numerical results on an idealized synthetic example provide a proof of concept. This opens the door to a broad range of algorithmic possibilities and provides a basis for evaluating and comparing different strategies. While this case study is very specific, it shares a structure that transfers easily to many problems of current interest, particularly in machine learning.
△ Less
Submitted 24 January, 2024;
originally announced January 2024.
-
Convergence in Distribution of Randomized Algorithms: The Case of Partially Separable Optimization
Authors:
D. Russell Luke
Abstract:
We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators and distributions behind the iterates of stochastic algorithms, and in particular the regularity of Markov operators and rates of convergence of the distributi…
▽ More
We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators and distributions behind the iterates of stochastic algorithms, and in particular the regularity of Markov operators and rates of convergence of the distributions of the corresponding Markov chains. This provides a detailed characterization of the moments of the sequences beyond just the expected behavior. This also serves as a case study of how randomization restores favorable properties to algorithms that iterations of only partial information destroys. We demonstrate this on stochastic blockwise implementations of the forward-backward and Douglas-Rachford algorithms for nonconvex (and, as a special case, convex), nonsmooth optimization.
△ Less
Submitted 31 October, 2023;
originally announced October 2023.
-
Rates of Convergence for Chains of Expansive Markov Operators
Authors:
Neal Hermer,
D. Russell Luke,
Anja Sturm
Abstract:
We provide conditions that guarantee local rates of convergence in distribution of iterated random functions that are not nonexpansive mappings in locally compact Hadamard spaces. Our results are applied to stochastic instances of common algorithms in optimization, stochastic tomography for X-FEL imaging, and a stochastic algorithm for the computation of Fréchet means in model spaces for phylogene…
▽ More
We provide conditions that guarantee local rates of convergence in distribution of iterated random functions that are not nonexpansive mappings in locally compact Hadamard spaces. Our results are applied to stochastic instances of common algorithms in optimization, stochastic tomography for X-FEL imaging, and a stochastic algorithm for the computation of Fréchet means in model spaces for phylogenetic trees.
△ Less
Submitted 31 October, 2023; v1 submitted 10 June, 2022;
originally announced June 2022.
-
Nonexpansive Markov Operators and Random Function Iterations for Stochastic Fixed Point Problems
Authors:
Neal Hermer,
D. Russell Luke,
Anja Sturm
Abstract:
We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes earlier work studying the stochastic feasibility problem, namely, to find points that are, with probability 1, fixed points of the random functions. When no such po…
▽ More
We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes earlier work studying the stochastic feasibility problem, namely, to find points that are, with probability 1, fixed points of the random functions. When no such points exist, the stochastic feasibility problem is called inconsistent, but still under certain assumptions, the more general stochastic fixed point problem has a solution and the random function iterations converge to an invariant measure for the corresponding Markov operator. We show how common structures in deterministic fixed point theory can be exploited to establish existence of invariant measures and convergence in distribution of the Markov chain. This framework specializes to many applications of current interest including, for instance, stochastic algorithms for large-scale distributed computation, and deterministic iterative procedures with computational error. The theory developed in this study provides a solid basis for describing the convergence of simple computational methods without the assumption of infinite precision arithmetic or vanishing computational errors.
△ Less
Submitted 9 March, 2023; v1 submitted 31 May, 2022;
originally announced May 2022.
-
Formation of moiré interlayer excitons in space and time
Authors:
David Schmitt,
Jan Philipp Bange,
Wiebke Bennecke,
AbdulAziz AlMutairi,
Kenji Watanabe,
Takashi Taniguchi,
Daniel Steil,
D. Russell Luke,
R. Thomas Weitz,
Sabine Steil,
G. S. Matthijs Jansen,
Stephan Hofmann,
Marcel Reutzel,
Stefan Mathias
Abstract:
Moiré superlattices in atomically thin van-der-Waals heterostructures hold great promise for an extended control of electronic and valleytronic lifetimes, the confinement of excitons in artificial moiré lattices, and the formation of novel exotic quantum phases. Such moiré-induced emergent phenomena are particularly strong for interlayer excitons, where the hole and the electron are localized in d…
▽ More
Moiré superlattices in atomically thin van-der-Waals heterostructures hold great promise for an extended control of electronic and valleytronic lifetimes, the confinement of excitons in artificial moiré lattices, and the formation of novel exotic quantum phases. Such moiré-induced emergent phenomena are particularly strong for interlayer excitons, where the hole and the electron are localized in different layers of the heterostructure. In order to exploit the full potential of correlated moiré and exciton physics, a thorough understanding of the ultrafast interlayer exciton formation process and the real-space wavefunction confinement in the moiré potential is indispensable. However, direct experimental access to these parameters is limited since most excitonic quasiparticles are optically dark. Here we show that femtosecond photoemission momentum microscopy provides quantitative access to these key properties of the moiré interlayer excitons. We find that interlayer excitons are dominantly formed on the sub-50~fs timescale via interlayer tunneling at the K valleys of the Brillouin zones. In addition, we directly measure energy-momentum fingerprints of the moiré interlayer excitons by mapping their spectral signatures within the mini Brillouin zone that is built up by the twisted heterostructure. From these momentum-fingerprints, we gain quantitative access to the modulation of the exciton wavefunction within the moiré potential in real-space. Our work provides the first direct access to the interlayer moiré exciton formation dynamics in space and time and reveals new opportunities to study correlated moiré and exciton physics for the future realization of exotic quantum phases of matter.
△ Less
Submitted 9 December, 2021;
originally announced December 2021.
-
Convergence of Proximal Splitting Algorithms in CAT(k) Spaces and Beyond
Authors:
Florian Lauster,
D. Russell Luke
Abstract:
In the setting of CAT(k) spaces, common fixed point iterations built from prox mappings (e.g. prox-prox, Krasnoselsky-Mann relaxations, nonlinear projected-gradients) converge locally linearly under the assumption of linear metric subregularity. Linear metric subregularity is in any case necessary for linearly convergent fixed point sequences, so the result is tight. To show this, we develop a the…
▽ More
In the setting of CAT(k) spaces, common fixed point iterations built from prox mappings (e.g. prox-prox, Krasnoselsky-Mann relaxations, nonlinear projected-gradients) converge locally linearly under the assumption of linear metric subregularity. Linear metric subregularity is in any case necessary for linearly convergent fixed point sequences, so the result is tight. To show this, we develop a theory of fixed point mappings that violate the usual assumptions of nonexpansiveness and firm nonexpansiveness in p-uniformly convex spaces.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
-
$α$-Firmly Nonexpansive Operators on Metric Spaces
Authors:
Arian Bërdëllima,
Florian Lauster,
D. Russell Luke
Abstract:
We extend to $p$-uniformly convex spaces tools from the analysis of fixed point iterations in linear spaces. This study is restricted to an appropriate generalization of single-valued, pointwise $α$-averaged mappings. Our main contribution is establishing a calculus for these mappings in p-uniformly convex spaces, showing in particular how the property is preserved under compositions and convex co…
▽ More
We extend to $p$-uniformly convex spaces tools from the analysis of fixed point iterations in linear spaces. This study is restricted to an appropriate generalization of single-valued, pointwise $α$-averaged mappings. Our main contribution is establishing a calculus for these mappings in p-uniformly convex spaces, showing in particular how the property is preserved under compositions and convex combinations. This is of central importance to splitting algorithms that are built by such convex combinations and compositions, and reduces the convergence analysis to simply verifying $α$-firm nonexpansiveness of the individual components at fixed points of the splitting algorithms. Our convergence analysis differs from what can be found in the previous literature in that only $α$-firm nonexpansiveness with respect to fixed points is required. Indeed we show that, if the fixed point mapping is pointwise nonexpansive at all cluster points, then these cluster points are in fact fixed points, and convergence of the sequence follows. Additionally, we provide a quantitative convergence analysis built on the notion of gauge metric subregularity, which we show is necessary for quantifiable convergence estimates. This allows one for the first time to prove convergence of a tremendous variety of splitting algorithms in spaces with curvature bounded from above.
△ Less
Submitted 22 April, 2021;
originally announced April 2021.
-
Random Function Iterations for Stochastic Fixed Point Problems
Authors:
Neal Hermer,
D. Russell Luke,
Anja Sturm
Abstract:
We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes earlier work studying the stochastic feasibility problem}, namely, to find points that are, with probability 1, fixed points of the random functions [Hermer, Luke,…
▽ More
We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes earlier work studying the stochastic feasibility problem}, namely, to find points that are, with probability 1, fixed points of the random functions [Hermer, Luke, Sturm, 2019]. When no such points exist, the stochastic feasibility problem is called inconsistent, but still under certain assumptions, the more general stochastic fixed point problem has a solution and the random function iterations converge to an invariant measure for the corresponding Markov operator. There are two major types of convergence: almost sure convergence of the iterates to a fixed point in the case of stochastic feasibility, and convergence in distribution more generally. We show how common structures in deterministic fixed point theory can be exploited to establish existence of invariant measures and convergence of the Markov chain. We show that weaker assumptions than are usually encountered in the analysis of Markov chains guarantee linear/geometric convergence. This framework specializes to many applications of current interest including, for instance, stochastic algorithms for large-scale distributed computation, and deterministic iterative procedures with computational error. The theory developed in this study provides a solid basis for describing the convergence of simple computational methods without the assumption of infinite precision arithmetic or vanishing computational errors.
△ Less
Submitted 23 March, 2022; v1 submitted 13 July, 2020;
originally announced July 2020.
-
Efficient orbital imaging based on ultrafast momentum microscopy and sparsity-driven phase retrieval
Authors:
G. S. M. Jansen,
M. Keunecke,
M. Düvel,
C. Möller,
D. Schmitt,
W. Bennecke,
F. J. S. Kappert,
D. Steil,
D. R. Luke,
S. Steil,
S. Mathias
Abstract:
We present energy-resolved photoelectron momentum maps for orbital tomography that have been collected with a novel and efficient time-of-flight momentum microscopy setup. This setup is combined with a 0.5 MHz table-top femtosecond extreme-ultraviolet light source, which enables unprecedented speed in data collection and paves the way towards time-resolved orbital imaging experiments in the future…
▽ More
We present energy-resolved photoelectron momentum maps for orbital tomography that have been collected with a novel and efficient time-of-flight momentum microscopy setup. This setup is combined with a 0.5 MHz table-top femtosecond extreme-ultraviolet light source, which enables unprecedented speed in data collection and paves the way towards time-resolved orbital imaging experiments in the future. Moreover, we take a significant step forward in the data analysis procedure for orbital imaging, and present a sparsity-driven approach to the required phase retrieval problem, which uses only the number of non-zero pixels in the orbital. Here, no knowledge of the object support is required, and the sparsity number can easily be determined from the measured data. Used in the relaxed averaged alternating reflections algorithm, this sparsity constraint enables fast and reliable phase retrieval for our experimental as well as noise-free and noisy simulated photoelectron momentum map data.
△ Less
Submitted 23 March, 2020; v1 submitted 29 January, 2020;
originally announced January 2020.
-
Convergence Analysis of the Relaxed Douglas-Rachford Algorithm
Authors:
D. Russell Luke,
Anna-Lena Martins
Abstract:
Motivated by nonconvex, inconsistent feasibility problems in imaging, the relaxed alternating averaged reflections algorithm, or relaxed Douglas-Rachford algorithm (DR$λ$), was first proposed over a decade ago. Convergence results for this algorithm are limited either to convex feasibility or consistent nonconvex feasibility with strong assumptions on the regularity of the underlying sets. Using a…
▽ More
Motivated by nonconvex, inconsistent feasibility problems in imaging, the relaxed alternating averaged reflections algorithm, or relaxed Douglas-Rachford algorithm (DR$λ$), was first proposed over a decade ago. Convergence results for this algorithm are limited either to convex feasibility or consistent nonconvex feasibility with strong assumptions on the regularity of the underlying sets. Using an analytical framework depending only on metric subregularity and pointwise almost averagedness, we analyze the convergence behavior of DR$λ$ for feasibility problems that are both nonconvex and inconsistent. We introduce a new type of regularity of sets, called super-regular at a distance, to establish sufficient conditions for local linear convergence of the corresponding sequence. These results subsume and extend existing results for this algorithm.
△ Less
Submitted 23 May, 2019; v1 submitted 28 November, 2018;
originally announced November 2018.
-
Optimization on Spheres: Models and Proximal Algorithms with Computational Performance Comparisons
Authors:
D. Russell Luke,
Shoham Sabach,
Marc Teboulle
Abstract:
We present a unified treatment of the abstract problem of finding the best approximation between a cone and spheres in the image of affine transformations. Prominent instances of this problem are phase retrieval and source localization. The common geometry binding these problems permits a generic application of algorithmic ideas and abstract convergence results for nonconvex optimization. We organ…
▽ More
We present a unified treatment of the abstract problem of finding the best approximation between a cone and spheres in the image of affine transformations. Prominent instances of this problem are phase retrieval and source localization. The common geometry binding these problems permits a generic application of algorithmic ideas and abstract convergence results for nonconvex optimization. We organize variational models for this problem into three different classes and derive the main algorithmic approaches within these classes (13 in all). We identify the central ideas underlying these methods and provide thorough numerical benchmarks comparing their performance on synthetic and laboratory data. The software and data of our experiments are all publicly accessible. We also introduce one new algorithm, a cyclic relaxed Douglas-Rachford algorithm, which outperforms all other algorithms by every measure: speed, stability and accuracy. The analysis of this algorithm remains open.
△ Less
Submitted 5 October, 2018;
originally announced October 2018.
-
Random Function Iterations for Consistent Stochastic Feasibility
Authors:
Neal Hermer,
D. Russell Luke,
Anja Sturm
Abstract:
We study the convergence of stochastic fixed point iterations in the consistent case (in the sense of Butnariu and Flåm (1995)) in several different settings, under decreasingly restrictive regularity assumptions of the fixed point mappings. The iterations are Markov chains and, for the purposes of this study, convergence is understood in very restrictive terms. We show that sufficient conditions…
▽ More
We study the convergence of stochastic fixed point iterations in the consistent case (in the sense of Butnariu and Flåm (1995)) in several different settings, under decreasingly restrictive regularity assumptions of the fixed point mappings. The iterations are Markov chains and, for the purposes of this study, convergence is understood in very restrictive terms. We show that sufficient conditions for geometric (linear) convergence in expectation of stochastic projection algorithms presented in Nedić (2011), are in fact necessary for geometric (linear) convergence in expectation more generally of iterated random functions.
△ Less
Submitted 21 September, 2018; v1 submitted 16 August, 2018;
originally announced August 2018.
-
Characterizations of Super-regularity and its Variants
Authors:
Aris Daniilidis,
D. Russell Luke,
Matthew K. Tam
Abstract:
Convergence of projection-based methods for nonconvex set feasibility problems has been established for sets with ever weaker regularity assumptions. What has not kept pace with these developments is analogous results for convergence of optimization problems with correspondingly weak assumptions on the value functions. Indeed, one of the earliest classes of nonconvex sets for which convergence res…
▽ More
Convergence of projection-based methods for nonconvex set feasibility problems has been established for sets with ever weaker regularity assumptions. What has not kept pace with these developments is analogous results for convergence of optimization problems with correspondingly weak assumptions on the value functions. Indeed, one of the earliest classes of nonconvex sets for which convergence results were obtainable, the class of so-called super-regular sets introduced by Lewis, Luke and Malick (2009), has no functional counterpart. In this work, we amend this gap in the theory by establishing the equivalence between a property slightly stronger than super-regularity, which we call Clarke super-regularity, and subsmootheness of sets as introduced by Aussel, Daniilidis and Thibault (2004). The bridge to functions shows that approximately convex functions studied by Ngai, Luc and Théra (2000) are those which have Clarke super-regular epigraphs. Further classes of regularity of functions based on the corresponding regularity of their epigraph are also discussed.
△ Less
Submitted 15 August, 2018;
originally announced August 2018.
-
Phase retrieval with sparse phase constraint
Authors:
Hieu Thao Nguyen,
D. Russell Luke,
Oleg Soloviev,
Michel Verhaegen
Abstract:
For the first time, this paper investigates the phase retrieval problem with the assumption that the phase (of the complex signal) is sparse in contrast to the sparsity assumption on the signal itself as considered in the literature of sparse signal processing. The intended application of this new problem model, which will be conducted in a follow-up paper, is to practical phase retrieval problems…
▽ More
For the first time, this paper investigates the phase retrieval problem with the assumption that the phase (of the complex signal) is sparse in contrast to the sparsity assumption on the signal itself as considered in the literature of sparse signal processing. The intended application of this new problem model, which will be conducted in a follow-up paper, is to practical phase retrieval problems where the aberration phase is sparse with respect to the orthogonal basis of Zernike polynomials. Such a problem is called sparse phase retrieval (SPR) problem in this paper. When the amplitude modulation at the exit pupil is uniform, a new scheme of sparsity regularization on phase is proposed to capture the sparsity property of the SPR problem. Based on this regularization scheme, we design and analyze an efficient solution method, named SROP algorithm, for solving SPR given only a single intensity point-spread-function image. The algorithm is a combination of the Gerchberg-Saxton algorithm with the newly proposed sparsity regularization on the phase. The latter regularization step is mathematically a rotation but with direction varying in iterations. Surprisingly, this rotation is shown to be a metric projection on an auxiliary set which is independent of iterations. As a consequence, SROP algorithm is proved to be the cyclic projections algorithm for solving a feasibility problem involving three auxiliary sets. Analyzing regularity properties of the latter auxiliary sets, we obtain convergence results for SROP algorithm based on recent convergence theory for the cyclic projections algorithm. Numerical results show clear effectiveness of the new regularization scheme for solving the SPR problem.
△ Less
Submitted 26 January, 2019; v1 submitted 4 April, 2018;
originally announced April 2018.
-
Block-coordinate primal-dual method for the nonsmooth minimization over linear constraints
Authors:
D. Russell Luke,
Yura Malitsky
Abstract:
We consider the problem of minimizing a convex, separable, nonsmooth function subject to linear constraints. The numerical method we propose is a block-coordinate extension of the Chambolle-Pock primal-dual algorithm. We prove convergence of the method without resorting to assumptions like smoothness or strong convexity of the objective, full-rank condition on the matrix, strong duality or even co…
▽ More
We consider the problem of minimizing a convex, separable, nonsmooth function subject to linear constraints. The numerical method we propose is a block-coordinate extension of the Chambolle-Pock primal-dual algorithm. We prove convergence of the method without resorting to assumptions like smoothness or strong convexity of the objective, full-rank condition on the matrix, strong duality or even consistency of the linear system. Freedom from imposing the latter assumption permits convergence guarantees for misspecified or noisy systems.
△ Less
Submitted 15 January, 2018;
originally announced January 2018.
-
Necessary conditions for linear convergence of iterated expansive, set-valued mappings with application to alternating projections
Authors:
D. Russell Luke,
Marc Teboulle,
Nguyen H. Thao
Abstract:
We present necessary conditions for monotonicity, in one form or another, of fixed point iterations of mappings that violate the usual nonexpansive property. We show that most reasonable notions of linear-type monotonicity of fixed point sequences imply {\em metric subregularity}. This is specialized to the alternating projections iteration where the metric subregularity property takes on a distin…
▽ More
We present necessary conditions for monotonicity, in one form or another, of fixed point iterations of mappings that violate the usual nonexpansive property. We show that most reasonable notions of linear-type monotonicity of fixed point sequences imply {\em metric subregularity}. This is specialized to the alternating projections iteration where the metric subregularity property takes on a distinct geometric characterization of sets at points of intersection called {\em subtransversality}. Our more general results for fixed point iterations are specialized to establish the necessity of subtransversality for consistent feasibility with a number of reasonable types of sequential monotonicity, under varying degrees of assumptions on the regularity of the sets.
△ Less
Submitted 25 May, 2018; v1 submitted 28 April, 2017;
originally announced April 2017.
-
Symbolic computation with monotone operators
Authors:
Florian Lauster,
D. Russell Luke,
Matthew K. Tam
Abstract:
We consider a class of monotone operators which are appropriate for symbolic representation and manipulation within a computer algebra system. Various structural properties of the class (e.g., closure under taking inverses, resolvents) are investigated as well as the role played by maximal monotonicity within the class. In particular, we show that there is a natural correspondence between our clas…
▽ More
We consider a class of monotone operators which are appropriate for symbolic representation and manipulation within a computer algebra system. Various structural properties of the class (e.g., closure under taking inverses, resolvents) are investigated as well as the role played by maximal monotonicity within the class. In particular, we show that there is a natural correspondence between our class of monotone operators and the subdifferentials of convex functions belonging to a class of convex functions deemed suitable for symbolic computation of Fenchel conjugates which were previously studied by Bauschke & von Mohrenschildt and by Borwein & Hamilton. A number of illustrative examples utilizing the introduced class of operators are provided including computation of proximity operators, recovery of a convex penalty function associated with the hard thresholding operator, and computation of superexpectations, superdistributions and superquantiles with specialization to risk measures.
△ Less
Submitted 17 March, 2017;
originally announced March 2017.
-
A Globally Linearly Convergent Method for Pointwise Quadratically Supportable Convex-Concave Saddle Point Problems
Authors:
D. Russell Luke,
Ron Shefi
Abstract:
We study the \emph{Proximal Alternating Predictor-Corrector} (PAPC) algorithm introduced recently by Drori, Sabach and Teboulle to solve nonsmooth structured convex-concave saddle point problems consisting of the sum of a smooth convex function, a finite collection of nonsmooth convex functions and bilinear terms. We introduce the notion of pointwise quadratic supportability, which is a relaxation…
▽ More
We study the \emph{Proximal Alternating Predictor-Corrector} (PAPC) algorithm introduced recently by Drori, Sabach and Teboulle to solve nonsmooth structured convex-concave saddle point problems consisting of the sum of a smooth convex function, a finite collection of nonsmooth convex functions and bilinear terms. We introduce the notion of pointwise quadratic supportability, which is a relaxation of a standard strong convexity assumption and allows us to show that the primal sequence is R-linearly convergent to an optimal solution and the primal-dual sequence is globally Q-linearly convergent. We illustrate the proposed method on total variation denoising problems and on locally adaptive estimation in signal/image deconvolution and denoising with multiresolution statistical constraints.
△ Less
Submitted 28 February, 2017;
originally announced February 2017.
-
About subtransversality of collections of sets
Authors:
Alexander Y. Kruger,
D. Russell Luke,
Nguyen H. Thao
Abstract:
We provide dual sufficient conditions for subtransversality of collections of sets in an Asplund space setting. For the convex case, we formulate a necessary and sufficient dual criterion of subtransversality in general Banach spaces. Our more general results suggest an intermediate notion of subtransversality, what we call weak intrinsic subtransversality, which lies between intrinsic transversal…
▽ More
We provide dual sufficient conditions for subtransversality of collections of sets in an Asplund space setting. For the convex case, we formulate a necessary and sufficient dual criterion of subtransversality in general Banach spaces. Our more general results suggest an intermediate notion of subtransversality, what we call weak intrinsic subtransversality, which lies between intrinsic transversality and subtransversality in Asplund spaces.
△ Less
Submitted 1 July, 2017; v1 submitted 15 November, 2016;
originally announced November 2016.
-
Quantitative convergence analysis of iterated expansive, set-valued mappings
Authors:
D. Russell Luke,
Nguyen H. Thao,
Matthew K. Tam
Abstract:
We develop a framework for quantitative convergence analysis of Picard iterations of expansive set-valued fixed point mappings. There are two key components of the analysis. The first is a natural generalization of single-valued averaged mappings to expansive, set-valued mappings that characterizes a type of strong calmness of the fixed point mapping. The second component to this analysis is an ex…
▽ More
We develop a framework for quantitative convergence analysis of Picard iterations of expansive set-valued fixed point mappings. There are two key components of the analysis. The first is a natural generalization of single-valued averaged mappings to expansive, set-valued mappings that characterizes a type of strong calmness of the fixed point mapping. The second component to this analysis is an extension of the well-established notion of metric subregularity -- or inverse calmness -- of the mapping at fixed points. Convergence of expansive fixed point iterations is proved using these two properties, and quantitative estimates are a natural byproduct of the framework. To demonstrate the application of the theory, we prove for the first time a number of results showing local linear convergence of nonconvex cyclic projections for inconsistent (and consistent) feasibility problems, local linear convergence of the forward-backward algorithm for structured optimization without convexity, strong or otherwise, and local linear convergence of the Douglas--Rachford algorithm for structured nonconvex minimization. This theory includes earlier approaches for known results, convex and nonconvex, as special cases.
△ Less
Submitted 25 March, 2017; v1 submitted 18 May, 2016;
originally announced May 2016.
-
Set Regularities and Feasibility Problems
Authors:
Alexander Y. Kruger,
D. Russell Luke,
Nguyen H. Thao
Abstract:
We synthesize and unify notions of regularity, both of individual sets and of collections of sets, as they appear in the convergence theory of projection methods for consistent feasibility problems. Several new characterizations of regularities are presented which shed light on the relations between seemingly different ideas and point to possible necessary conditions for local linear convergence o…
▽ More
We synthesize and unify notions of regularity, both of individual sets and of collections of sets, as they appear in the convergence theory of projection methods for consistent feasibility problems. Several new characterizations of regularities are presented which shed light on the relations between seemingly different ideas and point to possible necessary conditions for local linear convergence of fundamental algorithms.
△ Less
Submitted 16 February, 2016;
originally announced February 2016.
-
Local Linear Convergence of the ADMM/Douglas--Rachford Algorithms without Strong Convexity and Application to Statistical Imaging
Authors:
Timo Aspelmeier,
C. Charitha,
D. Russell Luke
Abstract:
We consider the problem of minimizing the sum of a convex function and a convex function composed with an injective linear mapping. For such problems, subject to a coercivity condition at fixed points of the corresponding Picard iteration, iterates of the alternating directions method of multipliers converge locally linearly to points from which the solution to the original problem can be computed…
▽ More
We consider the problem of minimizing the sum of a convex function and a convex function composed with an injective linear mapping. For such problems, subject to a coercivity condition at fixed points of the corresponding Picard iteration, iterates of the alternating directions method of multipliers converge locally linearly to points from which the solution to the original problem can be computed. Our proof strategy uses duality and strong metric subregularity of the Douglas--Rachford fixed point mapping. Our analysis does not require strong convexity and yields error bounds to the set of model solutions. We show in particular that convex piecewise linear-quadratic functions naturally satisfy the requirements of the theory, guaranteeing eventual linear convergence of both the Douglas--Rachford algorithm and the alternating directions method of multipliers for this class of objectives under mild assumptions on the set of fixed points. We demonstrate this result on quantitative image deconvolution and denoising with multiresolution statistical constraints.
△ Less
Submitted 15 March, 2016; v1 submitted 18 August, 2015;
originally announced August 2015.
-
Lagrange Multipliers, (Exact) Regularization and Error Bounds for Monotone Variational Inequalities
Authors:
C. Charitha,
Joydeep Dutta,
D. Russell Luke
Abstract:
We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone mapping, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associa…
▽ More
We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone mapping, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associated with the existence of Lagrange multipliers for the regularized variational inequality. A regularization is said to be exact if a solution to the regularized problem is a solution to the unregularized problem for all parameters beyond a certain value. The Lagrange multipliers corresponding to a particular regularization of a variational inequality, on the other hand, are defined via the dual gap function. Our analysis suggests various conceptual, iteratively regularized numerical schemes, for which we provide error bounds, and hence stopping criteria, under the additional assumption that the solution set to the unregularized problem is what we call weakly sharp of order greater than one.
△ Less
Submitted 2 November, 2015; v1 submitted 14 October, 2014;
originally announced October 2014.
-
Proximal Heterogeneous Block Input-Output Method and application to Blind Ptychographic Diffraction Imaging
Authors:
Robert Hesse,
D. Russell Luke,
Shoham Sabach,
Matthew K. Tam
Abstract:
We propose a general alternating minimization algorithm for nonconvex optimization problems with separable structure and nonconvex coupling between blocks of variables. To fix our ideas, we apply the methodology to the problem of blind ptychographic imaging. Compared to other schemes in the literature, our approach differs in two ways: (i) it is posed within a clear mathematical framework with pra…
▽ More
We propose a general alternating minimization algorithm for nonconvex optimization problems with separable structure and nonconvex coupling between blocks of variables. To fix our ideas, we apply the methodology to the problem of blind ptychographic imaging. Compared to other schemes in the literature, our approach differs in two ways: (i) it is posed within a clear mathematical framework with practically verifiable assumptions, and (ii) under the given assumptions, it is provably convergent to critical points. A numerical comparison of our proposed algorithm with the current state-of-the-art on simulated and experimental data validates our approach and points toward directions for further improvement.
△ Less
Submitted 8 August, 2014;
originally announced August 2014.
-
Alternating Projections and Douglas-Rachford for Sparse Affine Feasibility
Authors:
Robert Hesse,
D. Russell Luke,
Patrick Neumann
Abstract:
The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved nonconvex relaxations. In this work we consider elementary methods based on projections for solving a sparse feasibility problem without employing convex heuristics. In a…
▽ More
The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved nonconvex relaxations. In this work we consider elementary methods based on projections for solving a sparse feasibility problem without employing convex heuristics. In a recent paper Bauschke, Luke, Phan and Wang (2014) showed that, locally, the fundamental method of alternating projections must converge linearly to a solution to the sparse feasibility problem with an affine constraint. In this paper we apply different analytical tools that allow us to show global linear convergence of alternating projections under familiar constraint qualifications. These analytical tools can also be applied to other algorithms. This is demonstrated with the prominent Douglas-Rachford algorithm where we establish local linear convergence of this method applied to the sparse affine feasibility problem.
△ Less
Submitted 14 March, 2014; v1 submitted 8 July, 2013;
originally announced July 2013.
-
Real-Time Phase Masks for Interactive Stimulation of Optogenetic Neurons
Authors:
Stephan C. Kramer,
Johannes Hagemann,
D. Russell Luke
Abstract:
Experiments with networks of optogenetically altered neurons require stimulation with high spatio-temporal selectivity. Computer-assisted holography is an energy-efficient method for robust and reliable addressing of single neurons on the millisecond-timescale inherent to biologial information processing. We show that real-time control of neurons can be achieved by a CUDA-based hologram computatio…
▽ More
Experiments with networks of optogenetically altered neurons require stimulation with high spatio-temporal selectivity. Computer-assisted holography is an energy-efficient method for robust and reliable addressing of single neurons on the millisecond-timescale inherent to biologial information processing. We show that real-time control of neurons can be achieved by a CUDA-based hologram computation.
△ Less
Submitted 1 February, 2013;
originally announced February 2013.
-
Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems
Authors:
Robert Hesse,
D. Russell Luke
Abstract:
We consider projection algorithms for solving (nonconvex) feasibility problems in Euclidean spaces. Of special interest are the Method of Alternating Projections (MAP) and the Douglas-Rachford or Averaged Alternating Reflection Algorithm (AAR). In the case of convex feasibility, firm nonexpansiveness of projection mappings is a global property that yields global convergence of MAP and for consiste…
▽ More
We consider projection algorithms for solving (nonconvex) feasibility problems in Euclidean spaces. Of special interest are the Method of Alternating Projections (MAP) and the Douglas-Rachford or Averaged Alternating Reflection Algorithm (AAR). In the case of convex feasibility, firm nonexpansiveness of projection mappings is a global property that yields global convergence of MAP and for consistent problems AAR. Based on (ε, δ)-regularity of sets developed by Bauschke, Luke, Phan and Wang in 2012, a relaxed local version of firm nonexpansiveness with respect to the intersection is introduced for consistent feasibility problems. Together with a coercivity condition that relates to the regularity of the intersection, this yields local linear convergence of MAP for a wide class of nonconvex problems,
△ Less
Submitted 24 June, 2013; v1 submitted 13 December, 2012;
originally announced December 2012.
-
Restricted normal cones and sparsity optimization with affine constraints
Authors:
Heinz H. Bauschke,
D. Russell Luke,
Hung M. Phan,
Xianfu Wang
Abstract:
The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved non convex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing co…
▽ More
The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved non convex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing convex heuristics. In a parallel paper we recently introduced the restricted normal cone which generalizes the classical Mordukhovich normal cone and reconciles some fundamental gaps in the theory of sufficient conditions for local linear convergence of the MAP algorithm. We use the restricted normal cone together with the notion of superregularity, which is naturally satisfied for the affine sparse optimization problem, to obtain local linear convergence results with estimates for the radius of convergence of the MAP algorithm applied to sparsity optimization with an affine constraint.
△ Less
Submitted 2 May, 2012;
originally announced May 2012.
-
Restricted normal cones and the method of alternating projections
Authors:
Heinz H. Bauschke,
D. Russell Luke,
Hung M. Phan,
Xianfu Wang
Abstract:
The method of alternating projections (MAP) is a common method for solving feasibility problems. While employed traditionally to subspaces or to convex sets, little was known about the behavior of the MAP in the nonconvex case until 2009, when Lewis, Luke, and Malick derived local linear convergence results provided that a condition involving normal cones holds and at least one of the sets is supe…
▽ More
The method of alternating projections (MAP) is a common method for solving feasibility problems. While employed traditionally to subspaces or to convex sets, little was known about the behavior of the MAP in the nonconvex case until 2009, when Lewis, Luke, and Malick derived local linear convergence results provided that a condition involving normal cones holds and at least one of the sets is superregular (a property less restrictive than convexity). However, their results failed to capture very simple classical convex instances such as two lines in three-dimensional space.
In this paper, we extend and develop the Lewis-Luke-Malick framework so that not only any two linear subspaces but also any two closed convex sets whose relative interiors meet are covered. We also allow for sets that are more structured such as unions of convex sets. The key tool required is the restricted normal cone, which is a generalization of the classical Mordukhovich normal cone. We thoroughly study restricted normal cones from the viewpoint of constraint qualifications and regularity. Numerous examples are provided to illustrate the theory.
△ Less
Submitted 2 May, 2012;
originally announced May 2012.
-
Prox-regularity of rank constraint sets and implications for algorithms
Authors:
D. Russell Luke
Abstract:
We present an analysis of sets of matrices with rank less than or equal to a specified number $s$. We provide a simple formula for the normal cone to such sets, and use this to show that these sets are prox-regular at all points with rank exactly equal to $s$. The normal cone formula appears to be new. This allows for easy application of prior results guaranteeing local linear convergence of the f…
▽ More
We present an analysis of sets of matrices with rank less than or equal to a specified number $s$. We provide a simple formula for the normal cone to such sets, and use this to show that these sets are prox-regular at all points with rank exactly equal to $s$. The normal cone formula appears to be new. This allows for easy application of prior results guaranteeing local linear convergence of the fundamental alternating projection algorithm between sets, one of which is a rank constraint set. We apply this to show local linear convergence of another fundamental algorithm, approximate steepest descent. Our results apply not only to linear systems with rank constraints, as has been treated extensively in the literature, but also nonconvex systems with rank constraints.
△ Less
Submitted 3 December, 2012; v1 submitted 2 December, 2011;
originally announced December 2011.
-
Local Linear Convergence of Approximate Projections onto Regularized Sets
Authors:
D. Russell Luke
Abstract:
The numerical properties of algorithms for finding the intersection of sets depend to some extent on the regularity of the sets, but even more importantly on the regularity of the intersection. The alternating projection algorithm of von Neumann has been shown to converge locally at a linear rate dependent on the regularity modulus of the intersection. In many applications, however, the sets in qu…
▽ More
The numerical properties of algorithms for finding the intersection of sets depend to some extent on the regularity of the sets, but even more importantly on the regularity of the intersection. The alternating projection algorithm of von Neumann has been shown to converge locally at a linear rate dependent on the regularity modulus of the intersection. In many applications, however, the sets in question come from inexact measurements that are matched to idealized models. It is unlikely that any such problems in applications will enjoy metrically regular intersection, let alone set intersection. We explore a regularization strategy that generates an intersection with the desired regularity properties. The regularization, however, can lead to a significant increase in computational complexity. In a further refinement, we investigate and prove linear convergence of an approximate alternating projection algorithm. The analysis provides a regularization strategy that fits naturally with many ill-posed inverse problems, and a mathematically sound stopping criterion for extrapolated, approximate algorithms. The theory is demonstrated on the phase retrieval problem with experimental data. The conventional early termination applied in practice to unregularized, consistent problems in diffraction imaging can be justified fully in the framework of this analysis providing, for the first time, proof of convergence of alternating approximate projections for finite dimensional, consistent phase retrieval problems.
△ Less
Submitted 14 September, 2011; v1 submitted 10 August, 2011;
originally announced August 2011.
-
Robust Mixing for Ab-Initio Quantum Mechanical Calculations
Authors:
L. D. Marks,
D. R. Luke
Abstract:
We study the general problem of mixing for ab-initio quantum-mechanical problems. Guided by general mathematical principles and the underlying physics, we propose a multisecant form of Broydens second method for solving the self-consistent field equations of Kohn-Sham density functional theory. The algorithm is robust, requires relatively little finetuning and appears to outperform the current s…
▽ More
We study the general problem of mixing for ab-initio quantum-mechanical problems. Guided by general mathematical principles and the underlying physics, we propose a multisecant form of Broydens second method for solving the self-consistent field equations of Kohn-Sham density functional theory. The algorithm is robust, requires relatively little finetuning and appears to outperform the current state of the art, converging for cases that defeat many other methods. We compare our technique to the conventional methods for problems ranging from simple to nearly pathological.
△ Less
Submitted 20 January, 2008;
originally announced January 2008.
-
Relaxed Averaged Alternating Reflections for Diffraction Imaging
Authors:
D. Russell Luke
Abstract:
We report on progress in algorithms for iterative phase retrieval. The theory of convex optimization is used to develop and to gain insight into counterparts for the nonconvex problem of phase retrieval. We propose a relaxation of averaged alternating reflectors and determine the fixed point set of the related operator in the convex case. A numerical study supports our theoretical observations a…
▽ More
We report on progress in algorithms for iterative phase retrieval. The theory of convex optimization is used to develop and to gain insight into counterparts for the nonconvex problem of phase retrieval. We propose a relaxation of averaged alternating reflectors and determine the fixed point set of the related operator in the convex case. A numerical study supports our theoretical observations and demonstrates the effectiveness of the algorithm compared to the current state of the art.
△ Less
Submitted 11 May, 2004;
originally announced May 2004.