schliessen

Filtern

 

Bibliotheken

Performance of first-order methods for smooth convex minimization: a novel approach

We introduce a novel approach for analyzing the worst-case performance of first-order black-box optimization methods. We focus on smooth unconstrained convex minimization over the Euclidean space. Our approach relies on the observation that by definition, the worst-case behavior of a black-box optim... Full description

Journal Title: Mathematical programming 2013, Vol.145 (1-2), p.451-482
Main Author: Drori, Yoel
Other Authors: Teboulle, Marc
Format: Electronic Article Electronic Article
Language: English
Subjects:
Publisher: Berlin/Heidelberg: Springer Berlin Heidelberg
ID: ISSN: 0025-5610
Zum Text:
SendSend as email Add to Book BagAdd to Book Bag
Staff View
recordid: cdi_proquest_miscellaneous_1551048168
title: Performance of first-order methods for smooth convex minimization: a novel approach
format: Article
creator:
  • Drori, Yoel
  • Teboulle, Marc
subjects:
  • Algorithms
  • Analysis
  • Calculus of Variations and Optimal Control
  • Optimization
  • Combinatorics
  • Convex analysis
  • Full Length Paper
  • Mathematical analysis
  • Mathematical and Computational Physics
  • Mathematical Methods in Physics
  • Mathematical programming
  • Mathematics
  • Mathematics and Statistics
  • Mathematics of Computing
  • Methods
  • Minimization
  • Numerical Analysis
  • Optimization
  • Optimization algorithms
  • Performance evaluation
  • Semidefinite programming
  • Studies
  • Theoretical
ispartof: Mathematical programming, 2013, Vol.145 (1-2), p.451-482
description: We introduce a novel approach for analyzing the worst-case performance of first-order black-box optimization methods. We focus on smooth unconstrained convex minimization over the Euclidean space. Our approach relies on the observation that by definition, the worst-case behavior of a black-box optimization method is by itself an optimization problem, which we call the performance estimation problem (PEP). We formulate and analyze the PEP for two classes of first-order algorithms. We first apply this approach on the classical gradient method and derive a new and tight analytical bound on its performance. We then consider a broader class of first-order black-box methods, which among others, include the so-called heavy-ball method and the fast gradient schemes. We show that for this broader class, it is possible to derive new bounds on the performance of these methods by solving an adequately relaxed convex semidefinite PEP. Finally, we show an efficient procedure for finding optimal step sizes which results in a first-order black-box method that achieves best worst-case performance.
language: eng
source:
identifier: ISSN: 0025-5610
fulltext: no_fulltext
issn:
  • 0025-5610
  • 1436-4646
url: Link


@attributes
NO1
SEARCH_ENGINEprimo_central_multiple_fe
SEARCH_ENGINE_TYPEPrimo Central Search Engine
RANK2.6938734
LOCALfalse
PrimoNMBib
record
control
sourceidgale_proqu
recordidTN_cdi_proquest_miscellaneous_1551048168
sourceformatXML
sourcesystemPC
galeidA368452901
sourcerecordidA368452901
originalsourceidFETCH-LOGICAL-1387t-cacbb9c302c2b13184a5178b317421b3a728613752452987b6f0df681038ed170
addsrcrecordideNp1kUGPFCEQhYnRxHH0B3gj8eKl1yroBtbbZuOqySaaqGdC07DDphtG6Nmov96atAdj4gUS-B7vUY-xlwgXCKDfNAQE3QHKDtRAyyO2w16qrle9esx2AGLoBoXwlD1r7R6ASGN27MvnUGOpi8s-8BJ5TLWtXalTqHwJ66FMjdM9b0sp64H7kh_CD76knJb0y62p5Lfc8Vwewszd8ViL84fn7El0cwsv_ux79u3m3dfrD93tp_cfr69uO7LWa-edH8dLL0F4MaJE07sBtRkl6l7gKJ0WRqHUg-gHcWn0qCJMURkEacKEGvbs9fYu2X4_hbbaJTUf5tnlUE7N4jAg9AaVIfTVP-h9OdVM6YgS5IBCCqIuNurOzcGmHMtaHcV0U1gSfT3EROdXUplzIhr4nuEm8LW0VkO0x5oWV39aBHvuxW69WJq2PfdigTRi0zRi812of0X5r-g3rSmNrg
sourcetypeAggregation Database
isCDItrue
recordtypearticle
pqid1525241232
display
typearticle
titlePerformance of first-order methods for smooth convex minimization: a novel approach
creatorDrori, Yoel ; Teboulle, Marc
creatorcontribDrori, Yoel ; Teboulle, Marc
descriptionWe introduce a novel approach for analyzing the worst-case performance of first-order black-box optimization methods. We focus on smooth unconstrained convex minimization over the Euclidean space. Our approach relies on the observation that by definition, the worst-case behavior of a black-box optimization method is by itself an optimization problem, which we call the performance estimation problem (PEP). We formulate and analyze the PEP for two classes of first-order algorithms. We first apply this approach on the classical gradient method and derive a new and tight analytical bound on its performance. We then consider a broader class of first-order black-box methods, which among others, include the so-called heavy-ball method and the fast gradient schemes. We show that for this broader class, it is possible to derive new bounds on the performance of these methods by solving an adequately relaxed convex semidefinite PEP. Finally, we show an efficient procedure for finding optimal step sizes which results in a first-order black-box method that achieves best worst-case performance.
identifier
0ISSN: 0025-5610
1EISSN: 1436-4646
2DOI: 10.1007/s10107-013-0653-0
3CODEN: MHPGA4
languageeng
publisherBerlin/Heidelberg: Springer Berlin Heidelberg
subjectAlgorithms ; Analysis ; Calculus of Variations and Optimal Control; Optimization ; Combinatorics ; Convex analysis ; Full Length Paper ; Mathematical analysis ; Mathematical and Computational Physics ; Mathematical Methods in Physics ; Mathematical programming ; Mathematics ; Mathematics and Statistics ; Mathematics of Computing ; Methods ; Minimization ; Numerical Analysis ; Optimization ; Optimization algorithms ; Performance evaluation ; Semidefinite programming ; Studies ; Theoretical
ispartofMathematical programming, 2013, Vol.145 (1-2), p.451-482
rights
0Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2013
1COPYRIGHT 2014 Springer
2Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2014
lds50peer_reviewed
oafree_for_read
citesFETCH-LOGICAL-1387t-cacbb9c302c2b13184a5178b317421b3a728613752452987b6f0df681038ed170
links
openurl$$Topenurl_article
thumbnail$$Usyndetics_thumb_exl
search
creatorcontrib
0Drori, Yoel
1Teboulle, Marc
title
0Performance of first-order methods for smooth convex minimization: a novel approach
1Mathematical programming
addtitleMath. Program
descriptionWe introduce a novel approach for analyzing the worst-case performance of first-order black-box optimization methods. We focus on smooth unconstrained convex minimization over the Euclidean space. Our approach relies on the observation that by definition, the worst-case behavior of a black-box optimization method is by itself an optimization problem, which we call the performance estimation problem (PEP). We formulate and analyze the PEP for two classes of first-order algorithms. We first apply this approach on the classical gradient method and derive a new and tight analytical bound on its performance. We then consider a broader class of first-order black-box methods, which among others, include the so-called heavy-ball method and the fast gradient schemes. We show that for this broader class, it is possible to derive new bounds on the performance of these methods by solving an adequately relaxed convex semidefinite PEP. Finally, we show an efficient procedure for finding optimal step sizes which results in a first-order black-box method that achieves best worst-case performance.
subject
0Algorithms
1Analysis
2Calculus of Variations and Optimal Control; Optimization
3Combinatorics
4Convex analysis
5Full Length Paper
6Mathematical analysis
7Mathematical and Computational Physics
8Mathematical Methods in Physics
9Mathematical programming
10Mathematics
11Mathematics and Statistics
12Mathematics of Computing
13Methods
14Minimization
15Numerical Analysis
16Optimization
17Optimization algorithms
18Performance evaluation
19Semidefinite programming
20Studies
21Theoretical
issn
00025-5610
11436-4646
fulltextfalse
rsrctypearticle
creationdate2013
recordtypearticle
recordideNp1kUGPFCEQhYnRxHH0B3gj8eKl1yroBtbbZuOqySaaqGdC07DDphtG6Nmov96atAdj4gUS-B7vUY-xlwgXCKDfNAQE3QHKDtRAyyO2w16qrle9esx2AGLoBoXwlD1r7R6ASGN27MvnUGOpi8s-8BJ5TLWtXalTqHwJ66FMjdM9b0sp64H7kh_CD76knJb0y62p5Lfc8Vwewszd8ViL84fn7El0cwsv_ux79u3m3dfrD93tp_cfr69uO7LWa-edH8dLL0F4MaJE07sBtRkl6l7gKJ0WRqHUg-gHcWn0qCJMURkEacKEGvbs9fYu2X4_hbbaJTUf5tnlUE7N4jAg9AaVIfTVP-h9OdVM6YgS5IBCCqIuNurOzcGmHMtaHcV0U1gSfT3EROdXUplzIhr4nuEm8LW0VkO0x5oWV39aBHvuxW69WJq2PfdigTRi0zRi812of0X5r-g3rSmNrg
startdate20130323
enddate20130323
creator
0Drori, Yoel
1Teboulle, Marc
general
0Springer Berlin Heidelberg
1Springer
2Springer Nature B.V
scope
0AAYXX
1CITATION
2BSHEE
33V.
47SC
57WY
67WZ
77XB
887Z
988I
108AL
118AO
128FD
138FE
148FG
158FK
168FL
17ABJCF
18ABUWG
19ARAPS
20AZQEC
21BENPR
22BEZIV
23BGLVJ
24DWQXO
25FRNLG
26F~G
27GNUQQ
28HCIFZ
29JQ2
30K60
31K6~
32K7-
33L.-
34L6V
35L7M
36L~C
37L~D
38M0C
39M0N
40M2P
41M7S
42P5Z
43P62
44PQBIZ
45PQBZA
46PQEST
47PQQKQ
48PQUKI
49PRINS
50PTHSS
51PYYUZ
52Q9U
sort
creationdate20130323
titlePerformance of first-order methods for smooth convex minimization: a novel approach
authorDrori, Yoel ; Teboulle, Marc
facets
frbrtype5
frbrgroupidcdi_FETCH-LOGICAL-1387t-cacbb9c302c2b13184a5178b317421b3a728613752452987b6f0df681038ed170
rsrctypearticles
prefilterarticles
languageeng
creationdate2013
topic
0Algorithms
1Analysis
2Calculus of Variations and Optimal Control; Optimization
3Combinatorics
4Convex analysis
5Full Length Paper
6Mathematical analysis
7Mathematical and Computational Physics
8Mathematical Methods in Physics
9Mathematical programming
10Mathematics
11Mathematics and Statistics
12Mathematics of Computing
13Methods
14Minimization
15Numerical Analysis
16Optimization
17Optimization algorithms
18Performance evaluation
19Semidefinite programming
20Studies
21Theoretical
toplevelpeer_reviewed
creatorcontrib
0Drori, Yoel
1Teboulle, Marc
collection
0CrossRef
1Academic OneFile (A&I only)
2ProQuest Central (Corporate)
3Computer and Information Systems Abstracts
4ABI/INFORM Collection
5ABI/INFORM Global (PDF only)
6ProQuest Central (purchase pre-March 2016)
7ABI/INFORM Global (Alumni Edition)
8Science Database (Alumni Edition)
9Computing Database (Alumni Edition)
10ProQuest Pharma Collection
11Technology Research Database
12ProQuest SciTech Collection
13ProQuest Technology Collection
14ProQuest Central (Alumni) (purchase pre-March 2016)
15ABI/INFORM Collection (Alumni Edition)
16Materials Science & Engineering Collection
17ProQuest Central (Alumni Edition)
18Advanced Technologies & Aerospace Collection
19ProQuest Central Essentials
20ProQuest Central
21Business Premium Collection
22Technology Collection
23ProQuest Central Korea
24Business Premium Collection (Alumni)
25ABI/INFORM Global (Corporate)
26ProQuest Central Student
27SciTech Premium Collection
28ProQuest Computer Science Collection
29ProQuest Business Collection (Alumni Edition)
30ProQuest Business Collection
31Computer Science Database
32ABI/INFORM Professional Advanced
33ProQuest Engineering Collection
34Advanced Technologies Database with Aerospace
35Computer and Information Systems Abstracts – Academic
36Computer and Information Systems Abstracts Professional
37ABI/INFORM Global
38Computing Database
39Science Database
40Engineering Database
41Advanced Technologies & Aerospace Database
42ProQuest Advanced Technologies & Aerospace Collection
43ProQuest One Business
44ProQuest One Business (Alumni)
45ProQuest One Academic Eastern Edition
46ProQuest One Academic
47ProQuest One Academic UKI Edition
48ProQuest Central China
49Engineering Collection
50ABI/INFORM Collection China
51ProQuest Central Basic
jtitleMathematical programming
delivery
delcategoryRemote Search Resource
fulltextno_fulltext
addata
au
0Drori, Yoel
1Teboulle, Marc
formatjournal
genrearticle
ristypeJOUR
atitlePerformance of first-order methods for smooth convex minimization: a novel approach
jtitleMathematical programming
stitleMath. Program
date2013-03-23
risdate2013
volume145
issue1-2
spage451
epage482
pages451-482
issn0025-5610
eissn1436-4646
codenMHPGA4
abstractWe introduce a novel approach for analyzing the worst-case performance of first-order black-box optimization methods. We focus on smooth unconstrained convex minimization over the Euclidean space. Our approach relies on the observation that by definition, the worst-case behavior of a black-box optimization method is by itself an optimization problem, which we call the performance estimation problem (PEP). We formulate and analyze the PEP for two classes of first-order algorithms. We first apply this approach on the classical gradient method and derive a new and tight analytical bound on its performance. We then consider a broader class of first-order black-box methods, which among others, include the so-called heavy-ball method and the fast gradient schemes. We show that for this broader class, it is possible to derive new bounds on the performance of these methods by solving an adequately relaxed convex semidefinite PEP. Finally, we show an efficient procedure for finding optimal step sizes which results in a first-order black-box method that achieves best worst-case performance.
copBerlin/Heidelberg
pubSpringer Berlin Heidelberg
doi10.1007/s10107-013-0653-0
oafree_for_read