Abstracts

Marco Mondelli:
Learning from Data of All Sorts (Synthetic, Private, Surrogate and Mixture): Lessons from High-Dimensional Regression

Machine learning models are routinely trained on all sorts of data: synthetic data obtained via generative models; private data held by external providers; surrogate data labeled by pretrained teacher models; and mixture data coming from heterogenous domains. While each of these settings has its own distinctive features, in the talk I will outline a common methodology based on asymptotic characterizations of the risk, as well as non-asymptotic deterministic equivalents. Our approach provides precise predictions and establishes rigorous scaling laws for the error (under classical source-capacity conditions), for a broad class of high-dimensional ridge regression models. These include model shift, covariance shift, heterogeneous noise levels, dataset sizes growing at different rates, and the addition of noise to enforce differential privacy. In turn, the theoretical results yield insights into ubiquitous questions in machine learning practice: How to select synthetic data? When to commit to buying privatized data, without explicitly assessing its performance? To what extent can surrogate data labeled by an imperfect teacher improve performance? When does data mixing produce synergies, significantly outperforming the usage of data from each domain in isolation? We will give quantitative answers to these questions and, going beyond linear models, we will validate our findings in practical datasets. 

Francesca Maggioni:
Bounds and Approximations for Large Scale Optimization Problems Under Uncertainty

Many real world decision problems are dynamic and affected by uncertainty. Stochastic Programming provides a powerful approach to handle this uncertainty within a multi-period decision framework. However, as the number of stages increases, the computational complexity of these problems grows exponentially, posing significant challenges. To tackle this, approximation techniques are often used to simplify the original problem, providing useful upper and lower bounds for the objective function’s optimal value.
This talk explores methods for generating bounds for a wide variety of problem structures affected by uncertainty. We begin by discussing bounds based on scenario grouping under the assumption that a sufficiently large scenario tree is given but is unsolvable, both in the context of stochastic programming and distributionally robust optimization. Next, we extend these techniques to address more complex problems, including multi-horizon stochastic optimization and decision-dependent stochastic optimization.
Finally, the talk introduces the integration of these bounding methods with Benders’ decomposition. To reduce the computational burden of generating a cut for each scenario, a Benders refinement-chain cuts method is proposed, where scenario subsets are used to generate group-wise optimality cuts. This aggregation significantly lowers the number of cuts required, while preserving valid lower bounds. Theoretical relationships between cuts generated at different refinement levels are established.
Numerical experiments on various energy and transportation optimization problems demonstrate the efficiency of the proposed approaches.

Tobias Sutter:
Recursive Frank-Wolfe Optimization with Learned Domains and Objectives

We study a data-driven variant of the classical Frank-Wolfe algorithm for convex optimization over compact convex domains. In contrast to the standard setting, both the feasible domain and the objective function are initially unknown and must be learned from data during the optimization process. Our method incorporates statistical estimators directly into the Frank-Wolfe iterations, thereby preserving the projection-free nature of the classical algorithm while adapting to uncertainty in the problem formulation. We prove convergence guarantees showing that the optimization error is controlled by the accuracy of the learned domain and objective estimators. Numerical experiments illustrate that the proposed recursive Frank-Wolfe method can achieve convergence behavior comparable to the classical algorithm with exact problem knowledge, while offering significant computational savings in data-driven settings.

This is joint work with Marcel Kaiser, University of Konstanz.