Date/Time : 28 Oct 2020 16:00
Venue : via Zoom
Speaker : Lukas Einkemmer, University of Innsbruck

Join Zoom Meeting https://nus-sg.zoom.us/j/84355509004?pwd=ZEpxWEFMVjd2d0hDamliVUMvY1ozdz09 Meeting ID: 843 5550 9004 Password: 357874 Abstract: Computational fluid dynamics has become an extremely successful framework to understand a range of natural phenomena. Despite these successes, it is increasingly realized that many phenomena in both space and laboratory plasmas require a kinetic description. However, kinetic models come with a range of computational challenges that have to be addressed before these simulations can be routinely employed. The most challenging aspect is certainly the up to six-dimensional phase space. A promising approach to break the so-called curse of dimensionality are low-rank techniques. Contrary to many other complexity reduction methods, such as sparse grids or particle methods, low-rank techniques are able to resolve the small structures that are inherent in such problems. In this talk, we will discuss some of the algorithmic, analytic, and computational advances that have been made in the last few years and explain how this will enable the large scale simulation of kinetic equations using dynamical low-rank approximation.

Add to calendar :

Date/Time : 21 Oct 2020 16:00
Venue : via Zoom
Speaker : Milan Korda, LAAS-CNRS

Join Zoom Meeting https://nus-sg.zoom.us/j/82236914659?pwd=ZzgyeDU5SjJ1bFRNd1lHVURlV2wzQT09 Meeting ID: 822 3691 4659 Password: 083341 Abstract: This talk will give an overview of the applications of the moment-sum-of-squares hierarchy in the fields of dynamical systems and control. We will explain how a number of problems such as optimal control, region of attraction or invariant set computation can be cast as linear programming problems in the space of Borel measures and how these linear programs can be approximated using the moment-sum-of-squares hierarchy of semidefinite programming problems with guaranteed convergence. The talk will finish by describing how sparsity of the underlying dynamics can be exploited, leading to a decoupled computation, thereby allowing for a significant reduction in computation time without sacrificing convergence guarantees.  

Add to calendar :

Date/Time : 14 Oct 2020 16:00
Venue : via Zoom
Speaker : Tiangang Cui, Monash University

Join Zoom Meeting https://nus-sg.zoom.us/j/83653279396?pwd=L2hNSzI5dENjYWpRSkZpOEFMT2NzZz09 Meeting ID: 836 5327 9396 Password: 150910 Characterising intractable high-dimensional random variables is one of the fundamental challenges in stochastic computation. It has broad applications in statistical physics, machine learning, uncertainty quantification, econometrics, and beyond. The recent surge of transport maps offers a mathematical foundation and new insights for tackling this challenge. In this talk, we present a functional tensor-train (FTT) based monotonicity-preserving construction of inverse Rosenblatt transport in high dimensions. It characterises intractable random variables via couplings with tractable reference random variables. By integrating our FTT-based approach into a nested approximation framework inspired by deep neural networks, we are able to significantly expand its capability to random variables with complicated nonlinear interactions and concentrated density functions. We demonstrate the efficacy of the FTT-based inverse Rosenblatt transport on a range of applications in statistical learning and uncertainty quantification, including parameter estimation for dynamical systems, PDE-constrained inverse problems, and Bayesian filtering. This is joint work with Dr. Sergey Dolgov (Bath) and Mr. Yiran Zhao (Monash).

Add to calendar :

Date/Time : 30 Sep 2020 16:00
Venue : via Zoom
Speaker : Haijun Yu, Chinese Academy of Science

Join Zoom Meeting https://nus-sg.zoom.us/j/92338820026?pwd=U043L0JrWU9XRXFDUTVmMXVxNVBJdz09 Meeting ID: 923 3882 0026 Password: 936000 Abstract: Deep neural networks with rectified linear units (ReLU) are getting very popular recently.   Some theoretical progresses on deep ReLU network approximation for functions in Sobolev space and Korobov space have recently been made by several groups.   In this talk, we show that deep networks with rectified power units (RePU) can give better approximations for smooth functions than deep ReLU networks. Our analysis is based on classical polynomial approximation theory and some efficient and stable algorithms we proposed to convert polynomials into deep RePU networks of optimal size without any approximation error.  Our constructive proofs reveal clearly a relation between deep RePU networks and spectral methods.

Add to calendar :

Date/Time : 16 Sep 2020 16:00
Venue : va Zoom
Speaker : Jinglai Li, University of Birmingham

Join Zoom Meeting https://nus-sg.zoom.us/j/92493981998?pwd=Q3VUR3ljUDBoT2JrNkN4N2h4YkJKZz09 Meeting ID: 924 9398 1998 Password: 186838 Abstract: The performance of Hamiltonian Monte Carlo (HMC) sampler depends critically on some algorithm parameters such as the total integration time and the numerical integration stepsize. The parameter tuning is particularly challenging when the mass matrix of the HMC sampler is adapted. We propose in this work a Kolmogorov-Sinai entropy (KSE) based design criterion to optimize these algorithm parameters, which can avoid some potential issues in the often used jumping-distance based measures. For near-Gaussian distributions, we are able to derive the optimal algorithm parameters with respect to the KSE criterion analytically. As a byproduct the KSE criterion also provides a theoretical justification for the need to adapt the mass matrix in HMC sampler. Building upon the results, we propose an adaptive HMC algorithm, and we then demonstrate the performance of the proposed algorithm with numerical examples.

Add to calendar :

Date/Time : 09 Sep 2020 16:00
Venue : via Zoom
Speaker : Zhu Zhanxing, Peking University

Join Zoom Meeting https://nus-sg.zoom.us/j/93325963781?pwd=VVhBWkljeGQycExSOVVBdU10UW40Zz09 Meeting ID: 933 2596 3781 Password: 300023 Abstract: Iit has been a long-standing debate that “Is deep learning alchemy or science? ”, since the success of deep learning mostly relies on various engineering design and tricks, lack of  theoretical foundation. Unfortunately, the underlying mechanism of deep learning is still mysterious, which severely limits its further development from both theoretical and application aspects. In this talk, I will introduce some of our attempts on theoretically understanding deep learning, mainly focusing on analyzing its training methods and tricks, including stochastic gradient descent, batch normalization and knowledge distillation.  (1) We analyze the implicit regularization property of stochastic gradient descent (SGD), i.e. interpreting why SGD could find well generalizing minima compared with other alternatives; (2) We comprehensively reveal the learning dynamics of batch normalization and weight decay, and show its benefits on avoiding vanishing/exploding gradients, not being trapped into sharp minima and loss drop when decaying learning rate. (3) We also show the underlying mechanism of knowledge distillation, including its transfer risk bound, data efficiency and imperfect teacher distillation.  These new findings shed some light on understanding the  deep learning towards opening this black-box and also inspires new algorithmic design.

Add to calendar :

Date/Time : 02 Sep 2020 16:00
Venue : via Zoom
Speaker : Sonoda Sho, RIKEN AIP, Japan

Join Zoom Meeting https://nus-sg.zoom.us/j/99186202215?pwd=ZFp5S21NVGVJclphVU82cm54TlpGQT09 Meeting ID: 991 8620 2215 Password: 026907 Abstract: Neural networks are known to be a black box due to the high-dimensionality, non-linearity, and non-convexity of parameters. In the so-called `integral representation’, at the cost of lifting up the parameters to infinite-dimensional function space, the parameter becomes linear and convex. The presenter aims to turn the black box into a white box by using integral representation. In this talk, he describes the integral representation first, then introduces recent application results for analyzing the parameters acquired through several learning algorithms.

Add to calendar :

Date/Time : 26 Aug 2020 16:00
Venue : via Zoom
Speaker : Tong Xin, NUS

Join Zoom Meeting https://nus-sg.zoom.us/j/92076578514?pwd=aWk4Nm1Cc3htS1hGWDVpTlZ2cUgrdz09 Meeting ID: 920 7657 8514 Password: 459566 Gradient descent (GD) is known to converge quickly for convex objective functions, but it can be trapped at local minima. On the other hand, Langevin dynamics (LD) can explore the state space and find global minima, but in order to give accurate estimates, LD needs to run with a small discretization step size and weak stochastic force, which in general slows down its convergence. This paper shows that these two algorithms can “collaborate” through a simple exchange mechanism, in which they swap their current positions if LD yields a lower objective function. This idea can be seen as the singular limit of the replica-exchange technique from the sampling literature. We show that this new algorithm converges to the global minimum linearly with high probability, assuming the objective function is strongly convex in a neighborhood of the unique global minimum. By replacing gradients with stochastic gradients, and adding a proper threshold to the exchange mechanism, our algorithm can also be used in online settings. We further verify our theoretical results through some numerical experiments, and observe superior performance of the proposed algorithm over running GD or LD alone. We will further explain how does replica exchange method improve LD’s spectral gap using Poincare type of inequality.

Add to calendar :

Date/Time : 19 Aug 2020 16:00
Venue : via Zoom
Speaker : Pang Tongyao , National University of Singapore

Join Zoom Meeting https://nus-sg.zoom.us/j/97080214054?pwd=cnJtSXF0YTZWZ1U2QU5YQWpwaU1tUT09 Meeting ID: 970 8021 4054 Password: 632494 Abstract: In this talk, I will start with the classical sparse estimation, whih has been extensively studied and applied to a wide range of applications, e.g., image restoration, in the past few decades. While most of the existing methods targeted at solving a maximum a posterior (MAP) variational model with the sparsity prior, we proposed to find the minimum mean square error (MMSE) solution which provides a good balance between sparsity and unbiasedness. We develop a coordinate minimization algorithm and connect it to a variational model to establish a comprehensive convergence analysis. In recent years, deep learning has shown its advantages over the traditional sparsity prior based methods for image restoration problems. However, most deep learning methods are supervised which require a high-quality training dataset. Bringing the idea of Bayesian inference from sparse estimation to deep learning further, we are able to develop a self-supervised deep learning method that calls no other training samples but still achieves SOTA performance. The key to success of our method lies in the approximation to the MMSE estimate, which helps to reduce model bias and enables ensemble prediction scheme with lower variance.

Add to calendar :