Webinars

Performance Optimization of Deep-learning workloads in HPC environments

With the advent of specialized accelerators for optimized deep-learning, efficient data feed to the accelerator and optimized data movement between accelerators became crucial to performance. We aim to engage with scientific application teams leveraging DL in their application to apply analysis techniques to identify performance bottlenecks and to develop high-performance application-specific solutions for optimized data movement.

Contact name: Khaled Ibrahim

Contact email: kzibrahim@lbl.gov

Theme: Scalable Learning - Platform

Keywords: performance, data movement optimization, AI accelerators 

Integrating Machine Learning with Scientific Spatial and Temporal Modeling

Combining data-driven machine learning approaches with scientific mechanistic modeling can enable greater efficiency and better generalization for scientific prediction. I will discuss the challenges associated with incorporating fundamental physical laws into the machine learning process (e.g., “physics-informed” neural networks). I will then show how changing the learning paradigm for these problems can greatly improve performance. 

Contact name: Aditi Krishnapriyan

Contact email: akrishnapriyan@lbl.gov

Theme: Surrogate modeling

Keywords: physics-informed neural networks  

MFA: Multivariate Functional Approximation

Scientific data may be transformed by recasting to a data model fundamentally different from the discrete pointwise or element-wise datasets produced by computational models. In Multivariate Functional Approximation, or MFA, scientific datasets are redefined in a hypervolume of piecewise-continuous basis functions. Compared with existing discrete models, the continuous functional model can save space while affording many spatiotemporal analyses without reverting back to the discrete form. The MFA model can represent numerous types of data because it is agnostic to the mesh, field, or discretization of the input dataset. In the spectrum of AI/ML approaches, the MFA can be considered as one type of surrogate model. Once trained, it provides regression and implicit evaluation anywhere in the domain---like an implicit neural network but less expensive to train---not limited to the input data points. Besides modeling function values, the same MFA model can be queried for  analytical exact closed-form evaluation of points, derivatives, and integrals to high order, anywhere inside the domain. Applications of the MFA include high-order visualization, derivative fields, comparison of datasets with different discretizations, remapping between coupled multiphysics simulations, smoothing, simplification, filtering, and super-resolution. 

Contact name: Tom Peterka

Contact email: tpeterka@mcs.anl.gov

Theme: Surrogate Modeling, Representation - Functional Approximation

Keywords: Data reduction

Robust and Efficient Probabilistic Machine Learning for Scientific Data

Modern deep neural networks are in general quite brittle, and hence less robust to noise in the input or adversarial perturbation applied to them. From an information-theoretic point of view, the ability of the model to achieve better generalization and robustness will depend on whether the model learns more semantically meaningful information and compresses the nuisance information. Typically, the DNNs are trained by reducing an empirical loss function which can lead to nuisance or irrelevant information being memorized, that can have a detrimental impact on the model’s robustness. We describe an information-theoretic Bayesian deep learning framework, that combined with structural sparsity inducing priors provides an efficient and robust approach to probabilistic modeling and uncertainty quantification for scientific data.

Contact name: Sandeep Madireddy

Contact email: smadireddy@anl.gov

Theme: Surrogate Modeling, Uncertainty Quantification

Keywords: Uncertainty quantification, robustness

Extreme-Scale Feature Tracking for Science

Feature tracking plays an essential role in understanding scientific data, such as finding rare events, deriving quantities and statistics, and reducing data to store.  FTK (Feature Tracking Kit) is a framework that simplifies, scales, and delivers various feature-tracking algorithms for scientific data.  FTK was initially motivated by the lack of general-purpose feature-tracking tools and difficulties in scaling feature-tracking algorithms. The critical innovation of FTK is the simplicial spacetime meshing scheme that generalizes spatial meshes to 4D spacetime, which simplifies the handling of corner cases and improves the scalability of feature tracking algorithms, enabling scalable and parallel processing.  We also integrate an AI-based paradigm to extract and track features that cannot be defined mathematically, easing feature identification and analysis for improved data understanding.

Contact name: Hanqi Guo

Contact email: hguo@anl.gov

Theme: Data Analysis

Keywords: feature extraction, identification, tracking 

Scalable, Distributed, and Asynchronous Training of GANs 

A well-known problem in ML is to develop training procedures for GANs with stable convergence properties. In this work, we develop new algorithms for training GANs with theoretical convergence guarantees that are based on a stochastic extension of projective splitting. We implement asynchronous distributed versions that are robust to “straggler effects” on large-scale HPC systems and are also communication efficient.   

Contact name: Patrick Johnstone 

Contact email: pjohnston@bnl.gov

Theme: Scalable Learning - Optimization

Keywords: Generative networks, distributed training

Physics Guided Bayesian Neural Inference with xAI

We propose a physics-constrained deep learning model to fast produce accurate PDE solutions while preserving many of the advantages of classic numerical solvers. From a probabilistic modeling perspective, the learning scheme resembles the amortized variational methods in Bayesian inference. To better emulate the current numerical solvers, we further integrate XAI techniques to verify and exploit neighboring information to reinforce locality in space and time.

Contact Name: Xihaier Luo

Contact Email: xluo@bnl.gov

Theme: Surrogate, UQ 

Keywords: Explainable AI, Physics-informed deep neural networks

Anomaly and Change Point Detection for Scientific Discovery

We have developed robust online change point and anomaly detection methods for scientific discovery. The most interesting events happen when the distribution is changed or abnormal / extreme events happen. With change point and anomaly detection, scientists can prioritize which data to look at.

Contact Name: Thomas Flynn

Contact Email: tflynn@bnl.gov

Theme: Representation Learning

Keywords: Anomaly detection, extreme event detection

Automatic Differentiation as a Tool for Computational Science

Automatic Differentiation (AutoDiff) is a powerful tool for the computation of derivatives of functions defined by computer programs. AutoDiff lies at the heart of frameworks for machine learning, modeling languages for mathematical optimization, and derivative-based uncertainty quantification techniques. In this webinar, we aim to introduce the basic concepts of automatic differentiation, describe implementation techniques, and discuss the effective use of AutoDiff tools and frameworks.

Contact Name: Paul Hovland / Jan Hückelheim

Contact E-mail: hovland@anl.gov

Theme: Scalable Learning - Platform

Keywords: Automatic Differentiation (AD)

Machine Learning based Volumetric Compression

Development of physics-Informed deep neural networks for volumetric data compression.

Contact Name: Sanjay Ranka

Contact Email: sranka@ufl.edu

Keywords: Physics-informed deep neural networks, data reduction/compression

Asynchronous I/O Strategy for Large-Scale Deep Learning Applications

We propose a buffering strategy to hide the I/O time during the training phrase of deep learning applications.  For CosmoFlow and similar scientific applications, we are able to completely hide the I/O time and reduce the overall training time by nearly a half.

Contact name: Wei-keng Liao / John Wu

Contact email: wkliao@northwestern.edu

Theme: Scalable Learning - IO

Keywords: I/O optimization, large-scale training