Webinars

Distributed-memory Training of Graph Neural Networks

This webinar will give an overview of algorithms and methods for training various graph neural networks (GNNs), such as graph convolutional networks and graph attention networks. It will cover the computational primitives that are needed for efficient computing on GNNs, with a particular focus on parallelism and memory consumption. I will then briefly mention the use of communication-avoiding algorithms for reducing communication and the graph sampling techniques for reducing memory consumption.

Contact name: Aydin Buluc

Contact email: abuluc@lbl.gov

Theme: Scalable Learning - Library, Representation - Graph Learning

Keywords: graph neural networks, communication-avoiding algorithms, performance

Performance Optimization of Deep-learning workloads in HPC environments

With the advent of specialized accelerators for optimized deep-learning, efficient data feed to the accelerator and optimized data movement between accelerators became crucial to performance. We aim to engage with scientific application teams leveraging DL in their application to apply analysis techniques to identify performance bottlenecks and to develop high-performance application-specific solutions for optimized data movement.

Contact name: Khaled Ibrahim

Contact email: kzibrahim@lbl.gov

Theme: Scalable Learning - Platform

Keywords: performance, data movement optimization, AI accelerators

Integrating Machine Learning with Scientific Spatial and Temporal Modeling

Combining data-driven machine learning approaches with scientific mechanistic modeling can enable greater efficiency and better generalization for scientific prediction. I will discuss the challenges associated with incorporating fundamental physical laws into the machine learning process (e.g., “physics-informed” neural networks). I will then show how changing the learning paradigm for these problems can greatly improve performance.

Contact name: Aditi Krishnapriyan

Contact email: akrishnapriyan@lbl.gov

Theme: Surrogate modeling

Keywords: physics-informed neural networks

Automated Machine Learning with DeepHyper

DeepHyper is a software package that uses artificial intelligence (AI) techniques and parallel computing to automate the design and development of machine learning (ML) models for scientific and engineering applications. DeepHyper reduces the barrier to entry for using AI/ML models by reducing manually intensive trial-and-error efforts for developing predictive models.

Contact name: Prasanna Balaprakash

Contact email: pbalapra@anl.gov

Theme: Scalable Learning - AutoML

Keywords: Hyperparameter tuning, neural architecture search

MFA: Multivariate Functional Approximation

Scientific data may be transformed by recasting to a data model fundamentally different from the discrete pointwise or element-wise datasets produced by computational models. In Multivariate Functional Approximation, or MFA, scientific datasets are redefined in a hypervolume of piecewise-continuous basis functions. Compared with existing discrete models, the continuous functional model can save space while affording many spatiotemporal analyses without reverting back to the discrete form. The MFA model can represent numerous types of data because it is agnostic to the mesh, field, or discretization of the input dataset. In the spectrum of AI/ML approaches, the MFA can be considered as one type of surrogate model. Once trained, it provides regression and implicit evaluation anywhere in the domain---like an implicit neural network but less expensive to train---not limited to the input data points. Besides modeling function values, the same MFA model can be queried for analytical exact closed-form evaluation of points, derivatives, and integrals to high order, anywhere inside the domain. Applications of the MFA include high-order visualization, derivative fields, comparison of datasets with different discretizations, remapping between coupled multiphysics simulations, smoothing, simplification, filtering, and super-resolution.

Contact name: Tom Peterka

Contact email: tpeterka@mcs.anl.gov

Theme: Surrogate Modeling, Representation - Functional Approximation

Keywords: Data reduction

Robust and Efficient Probabilistic Machine Learning for Scientific Data

Modern deep neural networks are in general quite brittle, and hence less robust to noise in the input or adversarial perturbation applied to them. From an information-theoretic point of view, the ability of the model to achieve better generalization and robustness will depend on whether the model learns more semantically meaningful information and compresses the nuisance information. Typically, the DNNs are trained by reducing an empirical loss function which can lead to nuisance or irrelevant information being memorized, that can have a detrimental impact on the model’s robustness. We describe an information-theoretic Bayesian deep learning framework, that combined with structural sparsity inducing priors provides an efficient and robust approach to probabilistic modeling and uncertainty quantification for scientific data.

Contact name: Sandeep Madireddy

Contact email: smadireddy@anl.gov

Theme: Surrogate Modeling, Uncertainty Quantification

Keywords: Uncertainty quantification, robustness

Machine Learning with Structural Priors for Surrogate Modeling of Dynamical Systems

We describe recent developments in scientific machine learning that utilize a combination of data-driven and first-principles based models for dramatically accelerating the simulation of dynamical systems. Examples include the augmentation of neural architectures so that they discover manifolds in ambient space, conserve energy, or use closures during evolution.

Contact name: Romit Maulik

Contact email: rmaulik@anl.gov

Theme: Surrogate Modeling, Representation Learning

Keywords: physics-informed neural networks, dynamical systems

Machine Learning-based Inversion of Nuclear Responses

One of the most important problems in nuclear physics is to characterize the interaction of atomic nuclei at a microscopic level. Nuclear quantum Monte-Carlo that are commonly used to infer these interactions do so by inverting the Laplace transform of the original response function (the function that describes the interaction of an atom as a function of energy transfer). However, inverting the Laplace transform is a notoriously ill-posed problem; and Bayesian techniques, such as maximum entropy do not capture the true interaction very well in the low energy region. In this work, we present a physics-informed artificial neural network architecture suitable for approximating the inverse of the Laplace transform. Utilizing simulated, albeit realistic, electromagnetic response functions, we show that this physics-informed artificial neural network outperforms maximum entropy in both the low-energy regions, thereby allowing for robust characterization of the nuclear interactions.

Contact name: Krishnan Raghavan

Contact email: kraghavan@anl.gov

Theme: Surrogate Modeling, Inverse or Data Analytics

Keywords: physics-informed neural networks, inverse problems

Extreme-Scale Feature Tracking for Science

Feature tracking plays an essential role in understanding scientific data, such as finding rare events, deriving quantities and statistics, and reducing data to store. FTK (Feature Tracking Kit) is a framework that simplifies, scales, and delivers various feature-tracking algorithms for scientific data. FTK was initially motivated by the lack of general-purpose feature-tracking tools and difficulties in scaling feature-tracking algorithms. The critical innovation of FTK is the simplicial spacetime meshing scheme that generalizes spatial meshes to 4D spacetime, which simplifies the handling of corner cases and improves the scalability of feature tracking algorithms, enabling scalable and parallel processing. We also integrate an AI-based paradigm to extract and track features that cannot be defined mathematically, easing feature identification and analysis for improved data understanding.

Contact name: Hanqi Guo

Contact email: hguo@anl.gov

Theme: Data Analysis

Keywords: feature extraction, identification, tracking

InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations

InSituNet is a deep learning based surrogate model to support parameter space exploration for ensemble simulations that are visualized in situ. It allows flexible exploration of parameter space for large-scale ensemble simulations by taking advantage of the recent advances in deep learning. InSituNet is designed as a convolutional regression model to learn the mapping from the simulation and visualization parameters to the visualization results. With the trained model, users can generate new images for different simulation parameters under various visualization settings, which enables in-depth analysis of the underlying ensemble simulations.

Contact name: Han-Wei Shen

Contact email: shen.94@osu.edu

Theme: Surrogate Modeling

Keywords: parameter exploration, image synthesis, ensemble simulation

Autonomic Data Management for Extreme Scale Staging Based In-situ Workflows

Extreme scale scientific workflows are composed of multiple applications that exchange data at runtime. Several data-related challenges are limiting the potential impact of such workflows. While data staging and in-situ models of execution have emerged as approaches to address data-related costs at extreme scales, increasing data volumes and complex data exchange patterns impact the effectiveness of such approaches. We explore machine learning based approaches to capture the data access patterns between components of staging-based in-situ application workflows, and to use these learned access patterns to autonomous data management between the storage layers of the staging service. In addition, we also explore how ML techniques can be leveraged to enable dynamic data delivery and I/O offloading in such staging based scientific workflows.

Contact Name: Pradeep Subedi

Contact Email: pradeep.subedi@utah.edu

Theme: Scalable Learning - Data management

Keywords: data management, data movement and I/O optimization, workflow optimization

Scalable, Distributed, and Asynchronous Training of GANs

A well-known problem in ML is to develop training procedures for GANs with stable convergence properties. In this work, we develop new algorithms for training GANs with theoretical convergence guarantees that are based on a stochastic extension of projective splitting. We implement asynchronous distributed versions that are robust to “straggler effects” on large-scale HPC systems and are also communication efficient.

Contact name: Patrick Johnstone

Contact email: pjohnston@bnl.gov

Theme: Scalable Learning - Optimization

Keywords: Generative networks, distributed training

Causal Analysis for Scientific Discovery

Identifying the root cause of scientific phenomena is a time consuming and laborious task. We have developed automated discovery of the causal relationships among sensors or factors in scientific simulation. We have developed a scalable unsupervised and supervised causal discovery framework and demonstrated on various scientific and industrial applications.

Contact Name: Shinjae Yoo

Contact email: sjyoo@bnl.gov

Theme: Explainable AI

Keywords: Explainable AI, causal discovery

Physics Guided Bayesian Neural Inference with xAI

We propose a physics-constrained deep learning model to fast produce accurate PDE solutions while preserving many of the advantages of classic numerical solvers. From a probabilistic modeling perspective, the learning scheme resembles the amortized variational methods in Bayesian inference. To better emulate the current numerical solvers, we further integrate XAI techniques to verify and exploit neighboring information to reinforce locality in space and time.

Contact Name: Xihaier Luo

Contact Email: xluo@bnl.gov

Theme: Surrogate, UQ

Keywords: Explainable AI, Physics-informed deep neural networks

Anomaly and Change Point Detection for Scientific Discovery

We have developed robust online change point and anomaly detection methods for scientific discovery. The most interesting events happen when the distribution is changed or abnormal / extreme events happen. With change point and anomaly detection, scientists can prioritize which data to look at.

Contact Name: Thomas Flynn

Contact Email: tflynn@bnl.gov

Theme: Representation Learning

Keywords: Anomaly detection, extreme event detection

Automatic Differentiation as a Tool for Computational Science

Automatic Differentiation (AutoDiff) is a powerful tool for the computation of derivatives of functions defined by computer programs. AutoDiff lies at the heart of frameworks for machine learning, modeling languages for mathematical optimization, and derivative-based uncertainty quantification techniques. In this webinar, we aim to introduce the basic concepts of automatic differentiation, describe implementation techniques, and discuss the effective use of AutoDiff tools and frameworks.

Contact Name: Paul Hovland / Jan Hückelheim

Contact E-mail: hovland@anl.gov

Theme: Scalable Learning - Platform

Keywords: Automatic Differentiation (AD)

Machine Learning based Volumetric Compression

Development of physics-Informed deep neural networks for volumetric data compression.

Contact Name: Sanjay Ranka

Contact Email: sranka@ufl.edu

Keywords: Physics-informed deep neural networks, data reduction/compression

Asynchronous I/O Strategy for Large-Scale Deep Learning Applications

We propose a buffering strategy to hide the I/O time during the training phrase of deep learning applications. For CosmoFlow and similar scientific applications, we are able to completely hide the I/O time and reduce the overall training time by nearly a half.

Contact name: Wei-keng Liao / John Wu

Contact email: wkliao@northwestern.edu

Theme: Scalable Learning - IO

Keywords: I/O optimization, large-scale training