Bjoern Enders

Integrating a Supercomputing Facility into the Electron Microscopy Pipeline for On-the-fly 4D-STEM Reconstructions

Stephanie M Ribet, Sam Welborn, Bjoern Enders, Chris Harris et autres

The field of electron microscopy has been revolutionized by the integration of computational methods into the experimental pipeline [1]. Data-driven approaches have allowed for advanced reconstructions of high-dimensional datasets from complex experimental pipelines. While offline data processing is required for computationally intensive, …

us, nl (code pays fourni par la source)

Integrating Microscopy, Large Language Models, and High Performance Computing

Peter Ercius, Sam Welborn, Morgan K Wall, Alexander J Pattison et autres

Advanced computation is becoming an increasingly important part of microscopy [1]. It allows for automation, high throughput operation, and the implementation of new types of imaging modalities. One downside of this new direction is the large increase in data volumes being acquired …

us (code pays fourni par la source)

Interactive and Urgent HPC: State of the Research

Albert Reuther, William D. Arndt, Johannes Blaschke, Christian Boehme et autres

When we think of how we use smartphones, e-commerce, collaboration platforms, LLMs, etc., most of our interactions with computers are interactive and often urgent. Similar trends of interactivity and urgency are coming to HPC, with applications from simulations to data analysis and …

Interactive and Urgent HPC: State of the Research

Albert Reuther, William D. Arndt, Johannes Blaschke, Christian Boehme et autres

When we think of how we use smartphones, e-commerce, collaboration platforms, LLMs, etc., most of our interactions with computers are interactive and often urgent. Similar trends of interactivity and urgency are coming to HPC, with applications from simulations to data analysis and …

us (code pays fourni par la source)

Accelerating Advanced Light Source Science Through Multi-Facility HPC Workflows

David Abramov, Samuel S. Welborn, Ryan Chard, Kuldeep Chawla et autres

Synchrotron light sources support a wide array of techniques to investigate materials, often producing complex, high-volume data that challenge traditional workflows. At the Advanced Light Source (ALS), we developed infrastructure to move microtomography data over ESnet to ALCF and NERSC, where CPU- …

us (code pays fourni par la source)

Accelerating Time-to-Science by Streaming Detector Data Directly into Perlmutter Compute Nodes

Samuel S. Welborn, Chris Harris, Peter A. Ercius, Deborah Bard et autres

us (code pays fourni par la source)

Understanding Data Movement Patterns in HPC: A NERSC Case Study

Anna Giannakou, Damian Hazen, Bjoern Enders, Lavanya Ramakrishnan et autres

Scientific experiments are producing unprecedented volumes of data with real-time High Performance Computing (HPC) needs. Understanding and ensuring efficient data movement in these emerging data-intensive workloads is becoming critical for successful workflow execution. The need for end-to-end that integrates compute, network, and …

us (code pays fourni par la source)

Streaming Large-Scale Microscopy Data to a Supercomputing Facility

Samuel S. Welborn, Chris Harris, Stephanie M. Ribet, Georgios Varnavides et autres

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy generates …

us (code pays fourni par la source)

Streaming Large-Scale Electron Microscopy Data to a Supercomputing Facility

Samuel S. Welborn, Chris Harris, Stephanie M. Ribet, Georgios Varnavides et autres

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy (NCEM) …

Accelerating Time-to-Science by Streaming Detector Data Directly into Perlmutter Compute Nodes

Samuel S. Welborn, Bjoern Enders, Chris Harris, Peter A. Ercius et autres

Recent advancements in detector technology have significantly increased the size and complexity of experimental data, and high-performance computing (HPC) provides a path towards more efficient and timely data processing. However, movement of large data sets from acquisition systems to HPC centers introduces …

Real‐time XFEL data analysis at SLAC and NERSC: A trial run of nascent exascale experimental data analysis

Johannes Blaschke, Aaron S. Brewster, Daniel W. Paley⧓, Derek A. Mendez et autres

X-ray scattering experiments using Free Electron Lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID-19 viral proteins). XFEL experiments are a challenge to computing in two ways: i) due to the high …

us (code pays fourni par la source)

Interactive and Urgent HPC: Challenges and Opportunities

Albert Reuther, Nick Brown, William D. Arndt, Johannes Blaschke et autres

As a broader set of applications from simulations to data analysis and machine learning require more parallel computational capability, the demand for interactive and urgent high performance computing (HPC) continues to increase. This paper overviews the progress made so far and elucidates …