-
A Methodology for Explainable Large Language Models with Integrated Gradients and Linguistic Analysis in Text Classification
Authors:
Marina Ribeiro,
Bárbara Malcorra,
Natália B. Mota,
Rodrigo Wilkens,
Aline Villavicencio,
Lilian C. Hubner,
César Rennó-Costa
Abstract:
Neurological disorders that affect speech production, such as Alzheimer's Disease (AD), significantly impact the lives of both patients and caregivers, whether through social, psycho-emotional effects or other aspects not yet fully understood. Recent advancements in Large Language Model (LLM) architectures have developed many tools to identify representative features of neurological disorders thro…
▽ More
Neurological disorders that affect speech production, such as Alzheimer's Disease (AD), significantly impact the lives of both patients and caregivers, whether through social, psycho-emotional effects or other aspects not yet fully understood. Recent advancements in Large Language Model (LLM) architectures have developed many tools to identify representative features of neurological disorders through spontaneous speech. However, LLMs typically lack interpretability, meaning they do not provide clear and specific reasons for their decisions. Therefore, there is a need for methods capable of identifying the representative features of neurological disorders in speech and explaining clearly why these features are relevant. This paper presents an explainable LLM method, named SLIME (Statistical and Linguistic Insights for Model Explanation), capable of identifying lexical components representative of AD and indicating which components are most important for the LLM's decision. In developing this method, we used an English-language dataset consisting of transcriptions from the Cookie Theft picture description task. The LLM Bidirectional Encoder Representations from Transformers (BERT) classified the textual descriptions as either AD or control groups. To identify representative lexical features and determine which are most relevant to the model's decision, we used a pipeline involving Integrated Gradients (IG), Linguistic Inquiry and Word Count (LIWC), and statistical analysis. Our method demonstrates that BERT leverages lexical components that reflect a reduction in social references in AD and identifies which further improve the LLM's accuracy. Thus, we provide an explainability tool that enhances confidence in applying LLMs to neurological clinical contexts, particularly in the study of neurodegeneration.
△ Less
Submitted 30 September, 2024;
originally announced October 2024.
-
KaMPIng: Flexible and (Near) Zero-Overhead C++ Bindings for MPI
Authors:
Tim Niklas Uhl,
Matthias Schimek,
Lukas Hübner,
Demian Hespe,
Florian Kurpicz,
Christoph Stelz,
Peter Sanders
Abstract:
The Message-Passing Interface (MPI) and C++ form the backbone of high-performance computing, but MPI only provides C and Fortran bindings. While this offers great language interoperability, high-level programming languages like C++ make software development quicker and less error-prone.
We propose novel C++ language bindings that cover all abstraction levels from low-level MPI calls to convenien…
▽ More
The Message-Passing Interface (MPI) and C++ form the backbone of high-performance computing, but MPI only provides C and Fortran bindings. While this offers great language interoperability, high-level programming languages like C++ make software development quicker and less error-prone.
We propose novel C++ language bindings that cover all abstraction levels from low-level MPI calls to convenient STL-style bindings, where most parameters are inferred from a small subset of parameters, by bringing named parameters to C++. This enables rapid prototyping and fine-tuning runtime behavior and memory management. A flexible type system and additional safety guarantees help to prevent programming errors.
By exploiting C++'s template metaprogramming capabilities, this has (near) zero overhead, as only required code paths are generated at compile time.
We demonstrate that our library is a strong foundation for a future distributed standard library using multiple application benchmarks, ranging from text-book sorting algorithms to phylogenetic interference.
△ Less
Submitted 24 September, 2024; v1 submitted 8 April, 2024;
originally announced April 2024.
-
ReStore: In-Memory REplicated STORagE for Rapid Recovery in Fault-Tolerant Algorithms
Authors:
Lukas Hübner,
Demian Hespe,
Peter Sanders,
Alexandros Stamatakis
Abstract:
Fault-tolerant distributed applications require mechanisms to recover data lost via a process failure. On modern cluster systems it is typically impractical to request replacement resources after such a failure. Therefore, applications have to continue working with the remaining resources. This requires redistributing the workload and that the non-failed processes reload data. We present an algori…
▽ More
Fault-tolerant distributed applications require mechanisms to recover data lost via a process failure. On modern cluster systems it is typically impractical to request replacement resources after such a failure. Therefore, applications have to continue working with the remaining resources. This requires redistributing the workload and that the non-failed processes reload data. We present an algorithmic framework and its C++ library implementation ReStore for MPI programs that enables recovery of data after process failures. By storing all required data in memory via an appropriate data distribution and replication, recovery is substantially faster than with standard checkpointing schemes that rely on a parallel file system. As the application developer can specify which data to load, we also support shrinking recovery instead of recovery using spare compute nodes. We evaluate ReStore in both controlled, isolated environments and real applications. Our experiments show loading times of lost input data in the range of milliseconds on up to 24 576 processors and a substantial speedup of the recovery time for the fault-tolerant version of a widely used bioinformatics application.
△ Less
Submitted 25 January, 2023; v1 submitted 2 March, 2022;
originally announced March 2022.
-
Developing a 50 MeV LPA-based Injector at ATHENA for a Compact Storage Ring
Authors:
E. Panofski,
C. Braun,
J. Dirkwinkel,
L. Hübner,
T. Hülsenbusch,
A. Maier,
P. Messner,
J. Osterhoff,
G. Palmer,
T. Parikh,
A. Walker,
P. Winkler,
T. Eichner,
L. Jeppe,
S. Jalas,
M. Kirchen,
M. Schnepp,
M. Trunk,
C. Werle,
E. Bründermann,
B. Härer,
A. -S. Müller,
C. Widmann,
M. C. Kaluza,
A. Sävert
Abstract:
The laser-driven generation of relativistic electron beams in plasma and their acceleration to high energies with GV/m-gradients has been successfully demonstrated. Now, it is time to focus on the application of laser-plasma accelerated (LPA) beams. The "Accelerator Technology HElmholtz iNfrAstructure" (ATHENA) of the Helmholtz Association fosters innovative particle accelerators and high-power la…
▽ More
The laser-driven generation of relativistic electron beams in plasma and their acceleration to high energies with GV/m-gradients has been successfully demonstrated. Now, it is time to focus on the application of laser-plasma accelerated (LPA) beams. The "Accelerator Technology HElmholtz iNfrAstructure" (ATHENA) of the Helmholtz Association fosters innovative particle accelerators and high-power laser technology. As part of the ATHENAe pillar several different applications driven by LPAs are to be developed, such as a compact FEL, medical imaging and the first realization of LPA-beam injection into a storage ring. The latter endeavour is conducted in close collaboration between Deutsches Elektronen-Synchrotron (DESY), Karlsruhe Institute of Technology (KIT) and Helmholtz Institute Jena (HIJ). In the cSTART project at KIT, a compact storage ring optimized for short bunches and suitable to accept LPA-based electron bunches is in preparation. In this conference contribution we will introduce the 50 MeV LPA-based injector and give an overview about the project goals. The key parameters of the plasma injector will be presented. Finally, the current status of the project will be summarized.
△ Less
Submitted 21 June, 2021;
originally announced June 2021.
-
LUX -- A Laser-Plasma Driven Undulator Beamline
Authors:
N. Delbos,
C. Werle,
I. Dornmair,
T. Eichner,
L. Hübner,
S. Jalas,
S. W. Jolly,
M. Kirchen,
V. Leroux,
P. Messner,
M. Schnepp,
M. Trunk,
P. A. Walker,
P. Winkler,
A. R. Maier
Abstract:
The LUX beamline is a novel type of laser-plasma accelerator. Building on the joint expertise of the University of Hamburg and DESY the beamline was carefully designed to combine state-of-the-art expertise in laser-plasma acceleration with the latest advances in accelerator technology and beam diagnostics. LUX introduces a paradigm change moving from single-shot demonstration experiments towards a…
▽ More
The LUX beamline is a novel type of laser-plasma accelerator. Building on the joint expertise of the University of Hamburg and DESY the beamline was carefully designed to combine state-of-the-art expertise in laser-plasma acceleration with the latest advances in accelerator technology and beam diagnostics. LUX introduces a paradigm change moving from single-shot demonstration experiments towards available, stable and controllable accelerator operation. Here, we discuss the general design concepts of LUX and present first critical milestones that have recently been achieved, including the generation of electron beams at the repetition rate of up to 5 Hz with energies above 600 MeV and the generation of spontaneous undulator radiation at a wavelength well below 9 nm.
△ Less
Submitted 23 January, 2018;
originally announced January 2018.
