Obálka

Real-time data processing for serial crystallography experiments

Uloženo v:
Vydáno v:IUCrJ vol. 12, no. 1 (Jan 2025), p. 97
Hlavní autor: White, Thomas
Další autoři: Schoof, Tim, Yakubov, Sergey, Tolstikova, Aleksandra, Middendorf, Philipp, Karnevskiy, Mikhail, Mariani, Valerio, Henkel, Alessandra, Klopprogge, Bjarne, Hannappel, Juergen, Oberthuer, Dominik, Ivan De Gennaro Aquino, Egorov, Dmitry, Munke, Anna, Sprenger, Janina, Pompidor, Guillaume, Taberman, Helena, Gruzinov, Andrey, Meyer, Jan, Hakanpää, Johanna, Gasthuber, Martin
Vydáno:
International Union of Crystallography
Témata:
Data processing
Crystallography
Computation
Pixels
Frames (data processing)
Real time
Experiments
Frames per second
On-line přístup:Citation/Abstract
Full Text - PDF
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstrakt:We report the use of streaming data interfaces to perform fully online data processing for serial crystallography experiments, without storing intermediate data on disk. The system produces Bragg reflection intensity measurements suitable for scaling and merging, with a latency of less than 1 s per frame. Our system uses the CrystFEL software in combination with the ASAP::O data framework. In a series of user experiments at PETRA III, frames from a 16 megapixel Dectris EIGER2 X detector were searched for peaks, indexed and integrated at the maximum full-frame readout speed of 133 frames per second. The computational resources required depend on various factors, most significantly the fraction of non-blank frames ('hits'). The average single-thread processing time per frame was 242 ms for blank frames and 455 ms for hits, meaning that a single 96-core computing node was sufficient to keep up with the data, with ample headroom for unexpected throughput reductions. Further significant improvements are expected, for example by binning pixel intensities together to reduce the pixel count. We discuss the implications of real-time data processing on the 'data deluge' problem from recent and future photon-science experiments, in particular on calibration requirements, computing access patterns and the need for the preservation of raw data.
ISSN:2052-2525
DOI:10.1107/S2052252524011837
Zdroj:Materials Science Database