CrystFEL - Data processing for serial crystallography

Overview

CrystFEL is a suite of programs for processing data from serial crystallography experiments, performed at synchrotron and X-ray free-electron laser facilities, as well as in your home lab using an electron microscope.

Getting started

See INSTALL.md for installation instructions, including our container registry, installation via package manager and details of pre-existing installations at X-ray facilities around the world.

CrystFEL can be used from the command line or via a graphical user interface. To start the graphical user interface, run crystfel.

There is a video tutorial, as well as a text tutorial to get you started with processing via the GUI.

For command-line use, standard man pages are available. Start with man crystfel. The manual pages are also available on the web.

Documentation

Basic Tutorial
Which indexing method(s) should I use?
How to choose the right point group for merging
How to increase data processing speed
Real-time data processing
Processing electron diffraction data
Symmetry classification for serial crystallography
Matrix conventions used in CrystFEL code - for developers, written mostly for my own benefit.
Hit rate graph
Examples folder - contains some template input files.
Contributing to CrystFEL - including how to cite CrystFEL and how to find good first issues to work on.
Citation list - please send us details of your paper, if it's missing!
Scripts folder - a miscellany of smaller programs to help at various stages of data processing.

Journal articles and book chapters

Processing serial crystallography data with CrystFEL: a step-by-step guide - covers command-line processing only (pre-dates the GUI).
Recent developments in CrystFEL - now somewhat out of date, but contains some useful information about the algorithms used.
Crystallography and Molecular Imaging using X-ray Lasers - an introduction to the biological aspects and possibilities, written for physicists (in contrast to most other articles, which introduce the physical aspects for biologists!).
Original paper about CrystFEL from 2012. Not open access, but a "reprint" is available here.
Climbing the Data Mountain: Processing of SFX Data - emphasizes data volume issues for XFELs. Unfortunately not open access.
Processing of XFEL Data - describes the entire processing pipeline. Unfortunately not open access.

Awards

In 2017, the development of CrystFEL was recognised with the Max von Laue Prize from the German Society for Crystallography (DGK).

Citing CrystFEL

Please see CONTRIBUTING.md for citation instructions.

OnDA Monitor for real-time monitoring of data quality. Read the paper.

Pixel Anomaly Detection Tool uses machine learning techniques to find misbehaving detector pixels. Read the paper.

Careless scales and merges data using variational inference. Use its stream2mtz script to import data from CrystFEL. Read the paper.

DatView helps with multivariate analysis of large datasets. Read the paper. More information here including a tutorial and manual.

Funding acknowledgements

Development of CrystFEL is primarily funded by the Helmholtz Association via DESY.

Partial funding for CrystFEL has been provided by:

The consortium DAPHNE4NFDI in the context of the work of the NFDI e.V. The consortium is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - project number 460248799 (2022-).
European Union’s Horizon 2020 research and innovation programme under grant agreement No 857641 (ExPaNDS) (2019-2023).
X-Probe, a project of the European Union's 2020 Research and Innovation Program Under the Marie Skłodowska-Curie grant agreement 637295 (2015-2018).
The BMBF German-Russian Cooperation SyncFELMed, grant 05K14CHA (2014-2017).
BioStruct-X, a project funded by the Seventh Framework Programme (FP7) of the European Commission (2011-2016).

Licence

See AUTHORS as well as individual source code files for full details of contributors.

CrystFEL is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CrystFEL. If not, see http://www.gnu.org/licenses/.