diff options
| -rw-r--r-- | ChangeLog | 7 | ||||
| -rw-r--r-- | Makefile.am | 2 | ||||
| -rw-r--r-- | announcement-0.6.1 | 43 | ||||
| -rw-r--r-- | configure.ac | 2 | ||||
| -rw-r--r-- | doc/reference/libcrystfel/CrystFEL-docs.sgml | 2 | ||||
| -rw-r--r-- | libcrystfel/Makefile.am | 2 | ||||
| -rw-r--r-- | relnotes-0.6.0 | 257 | ||||
| -rw-r--r-- | relnotes-0.6.1 | 165 |
8 files changed, 217 insertions, 263 deletions
@@ -1,5 +1,5 @@ -Changes in this development version of CrystFEL ------------------------------------------------ +CrystFEL version 0.6.1, 3rd August 2015 +--------------------------------------- - partialator: big improvements, now works well on most data sets. - indexamajig: prediction refinement added. @@ -9,11 +9,14 @@ Changes in this development version of CrystFEL - process_hkl --even-only and --odd-only were added. - External bad pixel masks can now be used (see "mask_file" in geometry file). - indexamajig: Can now use --int-diag with -j n where n>1 +- Image data filters (e.g. --noise-filter) were fixed. +- indexamajig sandbox was redesigned and made more robust. - compare_hkl now uses a B-factor for scaling the datasets. - pattern_sim: spectrum normalisation added. - scripts/ave-resolution and scripts/create-frame-number added. - scripts/plot-predict-refine and scripts/plot-radius-resolution added. - scripts/detector-shift added. +- scripts/turbo-index added. - "reax" and "grainspotter" indexing methods removed. - Many other small but important bug fixes and improvements to data quality. diff --git a/Makefile.am b/Makefile.am index a6acd357..32cc8a63 100644 --- a/Makefile.am +++ b/Makefile.am @@ -29,7 +29,7 @@ TESTS = tests/list_check $(MERGE_CHECKS) $(PARTIAL_CHECKS) \ tests/prediction_gradient_check EXTRA_DIST += $(MERGE_CHECKS) $(PARTIAL_CHECKS) -EXTRA_DIST += relnotes-0.6.0 +EXTRA_DIST += relnotes-0.6.1 announcement-0.6.1 if BUILD_HDFSEE bin_PROGRAMS += src/hdfsee diff --git a/announcement-0.6.1 b/announcement-0.6.1 new file mode 100644 index 00000000..85137402 --- /dev/null +++ b/announcement-0.6.1 @@ -0,0 +1,43 @@ +Dear CrystFEL users and interested crystallographers, + +CrystFEL version 0.6.1 has been released. The most interesting changes +in this version are the new "prediction refinement" stage in +"indexamajig" and improvements to the scaling and post-refinement +program "partialator". There are also a couple of new indexing methods. + +The prediction refinement stage improves the quality of spot prediction +and hence the initial parameter estimates of the spot partialities. It +is also able to update the central beam position. + +The improvements to partialator include a significantly more stable +scaling procedure. Partialator is no longer considered experimental as +of this version, so please try it and send feedback. With the +combination of this and the new prediction refinement, many users are +already finding significant improvements in data quality. See the +release notes (link below) and the updated tutorial on the CrystFEL +website for more details. + +The new indexing methods are "asdf", which is an algorithm built into +CrystFEL which therefore requires no external programs, and +"mosflm-cell" which uses the prior unit cell algorithm in recent (7.2.0 +or later) versions of mosflm. "mosflm-cell" is now the default +behaviour for mosflm - if it doesn't work for you, explicitly specify +"mosflm-nocell" to restore the old behaviour. + +In addition, there are many smaller improvements, new features and bug +fixes. + +Thanks for all of your past and future feedback and contributions, and +your continued use of CrystFEL. As ever, please contact me directly if +you would prefer to be unsubscribed from this mailing list. + +CrystFEL website: +http://www.desy.de/~twhite/crystfel + +Release notes for version 0.6.1: +http://www.desy.de/~twhite/crystfel/relnotes-0.6.1 + +CrystFEL tutorial: +http://www.desy.de/~twhite/crystfel/tutorial + +Tom diff --git a/configure.ac b/configure.ac index 0971acb6..9d4d7edf 100644 --- a/configure.ac +++ b/configure.ac @@ -1,6 +1,6 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT([crystfel],[0.6.0],[taw@physics.org]) +AC_INIT([crystfel],[0.6.1],[taw@physics.org]) AC_CONFIG_AUX_DIR([build-aux]) AC_CONFIG_HEADERS([config.h]) AM_INIT_AUTOMAKE([subdir-objects]) diff --git a/doc/reference/libcrystfel/CrystFEL-docs.sgml b/doc/reference/libcrystfel/CrystFEL-docs.sgml index 6c7baa74..9ef5d945 100644 --- a/doc/reference/libcrystfel/CrystFEL-docs.sgml +++ b/doc/reference/libcrystfel/CrystFEL-docs.sgml @@ -8,7 +8,7 @@ <bookinfo> <title>CrystFEL Reference Manual</title> <releaseinfo> - For libcrystfel from CrystFEL 0.6.0. + For libcrystfel from CrystFEL 0.6.1. </releaseinfo> <abstract> This is the internal documentation for CrystFEL. Unless you are looking at diff --git a/libcrystfel/Makefile.am b/libcrystfel/Makefile.am index 1a0c3efe..77e5991b 100644 --- a/libcrystfel/Makefile.am +++ b/libcrystfel/Makefile.am @@ -1,6 +1,6 @@ lib_LTLIBRARIES = libcrystfel.la libcrystfel_la_LIBADD = ../lib/libgnu.la @LIBCRYSTFEL_LIBS@ $(PTY_LIB) -libcrystfel_la_LDFLAGS = -version-info 6:0:0 +libcrystfel_la_LDFLAGS = -version-info 7:0:0 libcrystfel_la_SOURCES = src/reflist.c src/utils.c src/cell.c src/detector.c \ src/thread-pool.c src/image.c src/hdf5-file.c \ diff --git a/relnotes-0.6.0 b/relnotes-0.6.0 deleted file mode 100644 index 22efa56d..00000000 --- a/relnotes-0.6.0 +++ /dev/null @@ -1,257 +0,0 @@ -CrystFEL - Crystallography with a FEL -------------------------------------- - -Release notes for version 0.6.0 - -Copyright © 2012-2015 Deutsches Elektronen-Synchrotron DESY, - a research centre of the Helmholtz Association. - -Authors: - Thomas White <taw@physics.org> - Richard Kirian <rkirian@asu.edu> - Kenneth Beyerlein <kenneth.beyerlein@desy.de> - Andrew Aquila <andrew.aquila@cfel.de> - Andrew Martin <andrew.martin@desy.de> - Lorenzo Galli <lorenzo.galli@desy.de> - Chun Hong Yoon <chun.hong.yoon@desy.de> - Kenneth Beyerlein <kenneth.beyerlein@desy.de> - Karol Nass <karol.nass@desy.de> - Nadia Zatsepin <nadia.zatsepin@asu.edu> - Anton Barty <anton.barty@desy.de> - Cornelius Gati <cornelius.gati@desy.de> - Fedor Chervinskii <fedor.chervinskii@gmail.com> - Alexandra Tolstikova <alexandra.tolstikova@desy.de> - Wolfgang Brehm <wolfgang.brehm@gmail.com> - Valerio Mariani <valerio.mariani@desy.de> - Parker de Waal <Parker.deWaal@vai.org> - Takanori Nakane <nakane.t@gmail.com> - Keitaro Yamashita <k.yamashita@spring8.or.jp> - Oleksandr Yefanov <oleksandr.yefanov@cfel.de> - -CrystFEL is free software: you can redistribute it and/or modify it under the -terms of the GNU General Public License as published by the Free Software -Foundation, either version 3 of the License, or (at your option) any later -version. - -CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY -WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A -PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -CrystFEL. If not, see <http://www.gnu.org/licenses/>. - - -Overview --------- - -The most important new features in this version of CrystFEL are: - -- Support for multi-event HDF5 files (e.g. CXI format as used by CXIDB) - -- geoptimiser: a new tool for precisely optimising the detector geometry - -- Removal of beam files / automatic determination of spot prediction parameters - -- whirligig: a new tool for finding clusters of similarly-oriented - crystal snapshots, e.g. for finding "mini rotation series" in data - from slow extrusion sample delivery methods. - -- Introduction of "CrystFEL unit cell files". - -These new features have individual sections below. In addition, there are many -other new developments. See the ChangeLog or the changes page on the website -for more details. There were, of course, the usual large number of smaller -refinements and bug fixes. - - -Support for multi-event HDF5 files ----------------------------------- - -CrystFEL's handling of HDF5 files has been made much more flexible. Most -importantly, it now offers the ability to handle HDF5 files which contain more -than one event (frame). Until now, CrystFEL has required that each event be -contained in its own file, which can place a lot of unnecessary strain on -filesystems. Recent versions of Cheetah allow you to create larger files which -contain many events each using the CXI format (see http://www.cxidb.org for more -details). CrystFEL now supports this as well as almost any reasonable -multi-event HDF5 layout. - -If you choose to continue using many small files, you should not notice much -difference. However, some small updates will be required. Firstly, -indexamajig's "-e" and "--image" command-line parameters have been removed. -Instead, you need to edit your geometry file and add a line such as this near -the top: - - data = /data/rawdata - -Geometry files now contain all the information needed to interpret the contents -of the HDF5 files as a physical setup, including the photon energy. Beam -parameter files have been removed. See below for more information about this. - -When using multi-event files in CXI format, the peak lists are read differently. -Use --peaks=cxi to retrieve peak lists from CXI files. - -hdfsee has been extended to support multi-event files, including navigation -between events. You should always provide a geometry file on the command line, -otherwise it won't know how to interpret the contents of your HDF5 file. - -A new tool in CrystFEL 0.6.0, list_events, is provided to simplify the process -of creating lists of individual events. However, most users will simply be -able to list the multi-event filenames themselves in the input to indexamajig, -which will then process all of the events in the file. See "man indexamajig" -and "man list_events" for more details. - -Finally, it may be necessary to update your check-near-bragg and -check-peak-detection scripts. Simply make a fresh copy from the CrystFEL -"scripts" folder or download them from the website. - - -geoptimiser: a tool for optimising the detector geometry --------------------------------------------------------- - -CrystFEL 0.6.0 introduces a new tool, geoptimiser, for optimising the detector -geometry. You simply give it a stream containing indexing results obtained with -the approximate geometry, and it gives you a refined geometry. It can refine -panel translations, rotations and camera lengths. - -To use geoptimiser, you need to add some extra information about the -mechanical construction of your detector. This is used to constrain the -refinement appropriately, for example to ensure that panels which share sensor -silicon do not move relative to one another. If you're processing CSPAD data, -you can simply copy all the rigid_group lines from one of the CSPAD geometry -file examples (folder doc/examples) into your own geometry file. - -Refer to "man geoptimiser" and "man crystfel_geometry" for more information. - - -Removal of beam files / automatic determination of spot prediction parameters ------------------------------------------------------------------------------ - -As of version 0.6.0, "beam files" have been removed completely. This should -simplify usage for most people and remove a lot of ambiguity. Programs which -need X-ray beam parameters, such as pattern_sim, now have additional command- --line arguments to provide them. indexamajig now determines these parameters -automatically for the purposes of spot prediction. - -We are interested in feedback about the automatic spot prediction parameter -determination. If it appears not to work well for you, you can restore the old -behaviour by using the new command-line options for indexamajig: ---fix-profile-radius, --fix-bandwidth and --fix-divergence. Simply set these -parameters to the values you had in your old beam file. - -The photon energy, or information about where to get it, now needs to be in the -geometry file. This can be done with a line like this: - - photon_energy = /LCLS/photon_energy_eV - -or, for a fixed value: - - photon_energy = 8300 - - -whirligig: a tool for finding "mini rotation series" ----------------------------------------------------- - -CrystFEL 0.6.0 introduces yet another tool, whirligig, which can be used for -locating "mini rotation series" in the output from indexamajig. This might be -used to perform an experiment similar to that described by Gati et al., IUCrJ -1 (2014) p87. In this initial version, whirligig finds runs of consecutive -frames which contain crystals in similar orientations. It writes the -corresponding filenames and event/crystal identifiers to log files, which might -be useful for further analysis. - -This is the program described by Nogly et al., IUCrJ 2 (2015). Refer to "man -whirligig" for usage information. - - -Introduction of CrystFEL unit cell files ----------------------------------------- - -CrystFEL offers you the ability to index patterns using Bravais lattice -information but without unit cell parameters, for example: "index these using -only tetragonal primitive lattices, but any parameters". However, it's awkward -to give this information using a PDB file: you essentially have to "trick" it -into interpreting the parameters correctly, then throw away the parameter -information. - -Version 0.6.0 of CrystFEL introduces a new way of specifying unit cell -information. These new unit cell files look like this: - - CrystFEL unit cell file version 1.0 - - lattice_type = cubic - centering = I - - a = 66.2 A - b = 66.2 A - c = 66.2 A - - al = 90.0 deg - be = 90.0 deg - ga = 90.0 deg - -In the event that you want to specify the lattice type information alone, you -can simply omit the cell parameters: - - CrystFEL unit cell file version 1.0 - - lattice_type = tetragonal - centering = P - unique_axis = c - -Note that a unique axis must be specified for all types of cell where this makes -sense, and can be omitted otherwise. This was done in the first example above, -which is cubic and therefore has no unique axis. - -You can, of course, continue to use PDB files just like before. - - -API changes ------------ - -The following changes have been made to the libcrystfel API: - -New functions: - - The event API (see libcrystfel/src/events.h) - - load_cell_from_file() - - cell_has_parameters() - - find_orig_panel() - - crystal_{get,set}_num_implausible_reflections() were added - - panel_is_in_rigid_group() - - rigid_group_is_in_collection() - - single_panel_data_source() - - find_rigid_group_collection_by_name() - - write_detector_geometry_2() - - find_intersections_to_res() - - sphere_fraction() - - gaussian_fraction() - - hdf5_read2() - - check_path_existence() - - get_peaks_cxi() - - hdfile_get_value() - - fill_event_list() - - image_reflection_closest() - - intmat_identity() - - extract_f_from_stuff() - -Removed functions: - - cell_set_cartesian_{a,b,c}() - - cell_{get,set}_pointgroup() - - twod_mapping() - - get_value() - -Changed function prototypes: - - record_image() - - get_detector_geometry() - - fill_in_values() - - write_detector_geometry() - - hdf5_write_image() - - hdf5_read() - - hdfile_set_image() - - hdfile_get_string_value() - - {get,set}_partial() - - write_chunk() - - prepare_indexing() and {dirax,mosflm,reax,grainspotter,xds}_prepare() - -beam-parameters.h was removed, and "struct beam_params" is now defined in -image.h. diff --git a/relnotes-0.6.1 b/relnotes-0.6.1 new file mode 100644 index 00000000..4c189cd6 --- /dev/null +++ b/relnotes-0.6.1 @@ -0,0 +1,165 @@ +CrystFEL - Crystallography with a FEL +------------------------------------- + +Release notes for version 0.6.1 + +Copyright © 2012-2015 Deutsches Elektronen-Synchrotron DESY, + a research centre of the Helmholtz Association. + +Authors: + Thomas White <taw@physics.org> + Richard Kirian <rkirian@asu.edu> + Kenneth Beyerlein <kenneth.beyerlein@desy.de> + Andrew Aquila <andrew.aquila@cfel.de> + Andrew Martin <andrew.martin@desy.de> + Lorenzo Galli <lorenzo.galli@desy.de> + Chun Hong Yoon <chun.hong.yoon@desy.de> + Kenneth Beyerlein <kenneth.beyerlein@desy.de> + Karol Nass <karol.nass@desy.de> + Nadia Zatsepin <nadia.zatsepin@asu.edu> + Anton Barty <anton.barty@desy.de> + Cornelius Gati <cornelius.gati@desy.de> + Fedor Chervinskii <fedor.chervinskii@gmail.com> + Alexandra Tolstikova <alexandra.tolstikova@desy.de> + Wolfgang Brehm <wolfgang.brehm@gmail.com> + Valerio Mariani <valerio.mariani@desy.de> + Parker de Waal <Parker.deWaal@vai.org> + Takanori Nakane <nakane.t@gmail.com> + Keitaro Yamashita <k.yamashita@spring8.or.jp> + Oleksandr Yefanov <oleksandr.yefanov@cfel.de> + +CrystFEL is free software: you can redistribute it and/or modify it under the +terms of the GNU General Public License as published by the Free Software +Foundation, either version 3 of the License, or (at your option) any later +version. + +CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A +PARTICULAR PURPOSE. See the GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along with +CrystFEL. If not, see <http://www.gnu.org/licenses/>. + + +Overview +-------- + +The most important new features in this version of CrystFEL are: + +- Introduction of "prediction refinement". + +- Scaling and post-refinement with "partialator". + +- New indexing methods: mosflm-cell and asdf. + +- process_hkl --even-only and --odd-only instead of "alternate-stream". + +These new features have individual sections below. In addition, there are many +other new developments. See the ChangeLog or the changes page on the website +for more details. There were, of course, the usual large number of smaller +refinements and bug fixes. + + +Introduction of "prediction refinement" +--------------------------------------- + +A new refinement step has been introduced between the indexing and prediction/ +integration stages of indexamajig. The refinement aims to maximise the +agreement between the predicted spot positions and the peak positions found by +the peak search step, at the same time as putting the peaks as close to the +Bragg condition as possible. This increases the accuracy of spot prediction, +which has positive effects on the downstream processing stages. + +The prediction refinement stage allows the central beam position to move +relative to the entire detector. The required offset for each crystal is +recorded in the stream and can be visualised using scripts/detector-shift. + +If, somehow, the prediction refinement causes problems for you, you can disable +it using the indexamajig option "--no-refine". + + +Scaling and post-refinement with "partialator" +---------------------------------------------- + +The scaling and post-refinement program partialator is no longer considered +experimental in this version. Many users are already finding significant +improvements in their data simply by applying scaling without partiality: + +$ partialator -i my.stream -o my.hkl -y mypg --model=unity --iterations=3 + +A further improvement is sometimes obtained by applying partialities and doing +full post-refinement: + +$ partialator -i my.stream -o my.hkl -y mypg --model=scsphere --iterations=3 + +Note that partialator applies a per-crystal resolution cutoff to stabilise the +scaling calculation. Therefore, you should experiment with different values +for "--push-res". + +Further improvements to partiality modelling, scaling and merging will be +included in the near future. + + +New indexing methods +-------------------- + +This version of CrystFEL can make use of the known unit cell algorithm included +with very recent (7.2.0 or later) versions of mosflm. The known unit cell +algorithm is used by default if you provide lattice parameters to indexamajig. +If this isn't what you want, or if it doesn't work well for you, use +"mosflm-nocell" as your indexing method to restore the old behaviour. + +A new indexing method, asdf, has been added. This algorithm is built into +CrystFEL directly, so no temporary files or external programs are needed. + +The old indexing methods "grainspotter" and "reax" have been removed. + + +process_hkl --even-only and --odd-only +-------------------------------------- + +When using process_hkl for merging, you no longer need to use "alternate-stream" +to split the stream into two parts in order to calculate figures of merit such +as Rsplit. Instead, simply use the "--even-only" and "--odd-only" options to +merge only even-numbered and odd-numbered crystals from the stream: + +$ process_hkl -i my.stream -o my.hkl1 -y mypg --even-only +$ process_hkl -i my.stream -o my.hkl2 -y mypg --odd-only +$ compare_hkl my.hkl1 my.hkl2 -y mypg --fom=rsplit [..etc..] + +Note that the numbering of crystals for this purpose is simply according to the +order they appear in the stream. For most purposes, this is effectively random. + +Note further that partialator writes the ".hkl1" and ".hkl2" split-merged +datasets for you automatically. + + +API changes +----------- + +The following changes have been made to the libcrystfel API: + +New functions: + - asdf_prepare(), run_asdf(), asdf_cleanup() + - cell_get_volume() + - crystal_{get,set}_Bfac() + - crystal_{get,set,add}_notes() + - detector_has_clen_references() + - {x,y,r}_gradient() + - refine_prediction() + - refine_radius() + - {get,set}_panel() + - {get,set}_flag() + - solve_svd() + +Removed functions: + - select_intersections() + - update_partialities_2() + - get_excitation_error() + - extract_f_from_stuff() + - grainspotter_{prepare,index,cleanup}() + - reax_{prepare,index,cleanup}() + +Changed function prototypes: + - integrate_all_4() + - set_detector_pos() |