aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorThomas White <taw@physics.org>2018-05-15 15:32:09 +0200
committerThomas White <taw@physics.org>2018-05-17 14:38:17 +0200
commit87fd650abf11da75e7abd2ff90821cf7cb041b91 (patch)
treee70b2890569dd860f4be487e494a50aac791c49e
parent4d9926e230ad3c550d69f78f6e4d95c555fe7639 (diff)
This is CrystFEL 0.7.00.7.0
-rw-r--r--ChangeLog15
-rw-r--r--Makefile.am2
-rw-r--r--announcement-0.6.335
-rw-r--r--announcement-0.7.037
-rw-r--r--configure.ac2
-rw-r--r--doc/reference/libcrystfel/CrystFEL-docs.sgml2
-rw-r--r--libcrystfel/Makefile.am2
-rw-r--r--relnotes-0.6.3211
-rw-r--r--relnotes-0.7.0176
9 files changed, 225 insertions, 257 deletions
diff --git a/ChangeLog b/ChangeLog
index 2625c6a9..4562a94b 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,21 +1,22 @@
-Changes in this development version
------------------------------------
+CrystFEL version 0.7.0, 17th May 2018
+--------------------------------------
-- indexamajig: Auto-detection of indexing methods was added.
+- indexamajig: Auto-detection of indexing methods
- indexamajig: New options system - removes confusing "comb","bad","retry" etc.
- Most programs now use the symmetry in the reflection files, allowing "-y"
+ options to be omitted.
- Ginn partiality model was added.
- partialator: Switch to gradient-free minimiser.
+- geoptimiser: Fixes for rotation and new features for AGIPD.
+- Speed and accuracy improvements for TakeTwo indexing (Helen Ginn).
- process_hkl, partialator and ambigator now include the audit information from
the stream in their output files.
- pattern_sim: Write orientation and lattice basis vectors into HDF5 file.
- hdfsee: Add event ID and peaks/preds/rings to exported image filenames.
- options to be omitted.
- indexamajig --serial-start was added (useful when using turbo-index).
-- indexamajig --overpredict was added (useful for partiality
- modelling/post-refinement)
+- indexamajig --overpredict was added (needed for post-refinement)
- pattern_sim --flat was added (new reflection profile option)
-- cell_explorer: Show number of matching cells on terminal.
+- cell_explorer: Now shows the number of matching cells on terminal.
- Felix options were simplified.
- scripts/sum-hdf5-files and eiger-badmap were added
diff --git a/Makefile.am b/Makefile.am
index f5d38c2a..a669b1dd 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -29,7 +29,7 @@ TESTS = tests/list_check $(MERGE_CHECKS) $(PARTIAL_CHECKS) \
tests/prediction_gradient_check tests/scaling_check
EXTRA_DIST += $(MERGE_CHECKS) $(PARTIAL_CHECKS)
-EXTRA_DIST += relnotes-0.6.3 announcement-0.6.3
+EXTRA_DIST += relnotes-0.7.0 announcement-0.7.0
if BUILD_HDFSEE
bin_PROGRAMS += src/hdfsee
diff --git a/announcement-0.6.3 b/announcement-0.6.3
deleted file mode 100644
index db8d0adb..00000000
--- a/announcement-0.6.3
+++ /dev/null
@@ -1,35 +0,0 @@
-Dear CrystFEL users and interested crystallographers,
-
-It's been a long time since the last CrystFEL release, but we have not been
-resting. CrystFEL version 0.6.3 has been released.
-
-This version incorporates a large number of "backstage" improvements and
-alterations. The most interesting changes for users are:
-
-- The incorporation of "peak finder 8" from Cheetah (also used in OnDA)
-
-- Support for images in CBF format
-
-- Incorporation of "TakeTwo" indexing algorithm
-
-- Hit finder functionality (skip frames with few peaks)
-
-- Better support for three-dimensional detector geometries
-
-See the release notes for more discussion of these improvements, and the
-ChangeLog for full details of the other improvements.
-
-Thanks for all of your past and future feedback and contributions, and
-your continued use of CrystFEL. As ever, please contact me directly if
-you would prefer to be unsubscribed from this mailing list.
-
-CrystFEL website:
-https://www.desy.de/~twhite/crystfel
-
-Release notes for version 0.6.3:
-https://www.desy.de/~twhite/crystfel/relnotes-0.6.3
-
-CrystFEL tutorial:
-https://www.desy.de/~twhite/crystfel/tutorial
-
-Tom
diff --git a/announcement-0.7.0 b/announcement-0.7.0
new file mode 100644
index 00000000..619dc570
--- /dev/null
+++ b/announcement-0.7.0
@@ -0,0 +1,37 @@
+Dear CrystFEL users and interested crystallographers,
+
+CrystFEL version 0.7.0 has been released.
+
+This version adds several exciting new features while greatly simplifying the
+user experience. The most interesting changes are:
+
+- New options system for indexamajig, removing most of the profusion of indexing
+ method flags "comb", "bad", "retry" and so on. Indexamajig can also now
+ automatically determine which indexing methods can be used if you don't tell
+ it which ones to use. In addition, the options for the Felix indexing method
+ have been simplified.
+
+- The spectrum-based partiality model and numerical post-refinement algorithm
+ from Ginn et al. [Acta D71 2015 p1400] has been incorporated.
+
+- The symmetry of the merged reflection list is now automatically tracked through
+ the subsequent data processing stages, so you no longer need to give the "-y"
+ option to compare_hkl, check_hkl and other programs.
+
+See the release notes for more discussion of these improvements, and the
+ChangeLog for full details of the other improvements.
+
+Thanks for all of your past and future feedback and contributions, and
+your continued use of CrystFEL. As ever, please contact me directly if
+you would prefer to be unsubscribed from this mailing list.
+
+CrystFEL website:
+https://www.desy.de/~twhite/crystfel
+
+Release notes for version 0.7.0:
+https://www.desy.de/~twhite/crystfel/relnotes-0.7.0
+
+CrystFEL tutorial:
+https://www.desy.de/~twhite/crystfel/tutorial
+
+Tom
diff --git a/configure.ac b/configure.ac
index 30e803f7..ef003372 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,6 +1,6 @@
dnl Process this file with autoconf to produce a configure script.
-AC_INIT([crystfel],[0.6.3],[taw@physics.org])
+AC_INIT([crystfel],[0.7.0],[taw@physics.org])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_HEADERS([config.h])
AM_INIT_AUTOMAKE([subdir-objects])
diff --git a/doc/reference/libcrystfel/CrystFEL-docs.sgml b/doc/reference/libcrystfel/CrystFEL-docs.sgml
index 048f4167..6a1dabee 100644
--- a/doc/reference/libcrystfel/CrystFEL-docs.sgml
+++ b/doc/reference/libcrystfel/CrystFEL-docs.sgml
@@ -8,7 +8,7 @@
<bookinfo>
<title>CrystFEL Reference Manual</title>
<releaseinfo>
- For libcrystfel from CrystFEL 0.6.3.
+ For libcrystfel from CrystFEL 0.7.0.
</releaseinfo>
<abstract>
This is the internal documentation for CrystFEL. Unless you are looking at
diff --git a/libcrystfel/Makefile.am b/libcrystfel/Makefile.am
index 24cb7358..1d24d353 100644
--- a/libcrystfel/Makefile.am
+++ b/libcrystfel/Makefile.am
@@ -1,6 +1,6 @@
lib_LTLIBRARIES = libcrystfel.la
libcrystfel_la_LIBADD = ../lib/libgnu.la @LIBCRYSTFEL_LIBS@ $(PTY_LIB)
-libcrystfel_la_LDFLAGS = -version-info 9:0:0
+libcrystfel_la_LDFLAGS = -version-info 10:0:0
libcrystfel_la_SOURCES = src/reflist.c src/utils.c src/cell.c src/detector.c \
src/thread-pool.c src/image.c src/hdf5-file.c \
diff --git a/relnotes-0.6.3 b/relnotes-0.6.3
deleted file mode 100644
index 11a1f93b..00000000
--- a/relnotes-0.6.3
+++ /dev/null
@@ -1,211 +0,0 @@
-CrystFEL - Crystallography with a FEL
--------------------------------------
-
-Release notes for version 0.6.3
-
-Copyright © 2012-2017 Deutsches Elektronen-Synchrotron DESY,
- a research centre of the Helmholtz Association.
-
-Authors:
- Thomas White <taw@physics.org>
- Richard Kirian <rkirian@asu.edu>
- Kenneth Beyerlein <kenneth.beyerlein@desy.de>
- Andrew Aquila <andrew.aquila@cfel.de>
- Andrew Martin <andrew.martin@desy.de>
- Lorenzo Galli <lorenzo.galli@desy.de>
- Chun Hong Yoon <chun.hong.yoon@desy.de>
- Kenneth Beyerlein <kenneth.beyerlein@desy.de>
- Karol Nass <karol.nass@desy.de>
- Nadia Zatsepin <nadia.zatsepin@asu.edu>
- Anton Barty <anton.barty@desy.de>
- Cornelius Gati <cornelius.gati@desy.de>
- Fedor Chervinskii <fedor.chervinskii@gmail.com>
- Alexandra Tolstikova <alexandra.tolstikova@desy.de>
- Wolfgang Brehm <wolfgang.brehm@gmail.com>
- Valerio Mariani <valerio.mariani@desy.de>
- Parker de Waal <Parker.deWaal@vai.org>
- Takanori Nakane <nakane.t@gmail.com>
- Keitaro Yamashita <k.yamashita@spring8.or.jp>
- Oleksandr Yefanov <oleksandr.yefanov@cfel.de>
- Steve Aplin <steve.aplin@desy.de>
- Helen Ginn <helen@strubi.ox.ac.uk>
-
-CrystFEL is free software: you can redistribute it and/or modify it under the
-terms of the GNU General Public License as published by the Free Software
-Foundation, either version 3 of the License, or (at your option) any later
-version.
-
-CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY
-WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
-PARTICULAR PURPOSE. See the GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License along with
-CrystFEL. If not, see <http://www.gnu.org/licenses/>.
-
-
-Overview
---------
-
-The most important new features in this version of CrystFEL are:
-
-- Support for CBF files as input
-
-- TakeTwo indexing algorithm
-
-- Improved peak detection and basic hitfinder functionality
-
-- Detector panels not perpendicular to X-ray beam, and "rail direction"
-
-These changes have sections below. In addition, there were many bug fixes and
-other improvements. See the "ChangeLog" or the changes page on the CrystFEL
-website for details.
-
-
-Support for CBF files as input
-------------------------------
-
-Crystallographic Binary Files (CBF) can now be used as input for indexamajig
-and hdfsee. To enable this functionality, you must have "CBFlib" installed.
-Some distributions (eg Fedora) include CBFlib in their standard repositories -
-simply install "cbflib-devel" or similar. Then compile and install CrystFEL as
-usual. If CBFlib is installed somewhere non-standard, e.g. if you has to
-install it manually, you will need to give the path with the "--with-cbflib-dir"
-option to "./configure".
-
-After that, you can simply give the names of CBF files in the indexamajig input,
-and everything should work as usual! There are some catches, for example you
-cannot yet use a bad pixel mask with CBF files, but these will be fixed in a
-future release.
-
-
-TakeTwo indexing algorithm
---------------------------
-
-This version of CrystFEL includes the TakeTwo indexing algorithm by Helen Ginn.
-TakeTwo is a new indexing algorithm, designed specifically for diffraction
-snapshots and operating completely differently to conventional FFT-based methods.
-
-To use it, simply use "taketwo" as the indexing method, or add it to the list of
-indexing methods you have already:
- $ indexamajig --indexing=taketwo
- $ indexamajig --indexing=mosflm,taketwo,dirax
-
-If you use TakeTwo, please take careful note of the citation reminder which will
-be shown to you, and cite the following paper in any resulting publications:
- Ginn et al., Acta Cryst. (2016). D72, 956-965
-
-TakeTwo is considered experimental in this version of CrystFEL. We welcome any
-feedback, and will use the information to improve its performance for future
-versions.
-
-
-Improved peak detection and basic hitfinder functionality
----------------------------------------------------------
-
-"Peak finder 8" from Cheetah, also used in OnDA, has been incorporated into
-CrystFEL. It uses a radially varying background level to find peaks, which
-works very well with most diffraction data.
-
-Simply use "--peaks=peakfinder8", and set --threshold, --min-snr,
---min-pix-count and --local-bg-radius. See the manual for other options.
-
-Indexamajig can now skip over patterns which have a small number of peaks. To
-use this, use the option "--min-peaks=N", where N is the minimum number of peaks
-before indexamajig will proceed to indexing (and hopefully integration) with the
-pattern.
-
-By combining the improved peak detection with the hitfinder functionality and
-CBF reading, you can process data from several different types of detector,
-without converting the files to HDF5 format or using any external program for
-hit finding!
-
-If the hit rate is very low, you might want to make CrystFEL stream smaller by
-excluding non-hits from it using "--no-non-hits-in-stream". Without this
-option, standard information about the non-hits (e.g. photon energy and peaks
-found) will be recorded.
-
-
-Detector panels not perpendicular to X-ray beam, and "rail direction"
-----------------------------------------------------------------------
-
-CrystFEL now supports three-dimensional detector geometry. The "fs" and "ss"
-vectors (which are the real-space directions of the fast scan and slow scan
-directions in the image data) can now contain a z component as well as x and y.
-
-For many detectors, the "rail" direction, along which the detector moves when
-the "camera length" (sample-detector distance) is adjusted, is not perpendicular
-to the detector plane. The geometry file now allows you to specify this "rail"
-direction, containing x, y and z components. See the crystfel_geometry manual
-page for more details.
-
-
-API changes
------------
-
-The following changes have been made to the libcrystfel API. The biggest
-changes are the switch from the old "hdfile" API (which is still supported),
-to the new "imagefile" API, while allows multiple file types to be used.
-The indexing system was also updated, and there were changes to the detector
-geometry system, which no longer uses the "data slab" representation internally.
-
-New functions:
- - crystal_{get,set}_det_shift()
- - panel_number()
- - get_detector_geometry_2()
- - fill_in_adu()
- - adjust_centering_for_rail()
- - get_peaks_2()
- - get_peaks_cxi_2()
- - imagefile_open()
- - imagefile_read()
- - imagefile_read_simple()
- - imagefile_get_hdfile()
- - imagefile_get_type()
- - imagefile_copy_fields()
- - imagefile_close()
- - {new,free}_imagefile_field_list()
- - add_imagefile_field()
- - get_indm_from_string()
- - setup_indexing()
- - index_pattern_2()
- - search_peaks_peakfinder8()
- - reflist_add_command_and_version()
- - reflist_{add,get}_notes()
- - write_chunk_2()
- - pointgroup_warning()
-
-New structure definitions:
- - struct image
- - struct imagefile_field_list
-
-Removed functions:
- - get_q()
- - find_panel()
- - find_panel_number()
- - fill_in_values()
- - partial_event_substitution()
- - build_indexer_list()
- - prepare_indexing()
-
-Changed function prototypes:
- - in_bad_region()
- - simple_geometry()
- - reverse_2d_mapping()
- - hdfile_set_image()
- - image_add_feature()
- - image_feature_closest()
- - image_reflection_closest()
- - index_pattern()
- - cleanup_indexing()
- - write_chunk()
-
-Changed structure definitions:
- - struct panel: removed {min,max}_{fs,ss}, added adu_per_photon,
- added fsz,ssz, rail_{x,y,z} and clen_for_centering
- - struct detector: removed max_{fs,ss}
- - struct imagefeature: replaced "pn" with "p"
- - struct beam_params: "copyme" is not an imagefile_field_list
- (previously: copy_hdf5_field)
- - struct image: "copyme" is not an imagefile_field_list
- (previously: copy_hdf5_field). width and height remoted.
-
diff --git a/relnotes-0.7.0 b/relnotes-0.7.0
new file mode 100644
index 00000000..186694a0
--- /dev/null
+++ b/relnotes-0.7.0
@@ -0,0 +1,176 @@
+CrystFEL - Data processing for serial crystallography
+-----------------------------------------------------
+
+Release notes for version 0.7.0
+
+Copyright © 2012-2018 Deutsches Elektronen-Synchrotron DESY,
+ a research centre of the Helmholtz Association.
+
+Authors:
+ Thomas White <taw@physics.org>
+ Richard Kirian <rkirian@asu.edu>
+ Kenneth Beyerlein <kenneth.beyerlein@desy.de>
+ Andrew Aquila <andrew.aquila@cfel.de>
+ Andrew Martin <andrew.martin@desy.de>
+ Lorenzo Galli <lorenzo.galli@desy.de>
+ Chun Hong Yoon <chun.hong.yoon@desy.de>
+ Karol Nass <karol.nass@desy.de>
+ Nadia Zatsepin <nadia.zatsepin@asu.edu>
+ Anton Barty <anton.barty@desy.de>
+ Cornelius Gati <cornelius.gati@desy.de>
+ Fedor Chervinskii <fedor.chervinskii@gmail.com>
+ Alexandra Tolstikova <alexandra.tolstikova@desy.de>
+ Wolfgang Brehm <wolfgang.brehm@gmail.com>
+ Valerio Mariani <valerio.mariani@desy.de>
+ Parker de Waal <Parker.deWaal@vai.org>
+ Takanori Nakane <nakane.t@gmail.com>
+ Keitaro Yamashita <k.yamashita@spring8.or.jp>
+ Oleksandr Yefanov <oleksandr.yefanov@cfel.de>
+ Steve Aplin <steve.aplin@desy.de>
+ Helen Ginn <helen@strubi.ox.ac.uk>
+ Thomas Grant <tgrant@hwi.buffalo.edu>
+ Mamoru Suzuki <mamoru.suzuki@protein.osaka-u.ac.jp>
+
+CrystFEL is free software: you can redistribute it and/or modify it under the
+terms of the GNU General Public License as published by the Free Software
+Foundation, either version 3 of the License, or (at your option) any later
+version.
+
+CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along with
+CrystFEL. If not, see <http://www.gnu.org/licenses/>.
+
+
+Overview
+--------
+
+The most important new features in this version of CrystFEL are:
+
+- New options system for indexamajig, removing most of the profusion of indexing
+ method flags "comb", "bad", "retry" and so on. Indexamajig can also now
+ automatically determine which indexing methods can be used if you don't tell
+ it which ones to use. In addition, the options for the Felix indexing method
+ have now been greatly simplified.
+
+- The spectrum-based partiality model and numerical post-refinement algorithm
+ from Ginn et al [Acta D71 2015 p1400] was incorporated.
+
+- The symmetry of the merged reflection list is now tracked through automatically,
+ so you no longer need to give the "-y" option to compare_hkl and check_hkl.
+
+These changes have sections below. In addition, there were many bug fixes and
+other improvements. See the "ChangeLog" or the changes page on the CrystFEL
+website for details.
+
+
+New indexamajig options system
+------------------------------
+
+The new version of indexamajig will automatically determine which indexing
+methods to use based on the prior information (e.g. if a target unit cell was
+provided) and which indexing programs are available on the computer. To enable
+the automatic determination, simply omit any "--indexing=" option to indexamajig.
+Of course, you can override the automatic choice by specifying --indexing, just
+as it worked before.
+
+To options for finer control of the indexing process, which in previous versions
+were enabled or disabled by adding flags to the indexing methods
+(e.g. "mosflm-noretry"), now apply to all indexing methods at once and are
+selected using new options, listed below.
+
+The indexing method should contain only the method itself and prior information
+modifiers ('cell' or 'latt'). This simplifies things greatly by removing the
+very long indexing method descriptions ("mosflm-raw-latt-noretry-refine-nomulti"
+and so on).
+
+- To disable prediction refinement ('norefine'), use --no-refine.
+- To check cell axes only ('axes'), use --no-cell-combinations.
+- To disable all unit cell checks ('raw'), use --no-check-cell.
+- To disable peak alignment check ('bad'), use --no-check-peaks.
+- To disable indexing retry ('noretry'), use --no-retry.
+- To enable multi-lattice indexing by 'delete and retry', use --multi.
+
+
+Spectrum-based partiality model and post-refinement
+---------------------------------------------------
+
+This version includes an implementation of the spectrum-based partiality model
+described by Ginn et al., Acta D71 2015 p1400 and associated numerical post-
+refinement algorithm. To use it, use "partialator --model=xsphere". The old
+models (scsphere and scgaussian) have been removed. You will probably need to
+experiment with changing the bandwidth and reflection radius, using
+--force-bandwidth and --force-radius. Add --no-pr to disable post-refinement,
+while still using the partiality estimates. To use post-refinement, omit --no-pr,
+but you should also run indexamajig with "--overpredict". You may also need to
+disable B factor scaling in partialator with "--no-Bscale".
+
+If you use post-refinement, the new scripts "plot-pr" and "plot-contourmap" can
+be used to visualise the distributions of partialities and the contour maps of
+residual against parameters such as crystal rotation angle. The data for these
+graphs is in the "pr-logs" folder which will be created by partialator.
+
+Partiality modelling and post-refinement should still be considered experimental
+when applied to real data. The success of post-refinement depends on many
+factors, and only has a chance of working if there are no other problems with
+the dataset (e.g. inaccurate detector geometry). Further improvements should
+appear in future versions.
+
+
+Automatic tracking of reflection list symmetry
+----------------------------------------------
+
+The point group symmetry used for merging a reflection list has been stored in
+the reflection file (.hkl) for a long time. However, version 0.7.0 of CrystFEL
+uses the stored symmetry when performing tasks such as calculating figures of
+merit using check_hkl. You can simply omit the "-y" options which were needed
+previously. Of course, you can override the symmetry by specifying -y as before.
+
+
+API changes
+-----------
+
+The following changes have been made to the libcrystfel API.
+
+New functions:
+ - asdf_probe() and _probe for all other indexing methods.
+ - compare_cells()
+ - crystal_get_cell_const()
+ - crystal_get_profile_radius()
+ - predict_to_res()
+ - calculate_partialities()
+ - update_predictions()
+ - is_cbf_file()
+ - get_indm_from_string_2()
+ - detect_indexing_methods()
+ - integrate_all_5()
+ - read_reflections_2()
+ - {get,set}_{khalf,kpred,exerr}()
+ - first_refl_const(), next_refl_const()
+ - open_stream_for_write_4()
+ - stream_has_old_indexers()
+ - stream_audit_info()
+
+Removed functions:
+ - find_intersections()
+ - find_intersections_to_res()
+ - update_partilities()
+ - write_reflections_to_file()
+ - read_reflections_from_file()
+ - {get,sh}_partial()
+
+Changed function prototypes:
+ - asdf_prepare() and _prepare for all other indexing methods.
+ - cell_new_from_cell() (input UnitCell is now const)
+ - crystal_copy() (input Crystal is now const)
+ - x_gradient(), y_gradient()
+ - remove_flagged_crystals() (now returns number of crystals removed)
+ - setup_indexing()
+
+Changed structure/enum definitions:
+ - PartialityModel: removed PMODEL_SCSPHERE and PMODEL_SCGAUSSIAN,
+ added PMODEL_XSPHERE
+ - gparam: added GPARAM_ANG1, GPARAM_ANG2, GPARAM_WAVELENGTH, GPARAM_EOL
+ - Added IndexingFlags