aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorThomas White <taw@physics.org>2017-07-24 14:58:39 +0200
committerThomas White <taw@physics.org>2017-07-24 14:58:58 +0200
commitad6d6cfe15bcea5357d370928bc609f1dcca4ae1 (patch)
tree64b851ce6135de66e1df2d0806ccc3c3c5f5525e
parent6a25a919a8a886f446089bd67e26132dbf46627e (diff)
This is CrystFEL 0.6.30.6.3
-rw-r--r--ChangeLog4
-rw-r--r--Makefile.am2
-rw-r--r--announcement-0.6.244
-rw-r--r--announcement-0.6.335
-rw-r--r--configure.ac2
-rw-r--r--doc/reference/libcrystfel/CrystFEL-docs.sgml2
-rw-r--r--libcrystfel/Makefile.am2
-rw-r--r--relnotes-0.6.2154
-rw-r--r--relnotes-0.6.3211
9 files changed, 252 insertions, 204 deletions
diff --git a/ChangeLog b/ChangeLog
index 0c8d596e..ba504e5d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,5 @@
-Changes in this development version
------------------------------------
+CrystFEL version 0.6.3, 24th July 2017
+--------------------------------------
- indexamajig --peaks=peakfinder8 was added (Valerio Mariani, Oleksandr Yefanov)
- CBF files are now supported as input.
diff --git a/Makefile.am b/Makefile.am
index fda28fbc..78456727 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -29,7 +29,7 @@ TESTS = tests/list_check $(MERGE_CHECKS) $(PARTIAL_CHECKS) \
tests/prediction_gradient_check
EXTRA_DIST += $(MERGE_CHECKS) $(PARTIAL_CHECKS)
-EXTRA_DIST += relnotes-0.6.2 announcement-0.6.2
+EXTRA_DIST += relnotes-0.6.3 announcement-0.6.3
if BUILD_HDFSEE
bin_PROGRAMS += src/hdfsee
diff --git a/announcement-0.6.2 b/announcement-0.6.2
deleted file mode 100644
index e92ce453..00000000
--- a/announcement-0.6.2
+++ /dev/null
@@ -1,44 +0,0 @@
-Dear CrystFEL users and interested crystallographers,
-
-CrystFEL version 0.6.2 has been released. The most interesting changes
-in this version are improvements to the stability of scaling in partialator,
-the addition of "custom dataset splitting" for time-resolved/isomorphous
-replacement experiments and more, "retry" indexing to increase the indexing
-rate, and support for per-pixel saturation values needed to correctly handle
-dual-gain CSPAD data.
-
-The improvements to scaling stability should improve the results with most
-data sets, but particularly those where there have been problems with large
-numbers of crystals getting rejected due to excessively large relative B
-factors.
-
-"Custom dataset splitting" enables you to process multiple datasets in one
-large scaling (or post-refinement) job, and separate them afterwards. This
-eliminates any possible concerns over uniqueness of the solution with these new
-data processing methods.
-
-"Retry" indexing is where the indexing method is repeated, if it fails the first
-time, after deleting some of the weakest peaks. In many cases, this leads to a
-successfully indexed pattern. This can sometimes double the indexing rate!
-However, it makes it slower to process if the indexing rate is low, and you can
-disable it by adding "-noretry" to all the indexing methods.
-
-See the release notes for more discussion of these improvements.
-
-In addition, there are many smaller improvements, new features and bug
-fixes. See the ChangeLog for details.
-
-Thanks for all of your past and future feedback and contributions, and
-your continued use of CrystFEL. As ever, please contact me directly if
-you would prefer to be unsubscribed from this mailing list.
-
-CrystFEL website:
-http://www.desy.de/~twhite/crystfel
-
-Release notes for version 0.6.2:
-http://www.desy.de/~twhite/crystfel/relnotes-0.6.2
-
-CrystFEL tutorial:
-http://www.desy.de/~twhite/crystfel/tutorial
-
-Tom
diff --git a/announcement-0.6.3 b/announcement-0.6.3
new file mode 100644
index 00000000..db8d0adb
--- /dev/null
+++ b/announcement-0.6.3
@@ -0,0 +1,35 @@
+Dear CrystFEL users and interested crystallographers,
+
+It's been a long time since the last CrystFEL release, but we have not been
+resting. CrystFEL version 0.6.3 has been released.
+
+This version incorporates a large number of "backstage" improvements and
+alterations. The most interesting changes for users are:
+
+- The incorporation of "peak finder 8" from Cheetah (also used in OnDA)
+
+- Support for images in CBF format
+
+- Incorporation of "TakeTwo" indexing algorithm
+
+- Hit finder functionality (skip frames with few peaks)
+
+- Better support for three-dimensional detector geometries
+
+See the release notes for more discussion of these improvements, and the
+ChangeLog for full details of the other improvements.
+
+Thanks for all of your past and future feedback and contributions, and
+your continued use of CrystFEL. As ever, please contact me directly if
+you would prefer to be unsubscribed from this mailing list.
+
+CrystFEL website:
+https://www.desy.de/~twhite/crystfel
+
+Release notes for version 0.6.3:
+https://www.desy.de/~twhite/crystfel/relnotes-0.6.3
+
+CrystFEL tutorial:
+https://www.desy.de/~twhite/crystfel/tutorial
+
+Tom
diff --git a/configure.ac b/configure.ac
index 2a5f40cf..41071139 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,6 +1,6 @@
dnl Process this file with autoconf to produce a configure script.
-AC_INIT([crystfel],[0.6.2],[taw@physics.org])
+AC_INIT([crystfel],[0.6.3],[taw@physics.org])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_HEADERS([config.h])
AM_INIT_AUTOMAKE([subdir-objects])
diff --git a/doc/reference/libcrystfel/CrystFEL-docs.sgml b/doc/reference/libcrystfel/CrystFEL-docs.sgml
index 407f23b4..048f4167 100644
--- a/doc/reference/libcrystfel/CrystFEL-docs.sgml
+++ b/doc/reference/libcrystfel/CrystFEL-docs.sgml
@@ -8,7 +8,7 @@
<bookinfo>
<title>CrystFEL Reference Manual</title>
<releaseinfo>
- For libcrystfel from CrystFEL 0.6.2.
+ For libcrystfel from CrystFEL 0.6.3.
</releaseinfo>
<abstract>
This is the internal documentation for CrystFEL. Unless you are looking at
diff --git a/libcrystfel/Makefile.am b/libcrystfel/Makefile.am
index fef2e11f..24cb7358 100644
--- a/libcrystfel/Makefile.am
+++ b/libcrystfel/Makefile.am
@@ -1,6 +1,6 @@
lib_LTLIBRARIES = libcrystfel.la
libcrystfel_la_LIBADD = ../lib/libgnu.la @LIBCRYSTFEL_LIBS@ $(PTY_LIB)
-libcrystfel_la_LDFLAGS = -version-info 8:0:0
+libcrystfel_la_LDFLAGS = -version-info 9:0:0
libcrystfel_la_SOURCES = src/reflist.c src/utils.c src/cell.c src/detector.c \
src/thread-pool.c src/image.c src/hdf5-file.c \
diff --git a/relnotes-0.6.2 b/relnotes-0.6.2
deleted file mode 100644
index daf48f3f..00000000
--- a/relnotes-0.6.2
+++ /dev/null
@@ -1,154 +0,0 @@
-CrystFEL - Crystallography with a FEL
--------------------------------------
-
-Release notes for version 0.6.2
-
-Copyright © 2012-2016 Deutsches Elektronen-Synchrotron DESY,
- a research centre of the Helmholtz Association.
-
-Authors:
- Thomas White <taw@physics.org>
- Richard Kirian <rkirian@asu.edu>
- Kenneth Beyerlein <kenneth.beyerlein@desy.de>
- Andrew Aquila <andrew.aquila@cfel.de>
- Andrew Martin <andrew.martin@desy.de>
- Lorenzo Galli <lorenzo.galli@desy.de>
- Chun Hong Yoon <chun.hong.yoon@desy.de>
- Kenneth Beyerlein <kenneth.beyerlein@desy.de>
- Karol Nass <karol.nass@desy.de>
- Nadia Zatsepin <nadia.zatsepin@asu.edu>
- Anton Barty <anton.barty@desy.de>
- Cornelius Gati <cornelius.gati@desy.de>
- Fedor Chervinskii <fedor.chervinskii@gmail.com>
- Alexandra Tolstikova <alexandra.tolstikova@desy.de>
- Wolfgang Brehm <wolfgang.brehm@gmail.com>
- Valerio Mariani <valerio.mariani@desy.de>
- Parker de Waal <Parker.deWaal@vai.org>
- Takanori Nakane <nakane.t@gmail.com>
- Keitaro Yamashita <k.yamashita@spring8.or.jp>
- Oleksandr Yefanov <oleksandr.yefanov@cfel.de>
-
-CrystFEL is free software: you can redistribute it and/or modify it under the
-terms of the GNU General Public License as published by the Free Software
-Foundation, either version 3 of the License, or (at your option) any later
-version.
-
-CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY
-WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
-PARTICULAR PURPOSE. See the GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License along with
-CrystFEL. If not, see <http://www.gnu.org/licenses/>.
-
-
-Overview
---------
-
-The most important new features in this version of CrystFEL are:
-
-- Stability improvements to the scaling algorithm in partialator.
-
-- Addition of "custom dataset splitting" in partialator.
-
-- "retry" indexing.
-
-- Per-pixel saturation values
-
-These changes have sections below. In addition, there were many bug fixes and
-smaller improvements.
-
-
-Stability improvements in partialator
--------------------------------------
-
-The scaling algorithm in partialator has been altered such that only the strong
-reflections contribute to the scaling factors at all. This reduces the
-"drifting" of scale factors which happened before, and helps avoid the situation
-where a very large number of crystals get rejected with large B factors.
-Of course, all reflections contribute to the final merged data set as before.
-
-In CrystFEL 0.6.1, we needed to reduce the amount of noise going into the
-scaling algorithm to make it work. This was done by including reflections only
-up to the apparent (conservative) resolution limit of each crystal made by
-indexamajig, and could be overridden by using the "--push-res" option of
-partialator. The default --push-res value was zero, which meant to ignore any
-reflections higher than the limit. With the stabilised algorithm, this is no
-longer necessary, and hence the default --push-res value has been changed to
-infinity, i.e. no cutoff at all. You can restore the old behaviour with
---push-res=0.
-
-
-Custom dataset splitting in partialator
----------------------------------------
-
-When performing a time-resolved or isomorphous replacement experiment (plus some
-other types of experiment), it is preferable to ensure that the data for all
-time points has been processed identically. Rather than processing each time
-point independently with separate runs of partialator, it is better to process
-them all together and do the splitting into time points just before the final
-output. Consider, for example, the case of simple scaling (without a B
-factor): when merging independently, the resulting datasets would probably end
-up with different overall scaling factors. When comparing the results, you
-would need to take this difference into account. In practice, most programs
-can do that job easily, but what about if a B factor is included? And what if
-partialities are included - how unique is the solution?
-
-Partialator's new "custom dataset splitting" option allows you to provide a
-text file containing a list of filenames, event numbers and dataset
-names, one event (detector frame) per line. All crystals will be refined
-together, but they will be merged according to the dataset names you give. The
-parameters (scaling factors, partialities etc) determined during the joint
-refinement will be applied. For each dataset, a separate pair of split
-half-datasets will also be written, allowing you to calculate figures of merit
-such as Rsplit and CC1/2 for each one.
-
-
-"retry" indexing
-----------------
-
-If an indexing method fails to index a pattern (or if the solution doesn't get
-through the prediction refinement stage), this version of CrystFEL will delete
-the weakest few peaks and try again. This can increase the indexing yield a lot
-(sometimes a factor of two!), but decreases the speed. If you'd rather keep
-things fast, add "-noretry" to each of your indexing methods.
-
-
-Per-pixel saturation values
----------------------------
-
-When using dual-gain mode on the CSPAD (or any other detector which offers a
-similar feature), our current advice is to correct the pixel values in one of
-the regions by multiplying them by the relative gain factor. This will give
-image data which looks good, but the saturation values will be different in the
-two regions. To handle this, CrystFEL 0.6.2 allows you to prepare a map, in
-HDF5 format, giving the saturation value for each pixel individually. The
-script "gaincal-to-saturation-map" can be used to create a suitable file from
-the gain map (e.g. as generated and used by Cheetah). Specify the filename and
-HDF5 path of the saturation map using saturation_map and saturation_map_file
-in the geometry file - these work exactly like the bad pixel map. Reflections
-containing any pixel above this value will be rejected, in addition to any
-reflections which get rejected by the usual saturation criteria (so remove any
-max_adu directives if you want the saturation map to be the only decider!).
-
-
-API changes
------------
-
-The following changes have been made to the libcrystfel API:
-
-New functions:
- - write_cell()
- - find_orig_panel_number()
- - felix_{prepare,index,cleanup}()
- - sort_peaks()
- - remove_flagged_crystals()
- - open_stream_for_write_3()
-
-Removed functions:
- (none)
-
-Changed function prototypes:
- - prepare_indexing()
-
-"data", "flags" and "twotheta" were removed from "struct image", and "sat" was
-added.
diff --git a/relnotes-0.6.3 b/relnotes-0.6.3
new file mode 100644
index 00000000..11a1f93b
--- /dev/null
+++ b/relnotes-0.6.3
@@ -0,0 +1,211 @@
+CrystFEL - Crystallography with a FEL
+-------------------------------------
+
+Release notes for version 0.6.3
+
+Copyright © 2012-2017 Deutsches Elektronen-Synchrotron DESY,
+ a research centre of the Helmholtz Association.
+
+Authors:
+ Thomas White <taw@physics.org>
+ Richard Kirian <rkirian@asu.edu>
+ Kenneth Beyerlein <kenneth.beyerlein@desy.de>
+ Andrew Aquila <andrew.aquila@cfel.de>
+ Andrew Martin <andrew.martin@desy.de>
+ Lorenzo Galli <lorenzo.galli@desy.de>
+ Chun Hong Yoon <chun.hong.yoon@desy.de>
+ Kenneth Beyerlein <kenneth.beyerlein@desy.de>
+ Karol Nass <karol.nass@desy.de>
+ Nadia Zatsepin <nadia.zatsepin@asu.edu>
+ Anton Barty <anton.barty@desy.de>
+ Cornelius Gati <cornelius.gati@desy.de>
+ Fedor Chervinskii <fedor.chervinskii@gmail.com>
+ Alexandra Tolstikova <alexandra.tolstikova@desy.de>
+ Wolfgang Brehm <wolfgang.brehm@gmail.com>
+ Valerio Mariani <valerio.mariani@desy.de>
+ Parker de Waal <Parker.deWaal@vai.org>
+ Takanori Nakane <nakane.t@gmail.com>
+ Keitaro Yamashita <k.yamashita@spring8.or.jp>
+ Oleksandr Yefanov <oleksandr.yefanov@cfel.de>
+ Steve Aplin <steve.aplin@desy.de>
+ Helen Ginn <helen@strubi.ox.ac.uk>
+
+CrystFEL is free software: you can redistribute it and/or modify it under the
+terms of the GNU General Public License as published by the Free Software
+Foundation, either version 3 of the License, or (at your option) any later
+version.
+
+CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along with
+CrystFEL. If not, see <http://www.gnu.org/licenses/>.
+
+
+Overview
+--------
+
+The most important new features in this version of CrystFEL are:
+
+- Support for CBF files as input
+
+- TakeTwo indexing algorithm
+
+- Improved peak detection and basic hitfinder functionality
+
+- Detector panels not perpendicular to X-ray beam, and "rail direction"
+
+These changes have sections below. In addition, there were many bug fixes and
+other improvements. See the "ChangeLog" or the changes page on the CrystFEL
+website for details.
+
+
+Support for CBF files as input
+------------------------------
+
+Crystallographic Binary Files (CBF) can now be used as input for indexamajig
+and hdfsee. To enable this functionality, you must have "CBFlib" installed.
+Some distributions (eg Fedora) include CBFlib in their standard repositories -
+simply install "cbflib-devel" or similar. Then compile and install CrystFEL as
+usual. If CBFlib is installed somewhere non-standard, e.g. if you has to
+install it manually, you will need to give the path with the "--with-cbflib-dir"
+option to "./configure".
+
+After that, you can simply give the names of CBF files in the indexamajig input,
+and everything should work as usual! There are some catches, for example you
+cannot yet use a bad pixel mask with CBF files, but these will be fixed in a
+future release.
+
+
+TakeTwo indexing algorithm
+--------------------------
+
+This version of CrystFEL includes the TakeTwo indexing algorithm by Helen Ginn.
+TakeTwo is a new indexing algorithm, designed specifically for diffraction
+snapshots and operating completely differently to conventional FFT-based methods.
+
+To use it, simply use "taketwo" as the indexing method, or add it to the list of
+indexing methods you have already:
+ $ indexamajig --indexing=taketwo
+ $ indexamajig --indexing=mosflm,taketwo,dirax
+
+If you use TakeTwo, please take careful note of the citation reminder which will
+be shown to you, and cite the following paper in any resulting publications:
+ Ginn et al., Acta Cryst. (2016). D72, 956-965
+
+TakeTwo is considered experimental in this version of CrystFEL. We welcome any
+feedback, and will use the information to improve its performance for future
+versions.
+
+
+Improved peak detection and basic hitfinder functionality
+---------------------------------------------------------
+
+"Peak finder 8" from Cheetah, also used in OnDA, has been incorporated into
+CrystFEL. It uses a radially varying background level to find peaks, which
+works very well with most diffraction data.
+
+Simply use "--peaks=peakfinder8", and set --threshold, --min-snr,
+--min-pix-count and --local-bg-radius. See the manual for other options.
+
+Indexamajig can now skip over patterns which have a small number of peaks. To
+use this, use the option "--min-peaks=N", where N is the minimum number of peaks
+before indexamajig will proceed to indexing (and hopefully integration) with the
+pattern.
+
+By combining the improved peak detection with the hitfinder functionality and
+CBF reading, you can process data from several different types of detector,
+without converting the files to HDF5 format or using any external program for
+hit finding!
+
+If the hit rate is very low, you might want to make CrystFEL stream smaller by
+excluding non-hits from it using "--no-non-hits-in-stream". Without this
+option, standard information about the non-hits (e.g. photon energy and peaks
+found) will be recorded.
+
+
+Detector panels not perpendicular to X-ray beam, and "rail direction"
+----------------------------------------------------------------------
+
+CrystFEL now supports three-dimensional detector geometry. The "fs" and "ss"
+vectors (which are the real-space directions of the fast scan and slow scan
+directions in the image data) can now contain a z component as well as x and y.
+
+For many detectors, the "rail" direction, along which the detector moves when
+the "camera length" (sample-detector distance) is adjusted, is not perpendicular
+to the detector plane. The geometry file now allows you to specify this "rail"
+direction, containing x, y and z components. See the crystfel_geometry manual
+page for more details.
+
+
+API changes
+-----------
+
+The following changes have been made to the libcrystfel API. The biggest
+changes are the switch from the old "hdfile" API (which is still supported),
+to the new "imagefile" API, while allows multiple file types to be used.
+The indexing system was also updated, and there were changes to the detector
+geometry system, which no longer uses the "data slab" representation internally.
+
+New functions:
+ - crystal_{get,set}_det_shift()
+ - panel_number()
+ - get_detector_geometry_2()
+ - fill_in_adu()
+ - adjust_centering_for_rail()
+ - get_peaks_2()
+ - get_peaks_cxi_2()
+ - imagefile_open()
+ - imagefile_read()
+ - imagefile_read_simple()
+ - imagefile_get_hdfile()
+ - imagefile_get_type()
+ - imagefile_copy_fields()
+ - imagefile_close()
+ - {new,free}_imagefile_field_list()
+ - add_imagefile_field()
+ - get_indm_from_string()
+ - setup_indexing()
+ - index_pattern_2()
+ - search_peaks_peakfinder8()
+ - reflist_add_command_and_version()
+ - reflist_{add,get}_notes()
+ - write_chunk_2()
+ - pointgroup_warning()
+
+New structure definitions:
+ - struct image
+ - struct imagefile_field_list
+
+Removed functions:
+ - get_q()
+ - find_panel()
+ - find_panel_number()
+ - fill_in_values()
+ - partial_event_substitution()
+ - build_indexer_list()
+ - prepare_indexing()
+
+Changed function prototypes:
+ - in_bad_region()
+ - simple_geometry()
+ - reverse_2d_mapping()
+ - hdfile_set_image()
+ - image_add_feature()
+ - image_feature_closest()
+ - image_reflection_closest()
+ - index_pattern()
+ - cleanup_indexing()
+ - write_chunk()
+
+Changed structure definitions:
+ - struct panel: removed {min,max}_{fs,ss}, added adu_per_photon,
+ added fsz,ssz, rail_{x,y,z} and clen_for_centering
+ - struct detector: removed max_{fs,ss}
+ - struct imagefeature: replaced "pn" with "p"
+ - struct beam_params: "copyme" is not an imagefile_field_list
+ (previously: copy_hdf5_field)
+ - struct image: "copyme" is not an imagefile_field_list
+ (previously: copy_hdf5_field). width and height remoted.
+