aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--ChangeLog7
-rw-r--r--Makefile.am2
-rw-r--r--announcement-0.6.143
-rw-r--r--configure.ac2
-rw-r--r--doc/reference/libcrystfel/CrystFEL-docs.sgml2
-rw-r--r--libcrystfel/Makefile.am2
-rw-r--r--relnotes-0.6.0257
-rw-r--r--relnotes-0.6.1165
8 files changed, 217 insertions, 263 deletions
diff --git a/ChangeLog b/ChangeLog
index b20db9af..49f5e830 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,5 @@
-Changes in this development version of CrystFEL
------------------------------------------------
+CrystFEL version 0.6.1, 3rd August 2015
+---------------------------------------
- partialator: big improvements, now works well on most data sets.
- indexamajig: prediction refinement added.
@@ -9,11 +9,14 @@ Changes in this development version of CrystFEL
- process_hkl --even-only and --odd-only were added.
- External bad pixel masks can now be used (see "mask_file" in geometry file).
- indexamajig: Can now use --int-diag with -j n where n>1
+- Image data filters (e.g. --noise-filter) were fixed.
+- indexamajig sandbox was redesigned and made more robust.
- compare_hkl now uses a B-factor for scaling the datasets.
- pattern_sim: spectrum normalisation added.
- scripts/ave-resolution and scripts/create-frame-number added.
- scripts/plot-predict-refine and scripts/plot-radius-resolution added.
- scripts/detector-shift added.
+- scripts/turbo-index added.
- "reax" and "grainspotter" indexing methods removed.
- Many other small but important bug fixes and improvements to data quality.
diff --git a/Makefile.am b/Makefile.am
index a6acd357..32cc8a63 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -29,7 +29,7 @@ TESTS = tests/list_check $(MERGE_CHECKS) $(PARTIAL_CHECKS) \
tests/prediction_gradient_check
EXTRA_DIST += $(MERGE_CHECKS) $(PARTIAL_CHECKS)
-EXTRA_DIST += relnotes-0.6.0
+EXTRA_DIST += relnotes-0.6.1 announcement-0.6.1
if BUILD_HDFSEE
bin_PROGRAMS += src/hdfsee
diff --git a/announcement-0.6.1 b/announcement-0.6.1
new file mode 100644
index 00000000..85137402
--- /dev/null
+++ b/announcement-0.6.1
@@ -0,0 +1,43 @@
+Dear CrystFEL users and interested crystallographers,
+
+CrystFEL version 0.6.1 has been released. The most interesting changes
+in this version are the new "prediction refinement" stage in
+"indexamajig" and improvements to the scaling and post-refinement
+program "partialator". There are also a couple of new indexing methods.
+
+The prediction refinement stage improves the quality of spot prediction
+and hence the initial parameter estimates of the spot partialities. It
+is also able to update the central beam position.
+
+The improvements to partialator include a significantly more stable
+scaling procedure. Partialator is no longer considered experimental as
+of this version, so please try it and send feedback. With the
+combination of this and the new prediction refinement, many users are
+already finding significant improvements in data quality. See the
+release notes (link below) and the updated tutorial on the CrystFEL
+website for more details.
+
+The new indexing methods are "asdf", which is an algorithm built into
+CrystFEL which therefore requires no external programs, and
+"mosflm-cell" which uses the prior unit cell algorithm in recent (7.2.0
+or later) versions of mosflm. "mosflm-cell" is now the default
+behaviour for mosflm - if it doesn't work for you, explicitly specify
+"mosflm-nocell" to restore the old behaviour.
+
+In addition, there are many smaller improvements, new features and bug
+fixes.
+
+Thanks for all of your past and future feedback and contributions, and
+your continued use of CrystFEL. As ever, please contact me directly if
+you would prefer to be unsubscribed from this mailing list.
+
+CrystFEL website:
+http://www.desy.de/~twhite/crystfel
+
+Release notes for version 0.6.1:
+http://www.desy.de/~twhite/crystfel/relnotes-0.6.1
+
+CrystFEL tutorial:
+http://www.desy.de/~twhite/crystfel/tutorial
+
+Tom
diff --git a/configure.ac b/configure.ac
index 0971acb6..9d4d7edf 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,6 +1,6 @@
dnl Process this file with autoconf to produce a configure script.
-AC_INIT([crystfel],[0.6.0],[taw@physics.org])
+AC_INIT([crystfel],[0.6.1],[taw@physics.org])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_HEADERS([config.h])
AM_INIT_AUTOMAKE([subdir-objects])
diff --git a/doc/reference/libcrystfel/CrystFEL-docs.sgml b/doc/reference/libcrystfel/CrystFEL-docs.sgml
index 6c7baa74..9ef5d945 100644
--- a/doc/reference/libcrystfel/CrystFEL-docs.sgml
+++ b/doc/reference/libcrystfel/CrystFEL-docs.sgml
@@ -8,7 +8,7 @@
<bookinfo>
<title>CrystFEL Reference Manual</title>
<releaseinfo>
- For libcrystfel from CrystFEL 0.6.0.
+ For libcrystfel from CrystFEL 0.6.1.
</releaseinfo>
<abstract>
This is the internal documentation for CrystFEL. Unless you are looking at
diff --git a/libcrystfel/Makefile.am b/libcrystfel/Makefile.am
index 1a0c3efe..77e5991b 100644
--- a/libcrystfel/Makefile.am
+++ b/libcrystfel/Makefile.am
@@ -1,6 +1,6 @@
lib_LTLIBRARIES = libcrystfel.la
libcrystfel_la_LIBADD = ../lib/libgnu.la @LIBCRYSTFEL_LIBS@ $(PTY_LIB)
-libcrystfel_la_LDFLAGS = -version-info 6:0:0
+libcrystfel_la_LDFLAGS = -version-info 7:0:0
libcrystfel_la_SOURCES = src/reflist.c src/utils.c src/cell.c src/detector.c \
src/thread-pool.c src/image.c src/hdf5-file.c \
diff --git a/relnotes-0.6.0 b/relnotes-0.6.0
deleted file mode 100644
index 22efa56d..00000000
--- a/relnotes-0.6.0
+++ /dev/null
@@ -1,257 +0,0 @@
-CrystFEL - Crystallography with a FEL
--------------------------------------
-
-Release notes for version 0.6.0
-
-Copyright © 2012-2015 Deutsches Elektronen-Synchrotron DESY,
- a research centre of the Helmholtz Association.
-
-Authors:
- Thomas White <taw@physics.org>
- Richard Kirian <rkirian@asu.edu>
- Kenneth Beyerlein <kenneth.beyerlein@desy.de>
- Andrew Aquila <andrew.aquila@cfel.de>
- Andrew Martin <andrew.martin@desy.de>
- Lorenzo Galli <lorenzo.galli@desy.de>
- Chun Hong Yoon <chun.hong.yoon@desy.de>
- Kenneth Beyerlein <kenneth.beyerlein@desy.de>
- Karol Nass <karol.nass@desy.de>
- Nadia Zatsepin <nadia.zatsepin@asu.edu>
- Anton Barty <anton.barty@desy.de>
- Cornelius Gati <cornelius.gati@desy.de>
- Fedor Chervinskii <fedor.chervinskii@gmail.com>
- Alexandra Tolstikova <alexandra.tolstikova@desy.de>
- Wolfgang Brehm <wolfgang.brehm@gmail.com>
- Valerio Mariani <valerio.mariani@desy.de>
- Parker de Waal <Parker.deWaal@vai.org>
- Takanori Nakane <nakane.t@gmail.com>
- Keitaro Yamashita <k.yamashita@spring8.or.jp>
- Oleksandr Yefanov <oleksandr.yefanov@cfel.de>
-
-CrystFEL is free software: you can redistribute it and/or modify it under the
-terms of the GNU General Public License as published by the Free Software
-Foundation, either version 3 of the License, or (at your option) any later
-version.
-
-CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY
-WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
-PARTICULAR PURPOSE. See the GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License along with
-CrystFEL. If not, see <http://www.gnu.org/licenses/>.
-
-
-Overview
---------
-
-The most important new features in this version of CrystFEL are:
-
-- Support for multi-event HDF5 files (e.g. CXI format as used by CXIDB)
-
-- geoptimiser: a new tool for precisely optimising the detector geometry
-
-- Removal of beam files / automatic determination of spot prediction parameters
-
-- whirligig: a new tool for finding clusters of similarly-oriented
- crystal snapshots, e.g. for finding "mini rotation series" in data
- from slow extrusion sample delivery methods.
-
-- Introduction of "CrystFEL unit cell files".
-
-These new features have individual sections below. In addition, there are many
-other new developments. See the ChangeLog or the changes page on the website
-for more details. There were, of course, the usual large number of smaller
-refinements and bug fixes.
-
-
-Support for multi-event HDF5 files
-----------------------------------
-
-CrystFEL's handling of HDF5 files has been made much more flexible. Most
-importantly, it now offers the ability to handle HDF5 files which contain more
-than one event (frame). Until now, CrystFEL has required that each event be
-contained in its own file, which can place a lot of unnecessary strain on
-filesystems. Recent versions of Cheetah allow you to create larger files which
-contain many events each using the CXI format (see http://www.cxidb.org for more
-details). CrystFEL now supports this as well as almost any reasonable
-multi-event HDF5 layout.
-
-If you choose to continue using many small files, you should not notice much
-difference. However, some small updates will be required. Firstly,
-indexamajig's "-e" and "--image" command-line parameters have been removed.
-Instead, you need to edit your geometry file and add a line such as this near
-the top:
-
- data = /data/rawdata
-
-Geometry files now contain all the information needed to interpret the contents
-of the HDF5 files as a physical setup, including the photon energy. Beam
-parameter files have been removed. See below for more information about this.
-
-When using multi-event files in CXI format, the peak lists are read differently.
-Use --peaks=cxi to retrieve peak lists from CXI files.
-
-hdfsee has been extended to support multi-event files, including navigation
-between events. You should always provide a geometry file on the command line,
-otherwise it won't know how to interpret the contents of your HDF5 file.
-
-A new tool in CrystFEL 0.6.0, list_events, is provided to simplify the process
-of creating lists of individual events. However, most users will simply be
-able to list the multi-event filenames themselves in the input to indexamajig,
-which will then process all of the events in the file. See "man indexamajig"
-and "man list_events" for more details.
-
-Finally, it may be necessary to update your check-near-bragg and
-check-peak-detection scripts. Simply make a fresh copy from the CrystFEL
-"scripts" folder or download them from the website.
-
-
-geoptimiser: a tool for optimising the detector geometry
---------------------------------------------------------
-
-CrystFEL 0.6.0 introduces a new tool, geoptimiser, for optimising the detector
-geometry. You simply give it a stream containing indexing results obtained with
-the approximate geometry, and it gives you a refined geometry. It can refine
-panel translations, rotations and camera lengths.
-
-To use geoptimiser, you need to add some extra information about the
-mechanical construction of your detector. This is used to constrain the
-refinement appropriately, for example to ensure that panels which share sensor
-silicon do not move relative to one another. If you're processing CSPAD data,
-you can simply copy all the rigid_group lines from one of the CSPAD geometry
-file examples (folder doc/examples) into your own geometry file.
-
-Refer to "man geoptimiser" and "man crystfel_geometry" for more information.
-
-
-Removal of beam files / automatic determination of spot prediction parameters
------------------------------------------------------------------------------
-
-As of version 0.6.0, "beam files" have been removed completely. This should
-simplify usage for most people and remove a lot of ambiguity. Programs which
-need X-ray beam parameters, such as pattern_sim, now have additional command-
--line arguments to provide them. indexamajig now determines these parameters
-automatically for the purposes of spot prediction.
-
-We are interested in feedback about the automatic spot prediction parameter
-determination. If it appears not to work well for you, you can restore the old
-behaviour by using the new command-line options for indexamajig:
---fix-profile-radius, --fix-bandwidth and --fix-divergence. Simply set these
-parameters to the values you had in your old beam file.
-
-The photon energy, or information about where to get it, now needs to be in the
-geometry file. This can be done with a line like this:
-
- photon_energy = /LCLS/photon_energy_eV
-
-or, for a fixed value:
-
- photon_energy = 8300
-
-
-whirligig: a tool for finding "mini rotation series"
-----------------------------------------------------
-
-CrystFEL 0.6.0 introduces yet another tool, whirligig, which can be used for
-locating "mini rotation series" in the output from indexamajig. This might be
-used to perform an experiment similar to that described by Gati et al., IUCrJ
-1 (2014) p87. In this initial version, whirligig finds runs of consecutive
-frames which contain crystals in similar orientations. It writes the
-corresponding filenames and event/crystal identifiers to log files, which might
-be useful for further analysis.
-
-This is the program described by Nogly et al., IUCrJ 2 (2015). Refer to "man
-whirligig" for usage information.
-
-
-Introduction of CrystFEL unit cell files
-----------------------------------------
-
-CrystFEL offers you the ability to index patterns using Bravais lattice
-information but without unit cell parameters, for example: "index these using
-only tetragonal primitive lattices, but any parameters". However, it's awkward
-to give this information using a PDB file: you essentially have to "trick" it
-into interpreting the parameters correctly, then throw away the parameter
-information.
-
-Version 0.6.0 of CrystFEL introduces a new way of specifying unit cell
-information. These new unit cell files look like this:
-
- CrystFEL unit cell file version 1.0
-
- lattice_type = cubic
- centering = I
-
- a = 66.2 A
- b = 66.2 A
- c = 66.2 A
-
- al = 90.0 deg
- be = 90.0 deg
- ga = 90.0 deg
-
-In the event that you want to specify the lattice type information alone, you
-can simply omit the cell parameters:
-
- CrystFEL unit cell file version 1.0
-
- lattice_type = tetragonal
- centering = P
- unique_axis = c
-
-Note that a unique axis must be specified for all types of cell where this makes
-sense, and can be omitted otherwise. This was done in the first example above,
-which is cubic and therefore has no unique axis.
-
-You can, of course, continue to use PDB files just like before.
-
-
-API changes
------------
-
-The following changes have been made to the libcrystfel API:
-
-New functions:
- - The event API (see libcrystfel/src/events.h)
- - load_cell_from_file()
- - cell_has_parameters()
- - find_orig_panel()
- - crystal_{get,set}_num_implausible_reflections() were added
- - panel_is_in_rigid_group()
- - rigid_group_is_in_collection()
- - single_panel_data_source()
- - find_rigid_group_collection_by_name()
- - write_detector_geometry_2()
- - find_intersections_to_res()
- - sphere_fraction()
- - gaussian_fraction()
- - hdf5_read2()
- - check_path_existence()
- - get_peaks_cxi()
- - hdfile_get_value()
- - fill_event_list()
- - image_reflection_closest()
- - intmat_identity()
- - extract_f_from_stuff()
-
-Removed functions:
- - cell_set_cartesian_{a,b,c}()
- - cell_{get,set}_pointgroup()
- - twod_mapping()
- - get_value()
-
-Changed function prototypes:
- - record_image()
- - get_detector_geometry()
- - fill_in_values()
- - write_detector_geometry()
- - hdf5_write_image()
- - hdf5_read()
- - hdfile_set_image()
- - hdfile_get_string_value()
- - {get,set}_partial()
- - write_chunk()
- - prepare_indexing() and {dirax,mosflm,reax,grainspotter,xds}_prepare()
-
-beam-parameters.h was removed, and "struct beam_params" is now defined in
-image.h.
diff --git a/relnotes-0.6.1 b/relnotes-0.6.1
new file mode 100644
index 00000000..4c189cd6
--- /dev/null
+++ b/relnotes-0.6.1
@@ -0,0 +1,165 @@
+CrystFEL - Crystallography with a FEL
+-------------------------------------
+
+Release notes for version 0.6.1
+
+Copyright © 2012-2015 Deutsches Elektronen-Synchrotron DESY,
+ a research centre of the Helmholtz Association.
+
+Authors:
+ Thomas White <taw@physics.org>
+ Richard Kirian <rkirian@asu.edu>
+ Kenneth Beyerlein <kenneth.beyerlein@desy.de>
+ Andrew Aquila <andrew.aquila@cfel.de>
+ Andrew Martin <andrew.martin@desy.de>
+ Lorenzo Galli <lorenzo.galli@desy.de>
+ Chun Hong Yoon <chun.hong.yoon@desy.de>
+ Kenneth Beyerlein <kenneth.beyerlein@desy.de>
+ Karol Nass <karol.nass@desy.de>
+ Nadia Zatsepin <nadia.zatsepin@asu.edu>
+ Anton Barty <anton.barty@desy.de>
+ Cornelius Gati <cornelius.gati@desy.de>
+ Fedor Chervinskii <fedor.chervinskii@gmail.com>
+ Alexandra Tolstikova <alexandra.tolstikova@desy.de>
+ Wolfgang Brehm <wolfgang.brehm@gmail.com>
+ Valerio Mariani <valerio.mariani@desy.de>
+ Parker de Waal <Parker.deWaal@vai.org>
+ Takanori Nakane <nakane.t@gmail.com>
+ Keitaro Yamashita <k.yamashita@spring8.or.jp>
+ Oleksandr Yefanov <oleksandr.yefanov@cfel.de>
+
+CrystFEL is free software: you can redistribute it and/or modify it under the
+terms of the GNU General Public License as published by the Free Software
+Foundation, either version 3 of the License, or (at your option) any later
+version.
+
+CrystFEL is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along with
+CrystFEL. If not, see <http://www.gnu.org/licenses/>.
+
+
+Overview
+--------
+
+The most important new features in this version of CrystFEL are:
+
+- Introduction of "prediction refinement".
+
+- Scaling and post-refinement with "partialator".
+
+- New indexing methods: mosflm-cell and asdf.
+
+- process_hkl --even-only and --odd-only instead of "alternate-stream".
+
+These new features have individual sections below. In addition, there are many
+other new developments. See the ChangeLog or the changes page on the website
+for more details. There were, of course, the usual large number of smaller
+refinements and bug fixes.
+
+
+Introduction of "prediction refinement"
+---------------------------------------
+
+A new refinement step has been introduced between the indexing and prediction/
+integration stages of indexamajig. The refinement aims to maximise the
+agreement between the predicted spot positions and the peak positions found by
+the peak search step, at the same time as putting the peaks as close to the
+Bragg condition as possible. This increases the accuracy of spot prediction,
+which has positive effects on the downstream processing stages.
+
+The prediction refinement stage allows the central beam position to move
+relative to the entire detector. The required offset for each crystal is
+recorded in the stream and can be visualised using scripts/detector-shift.
+
+If, somehow, the prediction refinement causes problems for you, you can disable
+it using the indexamajig option "--no-refine".
+
+
+Scaling and post-refinement with "partialator"
+----------------------------------------------
+
+The scaling and post-refinement program partialator is no longer considered
+experimental in this version. Many users are already finding significant
+improvements in their data simply by applying scaling without partiality:
+
+$ partialator -i my.stream -o my.hkl -y mypg --model=unity --iterations=3
+
+A further improvement is sometimes obtained by applying partialities and doing
+full post-refinement:
+
+$ partialator -i my.stream -o my.hkl -y mypg --model=scsphere --iterations=3
+
+Note that partialator applies a per-crystal resolution cutoff to stabilise the
+scaling calculation. Therefore, you should experiment with different values
+for "--push-res".
+
+Further improvements to partiality modelling, scaling and merging will be
+included in the near future.
+
+
+New indexing methods
+--------------------
+
+This version of CrystFEL can make use of the known unit cell algorithm included
+with very recent (7.2.0 or later) versions of mosflm. The known unit cell
+algorithm is used by default if you provide lattice parameters to indexamajig.
+If this isn't what you want, or if it doesn't work well for you, use
+"mosflm-nocell" as your indexing method to restore the old behaviour.
+
+A new indexing method, asdf, has been added. This algorithm is built into
+CrystFEL directly, so no temporary files or external programs are needed.
+
+The old indexing methods "grainspotter" and "reax" have been removed.
+
+
+process_hkl --even-only and --odd-only
+--------------------------------------
+
+When using process_hkl for merging, you no longer need to use "alternate-stream"
+to split the stream into two parts in order to calculate figures of merit such
+as Rsplit. Instead, simply use the "--even-only" and "--odd-only" options to
+merge only even-numbered and odd-numbered crystals from the stream:
+
+$ process_hkl -i my.stream -o my.hkl1 -y mypg --even-only
+$ process_hkl -i my.stream -o my.hkl2 -y mypg --odd-only
+$ compare_hkl my.hkl1 my.hkl2 -y mypg --fom=rsplit [..etc..]
+
+Note that the numbering of crystals for this purpose is simply according to the
+order they appear in the stream. For most purposes, this is effectively random.
+
+Note further that partialator writes the ".hkl1" and ".hkl2" split-merged
+datasets for you automatically.
+
+
+API changes
+-----------
+
+The following changes have been made to the libcrystfel API:
+
+New functions:
+ - asdf_prepare(), run_asdf(), asdf_cleanup()
+ - cell_get_volume()
+ - crystal_{get,set}_Bfac()
+ - crystal_{get,set,add}_notes()
+ - detector_has_clen_references()
+ - {x,y,r}_gradient()
+ - refine_prediction()
+ - refine_radius()
+ - {get,set}_panel()
+ - {get,set}_flag()
+ - solve_svd()
+
+Removed functions:
+ - select_intersections()
+ - update_partialities_2()
+ - get_excitation_error()
+ - extract_f_from_stuff()
+ - grainspotter_{prepare,index,cleanup}()
+ - reax_{prepare,index,cleanup}()
+
+Changed function prototypes:
+ - integrate_all_4()
+ - set_detector_pos()