diff options
Diffstat (limited to 'doc/articles/online.rst')
| -rw-r--r-- | doc/articles/online.rst | 85 |
1 files changed, 85 insertions, 0 deletions
diff --git a/doc/articles/online.rst b/doc/articles/online.rst new file mode 100644 index 00000000..84e037ca --- /dev/null +++ b/doc/articles/online.rst @@ -0,0 +1,85 @@ +========================================== +Real-time data processing with indexamajig +========================================== + +Instead of reading from files, indexamajig can get its data over a ZeroMQ +socket. Use ``--zmq-input`` instead of ``--input`` or ``-i``. An error will +be generated if you use ``--zmq-input`` and ``--input`` or ``-i`` +simultaneously. + +Indexamajig assumes that the socket is a *pub/sub* socket. You will also need +to specify which message prefixes to subscribe to. Use ``--zmq-subscribe`` for +this:: + + indexamajig --zmq-input=tcp://127.0.0.1:5002 \ + --zmq-subscribe=ondaframedata \ + -o output.stream -g Eiger.geom ... + +You can use ``--zmq-subscribe`` multiple times to subscribe to multiple message +prefixes. + +The option ``--no-image-data`` will be honoured, if given. This makes it +possible to quickly check streaming data for "indexability". + + +Data format +=========== + +In this version, CrystFEL makes the following assumptions about the data +received via ZeroMQ: + +* The data is serialised using MessagePack +* The data consists of either a single MsgPack 'map' object, or an array of + maps. + If there are multiple map objects in the array, only the first one will be + used. The others will be ignored. +* The image data is given as a two-dimensional array (i.e. no 3D arrays with + 'panel number' as one of the dimensions). +* The image data itself is given as a MsgPack 'map' object with "type"=>"<i4". + +Note that *all* of the above assumptions are 'open for negotiation' and will be +relaxed in future CrystFEL versions, as new online data formats arise. Anyone +interested in streaming CBF files over ZeroMQ? + +You can specify which map objects to look at in the geometry file. The +following example will get the incident photon energy (in eV) and detector +distance (in mm) by looking up the ``beam_energy`` and ``detector_distance`` +keys, respectively, in the MsgPack map object. It will then look up +``detector_data`` to find the image data itself. See the next section for an +explanation of ``peak_list``:: + + photon_energy = beam_energy eV + adu_per_photon = 1 + clen = detector_distance mm + res = 5814.0 + peak_list = peak_list + + thepanel/data = detector_data + thepanel/min_fs = 0 + thepanel/max_fs = 2067 + thepanel/min_ss = 0 + thepanel/max_ss = 2161 + thepanel/corner_x = -1034 + thepanel/corner_y = -1081 + thepanel/fs = x + thepanel/ss = y + + +Peak lists +========== + +You can use all of the usual peak search methods for streaming data. + +In addition, you can use ``--peaks=msgpack`` to get the peak locations from +the MsgPack data. In this case, the ``peak_list`` directive in the geometry +file specifies the key for the peak information in the MsgPack map object. +The peak information itself is expected to be a map object with three keys: +``fs``, ``ss`` and ``intensity``. Each of these keys should correspond to an +array containing (respectively) the fast scan and slow scan coordinates of each +peak, and their intensities. Obviously, the three arrays must have equal sizes. + +Note that there is no way, in this structure, to communicate which detector +panel contains a peak, in the case where different detector panels cover the +same pixel ranges (in this case, the pixel data would from multiple data +blocks). In practice, this means that the ``data`` directives for all panels +need to be the same when using ``--peaks=msgpack``. |