New GenX Publication

Artur Glavic — Tue, 02 Aug 2022 15:50:21 -0000

Hi everyone,

I'm happy to announce that a new paper has been published about GenX 3.6 that is available open access at https://doi.org/10.1107/S1600576722006653 . I would appreciate if you would cite this reference for any future publications based on GenX 3:
A. Glavic and M. Björck J. Appl. Cryst. 55, 1063-1071 (2022)

In addition to summing up the important changes since the original GenX publication, the paper gives an introduction to possible corrections for experimental errors, beyond counting statistics, that can be the basis for a better statistic analysis of parameter uncertainties.
I have also created a YouTube channel where I link existing Tutorial videos and will guide you through the GenX documentation with individual future videos. ( https://www.youtube.com/user/knuekopp666 )

In other news there have been further developments to GenX since the last newsletter that I'd like to share with you:

The GenX build script now includes the generation of Mac OS binaries that are reported to work on intel and even more recent hardware. A snap build for various Linux distributions is available, too.
Option to download and update new versions (tested on Windows, experimental otherwise)
genx3server PyPI package to install without the need for wx GUI components. New genx_server command that can be used to refine models remotely while running a GUI locally (experimental)
Batch processing interface that allows to fit a sequence of datasets to the same model, header information from ORSO files can be used to plot parameter values against e.g. temperature or field values
Neutron polarization analysis function that takes polarization efficiency into account
Generic definition of plot units for x- and y-axes that can be specified by the user for custom models
Edit model script in external Editor
Fix fitting without boundaries and show graphical indication of active/inactive fit boundaries in grid
Generate SLD uncertainty graph from fit parameter errors
Customization of display of SLD graphs to make it more accessible
Improved handling of copy/paste in the parameter grid
Support for the final ORSO .ort data format specification, see https://www.reflectometry.org/file_format/specification
Dialog to show and edit metadata information based on ORSO header standard
Add a data loader for the PSI SINQ six text and Rigaku .ras format
Modify plot scales with mouse wheel (use shift/ctrl/alt for fine and horizontal adjustment)
New option (also from command line) to stop the fit when the relative parameter variation is below a threshold value
An option in the spec_inhom model to simulate neutron supermirrors and to hide layers from the SLD profile to make larger number of layers more readable/increase plot performance.
Open model on drag&drop
Updates to the documentation
Tech preview of alternative plotting backend with better performance but limited quality
Many bug fixes

You can't undo GenX 3.5, but it can undo your mistakes

Artur Glavic — Tue, 23 Nov 2021 00:46:34 -0000

Hi everyone,

as an early Christmas present of this year I have a new major revision of GenX with a bag full of goodies.

One feature that many may have longed for since the start of GenX is an undo/redo functionality. With this you can revert changes to the model script as well as single and multiple parameters. Actions can even be reverted retroactively without touching other changes you did later in a history dialog.
Update of several tutorials in the documentation and new tutorial for the usage of bumps for parameter uncertainty estimation.
Option to start logging to a file from within the GUI to simplify bug reporting.
Updated ORSO .ort format (now requires orsopy package) and new feature to build a model from .ort file meta data. (Also from exported .ort files, directly.)
Better feedback on startup progress, especially first time start when compiling numba functions.
For all Changes, see the README.txt file.

As always, let me know if you encounter any bugs or weird behavior.

GenX 3.4 is ready to rock

Artur Glavic — Wed, 29 Sep 2021 17:50:01 -0000

This summer brought a cascade of updates to your favorite fitting program. After a few hiccups due to major changes in version 3.4.0 this series has now reached a stable state again. This is a good occasion to inform you of the goodies I've been adding in the last four months moving to version 3.3 and 3.4:

The documentation has been updated with more videos and tutorials, including the SimpleReflectivity interface that was introduced in 3.0
Models for surface x-ray and off-specular scattering have been re-implemented and should make GenX 3 a full replacement for the older program
Improved data loaders with an automatic selection of suitable plug-ins (default, resolution, sns_mr, amor, d17_cosmos and orso)
Fittable "systematic error" parameters that include a penalty factor to the FOM.
Additional optimizer options (fast Levenberg-Marquardt and Bumps Marcov-Chain Monte-Carlo)
GUI improvements, including drag&drop of data-files onto the dataset list and new dialog to create plots with specific designs for publication
Significant performance improvements, especially for simple models
Many bug fixes

As always, further implementation of new features will depend on the amount of spare time I'll have in the near future. Please feel free to submit tickets with suggestions. Currently I have three goals for the near and more distant future:

Writing a new publication for GenX 3 describing the background of all the changes made since the initial publication from 2007.
Try to implement an AI based "model guesser" for simple structures <5 layers (v3.5)
Full implementation of the new ORSO standard file format (.ort) and scattering length density database (v3.5)
Re-implement GenX GUI in a more future-proof and capable framework, Qt6 (v4.0) as I have found that wxPython is very hard to work with, unstable and slow. This will be a major rewrite of the code base and might not port all features.

GenX 3.2 is out

Artur Glavic — Thu, 20 May 2021 17:46:01 -0000

It's time for another major release of your favorite reflectometry package with a bunch of new goodies to play with. These are the main improvements in v 3.2:

A new simple scripting API that you can access by using from genx import api in your code. It includes nice IPython widget support for your Jupyter notebook. Have a look at the examples provided with the package. (The genx package has to be installed in your python environment or available in PYTHONPATH.)
A new dialog for statistical analysis of fit parameters that is available from the toolbar. It's based on the integration of GenX to the bumps package that can also be used from scripting for more elaborate analysis tasks.
Handling of meta data for imported datasets and an alpha version of exporting to the new ORSO text format .ort. For properly implemented data loaders the model now keeps this information with you datasets and exports it including the model script and parameters for full trackability. A new information button can be used to browse through that data.
A new auto data loader that selects the proper loading procedure by file suffix. Right now only Amor@PSI and MR@SNS as well as the default and resolution loaders are supported. If you want your instrument format to be supported, please write a similar data loader or send me example files with some format specifications.
Several improvements to the GUI as well as to calculation and plotting performance under the hood.

Some of these improvements required major changes to a variety of core GenX classes, please let me know if you notice bugs and unexpected behavior by writing a ticket.

GenX on PyPI

Artur Glavic — Tue, 27 Apr 2021 20:03:33 -0000

Thanks to a contribution by Leon Lohse from UNI Göttingen, the GenX build system was updated which allowed direct upload to the Python Package Index (PyPI)
https://pypi.org/project/genx3/

On an existing python environment with wxPython installed you can now use pip to install GenX as user or root:

pip3 install --user genx3
or
sudo pip3 install genx3

GenX to join ORSO analysis benchmarking

Artur Glavic — Tue, 16 Feb 2021 09:09:34 -0000

As all of you know, GenX is scientific software and thus correctness is as important as features and usability. To make sure there is quality control for the fundamental functions of x-ray and neutron reflectometry we have therefore joined the effort of the Open Reflectometry Standards Organization's (ORSO) effort of benchmarking various reflectometry suites agains a common set of standard systems.

Together with our friends from Refl1D, RefNX and BornAgain a validation workflow was added to the github action of ORSO here. For now the validation contains 6 examples of un-polarized neutron reflectometry with and without resolution convolution.

GenX 3.1 Performance Teaser

Artur Glavic — Sun, 23 Aug 2020 15:52:50 -0000

Hi Everybody,

the implementation of performance improvements for GenX 3.1 has made faster progress than expected. I was able to achieve sizable performance gains by making use of the Numba Just In Time (JIT) compiler package. While the gains strongly depend on the model, computer hardware and operation system they can reach 3-30x improvement over the normal, Numpy based, calculation. For those of you with Nvidia® graphics cards, implementation of CUDA GPU computing can add another factor of 2-4x on top of that.

See a compilation of gain factors for different configurations below, models use 4000 data points and 202 layers:

More details can be found in the raw data Raw Data. There are some clear trends observed:
1. Gains are always higher for larger datasets and number of layers, which is not surprising due to the general python overhead. In practice, systems with >1000 data points will likely see considerable benefits already. (This includes the extra points for resolution, so for example a 200 point datasets with 5 resolution points)
2. Gains in single threaded calculations are very different for Linux and Windows systems. (1.25x vs. 3.3x) This may be due to better optimization in Numpy on that platform. For the neutron matrix method used for spin-flip simulations the gains factor is always higher by ~2x.
3. In multi-threaded operation, all systems gain similar to their number of physical cores. Intel® multi-threading does not seem to be too effective here.
4. Gains from GPU computing require pretty large datasets (as expected, as the gain is realized through massive parallelization). Compared to multi-threaded CPU calculations there is a strong dependence on the computer hardware. For the test with a strong CPU with low powered Quadro® card the performance is actually lower on the GPU, but for balanced systems another 2.5-3.5x improvement was achieved. Both of my systems for testing have relatively good CPUs, so an average PC with dedicated graphics might see even bigger gains here. (Or fits on very large datasets >10k points.)

From these results I have decided to automatically use Numba JIT, if it is available. For GPU computing there will be an option to activate, but I haven't decided where to include it, yet.

For everyone eager to test these new features, please check out the "dev" branch on git. I'd be happy to get other test results ;-). Make sure that numba and nvidia cuda toolkit are installed on your system.
(run genx/tests/numba_performance.py)

GenX 3 released

Artur Glavic — Wed, 12 Aug 2020 20:49:07 -0000

Hi Everybody,

I'm happy to announce that, finally, the GenX version 3.0.0 is now released. This version is based on the lates libraries and has some nice improvements like the new SimpleReflectivity interface for beginners and simple models.

As a lot of the background code had to be modified to make the program work with the library changes there will probably be new bugs that I did not catch. Please report them through the Ticket system on SourceForge.

My plan is from now on to provide major revision updates, which will bring new features and improvements, 1-2 times per year and minor revisions with bug-fixes and smaller features on a continous rollout whenever issues can quickly be addressed. If you want to support the GenX development with some of your code or by updating the obsolete documentation, please feel free to contact me.

The plans for the next major revision 3.1.0 are currently:
1. Improved calculation performance. (The weave package used for compiled code before is discontinued, so currently only pure python is supported.)
2. Some refactoring of the GUI code to ade stability and fixing of bugs
3. Plugin to export models to BornAgain and maybe use BornAgain as calculation backend if it leads to significant performance gain
4. Adding some test cases following guidelines from ORSO
5. ORSO standard reflectometry fromat support, if defined up to the release date
6. Some user issues and feature requests

Almost there....

Artur Glavic — Sun, 19 Jan 2020 19:39:06 -0000

Happy New Decade GenX users!

The release of GenX 3 is getting closer. The first version of the SimpleReflectivity interface is implemented and builds for Linux and Windows 10 are running. If you want a preview of this exciting release and help me with hunting down some of the bugs, feel free to test the beta release that was just uploaded.

Rise of the Phoenix - GenX 3 is coming

Artur Glavic — Tue, 05 Nov 2019 20:50:27 -0000

Good news everybody! While there hasn't been an update on GenX for two years things are again starting to move forward. I want to thank Matts, who is unfortunately no longer able to provide support, for all his hard work he put into this project. I will take over the maintenance and development with new enthusiasm.

In the recent past the wxWidgets and python libraries behind GenX have all gone through many changes and soon python 2.7 will no longer be supported. Windows 10 has also led to incompatibilities with the binaries. You can expect a new major release update to version 3 within the next couple of months that focuses on transitioning to the wxPython 4 (Phoenix), python 3 and fixing associated bugs. But there is more to come with some additional functionality to keep you excited about this new version:

Overhaul of the user interface; better high DPI display support; alternative widget layout for larger screens
Tightened plot layout with correct x-/y-axis labels and image copy with white background
Improved materials database with online query options
A default simulation x-axis for all datasets that have no data loaded or simulation parameters set to avoid those "empty" plot windows
New plugin "parameter vault" to store different parameter sets of a model for 1:1 comparison in one plot
New plugin "Magnetic SLD" to display a separate magnetization axis in the SLD plot and calculate layer integrated magnetization for comparison with macroscopic measurements
Completely new simplified reflectometry user interface plugin to allow better access for new users. This interface will produce models compatible with the standard plugin but completely hide the underlying script interface behind an easy accessible layer table.

So I hope you are as excited as me about this new development. If you have small annoyances or new features that you would like me to include in this release, please write a ticket!

All the best,
Artur

Recent posts to news