This patch includes fixes to : 1) substantially increase the efficiency of summing the histograms when using the GPU; 2) Add validation the reweighting and check that caching used by CPU calculation correctly updates; and, 3) add a collection of quality of life fixes (particularly, fewer warnings during compilation).

Fixes relative to 1.8.3

Issue #545 : Make the handling of ROOT config information portable between linux and macos. This no longer assumes how ROOT is installed, and uses root-config safely.

Update (Related to #540): Strengthen the NaN trap in the JointProbability calculation. An NaN there means the fit cannot continue, so throw an error. Continue to allow INF.

Update (Prompted by #541): Make compilation less alarming by quieting warnings. They were originally part of debugging.

Issue #536 : Add control over the size of a "kick" when starting a fit. The starting point can be fluctuated around the prior, and the size of the step is controlled by an optional argument to --kickmc

Issue #534 : Backport CPU and GPU calculation backing from main, and add validation that the two calculations agree within machine precision.

Issue #530 : Fix an inefficiency in the summation of the event weights on the GPU and double the speed of the likelihood calculation using the GPU.

Issue # 524 : Apply the global event weight cap when using the Cache::Manager to calculate the likelihood.

Full Changelog: 1.8.3...1.8.4

This patch includes fixes to : 1) Stop output of disabled parameters; 2) fix parameters using the configuration file; 3) reduce the number of compilation warnings with CUDA; and, 4) substantially increase the efficiency with a GPU.

Fixes relative to 1.8.2

Associated Fix: The CalculateGeneralSpline and CalculateGraph functions have been changed to use a brute-force binary search in place of the linear search for the correct index. This results in a speed up of about 50% for those routines.

Issue #510 : Make the CacheManager calculation much more thread save when being run on the CPU. This isn't the normal mode, but is an important cross check. The CPU now uses (lock free) atomic addition, multiplication and value setting.

Issue #513 : Fix JointProbability so that an infinite log likelihood produces a warning, but does not terminate the program. Infinities are valid (e.g. a zero probability). This does produce a warning since they shouldn't occur often during a normal run.

Issue #506 : Fix output statement in ParameterSet.cpp to remove duplicated output.

Issue #505 : Add the "isFixed" option for parameters in the configuration file.

Issue #508 : Improve error checking for Cache::Manager. This makes sure that LogThrow is preferred to std::runtime_error. There is an issue filed for simple-cpp-logger to make sure that the output is flushed before throwing, so that should make the error output much more readable.

Issue #502 : Update the validation code. A validation failure caused by the recent corrections to the monotonic spline calculation (#486 and #494) has been fixed. The expected result has been updated. Unit testing with GoogleTest has been added and used for the HEMI GPU interface code. Further tests are being implemented.

Issue #492 : Fix compiler warnings from NVCC. This is removing some unused variables. It also fixes a "fix" that applied some correct C++ conventions to code that is aimed at the GPU (the fix produces inefficient code on a SIMD processor, and was primarily aesthetic.

Issue #490 : Don't set the value for disabled parameters in the output histograms. This will prevent uninitialized values (i.e. NAN) from appearing in plots.

Full Changelog: 1.8.2...1.8.3

Fixes relative to 1.8.1

Issue #499 : Update to a new version of the cpp-generic-toolkit submodule. The cpp-generic-toolkit fix sets the branch status to enable branches that are used for the data.

Issue #494 : Rearrange the application of the Fritschle-Carlson criteria so that it can be tested. This also applies a fix to Fritschle-Carlson so that the end of the spline are handled in a more reasonable fashion.

Issue #493 : Add tests for ComputeGeneralSplines to the validation suite. As a special bonus it also adds tests for CalculateUniformSpline.h, and resolves unused variable warnings for both those functions.

Summary

• Resolve LTS 1.8.x issue493: Add general spline test by @ClarkMcGrew in #495
• Resolve LTS/1.8.x issue 494: Check spline monotonicity by @ClarkMcGrew in #496
• Resolve LTS/Issue499: Set the status of branches by @ClarkMcGrew in #500

Full Changelog: 1.8.1...1.8.2

Fixes relative to 1.8.0

Issue #485 : A job will correctly continue when the likelihood returns an infinite value. The job will stop if the likelihood returns a NaN.

Issue #486 : Fix the Catmull-Rom splines so that they have symmetric behavior. This changes the extrapolation behavior for both Catmull-Rom and Catmull-Rom,monotonic splines. The Catmull-Rom monotonic splines are updated to use the full Fritsche-Carlson criteria so that the interpolation is smoother.

What's Changed

• Fix GUNDAM 1.8.x Issue #486: Catmull Rom Extrapolation by @ClarkMcGrew in #487
• GUNDAM 1.8.0: Fix for issue #485: LLH is throwing when there is a Inf (should continue) by @ClarkMcGrew in #488

Full Changelog: 1.8.0...1.8.1

This is a development, and cannot be considered validated

Improving class structure and better defining herarchy and ownership (see figure bellow). This is primarily a development release and will not receive long term support.

As usual, backward compatibility of config files is ensured! This version ended up around twice faster than the previous one. 😄

Additions

• Adding multiparameter dials
• Adding Formula and precompiled dials
• Adding app to compare fit results (sensitivity studies for instance): gundamFitPlot
• Not throwing an error when a NaN values is propagated in the reweight pipeline.
  • Migrads might explore these regions while adjusting the step size
• Using ordered JSON unfolding
• CMake option DISABLE_ZLIB to compile without Zlib (this library is only used to make the propagator cache faster)

Fixes

• Fixed gundamRoot with latest versions of ROOT
• Fixing event-by-event dial loading with multiple datasets
• update.sh script now properly updates the submodules

Full changelog of 1.8.0-> 1.9.0

The 1.8 series has been designated for long term support, and will be patched as bugs are found, but new feature are going to be avoided.

Additions

• Every event variable can now be custom defined thanks to variableDict
• Adding dialIndexFormula in order to allow the user to select a given entry of a TClonesArray for event-by-event dials
• Adding branch commit number to the displayed version displayed and stored by GUNDAM apps
• Adding globalEventReweightCap to apply a maximum value to the weight an event could get.
• Adding commit hash + git branch on GUNDAM version printout when not on tagged version
• gundamCalcXsec can now be ran on priors covariance matrix
• gundamCalcXsec support arbitrary binning
• Better multithread support with proper signal wait
• Changed pbar to display GUNDAM intead of T2K since it is beeing used outside of T2K
• Additional prinouts for breaking down parameters and sample before the fit

Fixes

• Various misleading printouts
• Better CMake support
• Proper handling of nan values in JSON parsing for old versions
• Optimized data loader

Full changelog of 1.7.2-> 1.8.0

Reintroduction of the CalcXsec!

• gundamCalcXsec is now working with the new dial/cache system of the propagator introduced since 1.7.0
• Now supporting parameter dependent renomalizations (like flux renormalisation)
• Don't load the fit samples in the xsection calc propagator (way faster)
• Use config override feature to tweak the parameter definitions used during the fit

Fixes

• CacheManager is now running with the new dial/cache system (i.e. CUDA is back!)
• Now working with ROOT v6.28.04+
• Now cmake checks if the submodules are present
• Full synced event/dial cache sorting to insure reproducibility of toy throws and fit runs

Additions

• ./update.sh is a magical script that will help you update the code and its submodule effortlessly 🙃
• Updating doc

/!\ Submodules have been moved to the gundam-organization for better support.

Some old version of git might freak out while updating, so if you encounter some troubles while recompiling
(GenericToolbox::... could not find function for instance), you can either try to update manually the submodules
or start back from scratch: i.e.:

git clone https://github.com/gundam-organization/gundam.git
cd gundam
./update.sh --latest

Full changelog of 1.7.1-> 1.7.2

Fixes

• Fixing logger issue while building with C++14
• Fixing ParScan config loading ending up in a segfault

Additions

• Systematically check for GUNDAM version while building
• Adding gundamFitReader app that allows to printout the results of a fit using the fitter output file
• Adding gundamInputZipper app that allows to gather configs and resource file in a local place, and zip it. This allows to pass a completely portable set of inputs for another user who wants to run the fit.
• Adding override config feature that allows to edit a main configuration: gundamFitter -c config
• Adding LineScanner feature that allows to monitor the LLH response on a straight line in the parameter space
• Adding convergence monitor that draws the path taken by Minuit to find the minimum

Full changelog of 1.7.0-> 1.7.1

This is the first release of GUNDAM using the new Dial implementation. This is the first version that is expected to be feature complete for the T2K 2024 near detector fit.

What has not changed

1. Previous GUNDAM configurations should continue to work without modification. You can start using new features when you need them.

What's new

1. The event weight calculation has been moved to a new dial interface that requires less memory, and better optimized. The new dial interface is designed so that it will (eventually) support multi-dimensional dials, and other ways of doing reweighting.
   1. The new dial infrastructure allows multiple types of cubic splines to be used. The supported splines types are "not-a-knot" (equivalent to the default TSpline3), "natural", and Catmull-Rom (only for uniformly spaced knots).
   2. All types of splines can be constrained to be monotonic. A monotonic spline will be monotonic at all points except for "extrema" located at a knot. The slope at an extrema is zero, and the first derivative is continuous.
2. The fitter engine infrastructure has been improved to better support different types of "fitters". There are currently interfaces for the standard MINUIT based fitter, and an MCMC tool to estimate the posterior probability distribution.
3. Infrastructure to determine the P-Value of a fit result.
4. A testing infrastructure to facilitate continuous integration and validation.
5. Injection of specific parameter values and calculate the likelihood so that the likelihood can be compared between different fits and fitters.
6. The individual event weights as a function of each parameter can be saved in the output file for model validation.
7. Adding new tools gundamFitInfo that read the ouput file of a fit. It reads and display information on the runtime
8. Adding parameter injector feature that allows to import(/export) a given set of parameters in the Propagator
9. Adding line scanner feature in which to can scan the LLH response a long a straight line between two points
10. More info are saved in the output root file of the fitter
11. gundamFitCompare can now show the position of the values on the Scan plots

What's next

The next point release will probably be 1.7.1 and is expected to include

1. Updates to the GPU calculation for the new dial interface.
2. Continuously improving documentation.

Full changelog of 1.6.1-> 1.7.0

** last version before new dial library **

What's new?

• First working version of gundamCalcXsec
• Adding GenericToolbox and Logger in gundamRoot
• Adding option for validation against BANFF: devUseParLimitsOnEigen
• Fixing crashes with plot generator

Full changelog of 1.6.0-> 1.6.1