nuisance is hosted by Hepforge, IPPP Durham
NUISANCE

Fitting generators to data with nuismin

Overview

nuismin expands on nuiscomp (documented here) by introducing the ability to minimise the test-statistic (i.e. the 𝜒2) by varying generator parameters (e.g. the axial mass for CCQE interactions, MAQE) and aspects of the data (e.g. normalisations). We recommend reading nuiscomp documentation and trying comparisons with nuiscomp before running fits with nuismin.

For the code-inclined, nuismin lives in nuisance/app/nuismin.cxx and is driven by nuisance/src/Routines/MinimizerRoutines.cxx. You can view it here.

This expands on the documentation provided by running

$ ./nuismin

which is equivalent to

$ ./nuismin -h


An example nuismin run

This example compares GENIE events to MiniBooNE data, just like in nuiscomp, and now fitting for MAQE. Again, the example is near identical for a different generator, you just have to Prepare appropriately and replace GENIE in the cardfile with your generator descriptor, e.g. NEUT, NUWRO, GIBUU. You also need to find the parameter in the generator you want to fit and replace the parameter in the card file appropriately, described below.

Requirements for the example:

Make a plain file mbcomp_fit.card and fill it with:

<nuisance> <sample name="MiniBooNE_CCQE_XSec_1DQ2_nu" input="GENIE:PATH_TO_PREPARED_GENIE_EVENTS.root"/> <parameter name="MaCCQE" nominal="0" type="genie_parameter" state="FREE"/> </nuisance>

sample name specifies the measurement to compare to: in this case MiniBooNE CCQE cross-section in 1 dimensional in Q2 for neutrino.
input specifies the input generator and the path to the prepared file containing the events: in this case GENIE and our file is PATH_TO_PREPARE_GENIE_EVENTS.root
parameter name specifies the reweighting parameter as a string, where the strings (e.g. MaNCEL, MaCCQE) are the parameter names read in by NUISANCE, see here
nominal is the parameter value which the minimisation starts at, measured in units of 1 sigma uncertainty, unless specified in state
type is the parameter type, in this case genie_parameter (N.B. for NEUT ReWeight the type is instead neut_parameter)
state is the state of the parameter which is specifies if the parameters is fixed or free in the fit, and what the units of nominal is.
  • For a free parameter, use state="FREE"
  • For a fixed parameter use state="FIX"
See here for the nuiscomp documentation on state="ABS" and state="FRAC".

You can now run nuismin with this card file by doing

$ ./nuismin -c mbcomp_fit.card -o myoutput.root

where -c flags preceeds the card file and -o preceeds the output file nuismin will write to. nuismin minimisation takes considerably longer than nuiscomp since it does a large number of reconfigures of the generated events.


Specifying a measurement

Follows exactly the same as nuiscomp.

Options for a measurement

In nuiscomp we introduced the option of scaling an experiment using the norm keyword here. In nuismin the sample normalisation can be made a fitting parameter (with or without penalties) using the same syntax, provided type="FREE" is specified.

You can also change how the sample is treated statistically, if any corrections or masking should be applied, how it should be scaled, and so on. These can have dramatic effects on the minimisation and are documented here.

As an example of the impact of using a covariance matrix, we use a card file with

MINERvA_CC0pi_XSec_1DQ2QE_nu_norm = 4.89272 +- 0.911015 compared to a card file where we only use the diagonal errors
MINERvA_CC0pi_XSec_1DQ2QE_nu_norm = 1.21195 +- 0.0335652

Specifying a generator

To change the generator you first need to have generated events with said generator and linked NUISANCE to the generator. You can find notes on this here. You also need to have prepared generator files for NUISANCE use.

Once you have NUISANCE linked and prepared files you simply change the input field to match your generator (GENIE, NEUT, NUWRO, GIBUU or NUANCE), and new file location. For example input="NUWRO:PATH_TO_PREPARED_NUWRO_FILE.root".


Changing systematics via event reweighting in nuismin

The options for nuismin are again expanded from those in nuiscomp. Please see here for the introduction and basic syntax.

Recalling the mbcomp_fit.card from above, we set

<nuisance> <sample name="MiniBooNE_CCQE_XSec_1DQ2_nu" input="GENIE:PATH_TO_PREPARED_GENIE_EVENTS.root"/> <parameter name="MaCCQE" nominal="1.0" type="genie_parameter"/> </nuisance>

If no state is given for a systematic it is assumed to be fixed at the nominal value.

Making fake-data via event reweighting

Fake-data can be provided to nuismin in two ways:

  • Giving the -d parameter to nuismin (outlined later)
  • Specifying a fakeparameter field
The fakeparameter must be a reweighting parameter already present in the card file, and follows the exact structure, specifying the name and nominal. The nominal is the value with which the fakedata is generated. The fake-data becomes the data histogram in NUISANCE, which is used to calculate e.g. the goodness of fit.

A card file for MiniBooNE CCQE in 1D Q2 for neutrino, setting the fake-data to be GENIE with MaCCQE set to 1 sigma above the default, would be

<nuisance> < sample name="MiniBooNE_CCQE_XSec_1DQ2_nu" input="GENIE:PATH_TO_PREPARED_GENIE_EVENTS.root" /> < parameter name="MaCCQE" nominal="1.0" type="genie_parameter" /> < fakeparameter name="MaCCQE" nominal="1.0" type="genie_parameter" /> </nuisance>


Running nuismin

nuismin takes a number of command line arguments, one of which is required. It requires either

  • -c mycard.card where mycard.card is the provided card file (see above)
  • -i 'card file contents'

The -i option follows the same structure as a card file but is instead an argument. For example

$ ./nuismin -o output.root -i 'sample name="MiniBooNE_CCQE_XSec_1DQ2_nu" input="GENIE:PATH_TO_PREPARED_GENIE_EVENTS.root"'

would acheive the same effect as providing nuismin with the simple mbcomp.card earlier.

The complete list of arguments and their types is:

  • -c string: the cardfile to read
  • -o string: the output file to create (default: append ".root" to cardfile name)
  • -n int: the number of events to run on (default: all)
  • -d string: fakedata input, can be either (default: empty)
    • "MC": for setting the data to the nominal MC prediction
    • "fakedata.root": Direct path to root file containing a MC histogram called the measurement name with "_MC" at the end (the default format for NUISANCE MC histograms)
  • -f string: Fitting routines
    • Study methods:
    • FixAtLim:
    • FixAtLimBreak:
    • ErrorBands:
    • DataToys:
    • Chi2Scan1D:
    • Chi2Scan2D:

    • Fitting methods:
    • Migrad: (Minuit2, Migrad)
    • Simplex: (Minuit2, Simplex)
    • Combined: (Minuit2, Combined)
    • Brute: (Minuit2, Scan)
    • Fumili: (Minuit2, Fumili)
    • ConjugateFR: (GSLMultiMin, ConjugateFR)
    • ConjugatePR: (GSLMultiMin, ConjugatePR)
    • BFGS: (GSLMultiMin, BFGS)
    • BFGS2: (GSLMultiMin, BFGS2)
    • SteepDesc: (GSLMultiMin, SteepestDescent)
    • GSLSimAn: (GSLSimAn)
    • MCMC

    • Debugging:
    • LowStat: Can be included to routines to run with low statistics (testing or debugging mode), e.g. LowStatMigrad or LowStatChi2Scan1D
  • -i string: card file-like XML commands (default: empty)
  • -q string: config arguments, which overwrite nuisance/parameters/config.xml (default: empty)
  • e +/- error level relative that in nuisance/parameters/config.xml
  • v +/- verbosity level relative that in nuisance/parameters/config.xml

Running nuismin as

$ ./nuismin -c mycard.card -o myoutput.root

will read the card file mycard.card and produce the output ROOT file myoutput.root.

NUISANCE produces various output as it sets up, using different verbosity logging levels, defined in nuisance/parameters/config.xml.


Inspecintg nuismin output

During minimisation NUISANCE prints the state of the fit to screen, e.g.
JointFCN.cxx::ReconfigureFastUsingManager[l. 762] : [ERR WARN ] : Signal Flags Empty! Using normal manager. [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 495] : Event Manager Reconfigure [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 0 events. [M, W] = [13, 1] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 260000 events. [M, W] = [26, 1] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 520000 events. [M, W] = [26, 1] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 780000 events. [M, W] = [1, 1.11376] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 1040000 events. [M, W] = [46, 1] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 1300000 events. [M, W] = [13, 1] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 1560000 events. [M, W] = [26, 1] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 1820000 events. [M, W] = [26, 1] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 2080000 events. [M, W] = [1, 1.11376] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 574] : MINERvA_CC0pi_XSec_2Dptpz_nu : Processed 2340000 events. [M, W] = [46, 1] [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 700] : Filled 576316 signal events. [LOG Reconf]:--- : JointFCN.cxx::ReconfigureUsingManager[l. 718] : Time taken ReconfigureUsingManager() : 340 [LOG Minmzr]:- : JointFCN.cxx::ReconfigureSamples[l. 430] : Finished Reconfigure iter. 161 in 340s [LOG Minmzr]:- : JointFCN.cxx::GetLikelihood[l. 294] : Getting likelihoods... : -2logL [LOG Minmzr]:- : JointFCN.cxx::GetLikelihood[l. 311] : -> MINERvA_CC0pi_XSec_2Dptpz_nu : 1165.21/144 [LOG Minmzr]:- : JointFCN.cxx::GetLikelihood[l. 311] : -> MINERvA_CC0pi_XSec_1DQ2QE_nu : 182.714/16 [LOG Fitter]: : JointFCN.cxx::DoEval[l. 235] : Current Stat (iter. 162) = 1347.92 [LOG Reconf]:--- : FitWeight.cxx::Print[l. 283] : Fit Weight State: [LOG Reconf]:--- : FitWeight.cxx::Print[l. 285] : -> Par 0. ZExpA1CCQE -0.209961 [LOG Reconf]:--- : FitWeight.cxx::Print[l. 285] : -> Par 1. ZExpA2CCQE -6.90742 [LOG Reconf]:--- : FitWeight.cxx::Print[l. 285] : -> Par 2. ZExpA3CCQE 0.518002 [LOG Reconf]:--- : FitWeight.cxx::Print[l. 285] : -> Par 3. ZExpA4CCQE -9.99773 [LOG Reconf]:--- : JointFCN.cxx::ReconfigureSamples[l. 393] : Starting Reconfigure iter. 162 [LOG Fitter]: : JointFCN.cxx::ReconfigureFastUsingManager[l. 749] : -> Doing FAST using manager