Changes between Version 3 and Version 4 of HowToUseNUISMIN-GENIE

Timestamp:

Jun 13, 2017, 3:47:33 PM (7 years ago)

Author:

Patrick Stowell

Comment:

--

HowToUseNUISMIN-GENIE

@@ -237 +237 @@
 sample NAME_OF_SAMPLE  INPUT_TYPE:FILE_INPUT    
 }}}
+- **NAME_OF_SAMPLE** : Name of the sample we found using nuissamples
+- **INPUT_TYPE** : Type of the input file we are using (e.g. GENIE)
+- **FILE_INPUT** : Path to the input MC event we want to use for this sample.
+- **OPTION** : (Optional Argument) Option that can be used to change sample behaviour at runtime. By default this is left as DEFAULT.
+- **NORM_VALUE** : (Optional Argument)  Start value of normalisation parameter used to change the MC normalisation for this sample. By default this is left at 1.0.
+
+For further examples on how to include these structures in card files please see [wiki:CardFileExamples Card File Examples].
 
 So if our GENIE file is called 'MiniBooNE_FHC_numu_2.5M.root', our card file will be :
@@ -250 +257 @@
 
 === Reweighting GENIE Comparisons ===
-The comparisons application also allows different GENIE reweight parameters to be provided in our card file.
+The minimiser application allows different GENIE reweight parameters to be provided in our card file and floated freely until the likelihood is maximised.
 
 The format for fixed dial values is:
@@ -258 +265 @@
 or for free dial variations
 {{{
-genie_parameter  NAME   VALUE  LOW HIGH STEP STATE
+genie_parameter  NAME   VALUE  LOW  HIGH  STEP STATE
 }}}
 
@@ -271 +278 @@
 
 
-In our example we shall generate another set of comparisons this time with two parameters shifted.
+In our example we want to tune the following parameters
 - Charged Current Quasielastic Axial Mass : Axial mass parameter used in the quasielastic form factor
 - Charged Current Quasielastic Normalisation : Total normalisation of CCQE events.
@@ -286 +293 @@
    switch(syst) {
      case ( kXSecTwkDial_MaNCEL           ) : return "MaNCEL";               break;
-     
+     case ( kXSecTwkDial_EtaNCEL          ) : return "EtaNCEL";              break;
+     case ( kXSecTwkDial_NormCCQE         ) : return "NormCCQE";             break;
+     case ( kXSecTwkDial_NormCCQEenu      ) : return "NormCCQEenu";          break;
+     case ( kXSecTwkDial_MaCCQE           ) : return "MaCCQE";               break;
+     case ( kXSecTwkDial_MaCCQEshape      ) : return "MaCCQEshape";          break;
+     case ( kXSecTwkDial_ZNormCCQE        ) : return "ZNormCCQE";            break;
      ...
-
-     case ( kXSecTwkDial_NormCCRES        ) : return "NormCCRES";            break;
-     case ( kXSecTwkDial_MaCCRESshape     ) : return "MaCCRESshape";         break;
-     case ( kXSecTwkDial_MvCCRESshape     ) : return "MvCCRESshape";         break;
-     case ( kXSecTwkDial_MaCCRES          ) : return "MaCCRES";              break;
-     case ( kXSecTwkDial_MvCCRES          ) : return "MvCCRES";              break;
 }}}
 
 We can see the dials we are interested in this list, specified by the strings: 'MaCCQE' and 'NormCCQE' respectively.
 
-Now we want to change these dials to +1 for MaCCES and -1 sigma for NormCCRES. Therefore we add the following lines to our card file:
-
-{{{
-genie_parameter MaCCRES       +1.0  FIX
-genie_parameter NormCCRES   -1.0   FIX
-}}}
-
-so that it now looks like the following
+Now we want to change include these dials in our card file. To start with we will try tuning a single parameter, keeping NormCCQE fixed at +0.1 sigma instead. Although we are keeping it fixed, we define the limits on NormCCQE anyway, in case we want to vary it later.
+{{{
+genie_parameter MaCCQE    0.0  -3.0  3.0  1.0  FREE
+genie_parameter NormCCQE  0.1  -3.0  3.0  1.0  FIX
+}}}
+
+So now our card file looks like the following:
 
 **genie_tutorial.card**
 {{{
-genie_parameter MaCCRES       +1.0  FIX
-genie_parameter NormCCRES   -1.0   FIX
-
-sample MiniBooNE_CC1pip_XSec_1DTpi_nu  GENIE: gntp.2063030.ghep.root
-sample MiniBooNE_CC1pip_XSec_1DTu_nu GENIE: gntp.2063030.ghep.root  
-}}}
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-=== Setting up our reweight dials ===
-
-Now that we have our samples, the next thing we want to do is choose some dials to vary. If we look inside 
-
-
-==== Defining Free dials ====
-
-==== Checking dial response ====
-
-==== Specifying fixed dials ====
+genie_parameter MaCCQE    0.0  -3.0  3.0  1.0  FREE
+genie_parameter NormCCQE  0.1  -3.0  3.0  1.0  FIX
+
+sample MiniBooNE_CCQE_XSec_1DQ2_nu GENIE:MiniBooNE_FHC_numu_2.5M.root
+sample MiniBooNE_CCQE_XSec_1DQ2_antinu GENIE:MiniBooNE_FHC_numu_2.5M.root 
+}}}
+
 
 === Running the minimiser (Migrad) ===
+Now that we have a prepared card file from the previous section, we can now run the minimiser.
+
+Lets start it running, we will use a smaller number of events for this example so we can get some idea of the behaviour. We do this by passing the '-n' option at the command line. 
+
+**WARNING** this is only so you can quickly see the results of minuit and check that the minimiser and your card file is working correctly. For serious fitting you should never use the '-n' flag.
+
+{{{
+ $ nuiscomp -c genie_tutorial.card -o genie_minuit_maccqe-free_normccqe-fixed.root -n 50000
+}}}
+
+Now that we have left that running in the background, lets quickly discuss how Migrad works and some options you have to make NUISANCE run a bit quicker.
 
 ==== How Migrad works ====
 
+Migrad is the default minimisation routine that nuismin uses. It attempts to find a best fit parameter set that maximises a likelihood (minimises a chi-squared value). When we run nuismin, the NUISANCE routines classes tells Migrad that the parameters we have defined as FREE should be treated as registered as free variables and the samples we have listed should be used to form a likelihood.
+
+Then when Migrad is run, it will step around the parameter space provided to it, calculate a likelihood given the parameters at the new step, and use a gradient descent algorithm to work its way to a best fit parameter set. Each of these 'steps' is referred to as an iteration, and usually a large number of iterations are required so that Migrad can find a reliable minimum.
+
+
+To calculate the likelihood at each step NUISANCE has to recalculate all of the MC predictions for the given parameter set at the step, before using them to calculate a data-MC likelihood. Since we have to loop over the event sample at each iteration, this can take a while to actually converge, hence why for this example we have restricted the number of events.
+
+Eventually when Migrad does think it has found a best fit point, it will step around the minimum checking that it is a good valid minimum, and assuming Gaussian statistics, calculate errors and correlations on each of the parameters.
+
+One issue that a user should be aware of is that Migrad can sometimes fall into local minima, which is especially more likely if the likelihood surface is non-linear. This can result in bias fit results, and it is advisable that extra closure tests are performed by start the minimiser from different starting values (changing the NOMINAL value in the parameter card file definition) and checking that the fit results are stable.
+
+
 ==== Signal Reconfigures ====
 
+The fact the Migrad has to iterate many times can be very time consuming.
+With large event samples it is not uncommon for tunings with multiple parameters to take days since to calculate a likelihood for a given iteration NUISANCE has to recalculate **all** the MC predictions for the updated parameter set before comparing the data to MC. This involves reading every single event in our samples at each iteration, recalculating event weights using our reweight engines, filling each of the histograms with the updated weight, and then finally calculating a likelihood for all samples included. The two most time consuming pieces of this iteration are the reading and event weight calculation stages, and as such a lot of event handling functionality has been added to the core classes of NUISANCE to minimise iteration time. This is referred to as the 'EventManager'.
+
+The EventManager's job is to keep a global list of all possible inputs and what samples they are assigned to, making sure that no event is read from disk twice or event weight is calculated twice during a single iteration. When the EventManager is switched on 'config EventManager 1' which it should be by default, during a single event loop, first the EventManager will read an event and calculate a weight before distributing that event information to only the relevant samples saving a significant amount of time compared to performing an event loop for each sample individually.
+
+To increase the speed of the EventManager even further a 'SignalReconfigures=1' option has been added which is not enabled by default, but that makes sure that only events that have been flagged as signal by the sample classes actually get read from disk and weights calculated. This mode is available but has not been extensively tested with all samples yet, so **use it with caution**.
+
 ==== Saving Nominal Prediction ====
+A configuration option is alose provided that forces NUISANCE to run an additional reconfigure step when starting nuismin that will save the starting MC predictions into a seperate folder in the output file called 'nominal/', so that pre-fit vs post-fit comparisons
+can be made.
+
+This can be turned on by using the configuration option
+{{{
+config savenominal 1
+}}}
 
 === Analysing the output ===