Debugging fits

When a fit fails there are several things you can do to investigate. CMS users can have a look at these slides from a previous Combine tutorial. This section contains a few pointers for some of the methods mentioned in the slides.

Analyzing the NLL shape in each parameter

The FastScan mode of combineTool.py can be used to analyze the shape of the NLL as a function of each parameter in the fit model. The NLL is evaluated varying a single parameter at a time, the other parameters stay at the default values they have in the workspace. This produces a file with the NLL, plus its first and second derivatives, as a function of each parameter. Discontinuities in the derivatives, particularly if they are close to the minimum of the parameter, can be the source of issues with the fit.

The usage is as follows:

combineTool.py -M FastScan -w workspace.root:w

Note that this will make use of the data in the workspace for evaluating the NLL. To run this on an asimov data set, with r=1 injected, you can do the following:

combine -M GenerateOnly workspace.root -t -1 --saveToys --setParameters r=1

combineTool.py -M FastScan -w workspace.root:w -d higgsCombineTest.GenerateOnly.mH120.123456.root:toys/toy_asimov

higgsCombineTest.GenerateOnly.mH120.123456.root is generated by the first command; if you pass a value for -m or change the default output file name with -n the file name will be different and you should change the combineTool call accordingly.