An important tool for analytic operations is the SAM framework, which is an abbreviation for scores, actions and measures. For the purposes here, think of analytic models as a black box that takes data records as inputs and produces scores as outputs.
For example, a model in computational advertising takes information about a visitor to a website as input and produces scores about ads as outputs so that ad server can decide which ads to offer the visitor (the higher the score the more likely the visitor is to click on it). Here the action is which ad to display to the visitor. There are several common ways to measure the effectiveness of the model. One is the cost per click (CPC), which is the amount spent on ads during a time period, divided by the number of clicks during the period. Another commons measure is the cost per acquisition, which requires the definition of an acquisition, such as filling out a form or buying a product. The cost per acquisition is then the amount spent on ads during a time period divided by the number of acquisition events.
SAM. The SAM framework shifts the attention from the performance of the model, as measured, for example, by the accuracy and false detection of the model, to the actions that you are trying to achieve and the relevant measures:
The SAM framework is covered in Chapter 9 of my forthcoming book The Strategy and Practice of Analytics (SPA). A slightly more general variant of SAM is to think of models as producing scores as well as other outputs and for the SAM framework to use these outputs to decide upon actions. Examples of other outputs include confidence scores and reason codes. In both cases, whether using just scores or score and other outputs to select appropriate actions, measures are used to quantify the value of the actions selected. Once measures are defined, standard techniques in optimizations can be used to choose actions that minimize or maximize the corresponding value as required.
COVID-19 SIR Models. Let’s turn now into COVID-19 epidemiological models.
One of the most basic epidemiological models for modeling COVID-19 is the SIR model, which models the number of susceptible S(t), infected I(t), and recovered (or removed) individuals R(t) in a population for each day t. For those that remember a bit of college calculus, there is a very readable introduction to the SIR model provided without a paywall by the MAA. The output of the SIR model are three curves that
When applying the SAM framework to COVID-19 modeling, the inputs can be viewed as the vector of parameters defining the model and the outputs as the number of infected individuals on a particular day t or the total number of infected individuals over a period of days. The actions might be interventions, such as sheltering in place, everyone wearing face masks, or other such measures. The measures might be the decrease in infections resulting from the interventions, or the decrease in deaths.
One of the more comprehensive studies of interventions for COVID-19 is the March 16, 2020 study by Imperial College COVID-19 Response Team.
To summarize, when we view COVID-19 modeling from a SAM point of view, the scores are the predictions of the SIR models as usual, while the actions are mitigations, and the measures are the number of infected individuals or the number of deaths that result from the mitigations. As is often the case, choosing appropriate actions is an optimization problem and requires an algorithm or model on its own.