{% include "header.html" %}
Binary Classification Analysis Options
Payoff Matrix (Deterministic) Intervention Analysis (Probabilistic)
Use this method if you can say directly what the costs and benefits are of each of the potential outcomes of your the decisions you will make. For example, the cost of a false positive, the benefit of a true positive. Use this method if there is a probabilistic element to the impacts of the actions you will take based on the model predictions. For example, you will try to intervene to prevent a customer from churning when they are predicted to. However, you will have to estimate the likelihood that the intervention will succeed.
Regression/Time Series Analysis Options
Proportional Analysis Thresholded Analysis
Use this method if the impact of your model can be estimated by a payoff or value that is proportional to an error or accuracy metric. For example, each percent of error costs on average $1000 per SKU per month. Use this method if there is a hard cutoff in the impact for the model. For example, we incur a cost of $1000 when the model is more than 10% too high, and $2000 when it is more than 5% too low.
@john_c_hawkins
Notes
These methods need to make some very strong assumptions about the way your model will be deployed and used. Please read through the notes on each method and feel free to contribute ideas to the development of this tool.
{% include "footer.html" %}