Write down how the decision is made today and what its limitations are. A rule, a simple statistical method or an existing workflow can provide a useful baseline.
Evaluate candidate approaches under comparable conditions. Keep data leakage, delayed labels and changing populations in view. Examine errors by their consequences.
Choose an approach using evidence that relates to the intended use. Retain the baseline and evaluation setup so future changes can be compared fairly.