Fraud Detection

Mobile money is the financial system where I live. In Ghana it moves more value than every traditional payment channel put together, and the fraud that rides along with it (SIM swap attacks, agent impersonation, the wrong number reversal scam) tends to land on the people who can least absorb it. I try to work on problems that are in front of me, and this one is very much in front of me.

The catch is that Ghana has no openly available mobile money transaction data. Not anonymised, not aggregated, not anything. So I used PaySim, a synthetic dataset calibrated against real African mobile money patterns, because it was the only honest option available. That constraint is worth stating plainly rather than hiding in a footnote.

The real difficulty was the class imbalance. Fraud is rare, which means a model can be almost perfectly accurate and completely useless at the same time.

The thing that stuck with me is that three baseline models all cleared 0.97 ROC-AUC while their false positive counts differed by three orders of magnitude. Same headline metric, wildly different systems. One of them is deployable and the others would drown an operations team in alerts. So the evaluation moved to precision and recall, and to what the false positive count actually does to a human being who has to work through the queue.

Around the model I built the full stack the brief asked for. MLflow for experiment tracking, FastAPI to serve predictions, Docker for the container, and ZenML to orchestrate the pipeline end to end, which runs over all 6.36 million rows in a little over seven minutes.

Reported numbers deserve suspicion, including my own. The test scores on this project are very high, and a good part of that is PaySim being a simulator with a near perfect deterministic fraud signature baked into it, not evidence that I have built something that would survive contact with real transactions. Saying so out loud, in the report, felt more useful than the number itself.

The second thing is that the MLOps was not overhead. Being able to trace exactly which model produced which prediction with which data is what would make a system like this defensible to a regulator, and it turns out to be achievable on a budget a mid sized African operator could actually afford.