Innovation and Compliance at Global Scale: Predictive Analytics Meets DevOps Automation

Abstract

The amount of transactions processed each year by international payment platforms exceeds trillions. These payment platforms continuously strive to create innovations while also complying with regulation requirements such as PCI- DSS, GDPR and PSD2 by providing strict auditability, but at the same time must provide real-time processing and latency of less than 100ms. To accomplish this, we propose an architecture that is unified and uses technologies such as Spark, MLflow, and ArgoCD to achieve significantly high performance with low latency and high compliance rates. We address several engineering challenges in this architecture concerning the fragmentation of ML features through the use of advanced data pipelines; the implementation of policy-as-code to ensure the integrity of our models and to mitigate bias; and the reduction of operational latency through the use of federated inference across clusters that are compliant with GDPR. Developing quick updates to models and improving fraud detection/recommendation systems through this structure has huge cost savings and efficiency improvements. Our development process stresses ethical AI governance with real-time explainability of models created within this structure and drift detection for those models; we will also look at delivering compliance automation, faster transaction authorizations in upcoming phases, and examining the unintended effects that have not yet occurred.