Capstone Project~15-18 hrs

Sub-Second Fraud Detection System

Identify anomalous transaction patterns across time windows using distributed state to flag fraudulent credit card swipes instantly.

4 Parts/12 Tools/8 Fraud Rules

paystream / flink-job-graph

INGEST

Kafka Source

Watermarks

Schema Registry

CDC Stream

→PROCESS

KeyedProcess

Windowing

ValueState

RocksDB

→DETECT

Velocity Rules

Geo-Anomaly

Escalation

Broadcast

→DEPLOY

Flink K8s Op

Checkpoints

Prometheus

Grafana

fig 1 — flink fraud detection job graph

THROUGHPUT

100K+

Events/min

STATE

RocksDB

Incremental Ckpt

GUARANTEES

EOS

Exactly-Once

DEPLOY

K8s

Flink Operator

What You'll Build

A complete fraud detection platform — from local Flink cluster to production Kubernetes with full observability and exactly-once processing.

Sub-100ms Detection

True streaming with DataStream API — velocity checks, geo-anomaly, rapid escalation, and 8 fraud rules at sub-100ms latency

Stateful Processing

RocksDB state backend managing 10GB+ per-customer state with incremental checkpointing and 95% faster recovery

Event Time & Windows

Bounded-lateness watermarks with 30s tolerance, tumbling and sliding windows, session windows, and late data side outputs

Production Deployment

Flink Kubernetes Operator, HA with ZooKeeper, Prometheus/Grafana monitoring, and GitHub Actions CI/CD pipeline

Progressive Build Path

4 parts, each building on the last. Watch your fraud detection system grow progressively.

Foundation & Stream Basics

3-4 hours

Set up local Flink + Kafka with Docker, build your first DataStream pipeline, implement basic fraud rules with map/filter/keyBy, and write unit tests with Flink test harness.

1 pipeline, 1 fraud rule — foundation ready

Windows, State & Watermarks

4-5 hours

Master event time processing with bounded-lateness watermarks, tumbling and sliding windows for velocity detection, ValueState for per-customer tracking, and RocksDB state backend.

4 fraud rules, 2 window types operational

Connectors & Exactly-Once

4-5 hours

Connect Kafka (transactional sink), Elasticsearch, PostgreSQL CDC via Debezium, broadcast state for dynamic rules, and async I/O for ML model scoring.

5 connectors, exactly-once guarantees

K8s Deployment & Monitoring

4-5 hours

Deploy with Flink Kubernetes Operator, incremental checkpointing to S3, HA with ZooKeeper, Prometheus/Grafana monitoring, and GitHub Actions CI/CD.

LIVE on Kubernetes with monitoring

Infrastructure Standards

Production patterns you'll implement across the Flink streaming platform.

THROUGHPUT

100K+events/min

True streaming with sub-100ms latency, not micro-batching — real-time fraud detection at scale

STATE

10GB+RocksDB

Incremental checkpointing to S3 with RocksDB backend for 95% faster recovery times

GUARANTEES

EOSexactly-once

Transactional Kafka sinks, checkpointing, and barrier alignment for zero duplicates

DEPLOY

K8sFlink operator

Native Kubernetes deployment with HA, auto-scaling, and Prometheus/Grafana monitoring

Environment Setup

Launch the Flink cluster and submit your first fraud detection job.

paystream-flink

# Clone PayStream & launch Flink cluster
$ git clone https://github.com/aide-hub/paystream-flink.git
$ cd paystream-flink

# Start Flink + Kafka + Elasticsearch + PostgreSQL
$ docker-compose -f docker-compose.flink.yml up -d

# Submit the fraud detection job
$ ./bin/flink run \
$   --class com.paystream.FraudDetectionJob \
$   --parallelism 4 \
$   target/fraud-detection-1.0.jar \
$   --watermark.max-lateness 30s --state.backend rocksdb

Tech Stack

Apache FlinkApache KafkaJava 17RocksDBKubernetesFlink K8s OperatorPrometheusGrafanaElasticsearchPostgreSQLDebeziumDocker

Prerequisites

Java basics (variables, classes, lambdas, streams API)
Docker fundamentals (containers, docker-compose)
Kafka concepts (topics, producers, consumers)
Kubernetes basics (pods, deployments, services)

Related Learning Path

Master Flink architecture, DataStream API, watermarks, state management, and production deployment patterns.

Flink Streaming Learning Path

Read: What is Apache Flink?

Ready to build real-time fraud detection?

Start with Part 1: Foundation & Stream Basics