Challenge
The challenge
User behavioural analytics only creates value when signals are fresh and queryable.
- High-volume event streams from products and clients
- Need for stream processing — not only nightly ETL
- Low-latency storage and query for behavioural and time-oriented workloads
- Services written for throughput and operability (Go) in the hot path
Approach
What BlackPotato did
- Designed and built core pieces of a real-time behavioural analytics platform
- Kafka as the durable event backbone
- Golang services for high-performance ingest and platform components
- Apache Flink for stream processing and behavioural aggregations / features
- QuestDB for time-series–oriented analytics storage and fast queries
- End-to-end path: events → process → store → real-time analytical access
Flow
Reference architecture
- User / product events
- Kafka
- Go services ↔ Flink stream processing
- QuestDB
- Real-time behavioural analytics / consumers
Architecture
Stack roles
Outcomes
Outcomes
- Real-time path for user behavioural data
- Clear separation of streaming transport, processing, and analytical store
- Stack suited to high ingest and timely query behavioural use cases