The article explores how TBMQ (ThingsBoard MQTT) transitioned from PostgreSQL to Redis for persistent message storage, emphasizing scalability, performance, and architectural enhancements. The migration aimed to address PostgreSQL’s limitations in handling high-throughput, session-based workloads. Redis offers horizontal scalability, making it a more suitable solution for such scenarios. The author, Dmytro Shvaika, details the implementation of Redis data structures and Lua scripting to ensure message order and consistency. This guide is worth reading because it provides practical insights into optimizing message storage for distributed systems. Readers will learn how to implement Redis-based message persistence with efficient data structures and asynchronous clients.
Key facts
- TBMQ migrated from PostgreSQL to Redis to improve scalability and performance for high-throughput, session, based workloads.
- Redis uses sorted sets and strings with TTL to manage message order, limits, and automatic cleanup.
- Lettuce, an asynchronous Redis client, was adopted to increase throughput to 60k messages per second.
- Lua scripting ensures atomic operations for message persistence, retrieval, and cleanup in Redis.
- Dynamic cleanup mechanisms prevent memory bloat and maintain efficient resource usage.
