A production-grade market data synchronization service that periodically retrieves the most liquid Binance perpetual futures symbols, validates historical data completeness, and ensures consistency across PostgreSQL storage layers and mirrored tables.
The system is designed to support downstream quantitative pipelines, including strategy engines, backtesting frameworks, and real-time signal generation. It emphasizes data integrity, symbol quality filtering, and continuous synchronization of high-volume crypto market data.
Core responsibilities
* Liquidity-based symbol selection (ranking/filtering)
* Continuous Binance Futures data ingestion
* Historical data validation (gap detection / completeness checks)
* PostgreSQL time-series storage design
* Data replication / mirror consistency
* Scheduler or daemon system (continuous sync)
* Backend reliability (retries, rate-limit handling)
* real-time stream (WebSocket instead of REST only)
* gap-repair system (auto backfill missing candles)
* schema versioning / migration system
* monitoring dashboard
* multi-exchange support (Binance + Bybit)