Real Time Sportsbook Odds Comparison Engine

The company's ambition was to build the most comprehensive and fastest odds comparison engine on the market, covering 50+ sportsbooks across every major sport. The fundamental technical obstacle was the sheer diversity of integration methods across those bookmakers. Roughly 20 of the target sportsbooks offered WebSocket connections with real time push delivery, each using proprietary message formats and authentication protocols. Another 15 exposed REST APIs with varying rate limits, pagination schemes, and update frequencies ranging from every 2 seconds to every 90 seconds. The remaining sources required negotiated data partnerships with third party odds feed providers who aggregated multiple smaller bookmakers into consolidated but inconsistently structured XML or JSON feeds. Each integration was essentially a bespoke engineering project.

Entity resolution across 50+ sources was the deepest unsolved problem. Every sportsbook uses its own naming conventions for teams, leagues, competitions, and market types. One source might list "Tottenham Hotspur" while another uses "Spurs," a third shows "TOT," and a fourth presents it in the local language entirely. The same fragmentation applied to leagues ("English Premier League" versus "EPL" versus "England 1" versus "Barclays Premier League"), market types ("Match Winner" versus "1X2" versus "Full Time Result" versus "Moneyline"), and even tournament stages. Without robust entity matching across all 50+ sources, the comparison data would be unreliable, pairing odds from the wrong fixtures or misaligning markets within the same event.

The throughput requirements were substantial. With 50+ sportsbooks each publishing updates across thousands of simultaneous events covering football, tennis, basketball, horse racing, cricket, and dozens of other sports, the system needed to ingest, normalize, and redistribute over 4 million individual odds movements per day. During peak periods such as a full Saturday afternoon of Premier League football combined with ATP tennis and NBA games, the update volume could spike to over 800,000 odds changes per hour. Every update needed to flow through the normalization pipeline and reach connected clients in under 400 milliseconds to maintain user trust.

Concurrency at the delivery layer posed its own scaling challenge. Major sporting events could drive 250,000 or more simultaneous users to the platform, all expecting live odds movement across dozens of bookmakers for the same set of events. The infrastructure needed to handle sudden traffic spikes, such as a last minute goal in a Champions League semifinal causing every connected user to refresh simultaneously, without any degradation in latency or data freshness.

Regulatory complexity added another layer of engineering work. The platform operated across multiple European jurisdictions, each with its own rules governing which bookmakers could be displayed to users in that country, what affiliate disclosure requirements applied, and how odds data could be commercially used. A user in Sweden needed to see a different set of licensed bookmakers than a user in the United Kingdom or Germany, and this jurisdiction aware filtering had to happen at the edge without adding latency to the odds delivery pipeline.

We architected the engine around a distributed ingestion layer built on .NET 8, where each sportsbook integration runs as an isolated worker service. WebSocket based sources connect through persistent managed connections that parse proprietary binary and JSON frame formats in real time. REST based sources use adaptive polling workers that dynamically adjust their request intervals based on each bookmaker's rate limits, observed update frequency, and current event activity. Third party feed providers connect through dedicated parsers that handle their specific XML or JSON schemas. Every worker publishes normalized odds messages onto a RabbitMQ message bus. This isolation model means that when one sportsbook changes their API, experiences an outage, or throttles requests, only that single feed is affected. Circuit breakers on each worker detect failures within seconds and trigger automatic reconnection with exponential backoff, while monitoring dashboards surface feed health metrics for the operations team.

The normalization engine is the core intellectual component of the platform. We built a multi stage entity resolution pipeline that processes every incoming odds message through three resolution layers. The first layer performs deterministic lookups against a curated canonical database mapping over 120,000 known team names, league identifiers, and market types to their standardized equivalents. The second layer applies a Python based ML model trained on 500,000+ historical event pairings that computes similarity scores using phonetic analysis, token overlap, abbreviation pattern matching, temporal proximity of event start times, and contextual signals such as sport type and competition tier. Matches above a 97% confidence threshold are accepted automatically; those in an uncertain range are queued for human review through an internal operations dashboard, and every human resolution feeds back into the training set for continuous improvement. The third layer handles market type normalization, mapping each bookmaker's proprietary market naming into a unified taxonomy of over 300 standardized market types across all sports.

For real time delivery, we implemented a Redis caching architecture where every normalized odds update is written to a structured key space organized by sport, event, market, and bookmaker. Redis pub/sub propagates each update to a cluster of SignalR hub servers that maintain persistent WebSocket connections to all connected clients. Users subscribe to specific events or competitions, and the SignalR infrastructure pushes only the relevant odds changes to each client, minimizing bandwidth consumption. Each odds entry carries a source timestamp, ingestion timestamp, and a computed staleness indicator so the frontend can visually distinguish between freshly updated prices and those from slower updating sources. The SignalR hubs are partitioned behind Azure Load Balancer with Redis backplane handling cross server message distribution, allowing the real time delivery layer to scale horizontally independently of the ingestion pipeline.

The user facing platform presents odds in a dense, scannable comparison grid optimized for rapid price discovery. Colour coded movement indicators show whether each price has shortened or drifted since the last update. Users can set threshold alerts for specific events and markets, and the platform's value detection engine continuously evaluates incoming odds to identify outlier prices where one bookmaker's offering significantly exceeds the market consensus. Jurisdiction aware filtering runs at the CDN edge layer, ensuring each user sees only bookmakers licensed in their region with dynamically rendered affiliate disclosures. The frontend uses virtualized rendering and targeted DOM updates driven by the SignalR event stream to handle grids displaying odds from 50+ bookmakers across hundreds of markets without performance degradation.

We deployed the full stack on Azure Kubernetes Service with horizontal pod autoscaling tied to custom metrics including active SignalR connections, RabbitMQ queue depth, ingestion worker lag, and Redis memory pressure. Proactive scaling triggers based on the sporting event calendar pre provision additional capacity 30 minutes before high profile kickoffs or race starts. SQL Server handles persistent storage for user accounts, alert configurations, bookmaker metadata, and historical odds data used for trend analysis, while Redis serves as the exclusive hot layer for all real time operations. The entire ingestion and delivery pipeline is instrumented with distributed tracing using OpenTelemetry, giving the operations team end to end visibility into the journey of every odds update from source to client.