BitTorrent allows for much more reliable DNS Failover capacity
Our client operates a web portal linking to a large and growing media database mostly in the form of mp3 files—24 Terabytes and counting. Wanting to enhance their back-end performance and create much-needed redundancy, they created vteam #377 to design and implement a system that:
- Keeps stored files readily available to the application.
- Replicates the entire contents of their data center.
- Supports DNS Failover, in the event of one server cluster crashing, another one can take its place immediately.
The solutions included:
Network Attached Storage (NAS) Sync: The network operations center (NOC) decided on a multi-drive Raid-60 setup for file storage.
Second Server Cluster: Acquired by the client and the NOC helped us get an exact replica of the hardware setup working and online.
Replication: Rsync is the standard way of syncing the data, but 24TB using Rsync would take 24 days. Unsurprisingly, no one was willing to go that route. Instead, we followed the best practices of the Fortune 500 and used BitTorrent to sync both NAS locations. It simultaneously syncs data while verifying the integrity of the files and it can do so in one or both directions.
DNS Failover: The client purchased and deployed the hardware.
MySQL database replication: Replication was done in real-time between the two clusters. This diagram illustrates the process:
