Network Storage Migration: Bandwidth vs Throughput and Transfer Physics
Migrating multi-terabyte or petabyte datasets between on-premise SAN/NAS environments and public cloud repositories (such as AWS S3, Azure Blob, or Google Cloud Storage) is governed by strict mathematical and physical constraints. A persistent source of project delays in enterprise IT is the naive assumption that a 1 Gbps network connection can move 1 gigabit of actual data every second indefinitely. In practice, protocol encapsulation, transmission latency, TCP window scaling, and disk subsystem I/O create significant discrepancies between advertised network bandwidth and effective throughput.
Deconstructing the Math: Bits, Bytes, and Protocol Efficiency
Network bandwidth is universally measured in bits per second (bps), while storage systems and files are measured in Bytes (B), where 1 Byte equals 8 bits. Thus, a theoretical 1 Gbps (1,000 Mbps) network link can transmit at most 125 MB/s of raw binary data under ideal zero-overhead conditions.
Effective Throughput (bits/sec) = Advertised Bandwidth × Efficiency FactorEffective Throughput (MB/s) = Effective Throughput (bits/sec) / (8 × 1,000,000)Transfer Time (Seconds) = Total Payload in Bits / Effective Throughput (bits/sec)
In real-world networks, several unavoidable overhead layers reduce effective throughput to between 75% and 92% of the advertised link speed:
- TCP/IP and Ethernet Framing: Standard Ethernet frames (1500-byte MTU) incur packet headers, TCP acknowledgments (ACKs), and IP routing metadata, consuming 3% to 5% of raw bandwidth.
- Round-Trip Time (RTT) & Latency: High WAN latency between geographic regions limits the TCP congestion window (BDP - Bandwidth Delay Product). Unless TCP window scaling and parallel multi-stream transfers are enabled, single-stream throughput drops dramatically over long distances.
- Transport Layer Security (TLS/SSL): End-to-end encryption adds cryptographic hashing overhead and packet framing.
The Physical Appliance Crossover Point (Sneakernet Physics)
As dataset sizes expand past 40 to 80 terabytes, online network migration frequently encounters diminishing returns. Moving 100 TB over a dedicated 100 Mbps leased line requires approximately 108 days of continuous transmission. Even across a 1 Gbps internet line, migrating 100 TB consumes nearly 11 days at 100% saturation, potentially impacting production business traffic.
In such scenarios, computer scientist Andrew Tanenbaum's famous adage holds true: "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." Cloud providers offer dedicated offline appliance shipping services—such as AWS Snowball, Azure Data Box, and Google Transfer Appliance. When your calculated transfer window exceeds 72 to 96 hours, ordering an encrypted physical storage appliance, copying data locally over 10GbE or 40GbE in hours, and shipping it via priority courier is faster, more reliable, and significantly less expensive.