Skip to main content
Chapter 2 of High Performance Browser Networking · Read the chapter
IP: Internet Protocal TCP: Transmission Control Protocal

​
Three-Way Handshake

  • Sender send SYN package = (x = random(), TCP flags, options…) -> Receiver
  • Receiver send SYN ACK package = (y = random(), x+1, flags, options, …) -> Sender
  • Sender send ACK package = (x+1, y+1) -> Receiver
After handshake, the application data can begin to flow between client and server. The client can send data immediately after send ACK package. The server must wait till it receive the ACK package to begin dispatch data.

​
TCP Fast Open

The handshake costs 1 RTT before any data moves. TFO (RFC 7413) tries to save that RTT: it puts the data inside the SYN packet. But TCP cannot allow this for everyone. Anyone can send a SYN with a fake IP address. If the server did work for every fake SYN, attackers would use the server to attack other people. So TFO uses a cookie: First time — normal handshake. The server also sends back a cookie. The cookie is the client’s IP address, encrypted with a secret key that only the server has. The client saves it. Nothing is faster this time. Next time — the client sends SYN + cookie + data. The server decrypts the cookie and gets an IP address. If that IP is the same as the packet’s real source IP, the client is real. The server sends the data to the app right away, and replies with SYN-ACK + response data. The cookie only proves one thing: this client got a packet from the server at this IP before. It does not prove the request is safe to run twice. Why it is hard to use:
  • Safe requests only. Data in a SYN can be copied and sent again by an attacker. GET is fine. A POST that moves money is not.
  • Small data only. It must fit in one packet — about 1460 bytes.
  • Both sides need it. Linux 3.6+/3.7+, macOS 10.11+, iOS 9+. Windows does not have it.
  • Routers in the middle. Many firewalls and NATs just drop a SYN that carries data. Then the client waits, times out, and starts a normal handshake — slower than if it never tried.
The book (2013) expects TFO to win. It did not. Chrome turned it on, then turned it off, because too many routers broke it.QUIC 0-RTT is the same idea, done again in a way that works. It runs on UDP, so routers in the middle cannot break it, and it also skips the TLS handshake — saving 2–3 RTTs instead of 1. See chapter 12.

​
Congestion Avoidance and Control

Congestion collapse:

​
Flow Control

Flow control prevents a sender from transmitting data faster than the receiver can process it.
In TCP, the receiver advertises its available buffer through rwnd.

​
Modern applications

  • TCP: Protects the receiver’s buffer.
  • HTTP/2: Controls data per stream and connection.
  • HTTP/3 and QUIC: Use MAX_DATA and MAX_STREAM_DATA.
  • Video streaming: Adjusts incoming data to playback capacity.
  • Message queues: Limits messages delivered to slow consumers.
  • Databases: Controls connections, queries, and result delivery.
  • Application streams: Implements backpressure in Node.js, Java, and async pipelines.
  • Microservices and APIs: Limits concurrent requests to protect services.

​
Slow-Start

  • A new connection does not know the network’s available capacity. TCP starts cautiously and increases its sending rate to avoid congestion.
  • Congestion window (cwnd): This limits how much unacknowledged data the sender can have in flight. It doubles every round-trip till it reach the receiver’s limit or detects congrestion.

​
Practical implications:

  • Reuse connections.
  • Reduce RTT with geographically closer servers or a CDN.
  • Use HTTP/2 or HTTP/3 multiplexing.
  • Keep critical initial responses small.
  • Avoid unnecessary connections to many hostnames.
  • Monitor packet loss, not just bandwidth.
Also distinguish slow start from the TCP handshake:
  • The handshake establishes the connection.
  • Slow start discovers how quickly the connection can safely transmit.
The main takeaway is: Initial connection performance depends heavily on RTT, congestion-window growth, and packet loss—not just advertised bandwidth.

​
Congestion Avoidance

TCP constantly adjusts its sending speed to match network conditions.
  • TCP starts slowly, then quickly sends more data each round trip.
  • Packet loss usually tells TCP that the network is overloaded.
  • When loss happens, TCP reduces how much data it sends at once.
  • It then increases the sending rate gradually until another loss occurs.
  • This repeated increase-and-drop behavior creates a “sawtooth” throughput pattern. Why this matters
For application developers:
  • Packet loss increases latency and lowers throughput.
  • Sending more data does not always make transfers faster.
  • Large responses and many network round trips make poor connections feel worse.
  • Different operating systems may behave differently because they use different congestion-control algorithms.
  • Modern servers and kernels can recover from packet loss more efficiently.
Proportional Rate Reduction (PRR) algorithm in simple terms Older TCP algorithms could reduce the sending rate too much after packet loss. Proportional Rate Reduction (PRR) adjusts the rate more precisely, helping the connection recover faster. Practical value as a programmer:
  • Reduce unnecessary requests and round trips.
  • Keep payloads small through compression and efficient data formats.
  • Design for slow, unstable, and lossy networks.
  • Use modern operating systems and updated kernels.
  • Measure latency and packet loss, not only bandwidth.
Main lesson: Network performance depends on how quickly TCP can detect congestion and recover—not just on the user’s advertised connection speed.

​
Bandwidth-Delay Product

TCP can send only a limited amount of data before waiting for acknowledgments (ACKs). That limit is the smaller of:
  • rwnd: receiver’s available buffer
  • cwnd: sender’s congestion limit
To fully use a connection: Required window size = bandwidth × round-trip time (RTT) Maximum throughput = window size ÷ RTT Examples:
  • A 16 KB window with 100 ms RTT limits throughput to about 1.31 Mbps.
  • A 10 Mbps connection with 100 ms RTT needs about a 122 KB window.
  • Even a fast LAN at 1 Gbps with 1 ms RTT needs about a 122 KB window.
What matters:
  • More bandwidth alone does not guarantee better throughput.
  • High latency or a small TCP window leaves the connection idle while waiting for ACKs.
  • Packet loss can shrink cwnd and reduce performance.
  • TCP usually tunes window sizes automatically, but window limits and TCP window scaling are worth checking when throughput is unexpectedly low.
Core idea: The TCP window must be large enough to keep the network connection continuously busy.

​
Head-of-Line Blocking

  • TCP guarantees reliable, ordered delivery.
  • If one packet is lost, TCP holds later packets until it arrives. This is head-of-line blocking.
  • HOL blocking simplifies application code but creates delays and jitter.
  • Use TCP when every piece of data must arrive correctly and in order.
  • Consider UDP when fresh data matters more than complete data, such as voice, video, live games, and real-time updates.
  • For real-time apps, skipping an old packet is often better than pausing everything to recover it.
Core lesson: Choose transport based on your application. Reliability is valuable, but sometimes low latency is more valuable.

​
Optimizing for TCP

TCP performance is often limited by latency, not bandwidth. Focus on:
  • Send less data and use compression.
  • Reuse connections instead of opening new ones.
  • Serve content closer to users with a CDN.
  • Keep server kernels updated.
  • Consider TCP features such as window scaling and TCP Fast Open.
  • Measure network behavior before changing low-level settings.