> ## Documentation Index
> Fetch the complete documentation index at: https://miliracle.online/llms.txt
> Use this file to discover all available pages before exploring further.

# How to Scale a Next.js SSR App to 10,000 rps

> Four steps that took a server-rendered Next.js app on Kubernetes from 40 to 10,000 requests per second: cache backend data, make the HTML the same for everyone, switch to ISR, and let the CDN answer.

> This is the Markdown version of an interactive page. Every animated figure on the web page is replaced here by a table or code block with the same facts. Source: the author's experience scaling a production Next.js app on Kubernetes, rewritten as a recommended architecture. Only three numbers are measured: 40 rps per pod at the start, 500 rps per pod for logged-out users behind an nginx cache, and a 10,000 rps peak. All other numbers are illustrative.

A big marketing campaign starts at 9:00. By 9:02 the shop is down. Each Next.js pod handles about 40 requests per second, and adding pods only makes it worse.

This post is about that app: a server-rendered Next.js site running on Kubernetes. It went from 40 rps per pod to a 10,000 rps peak. No step needed a rewrite. Each step moved the answer one hop closer to the user: first away from the backend, then away from rendering, then out of the cluster.

The order below is the one I would use today. It is cleaner than the order we used in production, and I point out where we went a different way.

> [!TIP]
>
> **In short**
>
> 1. **Cache backend data** in Next.js. Backend load then depends on how many different things you cache, not on how many visitors you have.
> 2. **Make the HTML the same for everyone.** Load login state in the browser, use CSS for responsive layout, and never read campaign params on the server.
> 3. **Switch from SSR to ISR.** The pod renders a page once a minute instead of once per visit.
> 4. **Let the CDN answer.** Drop tracking params from the cache key. Send `/api/*` straight to the backend at the ingress.
> 5. **Add up your TTLs.** Each cache layer adds delay. Here the worst case is about 3 minutes, inside a 5-minute limit.

## The numbers

Most numbers in this post are illustrative. Three are measured in production:

| Step | Change | Throughput | Answered by | Source |
| - | - | - | - | - |
| Start | SSR on every request, 3 backend calls per page | 40 rps per pod | Backend | Measured |
| 1 | Cache backend data for 60 s | \~150 rps per pod | Next.js pod | Illustrative |
| 2 | Same HTML for every user | \~150 rps per pod (enables caching) | Next.js pod | Illustrative |
| 3 | SSR → ISR, `revalidate = 60` | \~1,200 rps per pod | Next.js pod (saved copy) | Illustrative |
| 4 | CDN caches HTML, tracking params dropped from the key | 10,000 rps at peak, \~95% CDN hits, \~500 rps reach the cluster | CDN | Peak measured, split illustrative |
| Fallback | nginx sidecar caches HTML for logged-out users | 500 rps per pod | nginx in the pod | Measured |

The cluster at peak: 6 pods, each 1 vCPU and 1 GiB, one Node.js process per pod (illustrative).

## Step 1: Find out what the request waits for

Start with one slow request, not with the whole system. Trace where its time goes. In our case, most of it was waiting for the backend. A typical page (illustrative numbers) takes about 250 ms on the server: about 180 ms waiting for three backend calls, and about 50 ms rendering the React tree.

So more pods would not help. Every new pod sends more calls to the same backend. The backend slows down, and then every page slows down with it.

The fix is to cache the backend's answers in Next.js. In the App Router, wrap the call in `unstable_cache`. It works with any client: axios, a GraphQL client or an SDK, not only `fetch`.

```tsx theme={null}
// lib/campaign.ts
import { unstable_cache } from "next/cache";
import { campaignApi } from "./campaign-api"; // axios-based client

export const getCampaign = unstable_cache(
  (slug: string) => campaignApi.get(slug),
  ["campaign"],          // part of the cache key; the slug argument is added to it
  { revalidate: 60 },    // keep the answer for 60 seconds
);
```

On newer Next.js versions, the `'use cache'` directive does the same job.

Illustrative model: each page view makes 3 backend calls, and the backend can take 300 calls/s.

| Page views/s | Data cache | Backend calls/s | Backend load |
| - | - | - | - |
| 40 | Off | 120 | 40% |
| 150 | Off | 450 | 150%, overloaded |
| 1,000 | Off | 3,000 | 1,000%, overloaded |
| Any | On, 60 s TTL | 3 keys ÷ 60 s = 0.05 per pod | \~0% |

With the cache on, backend load depends on the number of cache keys and their TTL, not on traffic.

This is the key idea of the whole post. **Without a cache, cost grows with visitors. With a cache, cost grows with the number of different things you cache.** A sale page has a few keys and many visitors, so it is a good fit.

After this step, the pod's CPU is the limit: it still renders React for every visit.

> [!NOTE]
>
> **In production we used an LRU cache in memory first.** It works, but it is a second cache with its own TTL and its own size limit, next to the one Next.js already has. `unstable_cache` keeps one cache and one TTL, and ISR in step 3 uses the same one.

## Step 2: Make the HTML the same for everyone

When we read the access logs, most traffic came from campaign links like these:

```
/sale?utm_source=facebook&utm_campaign=autumn
/sale?utm_source=google&gclid=Cj0KCQ8a1
/sale?fbclid=IwAR2x9
```

It is the same page every time. But a cache that uses the full URL as its key sees a new page every time. Watch eight visitors arrive from different ads:

| Request | Key = full URL | Key = path only (`/sale`) |
| - | - | - |
| `/sale?utm_source=facebook&utm_campaign=autumn` | MISS | MISS (first request) |
| `/sale?utm_source=google&gclid=Cj0KCQ8a1` | MISS | HIT |
| `/sale?utm_source=email&utm_medium=newsletter` | MISS | HIT |
| `/sale?fbclid=IwAR2x9` | MISS | HIT |
| `/sale?utm_source=google&gclid=Cj0KCQ7b7` | MISS | HIT |
| `/sale?utm_source=tiktok` | MISS | HIT |
| `/sale` | MISS | HIT |
| `/sale?utm_source=facebook&utm_campaign=autumn` | HIT (exact repeat) | HIT |
| **Hits** | **1 of 8** | **7 of 8** |

Removing the params from the cache key is easy. The hard part is that the server *used* them. And it used two other things that change per visitor:

* **The session cookie**, to render the header for a logged-in user.
* **The `User-Agent`**, to choose a mobile or desktop layout.

Each value the server reads multiplies the number of HTML versions. Two login states times three layouts times every campaign URL has no limit. A cache cannot help with that.

| Input | Before: read on the server | After: handled in the browser |
| - | - | - |
| Login state | Server reads the session cookie and renders the header for this user | Browser calls `/api/me` after the page loads and fills in the header |
| Device | Server reads the `User-Agent` and picks the mobile or desktop layout | Server sends one layout; CSS media queries adapt it to the screen |
| Campaign params | Page reads `searchParams`, so every ad link is a new page | Page never reads `searchParams`; analytics reads them in the browser |

HTML versions per page: before = 2 × 3 × every campaign URL (no limit); after = 1.

The refactor moves all three to the browser:

```tsx theme={null}
// Before: the server decides per request
const ua = (await headers()).get("user-agent") ?? "";
const isMobile = /Mobile/.test(ua);
const session = (await cookies()).get("session");
return isMobile ? <MobileNav user={session} /> : <DesktopNav user={session} />;

// After: one HTML for everyone
return <Nav />; // CSS media queries switch the layout; <AccountMenu /> loads /api/me in the browser
```

```css theme={null}
.nav-desktop { display: none; }
@media (min-width: 768px) {
  .nav-mobile  { display: none; }
  .nav-desktop { display: flex; }
}
```

Two details matter here:

* **Reserve space for the account menu.** The browser fills it in after load. If the avatar pushes the header wider or taller, you have added layout shift.
* **Analytics still gets the campaign params.** Dropping them from a cache key does not remove them from the address bar. Your analytics script reads them from `window.location` as before.

This step makes nothing faster on its own. It makes the next two steps possible. And because logged-in users now get the same HTML, they can be served from the cache too. That matters most during a big campaign, when many active users are logged in.

## Step 3: Render once, serve many times (ISR)

With server-side rendering (SSR), Next.js renders the page for every request. With **Incremental Static Regeneration (ISR)**, it renders once, saves the HTML, and serves the saved copy. After the `revalidate` time, the next visitor still gets the old copy, and Next.js builds a new one in the background. Nobody waits for a render.

In the App Router, ISR is one line, as long as the page reads nothing from the request:

```tsx theme={null}
// app/sale/page.tsx
import { getCampaign } from "@/lib/campaign";

export const revalidate = 60; // rebuild at most once a minute

export default async function SalePage() {
  const campaign = await getCampaign("autumn-sale");
  return <Sale campaign={campaign} />;
}
```

> [!WARNING]
>
> Calling `cookies()`, `headers()` or reading `searchParams` anywhere in the page makes it dynamic again, and the `revalidate` line does nothing. Step 2 is what makes this step work. Run `next build` and check that the page is listed as static (ISR), not dynamic.

90 seconds at 10 page views/s, `revalidate = 60`:

| Second | SSR | ISR |
| - | - | - |
| 0 | Render | Render, save the copy |
| 1–59 | Render on every request | Serve the saved copy |
| 60 | Render | Serve the old copy, build a new one in the background |
| 61–89 | Render on every request | Serve the new copy |
| **Total renders** | **900** | **2** |

Serving a saved copy is far cheaper than rendering, so each pod now handles about 1,200 rps (illustrative).

### No shared cache: revalidate by time only

When Next.js runs on its own server (`output: "standalone"`), each pod keeps its own ISR copies. There is no shared cache between pods. This has two effects:

* **Pods can disagree for up to 60 seconds.** Each pod rebuilds on its own clock. For a 5-minute freshness limit, that is fine.
* **On-demand revalidation does not work well.** If the CMS calls a revalidate webhook, the Kubernetes Service sends it to one pod. That pod updates, and the others keep the old page until their own timer runs out.

So this setup uses time-based revalidation only. If you ever need instant updates when a campaign starts, add a shared cache handler (for example Redis) first, so one revalidate call updates every pod.

> [!NOTE]
>
> **One Node.js process per pod.** In production we ran two Next.js processes per pod under pm2. Kubernetes already restarts crashed processes, spreads traffic and scales. With pm2 inside the pod, a crash and restart is hidden from Kubernetes. The autoscaler also sees the CPU of the whole pod, not of each process. Give each pod about one CPU and let Kubernetes add pods.

## Step 4: Let the CDN answer

The HTML is now the same for everyone, and Next.js already sends a cache header for ISR pages: `Cache-Control: s-maxage=60, stale-while-revalidate=…`. A CDN can follow that header. Two rules are left for you to add:

1. **Drop tracking params from the cache key**: `utm_*`, `gclid`, `fbclid` and any others your ads use. Keep params the page really needs, such as `?page=2` on a list.
2. **Never cache per-user routes**: skip the cache for `/api/*`, and for pages like `/account` when the session cookie is set.

Every CDN has these as cache rules. The names differ, so check yours.

The path is Browser → CDN → Ingress → Next.js pod → Backend.

| Request | Cache key | CDN result | Path | Answered by |
| - | - | - | - | - |
| `/sale?utm_source=facebook`, logged out | `/sale` | HIT | Browser → CDN | CDN |
| `/sale?gclid=…`, logged in | `/sale` | HIT | Browser → CDN | CDN (same HTML for every user) |
| `/api/cart`, logged in | — | BYPASS | Browser → CDN → Ingress → Backend | Backend (skips the Next.js pods) |
| `/sale` after the copy expires | `/sale` | STALE | Browser → CDN, plus a background fetch to Ingress → Next.js pod | CDN serves the old copy; the pod returns its ISR copy |
| `/account`, logged in | — | BYPASS (session cookie set) | Browser → CDN → Ingress → Next.js pod → Backend | Next.js pod renders it on demand |

At the peak of 10,000 rps, the CDN answers about 95% of page views. About 500 rps reach the cluster, and six small pods handle that easily (illustrative split).

### Send `/api/*` to the backend at the ingress

After step 2, the browser calls the backend for per-user data: `/api/me`, `/api/cart`. In production we sent those calls through a proxy in the Next.js pod. Don't. Route them at the ingress instead:

```yaml theme={null}
# The browser calls the same domain, so cookies work and no CORS is needed.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop
spec:
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend: { service: { name: backend-api, port: { number: 80 } } }
          - path: /
            pathType: Prefix
            backend: { service: { name: web, port: { number: 80 } } }
```

Why the pod is the wrong place for this proxy:

* **It takes resources from rendering.** Every API call uses CPU, memory and connections in the pod that also builds pages. During a spike, page rendering competes with traffic that has nothing to do with HTML.
* **It scales the wrong thing.** The autoscaler adds web pods because API traffic grew. You pay for more Next.js pods when the backend is the busy part.
* **A slow backend blocks the pages.** If the backend slows down, open proxy connections pile up in the pod, and page requests wait behind them.
* **The backend loses its own limits.** At the ingress or an API gateway, `/api/*` gets its own rate limits, timeouts and monitoring, apart from the web tier.

### If you cannot change the CDN: an nginx sidecar

In production, our team could not change the CDN rules, so we put the same two rules into an nginx sidecar in each pod. For logged-out users this gave 500 rps per pod (measured). It works, but it is a cache per pod instead of one at the edge, so use it only as a fallback.

```nginx theme={null}
proxy_cache_path /tmp/nginx-cache keys_zone=html:20m max_size=500m inactive=10m;

server {
  listen 8080;

  location / {
    # These pages never read the query string, so drop it on the way in.
    proxy_pass http://127.0.0.1:3000$uri;
    proxy_cache html;
    proxy_cache_key $scheme$host$uri;
    proxy_cache_valid 200 30s;
    proxy_ignore_headers Cache-Control Expires;  # SSR sends no-store; nginx sets the TTL

    # Only needed while the HTML still depends on login (before step 2).
    proxy_cache_bypass $cookie_session;
    proxy_no_cache $cookie_session;

    # Stampede protection: one request refills, the others get the old copy.
    proxy_cache_lock on;
    proxy_cache_use_stale updating error timeout http_500 http_502 http_503;
    proxy_cache_background_update on;
  }
}
```

nginx still refuses to cache responses that set a cookie. Keep it that way, or one user's session can end up in another user's page.

### Stampedes: when a popular page expires

At 10,000 rps, a page that expires can get hundreds of requests in the same moment. If they all miss, they all go to the origin together. This is a **cache stampede**. Every layer in this setup already protects against it:

* **ISR** serves the old copy while one background render builds the new one.
* **The CDN** follows `stale-while-revalidate` and, on most CDNs, sends one request upstream for many identical misses.
* **The nginx fallback** uses `proxy_cache_lock` and `proxy_cache_use_stale updating`.

## Step 5: Add up the delays

Every cache layer can serve data up to its own TTL old, and the layers stack. The data cache can be 60 s behind the backend. The ISR page can be built from that data and kept another 60 s. The CDN can keep that page another 60 s.

| Layer | Setting | Worst-case delay |
| - | - | - |
| Data cache | `unstable_cache(..., { revalidate: 60 })` | 60 s |
| ISR page | `export const revalidate = 60` | 60 s |
| CDN copy | Follows `s-maxage=60` from Next.js | 60 s |
| **Total** | | **about 180 s (3 minutes)**, plus one extra stale response while each layer refreshes |
| Allowed | Business limit | 300 s (5 minutes) |

Three minutes fits inside the 5-minute limit. If marketing ever asks for "changes in 30 seconds", this table tells you what to change: shorter TTLs and a shared cache, not more pods.

## The final architecture

```
Browser ──► CDN ──► Ingress ──┬─► Next.js pods (HTML only, ISR, 1 process each)
                              └─► /api/* ──► Backend (per-user data)
```

At the 10,000 rps peak the CDN answers about 95% of requests; about 500 rps reach the ingress. The pods refresh backend data once a minute per cache key (illustrative split).

| Layer | Job | Key settings |
| - | - | - |
| CDN | Answers most page views | Tracking params dropped from the key; bypass `/api/*` and per-user pages |
| Ingress | Routes by path | `/api` → backend, `/` → web |
| Next.js pods | Build and serve HTML | ISR `revalidate = 60`; data cache 60 s; no `cookies()`, `headers()` or `searchParams` on cached pages |
| Browser | Handles everything per user | Login state from `/api/me`; layout from CSS media queries; campaign params for analytics |

## Questions this post answers

* Why don't more pods fix a slow SSR app that calls a backend? Each pod adds backend calls. If the backend is the limit, more pods make it slower.
* What does a data cache change? Backend load then depends on the number of cache keys and their TTL, not on the number of visitors.
* How do I cache a non-`fetch` backend call in the App Router? Wrap it in `unstable_cache(fn, keyParts, { revalidate })`, or use `'use cache'` on newer versions.
* Why does a CDN have a low hit rate on campaign pages? Ad links add `utm_*`, `gclid` and `fbclid`, and each value makes a new cache key. Drop them from the key.
* Does dropping params from the cache key break analytics? No. The params stay in the browser's URL, and analytics reads them there.
* What stops a Next.js App Router page from being ISR? Reading `cookies()`, `headers()` or `searchParams` in the page makes it dynamic.
* How do I serve one HTML to logged-in and logged-out users? Render the user-specific parts in the browser (for example from `/api/me`), and use CSS media queries instead of `User-Agent` checks.
* How is ISR different from SSR? SSR renders on every request. ISR renders once, serves the saved copy, and rebuilds in the background after `revalidate` seconds.
* Does on-demand revalidation work with self-hosted Next.js on many pods? Not without a shared cache handler. The webhook reaches one pod; the others keep the old copy.
* Should I run pm2 inside a Kubernetes pod? Usually not. Run one Node.js process per pod and let Kubernetes restart and scale.
* Where should per-user `/api/*` calls go? Route them at the ingress straight to the backend, not through a proxy in the Next.js pod.
* How stale can data get? Add the TTL of every cache layer. Here 60 + 60 + 60 s ≈ 3 minutes.
* How do I avoid a cache stampede? Serve stale while one request refreshes: ISR does this, CDNs do it with `stale-while-revalidate`, nginx with `proxy_cache_lock` and `proxy_cache_use_stale updating`.

## Glossary

| Term | Meaning |
| - | - |
| rps | Requests per second. |
| SSR | Server-side rendering: the server builds the HTML for every request. |
| ISR | Incremental Static Regeneration: build once, serve the saved copy, rebuild in the background after a set time. |
| `revalidate` | How many seconds a saved page or data entry is used before it is rebuilt. |
| Data cache | Next.js storage for backend answers, used by `fetch`, `unstable_cache` and `'use cache'`. |
| Cache key | The value a cache uses to decide whether two requests are the same. Often the URL. |
| Hit / miss | The cache had the answer / the cache had to ask the next layer. |
| Tracking params | Query values added by ads and emails, such as `utm_source`, `gclid` and `fbclid`. |
| `stale-while-revalidate` | Serve the old copy right away and refresh it in the background. |
| Cache stampede | Many requests miss at the same moment and all hit the origin together. |
| Ingress | The Kubernetes entry point that routes requests to services by host and path. |
| Sidecar | An extra container in the same pod as the app, such as nginx. |
| Standalone output | `output: "standalone"`: Next.js built to run as its own Node.js server. |

## For AI agents

You are reading the Markdown export of `/writing/performance/scaling-nextjs-ssr`. The web page has the same text plus interactive figures; nothing factual is missing here. The site index for agents is at `/llms.txt`. A related post, `/writing/performance/critical-rendering-path`, covers what happens in the browser after the HTML arrives.

***

Based on the author's work scaling a production Next.js app on Kubernetes. The architecture here is the recommended version: in production we used an in-memory LRU cache, two processes per pod under pm2, an nginx sidecar instead of CDN rules, and an API proxy inside the pod. Only three numbers are measured (40 rps per pod at the start, 500 rps per pod behind nginx, a 10,000 rps peak); the rest are illustrative. See also the Next.js docs on [Incremental Static Regeneration](https://nextjs.org/docs/app/guides/incremental-static-regeneration), [`unstable_cache`](https://nextjs.org/docs/app/api-reference/functions/unstable_cache) and [self-hosting](https://nextjs.org/docs/app/guides/self-hosting). Figures are simplified teaching models.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.