How to Scale a Next.js SSR App to 10,000 rps

A big marketing campaign starts at 9:00. By 9:02 the shop is down. Each Next.js pod handles about 40 requests per second, and adding pods only makes it worse. This post is about that app: a server-rendered Next.js site running on Kubernetes. It went from 40 rps per pod to a 10,000 rps peak. No step needed a rewrite. Each step moved the answer one hop closer to the user: first away from the backend, then away from rendering, then out of the cluster. The order below is the one I would use today. It is cleaner than the order we used in production, and I point out where we went a different way.

In short

Cache backend data in Next.js. Backend load then depends on how many different things you cache, not on how many visitors you have.
Make the HTML the same for everyone. Load login state in the browser, use CSS for responsive layout, and never read campaign params on the server.
Switch from SSR to ISR. The pod renders a page once a minute instead of once per visit.
Let the CDN answer. Drop tracking params from the cache key. Send /api/* straight to the backend at the ingress.
Add up your TTLs. Each cache layer adds delay. Here the worst case is about 3 minutes, inside a 5-minute limit.

The numbers

Most numbers in this post are illustrative. Three are measured in production:

Measured	Value
Start (SSR, no cache)	40 rps per pod
nginx HTML cache, logged-out users	500 rps per pod
Peak during a campaign	10,000 rps

Step 1: Find out what the request waits for

Start with one slow request, not with the whole system. Trace where its time goes. In our case, most of it was waiting for the backend. A typical page (illustrative numbers) takes about 250 ms on the server: about 180 ms waiting for three backend calls, and about 50 ms rendering the React tree. So more pods would not help. Every new pod sends more calls to the same backend. The backend slows down, and then every page slows down with it. The fix is to cache the backend’s answers in Next.js. In the App Router, wrap the call in unstable_cache. It works with any client: axios, a GraphQL client or an SDK, not only fetch.

// lib/campaign.ts
import { unstable_cache } from "next/cache";
import { campaignApi } from "./campaign-api"; // axios-based client

export const getCampaign = unstable_cache(
  (slug: string) => campaignApi.get(slug),
  ["campaign"],          // part of the cache key; the slug argument is added to it
  { revalidate: 60 },    // keep the answer for 60 seconds
);

On newer Next.js versions, the 'use cache' directive does the same job. This is the key idea of the whole post. Without a cache, cost grows with visitors. With a cache, cost grows with the number of different things you cache. A sale page has a few keys and many visitors, so it is a good fit. After this step, the pod’s CPU is the limit: it still renders React for every visit.

In production we used an LRU cache in memory first. It works, but it is a second cache with its own TTL and its own size limit, next to the one Next.js already has. unstable_cache keeps one cache and one TTL, and ISR in step 3 uses the same one.

Step 2: Make the HTML the same for everyone

When we read the access logs, most traffic came from campaign links like these:

/sale?utm_source=facebook&utm_campaign=autumn
/sale?utm_source=google&gclid=Cj0KCQ8a1
/sale?fbclid=IwAR2x9

It is the same page every time. But a cache that uses the full URL as its key sees a new page every time. Watch eight visitors arrive from different ads: Removing the params from the cache key is easy. The hard part is that the server used them. And it used two other things that change per visitor:

The session cookie, to render the header for a logged-in user.
The User-Agent, to choose a mobile or desktop layout.

Each value the server reads multiplies the number of HTML versions. Two login states times three layouts times every campaign URL has no limit. A cache cannot help with that. The refactor moves all three to the browser:

// Before: the server decides per request
const ua = (await headers()).get("user-agent") ?? "";
const isMobile = /Mobile/.test(ua);
const session = (await cookies()).get("session");
return isMobile ? <MobileNav user={session} /> : <DesktopNav user={session} />;

// After: one HTML for everyone
return <Nav />; // CSS media queries switch the layout; <AccountMenu /> loads /api/me in the browser

.nav-desktop { display: none; }
@media (min-width: 768px) {
  .nav-mobile  { display: none; }
  .nav-desktop { display: flex; }
}

Two details matter here:

Reserve space for the account menu. The browser fills it in after load. If the avatar pushes the header wider or taller, you have added layout shift.
Analytics still gets the campaign params. Dropping them from a cache key does not remove them from the address bar. Your analytics script reads them from window.location as before.

This step makes nothing faster on its own. It makes the next two steps possible. And because logged-in users now get the same HTML, they can be served from the cache too. That matters most during a big campaign, when many active users are logged in.

Step 3: Render once, serve many times (ISR)

With server-side rendering (SSR), Next.js renders the page for every request. With Incremental Static Regeneration (ISR), it renders once, saves the HTML, and serves the saved copy. After the revalidate time, the next visitor still gets the old copy, and Next.js builds a new one in the background. Nobody waits for a render. In the App Router, ISR is one line, as long as the page reads nothing from the request:

// app/sale/page.tsx
import { getCampaign } from "@/lib/campaign";

export const revalidate = 60; // rebuild at most once a minute

export default async function SalePage() {
  const campaign = await getCampaign("autumn-sale");
  return <Sale campaign={campaign} />;
}

Calling cookies(), headers() or reading searchParams anywhere in the page makes it dynamic again, and the revalidate line does nothing. Step 2 is what makes this step work. Run next build and check that the page is listed as static (ISR), not dynamic.

Serving a saved copy is far cheaper than rendering, so each pod now handles about 1,200 rps (illustrative).

No shared cache: revalidate by time only

When Next.js runs on its own server (output: "standalone"), each pod keeps its own ISR copies. There is no shared cache between pods. This has two effects:

Pods can disagree for up to 60 seconds. Each pod rebuilds on its own clock. For a 5-minute freshness limit, that is fine.
On-demand revalidation does not work well. If the CMS calls a revalidate webhook, the Kubernetes Service sends it to one pod. That pod updates, and the others keep the old page until their own timer runs out.

So this setup uses time-based revalidation only. If you ever need instant updates when a campaign starts, add a shared cache handler (for example Redis) first, so one revalidate call updates every pod.

One Node.js process per pod. In production we ran two Next.js processes per pod under pm2. Kubernetes already restarts crashed processes, spreads traffic and scales. With pm2 inside the pod, a crash and restart is hidden from Kubernetes. The autoscaler also sees the CPU of the whole pod, not of each process. Give each pod about one CPU and let Kubernetes add pods.

Step 4: Let the CDN answer

The HTML is now the same for everyone, and Next.js already sends a cache header for ISR pages: Cache-Control: s-maxage=60, stale-while-revalidate=…. A CDN can follow that header. Two rules are left for you to add:

Drop tracking params from the cache key: utm_*, gclid, fbclid and any others your ads use. Keep params the page really needs, such as ?page=2 on a list.
Never cache per-user routes: skip the cache for /api/*, and for pages like /account when the session cookie is set.

Every CDN has these as cache rules. The names differ, so check yours. At the peak of 10,000 rps, the CDN answers about 95% of page views. About 500 rps reach the cluster, and six small pods handle that easily (illustrative split).

Send `/api/*` to the backend at the ingress

After step 2, the browser calls the backend for per-user data: /api/me, /api/cart. In production we sent those calls through a proxy in the Next.js pod. Don’t. Route them at the ingress instead:

# The browser calls the same domain, so cookies work and no CORS is needed.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop
spec:
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend: { service: { name: backend-api, port: { number: 80 } } }
          - path: /
            pathType: Prefix
            backend: { service: { name: web, port: { number: 80 } } }

Why the pod is the wrong place for this proxy:

It takes resources from rendering. Every API call uses CPU, memory and connections in the pod that also builds pages. During a spike, page rendering competes with traffic that has nothing to do with HTML.
It scales the wrong thing. The autoscaler adds web pods because API traffic grew. You pay for more Next.js pods when the backend is the busy part.
A slow backend blocks the pages. If the backend slows down, open proxy connections pile up in the pod, and page requests wait behind them.
The backend loses its own limits. At the ingress or an API gateway, /api/* gets its own rate limits, timeouts and monitoring, apart from the web tier.

If you cannot change the CDN: an nginx sidecar

In production, our team could not change the CDN rules, so we put the same two rules into an nginx sidecar in each pod. For logged-out users this gave 500 rps per pod (measured). It works, but it is a cache per pod instead of one at the edge, so use it only as a fallback.

proxy_cache_path /tmp/nginx-cache keys_zone=html:20m max_size=500m inactive=10m;

server {
  listen 8080;

  location / {
    # These pages never read the query string, so drop it on the way in.
    proxy_pass http://127.0.0.1:3000$uri;
    proxy_cache html;
    proxy_cache_key $scheme$host$uri;
    proxy_cache_valid 200 30s;
    proxy_ignore_headers Cache-Control Expires;  # SSR sends no-store; nginx sets the TTL

    # Only needed while the HTML still depends on login (before step 2).
    proxy_cache_bypass $cookie_session;
    proxy_no_cache $cookie_session;

    # Stampede protection: one request refills, the others get the old copy.
    proxy_cache_lock on;
    proxy_cache_use_stale updating error timeout http_500 http_502 http_503;
    proxy_cache_background_update on;
  }
}

nginx still refuses to cache responses that set a cookie. Keep it that way, or one user’s session can end up in another user’s page.

Stampedes: when a popular page expires

At 10,000 rps, a page that expires can get hundreds of requests in the same moment. If they all miss, they all go to the origin together. This is a cache stampede. Every layer in this setup already protects against it:

ISR serves the old copy while one background render builds the new one.
The CDN follows stale-while-revalidate and, on most CDNs, sends one request upstream for many identical misses.
The nginx fallback uses proxy_cache_lock and proxy_cache_use_stale updating.

Step 5: Add up the delays

Every cache layer can serve data up to its own TTL old, and the layers stack. The data cache can be 60 s behind the backend. The ISR page can be built from that data and kept another 60 s. The CDN can keep that page another 60 s. Three minutes fits inside the 5-minute limit. If marketing ever asks for “changes in 30 seconds”, this table tells you what to change: shorter TTLs and a shared cache, not more pods.

The final architecture

Layer	Job	Key settings
CDN	Answers most page views	Tracking params dropped from the key; bypass `/api/*` and per-user pages
Ingress	Routes by path	`/api` → backend, `/` → web
Next.js pods	Build and serve HTML	ISR `revalidate = 60`; data cache 60 s; no `cookies()`, `headers()` or `searchParams` on cached pages
Browser	Handles everything per user	Login state from `/api/me`; layout from CSS media queries; campaign params for analytics

Glossary

Term	Meaning
rps	Requests per second.
SSR	Server-side rendering: the server builds the HTML for every request.
ISR	Incremental Static Regeneration: build once, serve the saved copy, rebuild in the background after a set time.
`revalidate`	How many seconds a saved page or data entry is used before it is rebuilt.
Data cache	Next.js storage for backend answers, used by `fetch`, `unstable_cache` and `'use cache'`.
Cache key	The value a cache uses to decide whether two requests are the same. Often the URL.
Hit / miss	The cache had the answer / the cache had to ask the next layer.
Tracking params	Query values added by ads and emails, such as `utm_source`, `gclid` and `fbclid`.
`stale-while-revalidate`	Serve the old copy right away and refresh it in the background.
Cache stampede	Many requests miss at the same moment and all hit the origin together.
Ingress	The Kubernetes entry point that routes requests to services by host and path.
Sidecar	An extra container in the same pod as the app, such as nginx.
Standalone output	`output: "standalone"`: Next.js built to run as its own Node.js server.

For AI agents

Based on the author’s work scaling a production Next.js app on Kubernetes. The architecture here is the recommended version: in production we used an in-memory LRU cache, two processes per pod under pm2, an nginx sidecar instead of CDN rules, and an API proxy inside the pod. Only three numbers are measured (40 rps per pod at the start, 500 rps per pod behind nginx, a 10,000 rps peak); the rest are illustrative. See also the Next.js docs on Incremental Static Regeneration, unstable_cache and self-hosting. Figures are simplified teaching models.

All posts

Frontend

Database

How to Scale a Next.js SSR App to 10,000 rps

The numbers

Step 1: Find out what the request waits for

Step 2: Make the HTML the same for everyone

Step 3: Render once, serve many times (ISR)

No shared cache: revalidate by time only

Step 4: Let the CDN answer

Send `/api/*` to the backend at the ingress

If you cannot change the CDN: an nginx sidecar

Stampedes: when a popular page expires

Step 5: Add up the delays

The final architecture

Glossary

For AI agents

​The numbers

​Step 1: Find out what the request waits for

​Step 2: Make the HTML the same for everyone

​Step 3: Render once, serve many times (ISR)

​No shared cache: revalidate by time only

​Step 4: Let the CDN answer

​Send /api/* to the backend at the ingress

​If you cannot change the CDN: an nginx sidecar

​Stampedes: when a popular page expires

​Step 5: Add up the delays

​The final architecture

​Glossary

​For AI agents

The numbers

Step 1: Find out what the request waits for

Step 2: Make the HTML the same for everyone

Step 3: Render once, serve many times (ISR)

No shared cache: revalidate by time only

Step 4: Let the CDN answer

Send `/api/*` to the backend at the ingress

If you cannot change the CDN: an nginx sidecar

Stampedes: when a popular page expires

Step 5: Add up the delays

The final architecture

Glossary

For AI agents