What an SEO audit found on a 3,000-page Next.js site
The site scored 89 out of 100, held there by one critical problem. Behind it were three faults a browser never shows, because they only appear to a crawler or only under a crawler’s load. What they were, how each was found, and what fixed it.
The site
Talval is an independent equity research site that publishes valuations for about 1,000 listed companies, with pages for each company, for superinvestors’ quarterly 13F filings and for daily verdict changes: nearly 3,000 pages, server-rendered with the Next.js App Router behind Cloudflare. It is built by the same person as Trakis, which is why we can show the code behind each finding. A Trakis audit reads up to 200 pages, chosen across every kind of page; these are the findings from 28 October 2026.
1. The sitemap listed pages the site refused
The one critical finding: /company/ABTC was in the sitemap and answered noindex. A sitemap that offers a page the page itself declines is a contradiction Search Console reports, and on its own it cost the grade 10.4 points.
The page had four reasons to send noindex, one of them a price older than 30 days. ABTC’s last close was 133 days old. The sitemap, though, asked a different question: does the company have a price at all? Six companies passed the sitemap and failed the page, five of them stopped on the same day, the mark of one feed gap.
The fix was to let the sitemap use the page’s own rule instead of a second, looser one. It went from 972 to 966 addresses, exactly those six, and a company whose prices return comes back on its own.
2. Crawlers saw the pricing page as “Loading plans…”
The audit flagged /pricing as thin at 114 words. It was not thin, it was broken: the whole page sat inside one Suspense boundary because a component read the URL’s query string, and in the App Router that makes everything up to the boundary render in the browser only. The HTML a crawler received was a heading and the words “Loading plans…”. The three plans, the feature table and the FAQ arrived only after JavaScript ran.
Development mode cannot show this, because there routes render on demand and the hook does not suspend. It took a crawler counting words. Reading the query string on the client after the first render kept the page static and put 584 words in the HTML.
3. Data pages thinned only for crawlers
Six pages came back with fewer than 250 words, one “under 150”, while a browser received 6,584 words on that same page. Two runs of the audit even picked different “thin” pages. The crawler was being served a different page from the one a visitor saw.
The site’s own API rate limit was the cause. Each server render was charged to the visitor’s address, so a crawler fetching 200 pages used up its own allowance of 120 API calls a minute. Refused calls rendered as an empty list with HTTP 200, and only successful responses were cached, so once a crawler’s allowance ran out every later page thinned again.
The fix was to give the reads that are the same for every visitor one shared cache entry. Ordinary traffic keeps it warm, and a crawler makes no API call at all: the same 200-page crawl went from 300 seconds to 38, with no failed page. The grade went from 89 to 99.6, then 100.
What the next audit caught
On 6 November the weekly audit’s list of changes reported three superinvestor pages that had lost 57 to 88% of their text, one from 1,580 words to 186, with their descriptions turned generic. Read again by hand, all three were whole. They had been read in a moment when their data did not load, and their error branch said “this filing couldn’t be loaded” with HTTP 200, which a crawler keeps as the page. Those pages now answer with a server error in that case, which a crawler retries instead of indexing.
What to take from it
- Count what the crawler receives. A page can be complete in a browser and nearly empty in its HTML. Word counts from a crawl are the cheapest way to see it.
- Let the sitemap ask the page’s question. When two pieces of code decide indexability separately, they drift apart.
- An error is not a page. A failed read that answers 200 gets indexed as content. Answer with a server error and the crawler comes back.
- A sample shows causes, not a fixed list. Two crawls of 200 pages read different pages. Chasing the count would have hidden that the crawler was being served a different site.
Audit your own site
The Trakis audit reads a site the way a search engine’s crawler does, groups what it finds by template and orders it by the traffic at risk. The example report shows what it looks like on a full site.
Last changed 9 November 2026.