Technical SEO Audit: Find What's Silently Killing Your Rankings
Only 62% of the illustrative firm's 3,900 URLs were indexed. Run the five-step technical audit — crawlability, indexation, speed, architecture, prioritization — on a free tool stack.
The most expensive SEO problems make no noise. A page returns a healthy status code, renders perfectly in a browser — and still never enters Google's index. Rankings slide a few positions a month, impressions flatline, and by the time anyone investigates, the compound cost is measured in quarters, not weeks. Technical SEO fails silently because nothing "breaks": the site works for people and fails for the crawler.
This guide is the audit framework we run before any serious SEO engagement, in five steps: crawlability, indexation, speed, architecture and internal links, then prioritization. Our running example is an illustrative 14-attorney personal-injury law firm we will call Halbrook Law, with about 3,900 URLs — typical mid-market territory. At marketplace scale the same problems get orders of magnitude worse, and we will point to what that looks like in a real recovery. Every number attached to Halbrook is modeled, with the assumptions shown; the tool stack at the end costs nothing to run.
Here is the short version before we crawl in.
Key Highlights
- A technical audit answers four questions in order: can Google reach your pages, read them, store them in the index, and find them again through your links.
- In the illustrative Halbrook audit, only 62% of 3,900 submitted URLs were in Google's index; 19% were crawled but not indexed.
- 37% of the firm's reachable pages sat four or more clicks from the home page, and 170 pages were orphans with no internal links at all.
- Bounce probability rises 32% as mobile load time goes from one to three seconds (Think with Google), and Deloitte measured an 8.4% conversion lift from a 0.1-second speed improvement in retail.
- Google's Core Web Vitals thresholds are LCP 2.5s, INP 200ms and CLS 0.1, measured at the 75th percentile of real users (Google).
- Since July 2024, Google crawls with a smartphone user-agent only — mobile rendering is the crawler's reality, not a side project.
- At marketplace scale, a crawl trap of 4.5 million junk URLs collapsed organic traffic 64% before a recovery reindexed 1.2 million commercial pages in 30 days (our case study).
- The Halbrook audit produced 214 findings — but 18 critical ones carried most of the recoverable traffic. Prioritization is the audit's real output.
What a Technical Audit Actually Is (and Why Problems Stay Silent)
A technical audit is not a review of your keywords or your content calendar. It inspects the machine underneath: how Googlebot reaches your pages, renders them, decides which ones belong in the index, and navigates between them. Content and links decide how well you rank; the technical layer decides whether you are playing at all. That is why audits come first — everything else compounds on top of them.
The problems stay silent because browsers are forgiving and crawlers are literal. A page that looks fine to a human can be invisible to Google because of one leftover noindex tag, a canonical pointing somewhere else, or content that only exists after JavaScript runs. None of it shows up as an error anyone sees; it shows up as traffic that never arrives.
Four symptoms usually trace back to technical causes:
- Organic traffic plateaus even though you keep publishing and earning links.
- New pages take weeks to appear in search, or never appear in Search Console's Page indexing report at all.
- Rankings swing violently after releases, redesigns or platform changes.
- Paid spend quietly grows to backfill traffic that used to be free.
Key takeaway: If search is a growth channel, the technical layer is load-bearing infrastructure. Audit it on a schedule — not after traffic falls off a cliff.
Audit Step 1: Crawlability — Can Googlebot Reach Your Pages?
Crawlability asks the first question: can Googlebot get to the URL at all, without being blocked, misled or trapped? The usual suspects are a robots.txt that accidentally disallows important sections (or the CSS and JavaScript Google needs to render), parameterized URLs that generate infinite crawlable permutations, and content that only exists after client-side JavaScript executes. Google's own crawl-budget documentation notes that sites with fewer than a couple of thousand URLs do not need to worry about crawl budget — but crawl waste and crawl traps hurt at almost any size.
At Halbrook's scale, the crawl check is simple; at scale it is an incident. In our marketplace recovery, a platform migration generated roughly 4.5 million junk URLs — facet permutations, session variants, error states — and Googlebot spent its finite attention on junk while commercial pages went stale; the full story is in our technical SEO case study. The lesson scales down: whatever your size, Google's attention is finite, and where it goes is a decision you should make deliberately.
The crawlability checklist:
- Fetch robots.txt and confirm it blocks nothing you sell; staging and admin sections only.
- Compare raw HTML against rendered HTML for key templates — if your content exists only after JavaScript, Google still sees it, but later and less reliably.
- Check the Crawl Stats report in Search Console for where Googlebot actually spends requests.
- Cap crawl traps at the pattern level: controlled parameters, session IDs stripped, internal search results blocked.
Key takeaway: Crawlability is about direction, not permission. Make sure Googlebot spends its finite attention on pages that earn money — and none of it on junk.
Audit Step 2: Indexation — Did Google Keep What It Crawled?
Being crawled is not being indexed. Google crawls a page, evaluates it against every other page it knows, and decides whether to store it — and that decision is where most silent losses live. The evidence base is Search Console's Page indexing report, which shows, for every submitted URL, exactly which bucket Google filed it in. The "site:" search operator is a smoke test at best; treat its results as a hunch, never as a finding.
For illustrative Halbrook, the Page indexing audit over the firm's 3,900 submitted URLs told the whole story in one chart:
That 38% gap is the silent killer in its natural habitat: real pages, live on a working site, absent from Google. Each bucket points at a different cause, with a different owner:
| Indexing bucket | What it usually means | Where the fix lives |
|---|---|---|
| Crawled, not indexed | Google judged the page thin, duplicate or not worth storing | Content depth, canonical hygiene, deduplication |
| Discovered, not crawled | Google knows the URL exists but has not spent attention on it | Internal links, sitemap priority, crawl waste |
| Duplicate, wrong canonical | The page declares a different page as the original | Canonical tags, protocol and parameter handling |
| Soft 404 errors | The page looks live but behaves like a missing page | Real content, or a genuine 410/redirect |
The causes behind those buckets are almost always short and fixable:
- Leftover noindex tags from staging environments carried into production.
- Canonicals pointing away from the real page, telling Google a duplicate is the original.
- Duplicate or thin pages — filters, sort orders, print views — diluting the set.
- Dishonest sitemaps: Google ignores lastmod values when they are unreliable, so stale timestamps cost you the one prioritization lever sitemaps offer.
Fix the indexation layer before touching anything else. Content improvements on pages that are not indexed return exactly nothing; every downstream fix depends on this one.
Key takeaway: The Page indexing report is the most honest page in SEO. Read it per template, quantify every bucket, and make "indexed share" a number your team tracks monthly.
Audit Step 3: Speed and Core Web Vitals — Where Seconds Become Revenue
Speed is the audit step with the clearest price tag. Google measures Core Web Vitals — LCP (largest content paint, threshold 2.5 seconds), INP (interaction to next paint, 200 milliseconds, which replaced FID in March 2024) and CLS (cumulative layout shift, 0.1) — at the 75th percentile of real field users. Since July 2024, Googlebot crawls with a smartphone user-agent only, so the mobile experience is not a variant of your site; it is the site Google sees.
The revenue evidence is old, consistent and inconvenient. Think with Google research found bounce probability rises 32% as mobile page load goes from one to three seconds, and Deloitte's "Milliseconds Make Millions" study measured an 8.4% lift in retail conversions from a 0.1-second improvement. Halbrook's modeled lab data shows the typical pattern — one heavy template dragging the whole site:
We watched the real-world version of that chart at a news publisher: mobile LCP fell from 4.8 to 1.4 seconds across 900,000 article pages, and organic impressions rose 63% in four months — the details are in our Core Web Vitals case study. The free tooling covers this step completely: PageSpeed Insights for field and lab data, the CrUX dataset for real-user trends, and Lighthouse in CI to catch regressions before they ship.
Key takeaway: Speed is a revenue input with published price tags, and the diagnosis is free. Find the worst template, fix it first, and put a performance budget in CI so it stays fixed.
Audit Step 4: Architecture and Internal Links — The Paths That Rank Pages
Google discovers pages by following links, which makes your internal link graph a ranking instrument, not a navigation afterthought. Three properties decide whether it works: depth (how many clicks each page sits from the home page), coverage (whether every page has at least one internal link — no orphans) and anchor quality (whether links use words that describe the destination). Pages deep in the architecture get crawled less, transfer less authority and quietly decay.
The illustrative Halbrook crawl-depth audit shows the usual mid-market profile:
Four fixes reshape that distribution in weeks:
- Build hub-and-spoke structure: one practice-area hub linking to every service page, every attorney and the related articles beneath it.
- Reconnect orphan pages — 170 in the illustrative audit — from the hubs and menus they belong in.
- Fix pagination canonicals: paginated pages should canonicalize to themselves, not to page one, or page two onward never gets indexed.
- Use descriptive anchors in body copy and breadcrumbs; "read our car accident guide" beats "click here" for both users and crawlers.
Architecture is the cheapest step in the audit because every fix is internal: no designers, no lawyers, no third parties. It is also the step most audits skip, which is a gift to whoever does it.
Key takeaway: Every page should be reachable in three clicks with a link that says what it is. Fix the graph, and Google finds, understands and ranks more of what you already built.
Audit Step 5: Prioritization — Turn 214 Findings Into One Quarter of Work
A good audit produces a long list; a useful audit produces an order of attack. We score every finding on two axes — impact (how much traffic sits behind the problem) and effort (engineering, design and approval cost) — then band the results into critical, high, medium and low. The pattern is remarkably stable: a minority of findings carries most of the recoverable traffic, and the majority of findings are hygiene that can wait for a scheduled sprint.
For illustrative Halbrook, the scoring looked like this:
| Band | Definition | Typical fixes from the audit | Count |
|---|---|---|---|
| Critical | Pages absent from the index, or traffic actively bleeding | Staging noindex in production, canonical misfires, blocked templates | 18 |
| High | Large traffic at risk, moderate effort | Orphan reconnection, pagination canonicals, worst-template speed | 47 |
| Medium | Efficiency and resilience gains | Redirect chains, sitemap lastmod automation, schema gaps | 71 |
| Low | Hygiene, do when touching that code anyway | Anchor text polish, alt text, trailing-slash normalization | 78 |
Sequenced by that table, the modeled recovery curve for a mid-market site looks like this:
Note the shape: flat for the first month while critical fixes ship and Google re-processes, then compounding. Technical SEO pays on a lag, which is precisely why it needs a prioritized plan and patient measurement rather than a weekly verdict. Our own timeline expectations for SEO work are laid out in how long SEO takes.
Key takeaway: An audit without prioritization is a wish list. Rank every finding by traffic at risk, ship the critical band first, and measure in months — not days.
The Common Killers: What First Audits Most Often Find
After enough first audits, a pattern of repeat offenders emerges. For illustrative Halbrook, the severity distribution of all 214 findings looked like this:
The critical band almost always contains some combination of the same killers:
- The staging site is indexed. A forgotten subdomain with duplicate content has been splitting trust signals for years; robots.txt and noindex fix it in an afternoon.
- Noindex survived the last release. A template shipped with
noindexin production — every page on that template vanishes from the index without any error anyone notices. - robots.txt blocks assets. Blocking CSS or JavaScript means Google renders a broken page and evaluates what it sees.
- Redirect chains and 302s. Five-hop redirect chains dilute signals, and temporary redirects where permanent ones belong keep pages in review limbo.
- Protocol and trailing-slash duplicates. HTTP, HTTPS, slash and non-slash variants of the same URL fragment the page's identity across four addresses.
- Orphans from the redesign. Pages the new navigation forgot still exist, still sell, and get zero internal links.
- Everything canonicalizes to page one. Paginated archives and category pages that all point at the first page quietly deindex most of a catalog.
One boundary condition worth knowing: if traffic collapsed overnight and Search Console shows a manual action notice, that is not a technical fault — it is a links or policy penalty, and the response is forensic. We documented that path in our penalty recovery case study.
Key takeaway: The killers are repetitive, which is good news: a checklist catches most of them, and almost none require new content, new links or new budget to fix.
How Often to Audit — and the Free Stack That Does It
The audit stack below runs on $0 in tools, which removes the last excuse. What costs money is attention, and that is what the cadence is really buying:
| Tool | What it checks | Cost |
|---|---|---|
| Google Search Console | Page indexing, crawl stats, sitemaps, mobile usability signals | Free |
| PageSpeed Insights + CrUX | Field and lab Core Web Vitals per template | Free |
| Screaming Frog crawler | Broken links, redirects, canonicals, orphans, metadata (free to 500 URLs) | Free tier |
| Ahrefs Webmaster Tools | Backlink overview, top pages, crawl report for verified sites | Free |
| Rich Results Test + Schema validator | Structured data correctness | Free |
| Server logs | What Googlebot actually requested, hour by hour | Already yours |
Our cadence for mid-market sites runs on three rhythms:
- Quarterly mini-audit: one crawl, the Page indexing report and Core Web Vitals — two focused days.
- Annual full audit: the five steps in this guide, with the prioritization ledger rebuilt from scratch.
- Event-triggered audits: always after migrations, redesigns, platform changes or any traffic anomaly — the GearLoop disaster was preventable with a post-launch indexation check, as its technical SEO case study shows.
Keep the findings ledger permanent — a running document, not a one-off PDF — so each audit updates the previous one instead of starting over. The performance side of the stack has its own deep dive in our Core Web Vitals guide.
Key takeaway: Audit on a rhythm: quarterly for the vitals, annually for the depths, and always after anything ships. The stack is free; the discipline is the product.
What Your Business Should Do
Here is the 90-day sequence we would run for a site that has never had a proper audit.
Days 1–30: Baseline the truth.
- Verify Search Console, pull the Page indexing report and quantify every bucket per template.
- Run a full crawl with Screaming Frog: status codes, canonicals, noindex tags, redirect chains, orphans, depth.
- Test the worst templates in PageSpeed Insights and record field LCP, INP and CLS as your baseline.
Days 31–60: Ship the critical band.
- Fix every indexation killer: staging exposure, leftover noindex tags, canonical misfires, duplicate templates.
- Reconnect orphan pages and rebuild hub-and-spoke internal links for your money pages.
- Attack the single slowest template; set a performance budget in CI so regressions block release.
Days 61–90: Re-measure and institutionalize.
- Re-pull the indexing report and crawl: indexed share, crawl waste and vitals should all have moved.
- Work down the high band — redirects, sitemap automation, pagination, schema — on a weekly sprint rhythm.
- Calendar the quarterly mini-audit, assign a named owner, and keep the findings ledger permanent.
The first audit is the only one that hurts. After it, the site becomes a machine you monitor instead of a mystery you mourn.
The Bottom Line
Technical SEO is the quiet half of the discipline: nobody praises an index that works, and everybody pays for one that doesn't. Crawlability, indexation, speed, architecture and prioritization — five steps, mostly free tools, and a cadence that keeps the silent killers from regrouping. The best time to audit was before the traffic fell; the second-best time is this quarter.
If you want a professional pair of eyes on your site's technical layer, that is a normal Tuesday for us. Book a free strategy call and we will walk your site through the five-step audit — or send us a message and we will tell you honestly whether you need us at all.
Free download: the Learning Guide (PDF)
Everything worth keeping from this article — key numbers, tables and the action checklist — in one print-ready, fillable PDF.
Join our newsletter — more growth tips, twice a month
Playbooks from real projects. No spam, unsubscribe anytime.
FAQ
How long does a technical SEO audit take?+
A focused mid-market audit — crawl, indexing report, vitals and prioritization — takes one to two working weeks, including the write-up. Fixing what it finds takes another four to eight weeks depending on engineering capacity, and Google needs additional weeks to re-crawl and re-process. Plan on seeing measurable index and speed movement inside one quarter.
Can I run a technical audit myself, or do I need an agency?+
You can absolutely run the first pass yourself — Search Console, Screaming Frog and PageSpeed Insights are free, and the five steps in this guide are the whole framework. Where agencies earn their fee is scale and judgment: prioritizing hundreds of findings by traffic at risk, and reading server logs when crawler behavior and dashboards disagree. Start solo; escalate when the findings list outgrows your engineering bandwidth.
How often should I audit my website?+
Quarterly for the health check — crawl, Page indexing report, Core Web Vitals — and a full five-step audit annually. Add an event-triggered audit after any migration, redesign or platform change, because that is where catastrophic indexation damage happens. Sites in the middle of active recovery work run weekly checks until the trajectory is stable.
Why did my rankings drop when nothing changed on the site?+
"Nothing changed" almost never survives an audit. Releases deploy without changelogs, hosting and CDN configurations drift, certificates and redirects expire, and competitors ship — while Google itself re-evaluates constantly. If the drop was sudden and Search Console shows a manual action, it is a penalty rather than a technical fault; otherwise, a crawl-and-compare audit against your last known-good state will find what moved.
Engineering-led team at Emerson Technology — building apps, web platforms and growth systems for ambitious businesses. Every guide comes from real client work.