How to Use Screaming Frog to Find Broken Links on Your Website

Few things frustrate a visitor like clicking a promising link and landing on a “page not found” error. Broken links waste your readers’ patience, send search engines into dead ends, and quietly leak the value of links other sites have given you. The frustrating part is that they accumulate invisibly. A deleted post here, a renamed product there, a mistyped address in an old article, and a year later a healthy-looking site can contain dozens of them.

Screaming Frog SEO Spider is the tool professionals reach for to find these problems quickly. It crawls your website the way a search engine does and lists every link with its status. This guide shows you exactly how to use it, step by step, to find broken links, export them into a manageable list, and fix them properly, including an honest explanation of what the free version can and cannot do.

What Screaming Frog Does and What You Need

Screaming Frog is a program you install on your own computer, available for Windows, Mac, and Linux. When you give it a website address, it follows links from page to page, recording each URL it finds, the status code each one returns, and details such as titles, headings, and images. Because it runs on your machine, crawl speed depends on your computer and connection rather than a monthly credit system.

The Free Version and Its URL Limit

The free version crawls up to 500 URLs per crawl. That number counts every page, image, and file the spider discovers, so a site with 150 pages and several images per page can reach the limit faster than you expect. For a small blog or business site, 500 URLs is still enough to find your broken links, and you can crawl key sections separately if needed. Larger sites, saving crawl configurations, and advanced features require the paid licence, which is a reasonable upgrade once you manage a big site or client work. Download the genuine program only from the official Screaming Frog SEO Spider page and user guide so you avoid bundled imitations.

Step by Step: Crawling Your Site for Broken Links

1. Install and open the program. The default settings are fine for a first broken link crawl, so do not change configuration options you do not understand yet.
2. Enter your homepage address. Type your full site address, including whether your site uses www or not, into the address bar at the top.
3. Press Start and let the crawl finish. Watch the progress bar. On a small site this takes a few minutes. Interrupting early gives you an incomplete, misleading picture.
4. Open the Response Codes tab. Once the crawl reaches 100 percent, this tab lists every URL grouped by the response it returned.
5. Filter by status code. Use the filter options to show client errors, the 4xx family. The entries marked 404 are pages that no longer exist.
6. Sort and review. Click the Status Code column to group errors together and confirm which broken addresses the crawl actually found on your site.

Understanding Status Codes Before You Act

• 200: the page works normally. No action needed.
• 301 and 302: the page redirects elsewhere. One redirect is fine, long chains of them are worth tidying.
• 404: the page is not found. This is your main target list.
• 5xx: a server error. These are more urgent than 404s because they suggest the server failed on a page that may still exist.

Finding Which Pages Link to Each Broken URL

A list of broken addresses alone is not enough, because you fix a broken link on the page that contains it, not on the missing page itself.

Using the Inlinks View

1. Select a broken URL in the Response Codes results.
2. Look at the lower window and open the Inlinks detail, which lists every crawled page that links to that broken address.
3. Note the source pages and anchor text for each broken target, so you know exactly what to edit later.
4. Repeat for each 404, or move to exporting if the list is long, which is usually the more practical route.

This distinction matters. A broken page that no internal page links to, and no other website links to, may simply stay a 404. A broken page linked from ten of your own articles needs fixing today.

Exporting Your Broken Link List

Bulk Export of 404 Addresses

For a working list, use the bulk export option for client error URLs. This produces a spreadsheet containing the broken addresses along with the source pages that link to them. Working from a spreadsheet lets you tick fixes off, share the list with a developer or writer, and re-check progress after you re-crawl.

Organising the Export Before Fixing

Sort your export into three groups before touching the site. Group one: broken internal pages that still should exist, usually because an address changed. Group two: links containing obvious typing mistakes. Group three: pages that were deliberately removed and are gone for good. Each group gets a different fix, which the next section covers, and grouping first prevents the classic error of redirecting everything randomly to the homepage.

If you are building a wider maintenance routine, the step-by-step tech guides on Network Ustad cover other free tools that pair well with a monthly crawl.

How to Fix Broken Links Properly

Update the Link at Its Source

The cleanest fix is editing the linking page so it points directly to the correct, live address. Do this whenever the intended destination still exists somewhere on your site. It keeps your internal linking fast and clear for both readers and crawlers, with no redirect in between.

Redirect Pages That Moved

If a page’s address changed permanently and other sites link to the old address, add a permanent 301 redirect from the old URL to the closest matching live page. Choose the most relevant destination, not automatically the homepage. Redirecting a missing guide about email setup to a homepage full of unrelated headlines helps nobody and search engines treat mass homepage redirects as effectively useless.

Remove or Replace Dead References

When the destination is gone and nothing sensible replaces it, edit the sentence around the link. Either link to a different helpful resource or rewrite the text so it no longer promises a link. Leaving the link and hoping nobody clicks it is not a strategy.

Handling Broken External Links

Your crawl can also surface links pointing out to other websites that have since vanished. You cannot redirect someone else’s site, so update your page to link to a working source, use an archived reference if appropriate, or remove the link and adjust the wording.

Mistakes to Avoid When Fixing Broken Links

• Redirecting everything to the homepage. It looks tidy and achieves almost nothing for readers or rankings. Always redirect to the closest relevant page.
• Fixing the target instead of the source. Recreating a deleted page just to satisfy one stray link is usually backwards. Edit the linking pages first.
• Crawling only part of the site and declaring it clean. An interrupted crawl misses sections. Always let the crawl finish and check the URL count against your expectations.
• Ignoring 5xx errors while chasing 404s. Server errors on live pages are more damaging than a missing old page. Escalate those to your host or developer.
• Never re-crawling after fixes. A short verification crawl proves the errors are actually gone and catches any new redirect chains your fixes created.

Conclusion

Finding broken links with Screaming Frog follows a simple rhythm: crawl the whole site, filter the Response Codes results to 404 client errors, use the inlinks data or a bulk export to find which pages contain each broken link, then fix at the source by updating, redirecting to a relevant page, or removing dead references. The free version’s 500-URL limit is honest and manageable for small sites, especially if you crawl in sections. Run this process every few months and after any redesign or content clean-up, and your site will stop leaking visitors and link value into dead ends.

Frequently Asked Questions

Is the free version of Screaming Frog enough for my site?

If your whole site, including images and files, stays under roughly 500 URLs per crawl, yes. For larger sites you can crawl important sections separately for free, but a paid licence becomes worthwhile when broken link checks are a regular part of your work.

What is the difference between a 404 and a 5xx error?

A 404 means the server responded but the specific page does not exist. A 5xx error means the server itself failed to handle the request, which can affect pages that should be working, so 5xx errors deserve faster attention.

Should every 404 page be redirected?

No. Pages that never had real links or traffic and were removed deliberately can safely remain 404. Redirect pages that moved, that other sites link to, or that your own content still references, and always choose a relevant destination.

How often should I check for broken links?

Every few months is sensible for most sites, and immediately after a redesign, a domain or address change, or a large content deletion. Broken links build up gradually, so regular short checks beat rare marathon sessions.

Can Screaming Frog find broken links on a site I do not own?

Technically it can crawl any public site within its limits, but be polite about crawl speed on sites that are not yours and respect their terms. For your own maintenance work, your own site is where the fixes are in your control anyway.