“Oops! That page can’t be found.“
Often when we google on the internet to found that the actual page is no longer exists. We call it “404 error page”. But what has caused this Google 404 error pages to happen and what is a 404 error.
Technically, A 404 error is an HTTP status code that means that the Googlebot tried to crawl a page that doesn’t exist on your site. Googlebot finds 404 pages when other sites or pages link to that non-existent page.
How does a ‘Google 404 error’ occur?
Here are some typical reasons that trigger an 404 messages appear. These include:
- The website content has been removed or moved to another Universal Resource Locator (URL)
- You change your permalink or the link structure of your website. (This mostly happens when changing design, or changing permalinks.)
- Somebody linked your website to a misspelled link or to an otherwise incorrect link.
- The URL or its content (such as files or images) was either deleted or moved (without adjusting any internal links accordingly)
- The URL was inadvertently written incorrectly (during the creation process or a redesign), linked incorrectly, or typed into the search bar incorrectly
- The server responsible for the website is down or the connection is broken
- The requested domain name can’t be converted to an IP by the domain name system (DNS)
- The entered domain name doesn’t exist (anymore)
Dead links are often referred to those links left for long periods of time since Webmasters have no knowledge that the linked content has been deleted or moved. Many websites still appear in the search engine results pages(SERPs) even though they aren’t available online anymore (or at least not at the specified URL). Other linked websites such as blogs, news sites, etc. are often not informed that the site has been removed or can now be found under a new URL. Many Webmasters don’t check their external links regularly and therefore a functioning link could easily become a dead one.
However, the Google 404 errors are probably the most misunderstood crawl error by the beginners.
We all know that error pages offer an obviously negative user experience, but how do they impact search engines especially Google?
- Are 404 pages harmful for SEO?
- Do Google penalize websites with too many 404 status code pages?
- This definitely will result in Google dropping those URLs from the search results, if not fixed.
In this article, I will look at 404 error pages from the perspective of both users and search engines, and will recommend some best practices that one can follow to improve their 404 error pages.
Google clearly states in their guidelines:
“Generally, 404 errors don’t affect your site’s ranking in Google, so you can safely ignore them.”
Google bots are smart, and they don’t care much about 404 error pages. But if you have too many 404 error pages Google will drop some of these URLs from the search results, if not fixed and may result in a considerable decrease in visitor numbers for the website. The visitor loses trust if the site is full of broken links and will definitely follow another link.
Another disadvantage is of the 404 error is that it can cause you to miss out on important links from other domains. This is not something that I worry much about, but when a user lands on a website by way of a referral link and sees a 404 pages, it usually makes them go away.
“When faced with 404s, my thinking is that unless the page:
A) Receives important links to it from external sources,
B) Is receiving a substantive quantity of visitor traffic, and/or
C) Has an obvious URL that visitors/links intended to reach
It’s OK to let it 404.”
Finding 404 links and fixing them for users as well as for SEO:
First, let’s us figure out what you can do with 404 error pages, and how to deal with them to your best advantage.
We first need to identify any 404 pages on your blog. There are numerous free tools available to help you find these broken links more easily. Five of the best and most well-known are:
1. Google Search Console (formerly known as ‘Google Webmaster Tools’): if you already have a Google account and have registered your website there, you should make use of the Google Search Console option. Any 404 errors found by the Google crawler are displayed in the web tool and can also be marked as corrected here too. Additional functions enable you to find errors in robots.txt files and use crawling statistics to work out how often your site has been crawled by Google crawlers.
Login to your Webmaster tool dashboard > Crawl errors > Not Found>
Click on any of the links and you will see “linked from” which will give you an idea of where these pages are linked from. This is also handy for finding potential link juice on your website.
2. Dead Link Checker: one of the simplest and fastest tools for finding both internally and externally linked 404 pages is the Dead Link Checker. With this web app you simply enter the URL of the site you want to inspect and then start the check. Here you have the choice of checking a single web page or a whole site. The app lists all the tracked error pages with status codes and URL.
3. Online Broken Link Checker: Check your blogs and websites to find out the dead links. You can scan an unlimited number of web pages. It works for both external and internal links. It indicates the broken links in your webpage’s HTML section. The tool runs on Windows, iOS, Linux / UNIX, and Mac OS X. You can scan up to 3000 pages for free, but hyperlinks in those pages are unlimited.
4. W3C Link Validation Tool: This tool finds out the problems related to anchors, links, web pages, CSS, and frequently to your whole website. To get the best output, first ensure that it is using CSS and Valid (X) HTML Markup. This free broken link checker is a part of Quality Web Tool and W3C’s Validators.
Once you have a list of 404 pages for your domain, here are several solutions for dealing with them:
If your important page is showing up as a 404 error page and you don’t want it to be, take these steps:
- Ensure the page is not deleted or in draft mode and has published from your Content Management System
- Ensure the 404 error page’s Universal Resource Locator (URL) is the right page and not another variation.
- Check whether this error shows up on the www vs non-www version of your site and the http vs https version of your site.
- If the error is arising due to a misspelled link from another domain, you can either ask the webmaster to update the link, or you can use the 301 redirection.
- Sometimes, you delete pages from your website and people still come to your website to find those page you deleted. If somebody tries to find a deleted page, they’ll get a 404 error. To fix this, you can usually restore the deleted page.
- If you don’t want to revive the page, but want to redirect it to another page, make sure you redirect it to the most appropriate related post/article or categories including your site home page using 301 redirect. If you are using WordPress, you can use a 404 to 301 WordPress Plugin or Redirection Plugin to monitor and redirect 404 error pages from your dashboard.
- If no relevant article is on your website related to the 404 link, simply let it be. Google will automatically remove such pages eventually.
- You can also manually delete such pages from the web index using the Google Search Console removal tool.
Here are some suggestions for creating an effective custom 404 pages that can help keep visitors on your site and help them find the information they’re looking for. The whole idea is to provide a better user experience and give your site a branding opportunity.
- Tell visitors clearly that the page they’re looking for is no longer present on the site.
- Use friendly and inviting language.
- Make sure your 404 page look and feel (including navigation) the same as the rest of your site.
- Be creative in designing your 404 page, you may want to check the 38 brilliantly designed 404 error page here.
- Consider adding links to your most popular articles or posts, as well as a link to your site’s home page.
- Think about providing a way for users to report a broken link.
- Add a Search Box.
- No matter how beautiful and useful your custom 404 pages, you probably don’t want it to appear in Google search results. In order to prevent 404 pages from being indexed by Google and other search engines, make sure that your web server returns an actual 404 HTTP status code when a missing page is requested.
- Use the Change of Address tool to tell Google about your site’s move.
Please do keep in mind is that if you have a site with hundreds of 404 pages, you do not want the search engine bots to waste their limited crawling resources on such pages. It is best to redirect whenever possible or to block bots from accessing these pages using Robots.txt, and remove them using the Google Search Console. This is ideal particularly when you have removed any directory or category/tag pages from your site.
Although 404 pages do not affect SEO, but it’s not a good idea to have a long list of 404 pages on your blog. My main concern with 404 pages is the poor user-experience they create. We must make the effort to address them as search engine optimization revolving around providing a good user experience.
Your turn, please tell us how do you address Google 404 error issues on your blog or website? Do you set a redirection or let it be 404?