What is a Sitemap? A Success Guide
Everything You Need to Know About Creating and Implementing Website Sitemaps
What is a sitemap? Nearly every website owner has heard of a sitemap but doesn’t know how to create, manage, and troubleshoot one to serve your website for years to come.
Unlock the secrets of improving your website’s online visibility with our in-depth Success Guide to What is a Sitemap? Whether you’re a seasoned pro or new to SEO, our comprehensive guide will explain how to create a sitemap, where to upload it, and how to troubleshoot possible issues.
Table of Contents
- What is a Sitemap?
- Website Sitemap Types: XML and HTML
- Do I Need a Website Sitemap?
- What Should Be Included in a Website Sitemap?
- What Should Not Be Included in a Website Sitemap?
- How Do I Create an XML Sitemap?
- How Do I Create an HTML Sitemap?
- How to Submit an XML Website Sitemap to Google Search Console
- Website Sitemap FAQ
What is a Sitemap?
A sitemap is a file featuring a list of web pages, images, videos, and other files on your website.
The sitemap helps search engines understand the relationship between the pages and files featured within it. It could also be described as a blueprint search engines use to help them better understand how to crawl and index your website more effectively.
Your sitemap file should contain your website’s most important pages and page information. Page information could include when the page was last updated, how often the page is changed, and any alternative language versions of a page. Your content information could include the content’s running time, the category it falls into, and the age rating.
Website Sitemap Types: XML and HTML
XML Sitemaps
XML sitemaps (Extensible Markup Language) focus more on helping search engines discover the pages on your website. It follows a specific format that makes crawling more efficient for search engines. These sitemaps give a complete map of your website and the importance of all the pages on a site.
Every website should have an XML sitemap regardless of its size. However, it’s crucial for larger websites due to crawl depth issues. An XML website sitemap helps prioritize the pages for search engines to focus on first.
HTML Sitemap
An HTML sitemap looks like a regular web page. It features an organized list of the most important pages on a website for people to find. HTML sitemaps focus more on website visitors instead of search engines.
This type of sitemap is a great opportunity to create order out of chaos that is worth making the effort out of for your site. Additionally, it can help improve your internal linking.
As a reminder, an HTML sitemap should not replace an XML sitemap. A website can have both, but a sitemap for SEO is an asset that can help increase performance.
Do I Need a Website Sitemap?
Yes, website sitemaps are extremely important and beneficial for websites and SEO. While it’s not a requirement, we strongly recommend that all websites create sitemaps regardless of size.
It’s important to note that while a sitemap for SEO is essential, Google doesn’t guarantee that it will crawl or index the URLs in the file in the order they’re placed.
Sitemaps for SEO are a valuable tool because they help:
- Uncover issues that need immediate attention.
- Search engines understand the most essential pages to crawl and index.
- Build trust with search engines when sitemaps are updated regularly.
If you must choose which website sitemap to feature first, create the XML sitemap and then work on the HTML sitemap for your site visitors as an additional aid in navigating your website.
Here’s what Google has to say on the matter.
You Might Need a Sitemap If:
- Your site is large. As a result, it’s more likely Google web crawlers might overlook crawling some of your new or recently updated pages.
- Your site has a large archive of content pages that are isolated or not well-linked to each other. If your site pages do not naturally reference each other, you can list them in a sitemap to ensure that Google does not overlook some of your pages.
- Your site is new and has few external links to it. Googlebot and other web crawlers crawl the web by following links from one page to another. As a result, Google might not discover your pages if no other sites link to them.
- Your site has a lot of rich media content (video, images) or is shown in Google News. Google can take additional information from sitemaps into account for search, where appropriate.
You Might Not Need a Sitemap If:
- Your site is “small”. By small, we mean about 500 pages or fewer on your site. (Only pages you think need to be in search results count toward this total.)
- Your site is comprehensively linked internally. This means that Google can find all the important pages on your site by following links from the home page.
- You don’t have many media files (video, image) or news pages that you want to show in search results. Sitemaps can help Google find and understand video and image files or news articles on your site. You might not need a sitemap if you don’t need these results to appear in Search.
What Should Be Included in a Website Sitemap?
- Pages that are returning a 2xx status code, meaning live pages.
- The most important web pages and posts to be crawled and indexed.
- High-quality, SEO-optimized images and other visual assets. Place them in their own sitemap files.
- The ‘last modification time’ field. This tells search engines when the page was last updated.
- Add hreflang into the sitemap if your website is using it already.
What Should Not Be Included in a Sitemap?
- URLs returning a 4xx, 3xx, and 5xx status code.
- Canonicalized URLs.
- URLs that are blocked within the robots.txt file.
- Noindexed URLs.
- Paginated URLs.
- Orphaned URLs.
- Pages with duplicate content.
- Pages with thin content that do not drive value or traffic to your website.
You do not want to overburden the XML website sitemap file with pages that don’t offer value to potential site visitors and will not generate as much traffic. The sitemap file should serve as a tool for search engines to understand the priority pages of your website. This increases the likelihood of those pages being crawled and indexed in a timely manner.
Websites can also organize their sitemaps into sub-sitemaps. This has the additional advantage of making sitemaps smaller and faster to process.
Need help in managing your client’s SEO strategies? Learn more about Boostability’s white label SEO services!
How Do I Create an XML Sitemap?
You can create an XML sitemap with the Yoast plugin for WordPress websites or use the Screaming Frog Sitemap Generator. Yoast is the simplest and quickest solution, but not every website is built on a WordPress CMS.
Follow the steps below to create your own XML website sitemap file using both Yoast and Screaming Frog.
Yoast Plugin on WordPress
If you have a WordPress website, the most common way to create a sitemap is through the Yoast plugin. Install Yoast onto your website, and it will automatically generate an XML sitemap. It also automatically updates your website sitemap file as new pages are published on your site.
Access your sitemap file within Yoast by:
- Going into the Yoast Settings
- You can either type “sitemap” into the search bar or find it within “Site Features.”
- The XML sitemap is nestled under the API category within “Site Features.”
- Click on the “View the XML Sitemap” button. Your sitemap file will look like this.
Screaming Frog
Screaming Frog is a website crawling tool with extensive capabilities. As a longtime favorite of ours for many auditing and SEO needs, it’s an excellent option for creating a website sitemap.
Follow the steps below to create a sitemap in Screaming Frog Spider:
- Crawl the entire website.
- Click “Sitemap” in the top navigation bar. Click on “XML Sitemap.”
- Specify which pages to include in the sitemap, like pages returning a 2xx status code. Do not check the 3xx, 4xx, and 5xx boxes — these are response codes you don’t want in your sitemap.
- After the configuration step, you can manually exclude any additional URLs from the XML sitemap file before it’s created. Highlight the ones you would like removed from the list, right-click, and select “Remove.”
- The next step will be to check the “last modified date” box so this information shows up in the XML website sitemap file. Click the dropdown and select “use server response.”
- Several optional tabs are included in the configuration process: priority, change frequency, images, and hreflang. Specifying the Priority and Change Frequency tags is unnecessary because Google is currently ignoring them.If you have hfreflang set up on your website, enable this for the XML website sitemap as well.The Images tab will allow you to include image URLs in this website sitemap file. However, we recommend creating a separate one for images to organize things better.
- Click “Next” to finalize the sitemap creation after completing the necessary configuration tabs. An XML sitemap file will be downloaded to your computer. Submit this to your web developer to publish onto your website and the robots.txt file.
How Do I Create an HTML Sitemap?
You will create an HTML sitemap like any other page within your CMS platform. Here are some best practices to consider:
- DO NOT noindex your HTML sitemap. You want search engines to crawl and index it in search results.
- Organize your content clearly. Group your content logically into main sections or categories. Additionally, use headings and subheadings to create a hierarchy that reflects the structure of your website.
- Use descriptive anchor text for the pages linked in the HTML sitemap. People should have an understanding of what the page will cover before even clicking on the link.
- Maintain a clean and simple design to improve the page’s user experience. Avoid overwhelming visitors with too much information. Use CSS styling to make the sitemap visually appealing and easy to read.
- Prioritize the most important pages you want website traffic to go to. Consider placing the most important or popular pages at the top of the sitemap. Use visual cues like bold text or different colors to highlight key sections.
- Include the pages that are important for people to see, even if they’re not optimized for SEO. Make sure your HTML sitemap includes links to all relevant pages on your website, like the Contact and About Us pages. Include not only main pages but also important subpages or sections.
- Link to the HTML sitemap within the footer. This ensures it’s easily accessible from any page by both search engines and people.
- Make your HTML sitemap responsive. People will be looking at your website on mobile devices. Your HTML sitemap should be well-designed and functional across ALL mobile devices.
- Optimize for SEO. While the primary purpose of an HTML sitemap is user navigation, it can also benefit SEO. Ensure that your sitemap includes relevant keywords. Submit your sitemap to search engines through their webmaster tools to aid in the indexing process.
- Update the HTML sitemap regularly. Whenever you add, remove, or update pages, reflect those changes in the HTML sitemap. An outdated sitemap can lead to confusion for both users and search engines.
How to Submit an XML Website Sitemap to Google Search Console
Once your XML website sitemap file is live on your website, submit it to Google Search Console so Google can crawl and index the pages. This can help your SEO by speeding up the process of Google finding and crawling it, as well as identifying possible issues to fix immediately.
Here’s how to submit an XML website sitemap to Google Search Console:
- Go to Google Search Console.
- Select the desired website property under the dropdown menu.
- Click “Sitemaps” under the “Indexing” menu.
- Copy and paste in the slug of the XML website sitemap (either /sitemap.xml or /sitemap_index.xml).
Google Search Console will show if the sitemap crawl was successful and how many URLs were found in the website sitemap file.
If you notice that important pages aren’t being crawled and indexed, double-check that they’re in your XML sitemap and resubmit the file to be crawled within Google Search Console.
Website Sitemap FAQ
1. How Big Should a Sitemap Be?
A website sitemap file should be less than 50,000 URLs and MUST NOT be larger than 50 MB. If you have more than 50,000 URLs you wish to include in a sitemap file, you will need to create smaller sub-files.
It’s a recommended and common practice to create separate sitemap files from Page and Post URLs to store images, videos, and other types of content you want indexed.
2. How Do I Know if a Website Has a Sitemap?
There are several ways to check if a website has a sitemap: Google Search Console, manually, search operators, or a robots.txt file.
Google Search Console
This only applies if you have access to the website’s Google Search Console account. If so, navigate “Sitemaps” within the left-hand navigation under the “Indexing” menu.
If the website has a sitemap that’s been submitted for crawling, it will show something like this:
Manually
Most sitemaps can be found by featuring /sitemap.xml after the domain. For example:
https://domain.com/sitemap.xml
The Yoast plugin in WordPress can slightly differ by using /sitemap_index.xml at the end of the URL. Give both a shot to see which one populates the XML sitemap for the website.
Search Operators
Search operators are useful for many SEO needs, including searching for a sitemap link on the search engine itself. A search operator consists of phrases and symbols that can be added to a search query, telling search engines to deliver specific results.
For example:
- “site:boostability.com filetype:xml”
- “site:boostability.com inurl:sitemap”
- “site:boostability.com intitle:sitemap”
Note: Simply replace “boostability.com” with the website domain you want to find the website sitemap for.
Robots.txt File
The robots.txt file is another way to find a website sitemap. Simply add “/robots.txt” to the end of the domain name in the URL to be taken to the file. The sitemap file is typically listed at the bottom of the robots.txt directives.
3. How Often Do You Need to Submit a Sitemap to Google?
Resubmit your XML website sitemap into Google Search Console for crawling when your website has undergone major changes or if you notice important pages are not being indexed.
It’s a good practice to review your sitemaps during your SEO maintenance to ensure that correct URLs are featured in the file and remove any redirects or broken links. Make it a regular practice to update your sitemap. Otherwise, Google will not trust it.
It’s essential to maintain your sitemap for SEO reasons. A sitemap file can also contain broken or redirected links if it’s not updated as pages are removed and new ones are created. As a result, the wrong URL can be featured in a sitemap file, which will not help increase website rankings.
