Do I need a sitemap?

Sitemaps make it easier for crawlers to find things on your site. Check out the standard format for sitemaps here. You can get more details on what search services have to say here:

If you have a simple Jekyll site with posts that are properly linked from your index, there may not be much of a benefit, but it’s easy to do, and if there is any payoff with search engines, it’s worth it.

jekyll-sitemap

The Jekyll team maintains the jekyll-sitemap plugin. Check out the source here. The instructions are very strait forward, I’m not going to repeat them here.

Once you’ve completed the installation, you will be able to go to http://localhost:4000/sitemap.xml Optionally, according to Bing webmaster help, you can add a line to a robots.txt file to reference the sitemap. To reference this the Jekyll way, create a file called robots.txt in the root of your blog and put this inside:

---
layout: null
---
Sitemap: {{ site.url }}/sitemap.xml

The first three lines tell Jekyll to parse and generate this file. The {{ site.url }} statement will pull the url setting defined in _config.yml. It’s better to do it this way to follow the DRY (Don’t Repeat Yourself) principle. If you have multiple environments, generating the sitemap based on environment is very useful. Even if you don’t plan on having multiple environments, if you ever change the domain your site uses, you only need to update the setting in _config.yml.

Getting Fancy

If you’re doing CI/CD, you can submit your sitemap directly to search engines as a part of a your release process. For example, with Bing, you can pass your sitemap directly to Bing:

curl -I https://www.bing.com/ping\?sitemap\=https%3A%2F%2Fspock.rocks/sitemap.xml

If you get a 200 back, you know it went through. This doesn’t mean your site will immediately be crawled, but you should expect search results to update more quickly than they would if you were waiting for the search engines to periodically poll your site.