Notifications
Clear all

Robots.txt Shopify Expert Question?

2 Posts
2 Users
0 Reactions
359 Views
TrafficdeveloperSEO1
(@trafficdeveloperseo1)
Estimable Member
Joined: 3 years ago
Posts: 46
Topic starter  
Any Shopify robot.txt experts out there got a question about blogs and indexing - by default
Shopify sets these parameters -
Disallow: /blogs/*+*
Disallow: /blogs/*%2B*
Disallow: /blogs/*%2b*
Disallow: /*/blogs/*+*
Disallow: /*/blogs/*%2B*
Disallow: /*/blogs/*%2b*
My question to any of the experts of Shopify out there is should these be removed if your blogs are not being indexed?

   
Quote
Nate Olsen
(@nateolsen)
Reputable Member Admin
Joined: 5 years ago
Posts: 277
 

It might look like it's blocking your actual blog posts but remember that Shopify's structure also includes the name of the blog. For example, many people have theirs like /blogs/news/articles so in this case, the robots.txt is blocking the /blogs/news archive page. So the default file won't stop your blogs from being crawled (Shopify isn't the best for SEO when compared to WordPress but they would never want to block the blog posts from being crawled/indexed haha) 😄 the best way to find out is using the Google robots.txt tester tool. Simply enter your file, then put the URL of the page you want to test in the bottom bar, then it'll tell you if it's allowed to crawl it or not based on your file:  https://www.google.com/webmasters/tools/robots-testing-tool?utm_source=support.google.com/webmasters/&utm_medium=referral&utm_campaign=%206062598


   
ReplyQuote
Share: