Notifications
Clear all

Robots.txt Shopify Expert Question?

2 Posts
2 Users
0 Likes
261 Views
TrafficdeveloperSEO1
(@trafficdeveloperseo1)
Posts: 44
Eminent Member
Topic starter
 
Any Shopify robot.txt experts out there got a question about blogs and indexing - by default
Shopify sets these parameters -
Disallow: /blogs/*+*
Disallow: /blogs/*%2B*
Disallow: /blogs/*%2b*
Disallow: /*/blogs/*+*
Disallow: /*/blogs/*%2B*
Disallow: /*/blogs/*%2b*
My question to any of the experts of Shopify out there is should these be removed if your blogs are not being indexed?
 
Posted : 08/12/2021 2:28 pm
Nate Olsen
(@nateolsen)
Posts: 249
Illustrious Member Admin
 

It might look like it's blocking your actual blog posts but remember that Shopify's structure also includes the name of the blog. For example, many people have theirs like /blogs/news/articles so in this case, the robots.txt is blocking the /blogs/news archive page. So the default file won't stop your blogs from being crawled (Shopify isn't the best for SEO when compared to WordPress but they would never want to block the blog posts from being crawled/indexed haha)Β πŸ˜„Β the best way to find out is using the Google robots.txt tester tool. Simply enter your file, then put the URL of the page you want to test in the bottom bar, then it'll tell you if it's allowed to crawl it or not based on your file:Β  https://www.google.com/webmasters/tools/robots-testing-tool?utm_source=support.google.com/webmasters/&utm_medium=referral&utm_campaign=%206062598

 
Posted : 08/12/2021 2:29 pm
Share: