AI Fit
LHG-AIFitProfiler
LHG-AIFitProfiler is the crawler behind AI Fit, a free Lake House Group tool that suggests AI workflows for an online store. If you found it in your server logs, this page explains what it read and how to stop it.
It identifies itself with this user agent:
LHG-AIFitProfiler/1.1 (+https://lakehousegrp.com/en/ai-fit/crawler/)When it visits
It reads a store when someone enters that store's address in AI Fit. Lake House Group also runs it on a list of stores to build the comparison figures AI Fit shows. It does not search the web for stores to read.
What it reads
Only public pages that anyone can open without an account:
- robots.txt, once when a check starts
- the home page
- the sitemaps the store publishes
- the public product feed (/products.json on Shopify, the Store API on WooCommerce)
- a sample of public product, collection, blog and information pages
It only sends GET requests. It does not run JavaScript, keep cookies, log in, fill in forms or add anything to a cart.
How it follows robots.txt
It reads your robots.txt when a check starts and follows it for the whole check: the rules for LHG-AIFitProfiler, or the rules for all crawlers (User-agent: *) when none name it. It does not request a page you disallow. A change to robots.txt applies from the next check, or within an hour during a longer one. It also follows the Crawl-delay and Request-rate lines.
How fast it goes
During one check of a store, it:
- sends one request at a time, never two at once
- starts each request at least half a second after the previous one, or later when your Crawl-delay or Request-rate asks for it
- sends at most 256 requests
- sends nothing more after a 429 (too many requests) or 503 answer until the time your Retry-After header gives has passed, or 30 seconds when there is none
- tries a refused request again at most twice
How to block it
Add these two lines to your robots.txt:
User-agent: LHG-AIFitProfiler
Disallow: /It will then read your robots.txt and nothing else from your store. To keep it out of one part of the store only, disallow that path instead, for example Disallow: /collections/. These lines do not affect search engines or other crawlers. Answering one of its page requests with 401 or 403 also makes it stop reading your store.
Questions
Write to us through our contact page or at info@lakehousegrp.com, with your store's address.