Straight answers about indexing, AI assistants, crawlers, IndexNow, fixes and your account. 36 questions, grouped by topic.
The most common reasons are that the pages look thin or too similar to other pages on your site, that Google has found more addresses than it wants to crawl, or that the pages are blocked, redirected or marked noindex. Google decides page by page, and it does not index everything it finds.
Learn more →It means Google visited and read the page, then chose not to add it to its index. This usually happens when Google sees little unique value in the page, for example when most of it is template text or it repeats other pages.
Learn more →It means Google knows the address exists but has not crawled it yet. On larger sites this often happens when there are many low-value addresses competing for Google's attention, so the important pages wait in line.
Learn more →No tool can, and claims that say otherwise are not true. What IndexPedia does is show why Google skips pages and what to change so Google sees them as worth indexing.
Learn more →No. Pages such as search results, filters, login screens and thin tag pages are often better left out. A smaller set of strong pages usually performs better than a large set of weak ones.
Learn more →It can take a few hours or several weeks. Google crawls popular sites that update often more frequently, and it may wait longer on pages that look like other pages. A clean sitemap, internal links to the new page and unique content help Google find the page and decide to keep it.
Learn more →It is a test, included in the Pro and Agency plans, that shows whether AI assistants recommend your business when customers ask for what you offer. IndexPedia asks the questions a customer would ask, live and with web search, and reads every answer. You see whether you were named, in what position, which competitors were named instead, and which websites the answers relied on.
Learn more →AI assistants build answers from what they can find and trust online: business websites, listing sites, review sites and local directories. Competitors that are named usually have clear, consistent facts across those sources. The check shows which sources were used, so you can see where you are missing.
Learn more →No one can guarantee it, but the steps that help are well known. Get listed, with the same name, address, hours and services, on the sources the check shows. Make sure your own website states clearly what you do and where.
Learn more →OAI-SearchBot crawls pages so they can appear in ChatGPT search results. ChatGPT-User visits a page when a ChatGPT user asks about it directly. GPTBot collects content that may be used to train OpenAI's models. Blocking GPTBot does not remove you from ChatGPT search; blocking OAI-SearchBot does.
Learn more →That is your decision, and AI Reach shows you which crawlers you are blocking so you can decide on purpose. Blocking search crawlers keeps your pages out of AI answers, which usually costs you visibility. Blocking training crawlers only affects model training. Many sites allow search crawlers and block training crawlers.
Learn more →No. Google-Extended is a robots.txt token that controls whether Google may use your content for Gemini model training. It does not change how Googlebot crawls your site or how you rank in Google Search.
Learn more →SEO is about showing up in a list of search results. AI visibility is about being named in the answer an AI assistant writes. The two overlap because AI assistants read the web through crawlers and search indexes. An AI answer has no page two, though, so a business is either named or left out. IndexPedia covers both.
Learn more →IndexNow is an open protocol that lets a website notify search engines the moment a page is added, updated or deleted. One notification is shared with all participating search engines, including Microsoft Bing, Yandex, Seznam.cz and Naver. Without it, search engines find changes only on their next crawl.
Learn more →No. Google does not take part in IndexNow. For Google, IndexPedia relies on your sitemap and, if you connect it, Search Console data to show which pages are indexed and why others are not.
Learn more →No. IndexNow makes sure search engines know about the page quickly. They still visit it and decide whether to index it based on its quality, the same as any other page.
Learn more →Microsoft Copilot answers with Bing's index, and ChatGPT search uses Bing among its sources. When your new pages reach Bing quickly, they can reach those AI answers sooner.
Learn more →The key is a short text file placed at the root of your site. It proves to search engines that the notifications really come from the owner of the site. IndexPedia gives you the file and checks that it is in place.
Learn more →Analytics tools run in the visitor's browser and ignore bots on purpose. AI crawlers do not run that code, so they never appear. The IndexPedia snippet runs on your server, where every request is seen.
Learn more →Each visit is checked against the IP addresses published by the crawler's owner, such as OpenAI, Perplexity, Google, Bing and Apple. Visits that use a famous name but come from other addresses are marked as not verified.
Learn more →Many sites let well-known crawlers through firewalls and rate limits. Scrapers copy their names to get the same treatment. The name in a request proves nothing; the address it comes from does.
Learn more →Block it by IP address in your firewall or hosting panel. Do not block by name, because that would also block the real crawler using the same name.
Learn more →In most hosting control panels, look for Logs, Raw Access or Access Logs. On your own server it is usually at /var/log/nginx/access.log or /var/log/apache2/access.log. Many hosts let you download the last few days as .gz files.
Learn more →The file is read once to build your report and then deleted. Only the report is kept. If you choose to, anonymous totals, with no addresses, pages or site name, are added to the public AI Crawler Radar.
Learn more →The Fix Kit contains the files and text your site needs, written from your own pages: a corrected robots.txt, an llms.txt guide for AI tools, schema markup for your homepage, shorter titles and new meta descriptions for pages that need them, and a clean list of sitemap URLs. Each item can be copied or downloaded.
Learn more →llms.txt is a proposed standard: a plain text file at the root of your site that tells AI tools what your site is and links to its most useful pages, in a format language models read easily. It sits beside robots.txt and your sitemap and does not replace them.
Learn more →The Fix Kit keeps the rules that protect private areas of your site and opens access for search and AI crawlers, then points to your sitemap. Read it before you replace your current file, especially if your site has areas that must stay private.
Learn more →Auto-fix finds what holds a page back in Google and writes the fix from the page itself: a better title and description, taking off a tag that tells Google to skip the page, making a page name itself as its main address, or adding links from pages Google already knows. You check each page side by side and press Save. Then each page shows the day Google saw the fix.
Learn more →No. Your files and your database are not touched. The fix is applied to the page as it is sent to the visitor, and switching a fix off returns the original immediately.
Learn more →Any website where you can add one line to the site header: Wix, Squarespace, Shopify, Webflow, Square Online, GoDaddy plans with Site-wide code, or a site that already uses Google Tag Manager. WordPress has a one-click plugin. The Connect page shows the steps for your builder. On a plan with no code box, Auto-fix cannot change the pages, and you copy the fixes into your builder's SEO settings instead.
Learn more →The free plan includes one site with up to 1,000 pages, automatic weekly checks of 100 pages, IndexNow submissions, AI crawler and AI visitor tracking, and the Fix Kit. The free site scan, AI Visibility Check, Log Analyzer and Bot checker can be used without an account.
Learn more →Pro is $19 a month for up to 5 sites and 50,000 pages per site, with daily checks of 1,000 pages, Google Search Console data and Auto-fix. Agency is $49 a month for up to 25 sites and 250,000 pages per site, with daily checks of 5,000 pages.
Learn more →Online payments aren't ready yet. Until they are, send a message from the contact page with your account email and the plan you want, and we move your account the same day. You can cancel at any time and keep your history.
Learn more →It means you uploaded your site's key file, which proves the site is yours. IndexNow requires the same file before it accepts your pages, so verifying covers both.
Learn more →Yes. IndexPedia checks public pages one at a time with a pause between them and identifies itself as IndexPediaBot. It never logs in, submits forms or changes your site.
Learn more →No, and no honest service can. Search engines and AI assistants make their own decisions. IndexPedia shows what stands in the way, fixes what can be fixed, and measures whether things improve.
Learn more →Just your domain. We find the pages Google is likely to skip. About 10 seconds.
