# Your community in search engines and AI

> The robots.txt, sitemap.xml and llms.txt every community serves on its own, what Google and AI assistants read and what stays out, how each space's visibility changes that, the search description, and how to submit the sitemap in Google Search Console and Bing Webmaster Tools.

Every Memberfy community, on its Memberfy address or on its [custom domain](/configuracoes/dominio-proprio), already gives search engines and AI assistants the files they look for. You don't need to create or update anything: the files are built from the community's menu as it looks to someone who is **not signed in**.

| File | What it has |
|---|---|
| **/robots.txt** | **Google, Bing and other crawlers.** What they shouldn't visit: only the API, the checkout, administration and the account. Everything else is allowed |
| **/sitemap.xml** | **Search engines.** The list of pages to index: the home page, the open spaces, the upcoming public events and the plans page (`/pricing`) Also the open courses, the calls for papers with submissions open and the certificates their holders let search engines show. |
| **/llms.txt** | **AI assistants (ChatGPT, Claude, Perplexity…).** The community in Markdown: what is open, with a link; what is closed, only with its name and what it takes to get in |
| **/llms-full.txt** | **AI assistants.** The same, with full descriptions instead of one line |

Example: `https://community.papodesysadmin.org/sitemap.xml` or `https://papodesysadmin.memberfy.net/llms.txt`.

## What it's for

- **Getting found**: people searching Google for your community's topic land on the open spaces, the events and the plans.
- **Being described well by AI**: when someone asks an assistant about your community, it has an organized source, with what is public, the prices and the way in.
- **Without exposing what is closed**: the content of a restricted area never goes in; at most its name and what it takes to see it.

## How it works

### What is indexed and what isn't

| In the sitemap and the index | Not in it |
|---|---|
| The home page | Administration, settings, checkout and the account (blocked in `robots.txt`) |
| The spaces open to people who aren't signed in (**Public** visibility) | Padlocked spaces: they show up in `llms.txt` by name only, and the page asks search engines not to index it |
| The upcoming public events in those spaces | Hidden spaces and group spaces |
| The plans page, when a plan is for sale | **Link only** plans |
| Published courses in open **Courses** spaces | |
| Calls for papers with submissions open, in open **Call for papers** spaces | |
| Certificates whose holder turned on **Show in search engines** (the verification page, without the name or e-mail) | |
| | Invite, unsubscribe and payment links, forms opened only by link, and member pages |

> [!NOTE]
> With **Guest Access** off (*"Allow visitors to view public content"*, the default), the community publishes nothing in these files beyond the home page, not even indexable certificates: search engines and AI tools only see the home page. See [Sign-up and sign-in](/membros/cadastro-e-entrada).

[Link](/espacos/link) spaces aren't in the sitemap (the page belongs to another site), but they show up in `llms.txt` with their target address.

**A page that doesn't exist answers 404.** An address with a section or space that doesn't exist (or that someone not signed in can't see) returns *not found* to search engines instead of a copy of the home page. That way an old link drops out of the index on its own.

### How each space's visibility affects this

What counts is the space's and the section's [visibility](/conceitos/visibilidade-e-acesso), and the **For those without access** option:

| The space, for someone not signed in | Sitemap | `llms.txt` |
|---|---|---|
| **Public** | In | Under **Public**, with link and description |
| **Members**, with a padlock | Out | Under **Requires sign-in or purchase**: the name, the description and *"requires an account"* |
| **Subscribers**, or **Private** with a plan, with a padlock | Out | The name and *"requires a plan"*, linking to the plans page |
| **Private** with a product, with a padlock | Out | The name and *"requires a purchase"* |
| **Private** for groups only | Out | Not listed: there's no way in from outside |
| **Hide** under **For those without access** | Out | Not listed |

If the community has no space open to people who aren't signed in, `llms.txt` has only the name, the description and *"private community; the content requires an account"*.

In `llms.txt`, open **Events** spaces list the upcoming events with date and time; [Store](/espacos/loja) spaces, the products with prices; and the **Plans** section has each public [plan](/monetizacao/planos) with the prices of its billing options. The file is in the community's language (**Settings › General › Language**).

> [!TIP]
> Want a space to show up on Google? Make it **Public**. Want it not to show up even by name? Use **Hide** under **For those without access**.

### The search description

The description Google shows under the title, which also opens `llms.txt`, comes, in this order:

1. From the **SEO Description**, in **Settings › Search and discovery › SEO Meta Tags** (see [Images and SEO](/configuracoes/imagens-e-seo));
2. If that's empty, from the **Community Description**, in **Settings › General**;
3. If both are empty, from a default sentence: *"Papo de Sysadmin: a community on Memberfy."*

The default sentence says nothing about the topic, and the description is what search engines and AI use to decide who to show the community to. Write one or two sentences saying what the community is, who it's for and what people find in it.

### When it changes

The files follow the community: a new open space, a new event or a new plan shows up within 5 minutes. Search engines reread on their own schedule, from a few hours to a few days.

### Who can

The files are automatic and have no settings. The **owner** and the **admin** change what appears in them, through the spaces' visibility, the plans and the description.

## Verify the community on Google and Bing

On the Memberfy address (`.memberfy.net`) there is no DNS to change: verification is by HTML tag. It also works on a custom domain.

*Role: owner or admin.*

1. In Google Search Console, add the property as a **URL prefix** (`https://papodesysadmin.memberfy.net`) and choose the **HTML tag** method. In Bing Webmaster Tools, choose **Meta tag**.
2. Copy the code or the whole tag: `<meta name="google-site-verification" content="…">` on Google, `<meta name="msvalidate.01" content="…">` on Bing.
3. In the community, under **Administration › Settings › Search and discovery**, in the **Search engine verification** card, paste it in the **Google Search Console** or **Bing Webmaster Tools** field and save. Only the code is kept (up to 100 letters, digits, `-` or `_`); another tag or other text shows an error on the field. Clearing the field removes the verification.
4. Go back to the search engine and click **Verify**: the tag is already in the head of the community's pages. Then submit the sitemap as in the steps below.

## Step by step: Google Search Console

Search Console shows which pages Google indexed, the searches that bring people in, and errors. Verification needs DNS access, so it applies to a **custom domain**; on the Memberfy address, Google finds the sitemap on its own through `robots.txt`.

*Role: whoever manages the domain's DNS.*

1. Open [search.google.com/search-console](https://search.google.com/search-console) and click **Add property**.
2. Choose **Domain** and type the main domain, without `https://` and without the subdomain: `papodesysadmin.org`.
3. Google shows a **TXT** record. Create it in your DNS provider's panel (Cloudflare, GoDaddy, Namecheap…), under the name `@` (or the root domain), and click **Verify**. DNS can take a few minutes.
4. In the left menu, open **Sitemaps**, type the full address, `https://community.papodesysadmin.org/sitemap.xml`, and click **Submit**.
5. Come back in a few days: **Pages** shows what was indexed and why each page was left out.

## Step by step: Bing Webmaster Tools

Bing also feeds other search engines and assistants that use its index.

1. Open [bing.com/webmasters](https://www.bing.com/webmasters) and sign in with a Microsoft, Google or Facebook account.
2. The shortest path is **Import from Google Search Console**: it brings the site and the sitemap already verified. To do it by hand, add `https://community.papodesysadmin.org` and choose verification by **DNS record** (CNAME).
3. Under **Sitemaps**, click **Submit sitemap** and paste `https://community.papodesysadmin.org/sitemap.xml`.

## Examples

**Excel School**, on `school.excelpractice.com`. The **Start here** section is **Public**, with the *Welcome* and *Free lessons* spaces; **Student area** is **Private** with the *Complete* plan, with a padlock; **March class** is **Private** for the class group only. Result:

- The sitemap has the home page, *Welcome*, *Free lessons*, the upcoming events from the open calendar and `/pricing`.
- `llms.txt` lists the two open spaces, the events, the *Complete* plan with its monthly and yearly price and, under **Requires sign-in or purchase**, *Student area — requires a plan*. *March class* doesn't appear.

## Tips

- Write the **Community Description** even if you fill in the **SEO Description**: the first one also shows inside the community.
- Give every open space a description: it becomes the page's description on Google and the space's line in `llms.txt`.
- Use a padlock (not **Hide**) on the paid areas you want to sell: the name and description reach AI with the way in, without revealing the content.
- With a custom domain, submit the custom domain's sitemap, not the Memberfy address's: that's the one the pages point to as the official address.

## Common errors and how to fix them

| Situation | What to do |
|---|---|
| A space isn't in the sitemap | Check that it's **Public**, and that its section is too: a space is never more open than its section |
| Search shows *"a community on Memberfy"* as the description | Fill in the **Community Description** or the **SEO Description** and wait for Google to reread the page |
| Search Console says *"Blocked by robots.txt"* on a page | It's an administration, checkout or account page: it shouldn't be indexed |
| Search Console says *"Excluded by 'noindex' tag"* | It's a padlocked space, a member page or an invite link: it's right to leave it out |
| Search Console says *"Not found (404)"* | A section or space was deleted, renamed or closed. If you renamed it, the new address goes into the sitemap on its own |
| I can't verify the `.memberfy.net` address | Use verification by tag: paste the Google or Bing code in the community's search settings (see [Verify the community](#verify-the-community-on-google-and-bing)) |

## FAQ

**Can I turn off the sitemap or `llms.txt`?**
No. They only have what anyone already sees without joining the community. To take a space out, change its visibility or use **Hide**.

**Does the content of posts and lessons go to AI?**
Not from closed areas: only the name and what it takes to get in. For **Public** spaces, the file has the name, the description and the link; an AI that follows the link reads the page like any visitor.

**Can I write my own `robots.txt`?**
No. Memberfy's blocks only the private areas and allows the rest.

**Are the prices in `llms.txt` the ones on the plans page?**
Yes, the same, in each option's currency. **Link only** plans aren't included.

## In the API

- `PUT /api/communities/{id}` accepts `googleSiteVerification` and `bingSiteVerification` (owner and admin): that engine's code or whole tag; only the code is kept, and empty or `null` clears it. Other text returns `400` with the field in `errors[].param`. Both come out in `GET /api/communities/settings`.
- `GET /api/public/community-summary` (no session; the community comes from `X-CommunityId`): `courses`, `callsForPapers` (with `closesAt`) and `certificates`, each item with `title`, `description` (one line, or `null`), `path` and `updatedAt`. Up to 200 per list, newest first; cached for 5 minutes and up to 120 reads per minute per IP. It never carries the certificate holder's name or e-mail. With **Guest Access** off (the default), the three lists come back empty, indexable certificates included.

## Related

- [Images and SEO](/configuracoes/imagens-e-seo)
- [Custom domain](/configuracoes/dominio-proprio)
- [Visibility and access](/conceitos/visibilidade-e-acesso)
- [Plans](/monetizacao/planos)
- [Google Analytics](/configuracoes/google-analytics)