> ## Documentation Index
> Fetch the complete documentation index at: https://developer.upsun.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Manage search indexing for an environment

> Protect SEO health by managing an environment's visibility to search engines.

Exposing multiple environments in the same project to search engines can harm your
production site's SEO rankings, for example, through duplicate content, broken links, or unpolished pages.

Whether search engines can index an environment depends on its domain:

* **No custom domain**: Always hidden from search engines. This default behavior cannot be changed.
* **Preview environment with a custom domain**: Hidden by default. You can make it visible.
* **Production environment with a custom domain (new project)**: When you add a custom domain, the site becomes visible to search engines by default. You can change this at any time.
* **Production environment with a custom domain (existing project)**: Retains its current setting when a custom domain is added. You can change this at any time.

> [!NOTE]
>
> If multiple environments have [custom domains](https://developer.upsun.com/docs/domains/steps), ensure that **at most** one is visible to avoid duplicate content and SEO dilution.

To change whether search engines can index an environment:

> [tab] In the Console
>
> Prerequisite: The environment must have a [custom domain](https://developer.upsun.com/docs/domains/steps).
>
> 1. Select the project you want to update.
> 2. From the **Environment** menu, select the environment.
> 3. Click  **Settings**.
> 4. Select or clear the **Hide from search engines** checkbox.

> [tab] Using the CLI
>
> Prerequisite: The environment must have a [custom domain](https://developer.upsun.com/docs/domains/steps).
>
> Run the following command:
>
> ```bash theme={null}
> upsun environment:info --environment <ENVIRONMENT_NAME> restrict_robots false
> ```

> [!NOTE]
>
> Upsun Cloud can't guarantee that indexers follow the instructions.
> To restrict access entirely, set up [HTTP access control](https://developer.upsun.com/docs/environments/http-access-control.md).

## How it works

When **Hide from search engines** is enabled, Upsun Cloud adds the following header
to responses, which instructs search engines to not index and to not traverse your site:

```txt theme={null}
X-Robots-Tag: noindex, nofollow
```

This setting can only be changed on environments with a custom domain.

## Alternative method: Use a robots.txt file

You can also send instructions to search engine indexers using a `robots.txt` file.
Your app can serve this as a static file from its disk or as a dynamic response from its `passthru`.
Control either with the [`location` section of your app configuration](https://developer.upsun.com/docs/configure-apps/image-properties/web#locations).

If your `robots.txt` file instructs search engines to ignore a page, that instruction takes effect regardless of whether Upsun Cloud is configured to send the `X-Robots-Tag` header.

## What's next

* [Add a custom domain to your production environment](https://developer.upsun.com/docs/domains/steps)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.