# Algolia 101: Key Search Technology Driving Business Success

### 1\. What you’ll build

You’ll (1) use **Algolia Crawler** to pull content from several domains into one Algolia application, (2) send that content into one or more indices, and (3) expose a **unified search-as-you-type UI** that can be embedded in WordPress, Shopify, Webflow or any other front-end.

---

### 2\. Prerequisites

| Requirement | Why you need it |
| --- | --- |
| Algolia account on the “Build” plan or higher | Access to Crawler & multi-index analytics |
| Admin API key (server side) + Search-only API key (front-end) | Security best-practice |
| Access to the DNS / `robots.txt` / sitemap of each domain | Lets the crawler reach and follow links |
| npm / yarn (for custom InstantSearch builds) | Build the shared search UI |
| Basic HTML edit rights in every CMS | Embed the widget |

---

### 3\. Create a single Algolia **application**

1. Log in → **New application** → choose nearest region.
    
2. In *Settings › Team & Access* invite stakeholders if you need separate API keys per environment.  
    *Why single app?* Algolia places no restriction on mapping sites to indices inside one application, so analytics and quota stay centralised. ([support.algolia.com](https://support.algolia.com/hc/en-us/articles/8777149205649-If-I-have-two-websites-that-will-use-separate-indices-should-I-create-a-separate-application-for-each-index?utm_source=chatgpt.com))
    

---

### 4\. Plan your indexing strategy

| Scenario | Recommended setup |
| --- | --- |
| Same search UX for all sites (“global” search) | **One index** and add an attribute `site` so users can filter results. |
| Per-site UX but shared backend | **One crawler** with **multiple actions** that write to *different indices* (one per domain). ([support.algolia.com](https://support.algolia.com/hc/en-us/articles/11978306097041-How-can-I-index-to-multiple-indices-in-one-crawler?utm_source=chatgpt.com)) |
| Hard isolation (different business units) | Separate Algolia *applications* + their own crawlers. |

---

### 5\. Configure **Algolia Crawler**

#### 5.1 Add a crawler

1. Dashboard → **Crawler › Create**.
    
2. Give the crawler a name (e.g. *multi-site-search*).
    

#### 5.2 Define start URLs

Add one start URL per domain, e.g.

```json
[
  "https://docs.myproduct.com/",
  "https://blog.myproduct.com/",
  "https://help.anotherbrand.io/"
]
```

#### 5.3 Create actions (one per domain or content-type)

Each action sets:

```plaintext
{
  "indexName": "global_content",   // or "blog_index"
  "pathsToMatch": ["https://blog.myproduct.com/**"],
  "selectors": {
    "record": {
      "title": "h1",
      "content": "main",
      "url": "url",
      "site": "static:blog"        // custom attribute for faceting
    }
  }
}
```

> You can edit all of this in the visual **Crawler editor** or paste JSON directly. ([algolia.com](https://www.algolia.com/doc/tools/crawler/getting-started/crawler-configuration/?utm_source=chatgpt.com))

#### 5.4 Respect crawl rules

* Make sure `robots.txt` allows Algolia’s user-agent (`Algolia Crawler`), or whitelist its IPs on private sites.
    
* For password-protected staging domains, set **Basic auth credentials** in *Crawler › Advanced settings*.
    

#### 5.5 Schedule & monitor

* **Schedule** tab → choose frequency (daily, hourly…).
    
* Use **Crawler logs** and **Algolia CLI**/API for CI-based monitoring. ([algolia.com](https://www.algolia.com/doc/tools/crawler/getting-started/crawler-configuration/?utm_source=chatgpt.com))
    

---

### 6\. Fine-tune the indices

1. In **Indices › Configuration** set **Searchable attributes** (title → content).
    
2. Enable **Attributes for faceting**: `site`, `language`, `type`.
    
3. Create **Synonyms** for branded terms (Settings › Synonyms).
    
4. Add **Rules** (e.g. pin the docs homepage for query “documentation”).
    

---

### 7\. Build a shared search UI

#### 7.1 Choose a flavour of **InstantSearch**

| Environment | Library |
| --- | --- |
| Vanilla JS / Webflow embed | InstantSearch.js |
| React (Next.js, Gatsby) | React InstantSearch |
| Vue / Nuxt | Vue InstantSearch |
| Shopify | The “Algolia AI Search & Discovery” app uses InstantSearch under the hood. ([algolia.com](https://www.algolia.com/doc/integration/shopify/building-search-ui/instant-search/?utm_source=chatgpt.com)) |

#### 7.2 Example React component with **federated search**

```jsx
import algoliasearch from 'algoliasearch/lite';
import { InstantSearch, Index, SearchBox, Hits } from 'react-instantsearch-hooks-web';

const searchClient = algoliasearch('APP_ID', 'SEARCH_ONLY_KEY');

export default function GlobalSearch() {
  return (
    <InstantSearch searchClient={searchClient} indexName="global_content">
      <SearchBox placeholder="Search across all sites…" />
      {/* If you kept separate indices, repeat <Index> blocks */}
      <Index indexName="global_content">
        <h2>All results</h2>
        <Hits hitComponent={Hit} />
      </Index>
      <Index indexName="blog_index">
        <h2>Blog</h2>
        <Hits hitComponent={Hit} />
      </Index>
    </InstantSearch>
  );
}

function Hit({ hit }) {
  return (
    <a href={hit.url} className="block p-4 border-b">
      <p className="text-sm text-gray-500">{hit.site}</p>
      <h3 className="font-medium">{hit.title}</h3>
      <p className="line-clamp-2">{hit.content}</p>
    </a>
  );
}
```

*Styling is Tailwind-ready; adjust to match each CMS theme.*

---

### 8\. Embed the widget in each CMS

#### 8.1 WordPress

1. Install **WP Search with Algolia**. ([algolia.com](https://www.algolia.com/doc/integration/wordpress/?utm_source=chatgpt.com))
    
2. Paste your **Application ID** & **API keys**.
    
3. In the plugin settings, disable its built-in crawler (you now control indexing centrally).
    
4. Insert the React bundle (or use the plugin’s shortcode) in your theme’s header or via Gutenberg block.
    

#### 8.2 Shopify

*Install the “Algolia AI Search & Discovery” app → Customize › Search → choose* ***Custom layout*** *→ paste React bundle or tweak Liquid snippets provided by the app.* ([algolia.com](https://www.algolia.com/doc/integration/shopify/building-search-ui/instant-search/?utm_source=chatgpt.com))

#### 8.3 Webflow

1. Host the compiled JS bundle on a CDN.
    
2. In *Page settings › Before* add the `<script src="...bundle.js"></script>` tag.
    
3. Optionally use *No-Code Supply* example for filters & pagination. ([nocodesupply.co](https://www.nocodesupply.co/item/algolia-instant-search-in-webflow?utm_source=chatgpt.com))
    

---

### 9\. Security & performance tips

* **Server keys stay server-side.** Always expose only a *search-only* key in your JS bundles.
    
* Turn on **HTTP referrer restrictions** for Search-only keys.
    
* Add **Algolia Insights** to track click & conversion events for relevance tuning.
    
* Use **Replica indices** if you need per-country ranking variations.
    

---

### 10\. Maintenance checklist

| Task | Frequency |
| --- | --- |
| Review crawler logs for blocked pages | Weekly |
| Re-run crawl after major CMS releases | On demand |
| Audit API key restrictions | Quarterly |
| Update synonyms & rules from analytics | Monthly |
| Back-up index settings via Algolia CLI (`algolia settings get`) | Monthly |

---

#### Next steps

* Explore **Algolia Recommend** for “related content” widgets once the indices stabilise.
    
* If some sites expose large JSON APIs, consider bypassing the crawler and pushing data via the **Batch API** for faster indexing.
    

With this workflow you end up with **one search service, one UX, many sites**—maintainable, analytics-friendly and blazing-fast for users.