mlbrgl / ghost-algolia

Provides integration between Ghost (CMS) and Algolia (Search as a Service)

Apr 2021: archiving as I am not planning on working on this in the foreseeable future.

---

Ghost / Algolia integration

Enables Ghost sites owners to index their content through Algolia.

☢️ 🍄 The master branch does not contain production-ready code at that stage. Please use the released versions instead.

What it does

When you work on a story, and publish it, the content of that story is sent to Algolia's indexing engine. Any change you make to that story or its state afterwards (updating content, deleting the story or unpublishing it) is automatically synchronised with your index.

Fragment indexing

Fragment indexing refers to breaking up an HTML document into smaller blocks (or fragments) before sending them to the indexing engine. Those fragments are generally composed of a heading (h1, h2, ...) and some text. You may read about the rationale behind fragment indexing on the KirbyAlgolia project page.

Here is how the fragmenting engine handles the different types of fragments, in terms of when the indexing events are fired:

line
line
--> INDEXING (headless fragment)
# heading
line
--> INDEXING
## heading
--> INDEXING (content-less heading)
### subheading
line
line
--> INDEXING
# unlikely heading
--> INDEXING by code convenience but very little value

Structure of a fragment

• objectID: automatically generated by Algolia (e.g. 565098020)
• post_uuid: automatically generated by Ghost (e.g. 8693c79d-7880-4e17-903d-7afd448e3517)
• heading: the heading of the fragment being indexed (e.g. My first paragraph)
• id: the ID of the fragment being indexed (e.g. my-new-blog-post#card-markdown--My-first-paragraph--1)
• importance: an integer reprensenting how deep in the article structure a fragment is located (e.g. 1). The deeper the less relevant.
• post_title: the title of the post being indexed (e.g. My new blog post)
• post_published_at: the published date of the post (e.g. 2017-09-03T19:14:03.000Z)
• content: the content of the fragment being indexed (e.g. The content of the first paragraph)

What it does not do

This app only deals with the indexing side of things. Adding the actual search widget is not part of the scope at this point. A good option to look into is InstantSearch.js.

Installation

1. Create free accounts on both Github and Netlify.

2. Configure Algolia's index

Create a new API key on Algolia's dashboard. You want to make sure that the generated key has the following authorizations on your index:

• Search (search)
• Add records (addObject)
• Delete records (deleteObject)

Next add the following attributes as searcheable attributes, in the ranking tab under the "Basic settings" section:

• post_title
• heading
• content
• post_uuid

Ignore any warnings about the attributes not being found in a sample of your records, as you should not have any records at that stage yet.

Finally, add importance as a custom ranking attribute in the ranking tab under the "Ranking Formula & Custom Ranking" section. This will allow the tie-break algorithm to give preference to higher fragments in the document structure. In other words, h1 tags will rank higher than h2 tags if they otherwise have the same textual score.

3. 
4. On Ghost's admin panel, create a new custom integration and the following webhook:

• Name: Post published, Event: Post published, target URL: the endpoint of the post-published function, found on Netlify's admin panel (https://YOUR-SITE-ID.netlify.com/.netlify/functions/post-published)

Usage

Real-time indexing

Triggering indexing is transparent once the app is installed and happens on the following ghost panel operations:

• publishing a new post (add a new record)

Cost: as many operations as fragments in the current post

Compatibility

Check the last release (on the releases page) to see what Ghost version is currently supported.

Roadmap

• event: updating a published post (update an existing record)
• event: unpublishing a post (remove a record)
• event: deleting a post (remove a record)
• bulk indexing

Alternative

For a similar process using Zapier: https://discourse.algolia.com/t/how-to-install-algolia-for-ghost-blogging-platform/1201/8. Please check the limitations, as this might not be suitable for your use case.*

ghost-algolia's Issues

Uploading all existing posts for Indexing

As far as I understood the mechanism correctly, new posts which will be published will get added to the index.

How to achieve the indexing of existing blog data with this integration?

Possible solutions could be

• upload the ghost exports file
• get the Ghost API key on Netlify app start and fetch all the index-able data (posts, tags)

Dependency deprecation warning: markdown-to-ast (npm)

On registry https://registry.npmjs.org/, the "latest" version (v6.0.3) of dependency markdown-to-ast has the following deprecation notice:

See https://github.com/textlint/textlint/issues/455

Marking the latest version of an npm package as deprecated results in the entire package being considered deprecated, so contact the package author you think this is a mistake.

Affected package file(s): package.json

If you don't care about this, you can close this issue and not be warned about markdown-to-ast's deprecation again. If you would like to completely disable all future deprecation warnings then add the following to your config:

"suppressNotifications": ["deprecationWarningIssues"]