springboot-elasticsearch-thymeleaf

The goal of this project is to implement an application called product-app. It consists of two Spring Boot services: product-api (backend) and product-ui (frontend). Data will be stored in Elasticsearch.

Applications

product-api

Spring Boot Web Java application that exposes a REST API to manages products. The information about products are stored in Elasticsearch. product-api uses Spring Data Elasticsearch to persist/query/delete data in Elasticsearch
product-ui

Spring Boot Web application that was implemented using Thymeleaf as HTML template. product-ui is a client of product-api. It uses Spring Cloud OpenFeign to write web service clients easily.
eureka-server

Spring Boot application, used as service discovery in this project

Prerequisites

Start Environment

Open a terminal and navigate to springboot-elasticsearch-thymeleaf root folder run
```
docker-compose up -d
```
Wait a little bit until the docker containers are up and healthy. To check it, run
```
docker-compose ps
```

Configure an index, alias and insert some products in ES

In a terminal, make sure you are in springboot-elasticsearch-thymeleaf root folder
Run the following script to create the index ecommerce.products.v1 with the alias ecommerce.products (you can use the default values by just pressing Enter on every user input)
```
./create-index.sh
```
If you want to insert some products, run
```
./insert-products.sh
```
If you want to fix the reference property mapping error (explained below), run
```
./reindex.sh
```

Reindex

The script ./reindex.sh is used to reindex from an index to another index. The default will reindex from ecommerce.products.v1 to ecommerce.products.v2. The only difference between elasticsearch/mapping-v1.json (used by ecommerce.products.v1) to elasticsearch/mapping-v2.json (used by ecommerce.products.v2) is the type of the reference property. In the former is set text and in the latter, keyword.

It’s interesting because the reference property has some special characters. An example of reference code is SBES@DDR4-10000. As it is a text, ES (using the standard analyzer) splits the content in tokens ['SBES', 'DDR4', 10000]. So, for example, if you are looking for a product with DDR4 RAM and, for some reason, the string DDR4 is present in the reference code of some product X, the product X will be selected, even if it doesn’t have DDR4 in its description. It is an error.

So, the script ./reindex.sh aims to fix it, setting the type keyword to reference property. The DDR4 problem won’t happen again because, from now on, ES won’t tokenize the content present in the reference property.

Start applications

Below are the steps to start and run the applications. We will need to open a terminal for each one. Make sure you are in springboot-elasticsearch-thymeleaf root folder while running the commands.

eureka-server

./mvnw clean spring-boot:run --projects eureka-server

product-api

./mvnw clean spring-boot:run --projects product-api -Dspring-boot.run.jvmArguments="-Dserver.port=9080"

product-ui

./mvnw clean spring-boot:run --projects product-ui -Dspring-boot.run.jvmArguments="-Dserver.port=9081"

Applications URLs

Application	URL
eureka-server	http://localhost:8761
product-api	http://localhost:9080/swagger-ui.html
product-ui	http://localhost:9081

Note: when accessing product-ui, if you get the exception shown in the picture below, wait a bit.

Demo

Below is a simple demo that shows a user interacting with product-ui

Shutdown

Go to eureka-server, product-api and product-ui terminals and press Ctrl+C
In a terminal, make sure you are in springboot-elasticsearch-thymeleaf root folder and run the command below to stop and remove docker-compose containers, networks and volumes
```
docker-compose down -v
```

Creating indexes and reindexing them using Elasticsearch REST API

In the following steps, we are going to, manually and using Elasticsearch REST API, create an index called ecommerce.products.v1, associate an alias called ecommerce.products for it and then reindex to another index called ecommerce.products.v2.

Make sure you have a clean Elasticsearch without the indexes and alias mentioned previously. Also, the following curl commands must be executed in springboot-elasticsearch-thymeleaf root folder.

Check ES is up and running

curl localhost:9200

It should return something like

{
  "name" : "fadee0b011b7",
  "cluster_name" : "docker-es-cluster",
  "cluster_uuid" : "iO1Ne0WXRDeQYZyGp3DaWQ",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "oss",
    "build_type" : "docker",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Create ecommerce.products.v1 index

curl -X PUT localhost:9200/ecommerce.products.v1 -H "Content-Type: application/json" -d @elasticsearch/mapping-v1.json

It should return

{ "acknowledged":true, "shards_acknowledged":true, "index":"ecommerce.products.v1" }

Check indexes

curl "localhost:9200/_cat/indices?v"

It should return something like

health status index                 uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   ecommerce.products.v1 1B3JXm6zQnKolob4mtwRUg   1   1          0            0       230b           230b

Check ecommerce.products.v1 index mapping

curl "localhost:9200/ecommerce.products.v1/_mapping?pretty"

It should return

{
  "ecommerce.products.v1": {
    "mappings": {
      "properties": {
        "categories": { "type": "keyword" },
        "created": { "type": "date", "format": "strict_date_time_no_millis||yyyy-MM-dd'T'HH:mmZZ" },
        "description": { "type": "text" },
        "name": { "type": "text" },
        "price": { "type": "float" },
        "reference": { "type": "text" },
        "reviews": {
          "properties": {
            "comment": { "type": "text" },
            "created": { "type": "date", "format": "strict_date_time_no_millis||yyyy-MM-dd'T'HH:mmZZ" },
            "stars": { "type": "short" }
          }
        }
      }
    }
  }
}

Create alias for ecommerce.products.v1 index

curl -X POST localhost:9200/_aliases -H 'Content-Type: application/json' \
     -d '{ "actions": [{ "add": {"alias": "ecommerce.products", "index": "ecommerce.products.v1" }}]}'

It should return

{ "acknowledged":true }

Check aliases

curl "localhost:9200/_aliases?pretty"

It should return

{ "ecommerce.products.v1": { "aliases": { "ecommerce.products": {} } } }

Create ecommerce.products.v2 index

curl -X PUT localhost:9200/ecommerce.products.v2 -H "Content-Type: application/json" -d @elasticsearch/mapping-v2.json

It should return

{ "acknowledged":true, "shards_acknowledged":true, "index":"ecommerce.products.v2" }

Checking indexes again

curl "localhost:9200/_cat/indices?v"

It should return something like

health status index                 uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   ecommerce.products.v2 Iq0adLgEQSaCTIOISIW4DA   1   1          0            0       230b           230b
yellow open   ecommerce.products.v1 1B3JXm6zQnKolob4mtwRUg   1   1          0            0       283b           283b

Reindex from ecommerce.products.v1 to ecommerce.products.v2

curl -X POST localhost:9200/_reindex -H 'Content-Type: application/json' \
     -d '{ "source": { "index": "ecommerce.products.v1" }, "dest": { "index": "ecommerce.products.v2" }}'

It should return something like

{
  "took": 86,
  "timed_out": false,
  "total": 0,
  "updated": 0,
  "created": 0,
  "deleted": 0,
  "batches": 0,
  "version_conflicts": 0,
  "noops": 0,
  "retries": { "bulk": 0, "search": 0 },
  "throttled_millis": 0,
  "requests_per_second": -1.0,
  "throttled_until_millis": 0,
  "failures": []
}

Adjust alias after reindex from ecommerce.products.v1 to ecommerce.products.v2

curl -X POST localhost:9200/_aliases -H 'Content-Type: application/json' \
     -d '{ "actions": [{ "remove": {"alias": "ecommerce.products", "index": "ecommerce.products.v1" }}, { "add": {"alias": "ecommerce.products", "index": "ecommerce.products.v2" }}]}'

It should return

{ "acknowledged":true }

Checking aliases again

curl "localhost:9200/_aliases?pretty"

It should return something like

{
  "ecommerce.products.v1": { "aliases": {} },
  "ecommerce.products.v2": { "aliases": { "ecommerce.products": {} } }
}

Delete ecommerce.products.v1 index

curl -X DELETE localhost:9200/ecommerce.products.v1

It should return

{ "acknowledged":true }

Checking aliases again

curl "localhost:9200/_aliases?pretty"

It should return

{ "ecommerce.products.v2": { "aliases": { "ecommerce.products": {} } } }

Simple search

curl "localhost:9200/ecommerce.products/_search?pretty"

It should return something like

{
  "took": 1,
  "timed_out": false,
  "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 },
  "hits": {
    "total": { "value": 0, "relation": "eq" },
    "max_score": null,
    "hits": []
  }
}

As I don’t have any products, the hits array field is empty

TODO

add some Ajax calls, for example, when adding a comment, so the page doesn’t need to be refreshed (https://grokonez.com/java-integration/integrate-jquery-ajax-post-get-spring-boot-web-service);
add pagination. now, it is just returning all products;
add functionality to delete products;

tosunkaya / springboot-elasticsearch-thymeleaf Goto Github PK