spring-data-search / spring-boot-starter-data-search

Spring-Data search API augmented with Natural Language support.

spring-boot-starter-data-search

Spring Data Search API augmented with Natural Language support.

Already supported:

Data Search is compatible with Spring Native (see the list of supported drivers and limitations)

Coming soon:

Demo:
spring-boot-starter-data-search-mongodb-demo
spring-boot-starter-data-search-jpa-demo

Table of Contents

• Data Search
• Features
  • Comparison operators
  • Supported values
• Detailed use case
• Getting Started
  • Requirements
  • Demo
    • Mongodb demo
    • JPA demo
  • Installation
    • Data Search Mongodb starter
    • Data Search JPA Starter
  • Configuration
    • Mongodb Repository
    • JPA Repository
  • Usage
  • Field Mapping
    • Flat Mapper
    • Advanced Mapper
  • Value Mapping
• Contributors
• License

Data Search

Data Search provides an enterprise & production ready API, prowered by Spring Boot and Antlr, to query your data, and perform advanced search with Natural Language.

Search example: users born after 1988-01-01, with an gmail or protonmail email address, having completed the email verification, and having an address in one of the following countries: France, Switzerland or China

birthDate >: '1988-01-01' and (emailAddress : *gmail.com or emailAddress: *protonmail.com) and emailAddressVerified: true and addresses.countryCode: 'FR,CH,CN'

Features

• Logical operators (or/ and)
• Parenthesis/ criteria prioritization
• Mongodb and all JPA compatible db engines
• Fields mapping/ DTO to Entities
• Filter by subentity fields/ deep search
• Advanced RegEx for Mongodb and like operator for JPA
• All comparison operators

Comparison operators

Equal :

emailAddressVerified : true

Not equal !:

emailAddressVerified !: true

In :

countryCode : 'FR,CH,CN'

Not in :

countryCode !: 'FR,CH,CN'

Starts with :

firstName: S*

Ends with :

firstName: *S

Contains :

firstName: *S*

Less than <

birthDate < '1988-01-01'

Less than or equal <:

birthDate <: '1988-01-01'

Greater than >

birthDate > '1988-01-01'

Greater than or equal >:

birthDate >: '1988-01-01'

Exists (is not null)

birthDate

Doesn't exist (is null) !

!birthDate

Supported values

String

firstName : Stan

Enum

title : MR

Boolean

emailAddressVerified : true

Number (Integer, Double, Long, BigDecimal)

ref >: 100 or coins > 6.76453

LocalTime

time >: '18:58:24' and time <: '18:58:24.999'

OffsetTime

time >: '18:58:24Z' and time <: '20:58:24.999+02:00'

LocalDate

birthDate >: '1988-01-01'

LocalDateTime

createdDate >: '2021-08-23T18:58:24' and createdDate <: '2021-10-12T18:58:24.000'

OffsetDateTime

createdDate >: "2021-08-23T18:58:24Z" and createdDate <: '2021-10-12T18:58:24.000+02:00'

Array

countryCode : 'FR,CH,CN'

RegEx

emailAddress : '/.*gmail.com/' Regular expression, supported only for mongodb (see documentation) |

Detailed use case

See detailed search use case here

Getting Started

Requirements

Java version 11 or higher (If java 8 support is needed, please vote for this issue)

SpringBoot version 2.1.0 or higher

Demo

Mongodb demo

https://github.com/commerce-io/spring-boot-starter-data-search-mongodb-demo

JPA demo

https://github.com/commerce-io/spring-boot-starter-data-search-jpa-demo

Installation

➡️ Get the latest releases here ⬅️

Data Search Mongodb starter

Maven

<dependency>
    <groupId>app.commerce-io</groupId>
    <artifactId>spring-boot-starter-data-search-mongodb</artifactId>
    <version>1.3.0</version>
</dependency>

Gradle implementation 'app.commerce-io:spring-boot-starter-data-search-mongodb:1.3.0'

Data Search JPA Starter

Maven

<dependency>
    <groupId>app.commerce-io</groupId>
    <artifactId>spring-boot-starter-data-search-jpa</artifactId>
    <version>1.3.0</version>
</dependency>

Gradle implementation 'app.commerce-io:spring-boot-starter-data-search-jpa:1.3.0'

Configuration

Data Search provides a custom repository. In order to use the provided repository, add the following annotation to the main class or any other configuration class.

Mongodb Repository

@Configuration
@EnableMongoRepositories(repositoryBaseClass = SearchRepositoryImpl.class)
public class DemoConfiguration {
}

JPA Repository

@Configuration
@EnableJpaRepositories(repositoryBaseClass = SearchRepositoryImpl.class)
public class DemoConfiguration {
}

Usage

Make your repositories extend SearchRepository

@Repository
public interface CustomerRepository extends SearchRepository<Customer, String> {
}

And use in a controller or from anywhere else

@RestController
@RequiredArgsConstructor
public class DemoController {

    private final CustomerRepository customerRepository;

    @Transactional(readOnly = true)
    @RequestMapping(
            method = RequestMethod.GET,
            value = "/customers",
            produces = {"application/json"}
    )
    public ResponseEntity<Page<Customer>> searchCustomers(
            @RequestParam(value = "search", required = false) String search,
            Pageable pageable) {

        Page<Customer> customerPage = customerRepository.findAll(
                search,
                pageable);

        return ok(customerPage);
    }
}

Field Mapping

Users will tend to filter by the search result fields, and sometimes, your DTO structure differs from your entity/ collection. Data search provides a mapper, to define your custom mappings rules in two different ways:

Flat Mapper

The flat mapper is a basic String to String mapping, which could be useful for simple use cases.

String search = "addressName: test";
Mapper addressMapper = Mapper.flatMapper()
            .mapping("addressName", "address.firstName")
            .build();

Page<CustomerEntity> page = customerRepository.findAll(search, pageable, addressMapper);

Advanced Mapper

The advanced mapper is used for complex structure mapping, and enable the reuse of mappers. Advanced mappers can be combined with flat mappers.

String search = "name: test OR address.addressName: test";
Mapper addressMapper = Mapper.flatMapper()
            .mapping("addressName", "firstName")
            .build();

Mapper mapper = Mapper.mapper()
                .mapping("name", "firstName")
                .mapping("address", "addressEntity", addressMapper)
                .build();

Page<CustomerEntity> page = customerRepository.findAll(search, pageable, mapper);

Value Mapping

In addition to the fields mapping, some values could be converted before returning the search result. Data Search supports values mapping:

public class LowerCaseValueMapping implements ValueMapping {

  @Override
  public String map(String from) {
    return from == null ? null : from.toLowerCase();
  }
}

String search = "addressName: TeSt";
Mapper addressMapper = Mapper.flatMapper()
        .mapping("addressName", "address.firstName", new LowerCaseValueMapping())
        .build();

Page<CustomerEntity> page = customerRepository.findAll(search, pageable, addressMapper);

Contributors

Thanks to these wonderful people (emoji key):

@adlmez 💻 🤔 👀 💡 ⚠️

Dependabot 🚇

All Contributors 🚇

This project follows the all-contributors specification. Contributions of any kind welcome!

License

This software is released under the Apache license. See LICENSE for more information.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

This project follows the all-contributors specification. Contributions of any kind welcome!

spring-boot-starter-data-search's Issues

add field mapping support

Is your feature request related to a problem? Please describe.
When the structure returned is different from the entity, the user will naturally try to filter using the returned structure fields.
It could be useful to support an optional field mapping, to convert the input field names to the target ones.

Describe the solution you'd like
The first option is to manage a basic flat mapping:
customer.address.street --> customerEntity.addressEntity.streetAddress

The second option is to have a dedicated mapper per object, to allow reuse of mappers.
addressMapper:
street --> streetAddress

customerMapper
address --> addressEntity using addressMapper

Describe alternatives you've considered
N/A

Additional context
N/A

Support @ElementCollection

Is your feature request related to a problem? Please describe.
At the moment, the library throws an exception when trying to query from a list annotated with @ElementCollection.

org.springframework.dao.InvalidDataAccessApiUsageException: Parameter value [Mr.] did not match expected type [java.util.Collection (n/a)];
nested exception is java.lang.IllegalArgumentException: Parameter value [Mr.] did not match expected type [java.util.Collection (n/a)]

// Add additional property in TestEntity.java
    @ElementCollection
    private List<String> testStringList;

//// SpringDataSearchJpaApplicationTest.java
// Add additional test case
    @Test
    void search_stringList() {
        initLargeTestData();

        Page<TestEntity> testEntityPage = testEntityRepository.findAll("testStringList: Mr.", Pageable.unpaged());
        // TODO add assertions
    }

    private TestEntity getTestEntity(...) {
        // init testEntity

        List<String> stringList = new ArrayList<>();
        stringList.add(testSubString);
        testEntity.setTestStringList(stringList);

        // init return testEntity
    }

Describe the solution you'd like

Ideally, the same syntax as shown in docs/use-cases.md (addresses.countryCode: CH) would also work on elements with the annotation @ElementCollection.

Describe alternatives you've considered

• Create a serialized copy of the list (only works for Strings, also data duplication is bad in general)

CVE-2021-44228

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228

Enum fields support

Is your feature request related to a problem? Please describe.
When searching with Enum fields, we have the following error: Parameter value [] did not match expected type

Describe the solution you'd like
The string value must be converted to Enum

Describe alternatives you've considered
N/A

Additional context
N/A

Java 8 support

Is your feature request related to a problem? Please describe.
Java 8 is still one of the most used java versions (source: Jetbrains)

Describe the solution you'd like
Add Java 8 support

Describe alternatives you've considered
N/A

Additional context
N/A