GithubHelp home page GithubHelp logo

meilisearch / meilisearch-symfony Goto Github PK

View Code? Open in Web Editor NEW
111.0 111.0 24.0 504 KB

Seamless integration of Meilisearch into your Symfony project.

Home Page: https://www.meilisearch.com

License: MIT License

PHP 99.42% Dockerfile 0.26% Shell 0.32%
client integration search symfony

meilisearch-symfony's Introduction

Meilisearch-Symfony

Meilisearch Symfony Bundle

Codecov coverage Latest Stable Version Test License Bors enabled

⚑ The Meilisearch bundle for Symfony

Meilisearch Symfony is a search bundle to integrate Meilisearch within your Symfony project.

Meilisearch is an open-source search engine. Discover what Meilisearch is!

Table of Contents

πŸ“– Documentation

Check out the Wiki of this repository to get started! πŸš€

Also, see our Documentation or our API References.

⚑ Supercharge your Meilisearch experience

Say goodbye to server deployment and manual updates with Meilisearch Cloud. No credit card required.

πŸ“ Requirements

  • Require PHP 7.4 and later.
  • Compatible with Symfony 5.4 and later.
  • Support Doctrine ORM.

For support of older versions, see older versions of this bundle.

πŸ€– Compatibility with Meilisearch

This package guarantees compatibility with version v1.x of Meilisearch, but some features may not be present. Please check the issues for more info.

πŸ’‘ Learn More

The following sections may interest you:

πŸ“– Also, check out the Wiki of this repository!

βš™οΈ Development Workflow and Contributing

Any new contribution is more than welcome in this project!

If you want to know more about the development workflow or want to contribute, please visit our contributing guidelines for detailed instructions!


Meilisearch provides and maintains many SDKs and Integration tools like this one. We want to provide everyone with an amazing search experience for any kind of project. If you want to contribute, make suggestions, or just know what's going on right now, visit us in the integration-guides repository.

meilisearch-symfony's People

Contributors

94noni avatar alallema avatar althaus avatar bors[bot] avatar bpolaszek avatar brunoocasali avatar chris53897 avatar chris8934 avatar codedge avatar connorhu avatar curquiza avatar dafish avatar david-simplyphp avatar dependabot-preview[bot] avatar dependabot[bot] avatar dibashthapa avatar emulienfou avatar james2001 avatar jonatanrdsantosmcf avatar joolsmcfly avatar ker0x avatar kermitsxb avatar meili-bors[bot] avatar meili-bot avatar mkilmanas avatar norkunas avatar revenkroz avatar tomdoo avatar wucherpfennig avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

meilisearch-symfony's Issues

docker-composer issue with tests on MacOs

I get a failure when I try to launch the tests with docker compose:

$ docker-compose up -d
Creating network "meilisearch-symfony_default" with the default driver
Creating meilisearch-symfony_meilisearch_1 ... done
Creating meilisearch-symfony_php_1         ... done
$ docker-compose exec -e MEILISEARCH_URL=http://meilisearch:7700 php composer test:unit
Parse error: syntax error, unexpected 'string' (T_STRING), expecting function (T_FUNCTION) or const (T_CONST) in /usr/src/vendor/ocramius/package-versions/src/PackageVersions/Installer.php on line 34

Command meili:create doesn't create index without settings

Description
Running php bin/console meili:create - didnt create index when no settings is present in configuration aka:

    indices:
        - name: shopping_gallery_tenant
          class: App\Entity\ShoppingGalleryTenant
          enable_serializer_groups:  true

Expected behavior
Index is created when no settings is present

Current behavior
Index is not created

Environment (please complete the following information):

  • OS: [e.g. Debian GNU/Linux]
  • Meilisearch version: v:0.23.1
  • meilisearch-symfony version: v0.7.2

Change master branch to main

Let's be allies and make this change that means a lot.

Here is a blog post that explain a little more why it's important, and how to easily do it. It will be a bit more complicated with automation, but we still should do it!

Remove "objectID" usage of this repository

This repository was highly built based on the Algolia's search bundle.

Algolia, to define the unique field of the documents uses the objectID field. Which is not the case of MeiliSearch. MeiliSearch uses the primary key: it infers it, or the user can define it.
More about the primary key of MeiliSerach: https://docs.meilisearch.com/learn/core_concepts/documents.html#primary-field

It means having a code base with objectID does not make sense for a MeiliSearch repository in terms of naming (related to #58) but also in terms of usage!
Indeed it leads us to some issues in the past, especially because the search() method is badly designed for MeiliSearch. See:

⚠️ Because you cannot know the primary key (so the unique field) when getting the search response by MeiliSearch we might think about a way to avoid using this information

Add raw search?

Looking at MeiliSearchService->search() it requires an ObjectManager.

While i do get the idea behind it, there should be a way to call the raw results without getting the entities from the database, something like this:

    public function searchRaw(
        string $className,
        string $query = '',
        array $requestOptions = []
    ): array {
        $this->assertIsSearchable($className);

        return $this->engine->search($query, $this->searchableAs($className), $requestOptions);
    }

Performance-wise its faster getting the data directly from MS than waiting for the "real" entities from MySQL.

What do you think?

Symfony 6.0.9 and PHP 8.1.0 -> can't composer install meilisearch/search-bundle

Hello πŸ‘‹

I'm using Symfony 6.0.9 with PHP 8.1.0.
When I run composer install meilisearch/search-bundle, I get these errors:

Using version ^0.7.2 for meilisearch/search-bundle
./composer.json has been updated
Running composer update meilisearch/search-bundle
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - illuminate/contracts[v8.0.0, ..., v8.11.2] require php ^7.3 -> your php version (8.1.0) does not satisfy that requirement.
    - illuminate/contracts[v8.12.0, ..., v8.83.16] require psr/container ^1.0 -> found psr/container[1.0.0, 1.1.0, 1.1.1, 1.1.2] but the package is fixed to 2.0.2 (lock file version) by a partial update and that version does not match. Make sure you list it as an argument for the update command.
    - meilisearch/search-bundle v0.7.2 requires illuminate/collections ^8.47 -> satisfiable by illuminate/collections[v8.47.0, ..., v8.83.16].
    - illuminate/collections[v8.47.0, ..., v8.83.16] require illuminate/contracts ^8.0 -> satisfiable by illuminate/contracts[v8.0.0, ..., v8.83.16].
    - Root composer.json requires meilisearch/search-bundle ^0.7.2 -> satisfiable by meilisearch/search-bundle[v0.7.2].

Use the option --with-all-dependencies (-W) to allow upgrades, downgrades and removals for packages currently locked to specific versions.
You can also try re-running composer require with an explicit version constraint, e.g. "composer require meilisearch/search-bundle:*" to figure out if any version is installable, or "composer require meilisearch/search-bundle:^2.1" if you know which you need.

Installation failed, reverting ./composer.json and ./composer.lock to their original content.

From what I understand, meilisearch/search-bundle is not (yet) compatible with PHP 8.1 but [with version 8.0](https://github.com/meilisearch/meilisearch-symfony/blob/main/composer .json#L21).

Are there plans to make meilisearch/search-bundle compatible on PHP 8.1?

Thanks !

Importing no indexes results with success

When running php bin/console meili:import it returns a successful Done! even if no indexes are defined via cli or via yaml file.

It should print a warning that no indices are defined, as it does when providing indices via cli but none is found.

Meilisearch requires doctrine bundle although the target application doesn't use it

Description
Basically, when an application uses doctrine mongodb odm for instance, installing meilisearch symfony will also install doctrine bundle which is not required by the app.

Expected behavior
Meilisearch bundle should not depend on doctrine-bundle but rely on configuration options to use the right DB

Screenshots or Logs
N/A

Environment (please complete the following information):

  • meilisearch-symfony version: v0.7.1

Symfony messenger support

As the title says, it would be good if Symfony messenger would be supported out of the box :)

Currently we are using a custom message and a message handler, but there's a slight problem that it doesn't properly work for the document removal part from the index. To do a reall ->remove call we use $objectManager->getReference().
But then the call is delegated to Engine and calls $entity->getSearchableArray(); but as the entity is already removed, doctrine throws error that the entity not found because proxy tries to load it's information to normalize the searchable array.

Multi language index per language

Description
My project is multi language. I have a product entity containing data in 4 languages. I would like to create an index per language.

Expected behavior
In my normalizer I would like to know the name of the index, this way I could extract the language from the index name and feed the correct language to the index. In meilisearch.yaml I would do this:

meilisearch:
indices:
- name: products_en
class: App\Entity\Product
index_if: isSearchable
- name: products_fr
class: App\Entity\Product
index_if: isSearchable

Current behavior
There is no way to retrieve the index name in the normalizer.

I could not find many information on multi language indexing. Maybe there is a better approach to multi language indexing than my suggested approach?

Entity mapping not working correctly?

I have try this bundle in a new project, but i cannot get it to work properly.

Maybe i have found a bug, in the MeiliSearchService at linte 218 their is a for-each to iterate over the ids.

$test= $searchService->search($this->getDoctrine()->getManager(), Test::class, 'hello');

https://github.com/meilisearch/meilisearch-symfony/blob/master/src/Services/MeiliSearchService.php#L218

foreach ($ids as $objectID) {

But if i do a var_dump on the $ids array, the array looks like:

array (size=7)
  'hits' => 
    array (size=20)
      0 => 
        array (size=7)
          'objectID' => int 7
          'id' => int 7
          'tag' => string 'hello' (length=5)
          'context' => null
          'actions' => null
          'parameters' => null
          'patterns' => 
            array (size=6)
              ...
      1 => 

If i change the for-each loop to the following, all is working as expected:

foreach ($ids['hits'] as $objectID) {

Is it a bug, or i am doing something wrong?
If it is a bug, i can create a PR to fix the issue.

Update .code-samples.meilisearch.yaml

⚠️ This issue is generated, it means the examples and the namings do not necessarily correspond to the language of this repository.
Also, if you are a maintainer, feel free to add any clarification and instruction about this issue.

Sorry if this is already partially/completely implemented, feel free to let me know about the state of this issue in the repo.

Related to meilisearch/integration-guides#185


Since there is no code-samples in this repository, you should add a new file .code-samples.meilisearch.yml into the root of this directory, and add a new key containing the landing_getting_started_1 contents.
Check out the #185 issue (Add landing_getting_started_1 code samples section) for more information

TODO:

  • Add new file .code-samples.meilisearch.yml
  • Add landing_getting_started_1 code samples

composer.lock conflicts with Docker configuration

Spinning up the containers via Docker Compose yields some issues:

You are using the deprecated option "--no-suggest". It has no effect and will break in Composer 3.
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Your lock file does not contain a compatible set of packages. Please run composer update.

  Problem 1
    - symfony/filesystem is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/filesystem v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 2
    - symfony/options-resolver is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/options-resolver v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 3
    - symfony/property-access is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/property-access v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 4
    - symfony/property-info is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/property-info v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 5
    - symfony/serializer is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/serializer v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 6
    - symfony/string is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/string v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 7
    - doctrine/dbal is locked to version 2.12.1 and an update of this package was not requested.
    - doctrine/dbal 2.12.1 requires php ^7.3 || ^8 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 8
    - phpunit/php-token-stream is locked to version 4.0.4 and an update of this package was not requested.
    - phpunit/php-token-stream 4.0.4 requires php ^7.3 || ^8.0 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 9
    - symfony/cache is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/cache v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 10
    - symfony/cache-contracts is locked to version v2.2.0 and an update of this package was not requested.
    - symfony/cache-contracts v2.2.0 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 11
    - symfony/config is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/config v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 12
    - symfony/console is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/console v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 13
    - symfony/dependency-injection is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/dependency-injection v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 14
    - symfony/doctrine-bridge is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/doctrine-bridge v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 15
    - symfony/error-handler is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/error-handler v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 16
    - symfony/event-dispatcher is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/event-dispatcher v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 17
    - symfony/event-dispatcher-contracts is locked to version v2.2.0 and an update of this package was not requested.
    - symfony/event-dispatcher-contracts v2.2.0 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 18
    - symfony/finder is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/finder v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 19
    - symfony/framework-bundle is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/framework-bundle v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 20
    - symfony/http-client is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/http-client v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 21
    - symfony/http-client-contracts is locked to version v2.3.1 and an update of this package was not requested.
    - symfony/http-client-contracts v2.3.1 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 22
    - symfony/http-foundation is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/http-foundation v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 23
    - symfony/http-kernel is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/http-kernel v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 24
    - symfony/process is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/process v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 25
    - symfony/routing is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/routing v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 26
    - symfony/service-contracts is locked to version v2.2.0 and an update of this package was not requested.
    - symfony/service-contracts v2.2.0 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 27
    - symfony/stopwatch is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/stopwatch v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 28
    - symfony/var-dumper is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/var-dumper v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 29
    - symfony/var-exporter is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/var-exporter v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 30
    - symfony/yaml is locked to version v5.2.3 and an update of this package was not requested.
    - symfony/yaml v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
  Problem 31
    - symfony/options-resolver v5.2.3 requires php >=7.2.5 -> your php version (7.2.0) does not satisfy that requirement.
    - friendsofphp/php-cs-fixer v2.18.2 requires symfony/options-resolver ^3.0 || ^4.0 || ^5.0 -> satisfiable by symfony/options-resolver[v5.2.3].
    - friendsofphp/php-cs-fixer is locked to version v2.18.2 and an update of this package was not requested.

The composer.lock seems to be based on PHP 7.2.5 whilst the Dockerfile uses 7.2.0.

Kind regards
Matthias

Please provide a small example for search with service

Hello

I'd like to make a basic search for my icons
Here's my code :

    #[Route('/search', name: 'app_search')]
    public function search(SearchService $searchService, Request $request, EntityManagerInterface $entityManager)
    {
        $searchQuery = $request->query->get('q');
        if (empty($searchQuery)) return $this->redirectToRoute('app_home');

        $hits = $searchService->rawSearch(Icon::class, $searchQuery)["hits"];
        dd($hits);
    }

This isn't optimal, I'm sure there is a way to search directly but I don't found anything in the documentation.

When I do this, I found 0 hits :

$hits = $searchService->search($entityManager, Icon::class, $searchQuery)

Can you provide a small example to how to do a basic search with SearchService

Thank you

Add PHPMD to improve code quality

Description
A good practice is to add PHPMD to the project so we can check the code against good practices.

Basic example
The code can be validated by running:

$ composer phpmd

Other
We could split the PHPMD into 2 phases, the first one to add PHPMD to the project and the second to fix the current problems.

Support Symfony 6

Make this package work with Symfony 6.

A contributor already tried to make this package work with Symfony 6 but we encountered some issues.
Check out this PR #139

Composer installation fails with missing ENV entries

Hey,

I've stumbled upon this project and as it looks quite promising, I've just gave it a try.

Sadly following the wiki just fails as running

composer require meilisearch/search-bundle symfony/http-client nyholm/psr7

will run into an 'Environment variable not found: "MEILISEARCH_URL".' error and composer will revert the changes.

So you have have to use the --no-scripts option or pre-define the ENV values. Maybe you'd like to update the wiki. On the other side I'd be great if a recipe could configure the .env automatically.

Kind regards
Matthias

Add support to the typo tolerance customization

⚠️ This issue is generated, it means the examples and the namings do not necessarily correspond to the language of this repository.
Also, if you are a maintainer, feel free to add any clarification and instruction about this issue.

Related to:


⚠️ The meilisearch/meilisearch-php#312 should be finished before starting this issue!

Add a typo_tolerance index settings resource to manage customization of the typo tolerance feature at the index level.

# meili_search.yaml
meili_search:
  indices:
    - name: posts
      class: App\Entity\Post
      settings:
        # attributesForFaceting: ['genres', 'publishedAt']
        typoTolerance: 
          enabled: true
          disableOnAttributes: ['title']
          disableOnWords: ['york']
          minWordSizeForTypos: 
            oneTypo: 5
            twoTypos: 9

More information about what the typo-tolerance field does could be checked in the spec.

You could check the other customization methods for reference like: synonyms.

TODO:

  • Add the typo_tolerance configuration
  • Add tests

MeiliSearchService `settingsMapping` looks unused

private array $settingsMapping;

$this->setSettingsMapping();

private function setSettingsMapping(): void
{
$mapping = [];
/** @var array $indexDetails */
foreach ($this->configuration->get('indices') as $indexDetails) {
$mapping[$indexDetails['class']] = $indexDetails['settings'];
}
$this->settingsMapping = $mapping;
}

Even IDE hints about this:
image

Allow to delete an object from index only having an identifier

Description
Currently MeiliSearch\Bundle\SearchService exports method for deleting an entity, but there is no way to delete it having an identifier only. The basic use case - delete hundreds of entities in batch, then dispatch a messenger message to remove them from index, but providing only integers, as you cannot load anymore entities that do not exist.

Basic example

public function removeById(ObjectManager $objectManager, $identifier); // not sure about typehint of $identifier

Other

nothing more.

Cannot update or delete documents with an Uuid

Hi!
I'm using this type of IDs in my entities :

/**
 * @ORM\Id
 * @ORM\GeneratedValue(strategy="UUID")
 * @ORM\Column(type="uuid")
 */
private $id;

On update or remove I get an MeiliSearch\Exceptions\InvalidArgumentException in this method :

private function assertValidDocumentId($documentId): void
{
    if (!\is_string($documentId) && !\is_int($documentId)) {
        throw InvalidArgumentException::invalidType('documentId', ['string', 'int']);
    }

    if (\is_string($documentId) && '' === trim($documentId)) {
        throw InvalidArgumentException::emptyArgument('documentId');
    }
}

I've got "Notice: Undefined index: errorCode" on excecuting bin/console meili:import command

On excecuting bin/console meili:import command i've got this:

In HandlesIndex.php line 64:
                                      
  Notice: Undefined index: errorCode  

My meili_search.yaml

meili_search:
    indices:
        - name: docs
          class: App\Entity\SearchRes
          enable_serializer_groups: true

SearchRes.php entity:

<?php

namespace App\Entity;
use App\Repository\SearchResRepository;
use Doctrine\ORM\Mapping as ORM;
use Symfony\Component\Serializer\Annotation\Groups;

/**
 * @ORM\Entity(repositoryClass=SearchResRepository::class)
 */
class SearchRes
{
    /**
     * @ORM\Id
     * @ORM\GeneratedValue
     * @ORM\Column(type="integer")
     */
    private $id;

    /**
     * @ORM\Column(type="text")
     */
    private $title;

    /**
     * @ORM\Column(type="string", length=255, nullable=true)
     */
    private $code;

    /**
     * @ORM\Column(type="string", length=255, nullable=true)
     */
    private $entityType;

    public function getId(): ?int
    {
        return $this->id;
    }

    /**
     * @Groups({"searchable"})
     */
    public function getTitle(): ?string
    {
        return $this->title;
    }

    public function setTitle(string $title): self
    {
        $this->title = $title;

        return $this;
    }

    /**
     * @Groups({"searchable"})
     */
    public function getCode(): ?string
    {
        return $this->code;
    }

    public function setCode(?string $code): self
    {
        $this->code = $code;

        return $this;
    }

    /**
     * @Groups({"searchable"})
     */
    public function getEntityType(): ?string
    {
        return $this->entityType;
    }

    public function setEntityType(?string $entityType): self
    {
        $this->entityType = $entityType;

        return $this;
    }
}

in .env:

MEILISEARCH_URL=http://127.0.0.1:7700
MEILISEARCH_API_KEY=Y0urVery-S3cureAp1K3y

Symfony 4, Ubuntu 21.10, PHP 7.4

Symfony HttpClient

Description
Symfony has it's own HttpClient component which I think should be used by default :)

Basic example
It's only one line in the services configuration

Other
N/A

Impossible to clear indices via CLI

  • I ran a local MS instance with an index named indexUID containing books with a title
  • I created a new Symfony project
  • I added:
MEILISEARCH_URL=http://127.0.0.1:7700
MEILISEARCH_API_KEY=masterKey

in my project

  • I added this file (/config/package/meili_search.yaml) in my project:
meili_search:
     indices:
          -   name: indexUID
               class: App\Entity\Book
               enable_serializer_groups: true

(I've also created an src/Entity/Book.php file following this part of the docs)

$ php bin/console meili:clear

And I got:

Index indexUID  couldn't be cleared
Done!

The index was indeed not removed.
Plus, an empty dev_indexUID was created in my MeiliSearch instance.

Versions:

  • MeiliSearch v0.12.0
  • meilisearch/search-bundle v0.2.0
  • Symfony v4.17.0

Service returns bad entities for correct search

Hi,

I've encountered another issue using the SearchService. The service returned different entities than the search client. I've debugged through the code and found this issue:

The search method has this line

$ids = $this->engine->search($query, $this->searchableAs($className), $requestOptions);

which fetches the results from the client. The data is then used in the loop to gather the primary keys:

            if (in_array($className, $this->aggregators, true)) {
                $entityClass = $className::getEntityClassFromObjectID($objectID);
                $id = $className::getEntityIdFromObjectID($objectID);
            } else {
                $id = $objectID;
                $entityClass = $className;
            }

In my case the code went into the else part and $objectID contained the complete data set. So it was an array and no plain literal. I've made the following change to get it working.

$id = is_array($objectID) ? $objectID['objectID'] : $objectID;

Did I miss something? The code doesn't look as it could work at all?

Kind regards
Matthias

Too many queries are made in the db

Description
For each search result a query is made in the database, and if the number of results is huge, so are the queries number.

Possible fix

  1. Collect all result ids
  2. Run one query in the DB
  3. Get all entities and sort them as it was in search result

Screenshots or Logs

Current state (results is limited to 10).
Screenshot 2021-11-28 at 18 00 12

After possible solution.
Screenshot 2021-11-28 at 17 59 18

Renaming some variables

This package was strongly based on the Algolia Symfony Bundler (https://github.com/algolia/search-bundle/ - MIT License).
MeiliSearch and Algolia are indeed similar but not exactly the same. It means there are some naming inconsistencies in the code base of this repository.

  • objectId should not be used. MeiliSearch does not generate any objectId but contains a primaryKey that depends on the user dataset. See an example of how to fetch the current primary key of an index
    πŸ’‘ More about the primary key
  • requestOptions should be renamed into searchParams (done with #70)
  • indexName should be renamed as indexUid (done with #69)

I would rather each point be done in different PRs πŸ™‚
Feel free to ask any question you need!

Refactor calls to deprecated methods

Currently the package uses calls to deprecated methods. This can be observed when running the meili:import command in a Symfony 5 application.

The call to Doctrines clear() method in MeiliSearchImportCommand line 157, line 160 should be refactored.

Tests suddendly fail for php 7.2 and 7.3

Tests are now failing with these errors for php 7.3 and PHP 7.2:

There were 3 errors:

1) MeiliSearch\Bundle\Test\TestCase\AggregatorTest::testAggregatorProxyClass
Symfony\Component\PropertyAccess\Exception\NoSuchPropertyException: Can't get a way to read the property "objectID" in class "MeiliSearch\Bundle\Test\Entity\ContentAggregator".

/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/property-access/PropertyAccessor.php:461
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/property-access/PropertyAccessor.php:117
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Normalizer/ObjectNormalizer.php:152
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Normalizer/AbstractObjectNormalizer.php:178
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Serializer.php:154
/Users/curquiza/Documents/meilisearch-symfony/src/SearchableEntity.php:84
/Users/curquiza/Documents/meilisearch-symfony/src/Engine.php:51
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:156
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:397
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:157
/Users/curquiza/Documents/meilisearch-symfony/src/EventListener/MeiliSearchIndexerSubscriber.php:46
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/doctrine-bridge/ContainerAwareEventManager.php:64
/Users/curquiza/Documents/meilisearch-symfony/vendor/doctrine/orm/lib/Doctrine/ORM/Event/ListenersInvoker.php:117
/Users/curquiza/Documents/meilisearch-symfony/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php:1127
/Users/curquiza/Documents/meilisearch-symfony/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php:402
/Users/curquiza/Documents/meilisearch-symfony/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php:371
/Users/curquiza/Documents/meilisearch-symfony/tests/TestCase/AggregatorTest.php:70

2) MeiliSearch\Bundle\Test\TestCase\CommandsTest::testSearchImportAggregator
Symfony\Component\PropertyAccess\Exception\NoSuchPropertyException: Can't get a way to read the property "objectID" in class "MeiliSearch\Bundle\Test\Entity\ContentAggregator".

/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/property-access/PropertyAccessor.php:461
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/property-access/PropertyAccessor.php:117
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Normalizer/ObjectNormalizer.php:152
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Normalizer/AbstractObjectNormalizer.php:178
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Serializer.php:154
/Users/curquiza/Documents/meilisearch-symfony/src/SearchableEntity.php:84
/Users/curquiza/Documents/meilisearch-symfony/src/Engine.php:51
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:156
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:397
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:157
/Users/curquiza/Documents/meilisearch-symfony/src/Command/MeiliSearchImportCommand.php:122
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/console/Command/Command.php:256
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/console/Tester/CommandTester.php:76
/Users/curquiza/Documents/meilisearch-symfony/tests/TestCase/CommandsTest.php:138

3) MeiliSearch\Bundle\Test\TestCase\SearchTest::testSearchImportAggregator
Symfony\Component\PropertyAccess\Exception\NoSuchPropertyException: Can't get a way to read the property "objectID" in class "MeiliSearch\Bundle\Test\Entity\ContentAggregator".

/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/property-access/PropertyAccessor.php:461
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/property-access/PropertyAccessor.php:117
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Normalizer/ObjectNormalizer.php:152
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Normalizer/AbstractObjectNormalizer.php:178
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/serializer/Serializer.php:154
/Users/curquiza/Documents/meilisearch-symfony/src/SearchableEntity.php:84
/Users/curquiza/Documents/meilisearch-symfony/src/Engine.php:51
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:156
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:397
/Users/curquiza/Documents/meilisearch-symfony/src/Services/MeiliSearchService.php:157
/Users/curquiza/Documents/meilisearch-symfony/src/Command/MeiliSearchImportCommand.php:122
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/console/Command/Command.php:256
/Users/curquiza/Documents/meilisearch-symfony/vendor/symfony/console/Tester/CommandTester.php:76
/Users/curquiza/Documents/meilisearch-symfony/tests/TestCase/SearchTest.php:126

ERRORS!
Tests: 14, Assertions: 35, Errors: 3.

They did not fail before, and we did not change anything, if someone knows what happened or how to fix it, it would be helpful!

Support Doctrine ORM inheritance

I'm excited to try Meilisearch, so I'm happy there's a symfony bundle! Sadly the project I picked to try it out on uses Doctrine's Single Table Inheritance, and this confuses the meili:import command, or more specifically Services\MeiliSearchService::isSearchable().

Under Doctrine's single table inheritance method, we create multiple entity classes that inherit from a root entity in the normal PHP way (class MyEntityType extends BaseEntity…), and then a so-called discriminator column saves which type an entity really is, and Doctrine takes care of hydrating the right entity type when you load things from the parent type's repository. To give an example, I've got an app where I've got a Card entity, and its two subtypes are Article and Link. Both Article and Link have all the (searchable) properties of Card, but Articles have a content blob and Links have a URL to a site. With my previous search solution, I indexed Card so that I could search all cards together, which makes sense in this app -- the differences between the entities are in how they're displayed, not in what we browse and search for.

The problem seems to be that isSearchable() does a string comparison of entity class names against a list of configured classes, rather than using PHP's instanceof operator. This means that although all my entities are descendants of Card, they're actually Link or Article entities so isSearchable returns false and nothing ever gets into my index. If I configure each entity to have its own index, then… well they'd be in separate indices and not searchable as one thing, which is what I'd like to continue doing.

If I were to try to resolve this myself, I'd change the isSearchable method to do something like this

    public function isSearchable($className): bool
    {
        if (is_object($className)) {
            $className = ClassUtils::getClass($className);
        }

        foreach ($this->searchableEntities as $searchableEntity) {
            if ($className instanceof $searchableEntity) {
                return true;
            }
        }

        return false;
    }

However I've not tested this, and I'm not familiar enough with the bundle or its uses to know if this could cause problems for other users, so I'm reluctant to submit a PR. If you think it's an acceptable change, I could try my hand at it though!

Aggregators do not create a new index

Description
Use case: implement a site-wide search on multiple models.

I had hoped to use multiple indices along with meilisearch/instant-meilisearch, but sadly, that is not supported yet. I took a peek at the Algolia docs, and they recommend using Aggregator-entities for this. Conveniently, this bundle provides similar Aggregator-classes, e.g. MeiliSearch\Bundle\Entity\Aggregator, so I went ahead and "registered" the entities I want indexed in the aggregator.

However, this does not create the desired aggregated index

Expected behavior
An index with the name defined in the config is created, and it contains the data of the sub-entities registered in the class extending from MeiliSearch\Bundle\Entity\Aggregator

Current behavior
The import-command reports "Done!", yet no index is actually created, neither for the aggregator, nor any aggregated entities.

Code
Below, find three entities: the first two are regular entities handled by Doctrine; the third is the aggregator.

<?php

declare(strict_types=1);

namespace App\Entity;

use App\Repository\PersonRepository;
use Doctrine\ORM\Mapping as ORM;
use Symfony\Component\Serializer\Annotation\Groups;

/**
 * @ORM\Entity(repositoryClass=PersonRepository::class)
 */
class Person
{
    /**
     * @ORM\Id
     * @ORM\GeneratedValue
     * @ORM\Column(type="integer")
     */
    private int $id;

    /**
     * @ORM\Column(type="string", length=255)
     * @Groups({"searchable"})
     */
    private string $firstName;

    /**
     * @ORM\Column(type="string", length=255)
     * @Groups({"searchable"})
     */
    private string $lastName;

   /* getters and setters omitted for brevity */
<?php

declare(strict_types=1);

namespace App\Entity;

use App\Repository\ProjectRepository;
use Doctrine\ORM\Mapping as ORM;
use Symfony\Component\Serializer\Annotation\Groups;

/**
 * @ORM\Entity(repositoryClass=ProjectRepository::class)
 */
class Project
{
    /**
     * @ORM\Id
     * @ORM\GeneratedValue
     * @ORM\Column(type="integer")
     */
    private int $id;

    /**
     * @ORM\Column(type="text")
     * @Groups({"searchable"})
     */
    private ?string $title;

    /* getters and setters omitted for brevity */
<?php declare(strict_types=1);

namespace App\Entity;

use Doctrine\ORM\Mapping as ORM;
use MeiliSearch\Bundle\Entity\Aggregator as AggregatorAlias;
use Symfony\Component\Serializer\Annotation\Groups;

/**
 * @ORM\Entity
 */
class Aggregator extends AggregatorAlias
{
    public static function getEntities(): array
    {
        return [
            Person::class,
            Project::class,
        ];
    }
}

This is the meili_search.yaml I'm using:

meili_search:
  url: '%env(MEILISEARCH_URL)%'
  api_key: '%env(MEILISEARCH_API_KEY)%'
  prefix: '%env(MEILISEARCH_PREFIX)%'
  indices:
    - name: data
      class: App\Entity\Aggregator
      enable_serializer_groups: true

Environment (please complete the following information):

  • OS: Arch Linux, Docker image getmeili/meilisearch:latest
  • MeiliSearch version: v.0.20.0
  • meilisearch-python version: unknown

Replace MEILISEARCH_HOST by MEILISEARCH_URL in the docker config

⚠️ This issue is generated, it means the examples and the namings do not necessarily correspond to the language of this repository.
Also, if you are a maintainer, feel free to add any clarification and instruction about this issue.

Sorry if this is already partially/completely implemented, feel free to let me know about the state of this issue in the repo.

Related to meilisearch/integration-guides#214


Replace the env var naming from the Dockerfile environment:

  • MEILISEARCH_HOST to MEILISEARCH_URL

Also, look for occurrences inside of the docker-compose.yml file.

After, run the tests according to the instructions in the CONTRIBUTING.md.

Also, check for occurences in the Github Actions workflows and inside of the source code.

Add test to check the value of "prefix"

Following this issue (#12) and this PR (#13), we need to add test checking the value of prefix (in meili_search.yml file), especially testing the empty value.

Any suggestions to implement these tests?

Delete command

Description
A command like import or clear to delete indexes.

Basic example
To complety delete one or more indexes

php bin/console meili:delete --indices=posts,comments

Other
If you are interrested I can make a pull request, I have already coded this command.

Ability to set request timeout

Sometimes when you import a lot of data, 5 sec as defined here is not enough and the command meili:import fails with the error:

[MeiliSearch\Exceptions\TimeOutException (408)]  
  Request timed out 

The idea is to add an option to configure a timeout when importing.

(feat): DocumentDataProvider to allow external data sources

Description

Hi everyone πŸ‘‹πŸ»

Small suggestion here that can help the bundle supporting extra document data sources, for now, this bundle only support loading document data from entities (without asynchronous supports but more on this later), even if this is the de facto approach when it comes to data handling in Symfony application (thanks to Doctrine), we often use external services like Redis, Kafka and so on, these services contains data and we often need to search on it.

The idea here is to introduce the idea of DocumentDataProvider (or just _X_DataProvider) along with a DataProviderRegistry which allow to use the data providers and allow each provider to return an array that contains data that we can load into indexes.

Basic example

Ok, so, here's a small example:

final class FooDataProvider implements DataProviderInterface
{
    public function getData(): array
    {
        // Fetch data from external data source and return a valid array
    }

    public static function getIndex(): string // static is not a requirement
    {
        return 'foo';
    }
}

Following the same approach, an EntityDataProvider can be created to move the logic from the existing command to a dedicated class.

The final part is a DataProviderRegistry which contains and allow to access and interact with the data providers (sort them, filter them and many more) which can be injected in the command to remove some logic on it and ease both the debug and evolution phase.

Regarding the asynchronous support, when using Messenger and regarding #62, adding this layer can help introduce the asynchronous support by allowing to dispatch a message which can call the registry to persist data into MS.

I've already implemented a solution here but the final implementation can be improved πŸ™‚

Other

--

Thanks for the feedback and have a great day πŸ™‚

Cannot import indices with document data

Description

Hi everyone πŸ‘‹πŸ»

It seems that the latest version of the bundle does not allows to load data into documents when using it in symfony/demo project.

Environment:

  • OS: macOS
  • PHP: 7.4
  • MeiliSearch version: latest
  • MeiliSearch PHP SDK: 0.16.0
  • meilisearch-symfony version: 0.2.6

Expected behavior

The data should be loaded in documents along with indices.

Current behavior

Current configuration:

meili_search:
  indices:
    - name: posts
      class: App\Entity\Post
      enable_serializer_groups: true

Current entity configuration:

/**
     * @var int
     *
     * @ORM\Id
     * @ORM\GeneratedValue
     * @ORM\Column(type="integer")
     *
     * @Groups({"searchable"})
     */
    private $id;

    /**
     * @var string
     *
     * @ORM\Column(type="string")
     * @Assert\NotBlank
     *
     * @Groups({"searchable"})
     */
    private $title;

    /**
     * @var string
     *
     * @ORM\Column(type="string")
     *
     * @Groups({"searchable"})
     */
    private $slug;

    /**
     * @var string
     *
     * @ORM\Column(type="string")
     * @Assert\NotBlank(message="post.blank_summary")
     * @Assert\Length(max=255)
     *
     * @Groups({"searchable"})
     */
    private $summary;

    /**
     * @var string
     *
     * @ORM\Column(type="text")
     * @Assert\NotBlank(message="post.blank_content")
     * @Assert\Length(min=10, minMessage="post.too_short_content")
     *
     * @Groups({"searchable"})
     */
    private $content;

When launching php bin/console meili:import, here's the error displayed:

In MeiliSearchImportCommand.php line 195:
                                   
  Notice: Undefined index: number  
                                   

meili:import [-i|--indices [INDICES]] [--update-settings]

Screenshots or Logs

Current logs (in Symfony):

doctrine.DEBUG: SELECT t0.id AS id_1, t0.title AS title_2, t0.slug AS slug_3, t0.summary AS summary_4, t0.content AS content_5, t0.published_at AS published_at_6, t0.author_id AS author_id_7 FROM symfony_demo_post t0 LIMIT 500 [] []
console.CRITICAL: Error thrown while running command "meili:import". Message: "Notice: Undefined index: number" {"exception":"[object] (ErrorException(code: 0): Notice: Undefined index: number at /__PROJECT__/vendor/meilisearch/search-bundle/src/Command/MeiliSearchImportCommand.php:195)","command":"meili:import","message":"Notice: Undefined index: number"} []
console.DEBUG: Command "meili:import" exited with code "1" {"command":"meili:import","code":1} []

Current logs (in Meilisearch):

[__DATE__ INFO  actix_web::middleware::logger] 127.0.0.1:54907 "GET /indexes/dev_posts HTTP/1.1" 200 146 "-" "Symfony HttpClient/Curl" 0.000302
[__DATE__ INFO  actix_web::middleware::logger] 127.0.0.1:54907 "POST /indexes/dev_posts/documents HTTP/1.1" 202 14 "-" "Symfony HttpClient/Curl" 0.004908
[__DATE__ INFO  actix_web::middleware::logger] 127.0.0.1:54907 "GET /indexes/dev_posts/updates/9 HTTP/1.1" 200 167 "-" "Symfony HttpClient/Curl" 0.000269
[__DATE__ INFO  actix_web::middleware::logger] 127.0.0.1:54907 "GET /indexes/dev_posts/updates/9 HTTP/1.1" 200 167 "-" "Symfony HttpClient/Curl" 0.000509
[__DATE__ INFO  meilisearch_lib::index::updates] document addition done: DocumentAdditionResult { nb_documents: 30 }

Here's a dump() when debugging the call $indexInstance->getUpdateStatus($apiResponse['updateId']); in MeiliSearchImportCommand at line 195:

array:5 [
  "status" => "processing"
  "updateId" => 8
  "type" => array:1 [
    "name" => "DocumentsAddition"
  ]
  "enqueuedAt" => "__DATE__"
  "startedProcessingAt" => "__DATE__"
]

It seems that when calling $formattedResponse[$indexName] += $updateStatus['type']['number'];, the number key is not available as type does not contains it.

I'm not sure about the internal usage of number (maybe displaying the number of row loaded), it seems that when MS is processing, the number key is not returned πŸ€”

Possible solution:

It seems that by default Indexes does not wait until the operation is processed:

public function waitForPendingUpdate($updateId, $timeoutInMs = 5000, $intervalInMs = 50): array
{
    $timeout_temp = 0;
    while ($timeoutInMs > $timeout_temp) {
        $res = $this->getUpdateStatus($updateId);
        if ('enqueued' != $res['status']) {
            return $res;
        }
        $timeout_temp += $intervalInMs;
        usleep(1000 * $intervalInMs);
    }
    throw new TimeOutException();
}

Here's what can be used:

public function waitForPendingUpdate($updateId, $timeoutInMs = 5000, $intervalInMs = 50): array
{
    $timeout_temp = 0;
    while ($timeoutInMs > $timeout_temp) {
        $res = $this->getUpdateStatus($updateId);
        if ('processed' === $res['status']) {
            return $res;
        }
        $timeout_temp += $intervalInMs;
        usleep(1000 * $intervalInMs);
    }
    throw new TimeOutException();
}

By waiting for updates to be processed, here's the output:

Indexed 30 / 30 App\Entity\Post entities into dev_posts index
Done!

I don't know if it helps but the fix seems to be relatively small.

Thanks again for the feedback and have a great day πŸ™‚

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.