GithubHelp home page GithubHelp logo

typesense / laravel-scout-typesense-driver Goto Github PK

View Code? Open in Web Editor NEW
120.0 8.0 38.0 154 KB

Laravel Scout Driver for Typesense

Home Page: https://typesense.org

License: MIT License

PHP 100.00%
typesense laravel laravel-package laravel-scout-driver laravel-scout

laravel-scout-typesense-driver's Introduction

Laravel Scout Typesense Driver

This package makes it easy to add full text search support to your models with Laravel 7.* to 11.*.

Important

The features from the Scout driver in this repo have been merged upstream into Laravel Scout natively.

So we've temporarily paused development in this repo and plan to instead address any issues or improvements in the native Laravel Scout driver instead.

If there are any Typesense-specific features that would be hard to implement in Laravel Scout natively (since we need to maintain consistency with all the other drivers), then at that point we plan to add those features into this driver and maintain it as a "Scout Extended Driver" of sorts. But it's too early to tell if we'd want to do this, so we're in a holding pattern on this repo for now.

In the meantime, we recommend switching to the native Laravel Scout driver and report any issues in the Laravel Scout repo.

Contents

Installation

The Typesense PHP SDK uses httplug to interface with various PHP HTTP libraries through a single API.

First, install the correct httplug adapter based on your guzzlehttp/guzzle version. For example, if you're on Laravel 8, which includes Guzzle 7, then run this:

composer require php-http/guzzle7-adapter

Then install the driver:

composer require typesense/laravel-scout-typesense-driver

And add the service provider:

// config/app.php
'providers' => [
    // ...
    Typesense\LaravelTypesense\TypesenseServiceProvider::class,
],

Ensure you have Laravel Scout as a provider too otherwise you will get an "unresolvable dependency" error

// config/app.php
'providers' => [
    // ...
    Laravel\Scout\ScoutServiceProvider::class,
],

Add SCOUT_DRIVER=typesense to your .env file

Then you should publish scout.php configuration file to your config directory

php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"

In your config/scout.php add:

'typesense' => [
    'api_key'         => 'abcd',
    'nodes'           => [
      [
        'host'     => 'localhost',
        'port'     => '8108',
        'path'     => '',
        'protocol' => 'http',
      ],
    ],
    'nearest_node'    => [
        'host'     => 'localhost',
        'port'     => '8108',
        'path'     => '',
        'protocol' => 'http',
    ],
    'connection_timeout_seconds'   => 2,
    'healthcheck_interval_seconds' => 30,    
    'num_retries'                  => 3,
    'retry_interval_seconds'       => 1,
  ],

Usage

If you are unfamiliar with Laravel Scout, we suggest reading it's documentation first.

After you have installed scout and the Typesense driver, you need to add the Searchable trait to your models that you want to make searchable. Additionaly, define the fields you want to make searchable by defining the toSearchableArray method on the model and implement TypesenseSearch:

<?php

namespace App;

use Illuminate\Database\Eloquent\Model;
use Typesense\LaravelTypesense\Interfaces\TypesenseDocument;
use Laravel\Scout\Searchable;

class Todo extends Model implements TypesenseDocument
{
    use Searchable;
    
     /**
     * Get the indexable data array for the model.
     *
     * @return array
     */
    public function toSearchableArray()
    {
        return array_merge(
            $this->toArray(), 
            [
                // Cast id to string and turn created_at into an int32 timestamp
                // in order to maintain compatibility with the Typesense index definition below
                'id' => (string) $this->id,
                'created_at' => $this->created_at->timestamp,
            ]
        );
    }

     /**
     * The Typesense schema to be created.
     *
     * @return array
     */
    public function getCollectionSchema(): array {
        return [
            'name' => $this->searchableAs(),
            'fields' => [
                [
                    'name' => 'id',
                    'type' => 'string',
                ],
                [
                    'name' => 'name',
                    'type' => 'string',
                ],
                [
                    'name' => 'created_at',
                    'type' => 'int64',
                ],
            ],
            'default_sorting_field' => 'created_at',
        ];
    }

     /**
     * The fields to be queried against. See https://typesense.org/docs/0.24.0/api/search.html.
     *
     * @return array
     */
    public function typesenseQueryBy(): array {
        return [
            'name',
        ];
    }    
}

Then, sync the data with the search service like:

php artisan scout:import App\\Models\\Todo

After that you can search your models with:

Todo::search('Test')->get();

Adding via Query

The searchable() method will chunk the results of the query and add the records to your search index. Examples:

$todo = Todo::find(1);
$todo->searchable();

$todos = Todo::where('created_at', '<', now())->get();
$todos->searchable();

Multi Search

You can send multiple search requests in a single HTTP request, using the Multi-Search feature.

$searchRequests = [
    [
      'collection' => 'todo',
      'q' => 'todo'
    ],
    [
      'collection' => 'todo',
      'q' => 'foo'
    ]
];

Todo::search('')->searchMulti($searchRequests)->paginateRaw();

Generate Scoped Search Key

You can generate scoped search API keys that have embedded search parameters in them. This is useful in a few different scenarios:

  1. You can index data from multiple users/customers in a single Typesense collection (aka multi-tenancy) and create scoped search keys with embedded filter_by parameters that only allow users access to their own subset of data.
  2. You can embed any search parameters (for eg: exclude_fields or limit_hits) to prevent users from being able to modify it client-side.

When you use these scoped search keys in a search API call, the parameters you embedded in them will be automatically applied by Typesense and users will not be able to override them.

<?php

namespace App;

use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;
use Typesense\LaravelTypesense\Concerns\HasScopedApiKey;
use Typesense\LaravelTypesense\Interfaces\TypesenseDocument;

class Todo extends Model implements TypesenseDocument
{
    use Searchable, HasScopedApiKey;
}

Usage

Todo::setScopedApiKey('xyz')->search('todo')->get();

Migrating from devloopsnet/laravel-typesense

  • Replace devloopsnet/laravel-typesense in your composer.json requirements with typesense/laravel-scout-typesense-driver
  • The Scout driver is now called typesense, instead of typesensesearch. This should be reflected by setting the SCOUT_DRIVER env var to typesense, and changing the config/scout.php config key from typesensesearch to typesense
  • Instead of importing Devloops\LaravelTypesense\*, you should import Typesense\LaravelTypesense\*
  • Instead of models implementing Devloops\LaravelTypesense\Interfaces\TypesenseSearch, they should implement Typesense\LaravelTypesense\Interfaces\TypesenseDocument

Authors

This package was originally authored by Abdullah Al-Faqeir and his company DevLoops: https://github.com/devloopsnet/laravel-scout-typesense-engine. It has since been adopted into the Typesense Github org.

Other key contributors include:

License

The MIT License (MIT). Please see the License File for more information.

laravel-scout-typesense-driver's People

Contributors

abdullahfaqeir avatar alexgschwend avatar arayiksmbatyan avatar bepsvpt avatar cephee avatar codacy-badger avatar felipehertzer avatar hi019 avatar jasonbosco avatar karakhanyans avatar laravel-shift avatar manavo avatar peterfox avatar wh5938316 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

laravel-scout-typesense-driver's Issues

Http\Discovery\NotFoundException No HTTPlug clients found.

I am currently experimenting with algolia, meilisearch and typsense, and typsense driver was the only one to fail after trying:

php artisan scout:import "App\Brand"

Http\Discovery\NotFoundException

No HTTPlug clients found. Make sure to install a package providing "php-http/client-implementation". Example: "php-http/guzzle6-adapter".

is this an expected behaviour?

PHP 8.2 deprecation warnings

Description

PHP 8.2 throws deprecation warning on using ${var} in strings.

Steps to reproduce

Just use PHP 8.2

Expected Behavior

No error shown.

Actual Behavior

PHP message: PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /var/www/starfish/vendor/typesense/laravel-scout-typesense-driver/src/Typesense.php on line 195

Metadata

Typesense Version: any
OS: linux

Undefined array key "code" error in Typesense.php:170 when updating searchable model

Description

When updating a searchable attribute for a searchable model, the searchable() update throws an error.

This issue occurs on the typesense driver 5.0.1 but not on 5.0.0.

The culprit seems to be in the Typesense\LaravelTypesense\Typesense class, in the importDocuments method, when the import method response structure does not match the looped check.

Is somehow the 5.0.1 version expected to work with a newer server version?

Steps to reproduce

To reproduce, update any searchable attribute for a searchable model.

Expected Behavior

Model updated/uploaded successfully to the typesense server, without errors.

Actual Behavior

Model updated/uploaded successfully to the typesense server, WITH an error.

[2021-12-03 14:53:48] local.ERROR: Undefined array key "code" {"userId":2,"exception":"[object] (ErrorException(code: 0): Undefined array key \"code\" at /home/bravo/_work/_proj/rolcrispa/vendor/typesense/laravel-scout-typesense-driver/src/Typesense.php:170)
[stacktrace]
#0 /home/redacted/vendor/typesense/laravel-scout-typesense-driver/src/Typesense.php(170): Illuminate\\Foundation\\Bootstrap\\HandleExceptions->handleError()
#1 /home/redacted/vendor/typesense/laravel-scout-typesense-driver/src/Engines/TypesenseEngine.php(86): Typesense\\LaravelTypesense\\Typesense->importDocuments()
#2 /home/redacted/vendor/laravel/scout/src/Searchable.php(62): Typesense\\LaravelTypesense\\Engines\\TypesenseEngine->update()
#3 /home/redacted/vendor/laravel/scout/src/Searchable.php(41): LaravelEnso\\Products\\Models\\Product->queueMakeSearchable()
#4 /home/redacted/vendor/laravel/framework/src/Illuminate/Macroable/Traits/Macroable.php(124): Illuminate\\Database\\Eloquent\\Collection->Laravel\\Scout\\{closure}()
#5 /home/redacted/vendor/laravel/scout/src/Searchable.php(169): Illuminate\\Support\\Collection->__call()
#6 /home/redacted/vendor/laravel/scout/src/ModelObserver.php(109): LaravelEnso\\Products\\Models\\Product->searchable()
#7 /home/redacted/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(528): Laravel\\Scout\\ModelObserver->saved()
#8 [internal function]: Illuminate\\Events\\Dispatcher->Illuminate\\Events\\{closure}()
#9 /home/redacted/vendor/laravel/framework/src/Illuminate/Database/DatabaseTransactionsManager.php(84): call_user_func()
#10 /home/redacted/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(529): Illuminate\\Database\\DatabaseTransactionsManager->addCallback()
#11 /home/redacted/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(424): Illuminate\\Events\\Dispatcher->Illuminate\\Events\\{closure}()
#12 /home/redacted/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(249): Illuminate\\Events\\Dispatcher->Illuminate\\Events\\{closure}()
#13 /home/redacted/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Concerns/HasEvents.php(189): Illuminate\\Events\\Dispatcher->dispatch()
#14 /home/redacted/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(1035): Illuminate\\Database\\Eloquent\\Model->fireModelEvent()
#15 /home/redacted/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(1006): Illuminate\\Database\\Eloquent\\Model->finishSave()
#16 /home/redacted/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(885): Illuminate\\Database\\Eloquent\\Model->save()
#17 /home/redacted/vendor/laravel-enso/products/src/Http/Controllers/Update.php(14): Illuminate\\Database\\Eloquent\\Model->update()
#18 /home/redacted/vendor/laravel-enso/product-eav/src/Http/Controllers/Update.php(15): LaravelEnso\\Products\\Http\\Controllers\\Update->__invoke()
#19 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): LaravelEnso\\ProductEav\\Http\\Controllers\\Update->__invoke()
#20 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\\Routing\\Controller->callAction()
#21 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Route.php(262): Illuminate\\Routing\\ControllerDispatcher->dispatch()
#22 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Route.php(205): Illuminate\\Routing\\Route->runController()
#23 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Router.php(695): Illuminate\\Routing\\Route->run()
#24 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}()
#25 /home/redacted/vendor/laravel-enso/localisation/src/Http/Middleware/SetLanguage.php(18): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#26 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): LaravelEnso\\Localisation\\Http\\Middleware\\SetLanguage->handle()
#27 /home/redacted/vendor/laravel-enso/permissions/src/Http/Middleware/VerifyRouteAccess.php(18): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#28 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): LaravelEnso\\Permissions\\Http\\Middleware\\VerifyRouteAccess->handle()2
#29 /home/redacted/vendor/laravel-enso/impersonate/src/Http/Middleware/Impersonate.php(22): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#30 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): LaravelEnso\\Impersonate\\Http\\Middleware\\Impersonate->handle()
#31 /home/redacted/vendor/laravel-enso/action-logger/src/Http/Middleware/ActionLogger.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#32 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): LaravelEnso\\ActionLogger\\Http\\Middleware\\ActionLogger->handle()
#33 /home/redacted/vendor/laravel-enso/core/src/Http/Middleware/VerifyActiveState.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#34 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): LaravelEnso\\Core\\Http\\Middleware\\VerifyActiveState->handle()
#35 /home/redacted/vendor/laravel-enso/control-panel-api/src/Http/Middleware/RequestMonitor.php(15): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#36 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): LaravelEnso\\ControlPanelApi\\Http\\Middleware\\RequestMonitor->handle()
#37 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(50): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#38 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle()
#39 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(127): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#40 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(103): Illuminate\\Routing\\Middleware\\ThrottleRequests->handleRequest()
#41 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(55): Illuminate\\Routing\\Middleware\\ThrottleRequests->handleRequestUsingNamedLimiter()
#42 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Routing\\Middleware\\ThrottleRequests->handle()
#43 /home/redacted/vendor/laravel/framework/src/Illuminate/Auth/Middleware/Authenticate.php(44): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#44 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Auth\\Middleware\\Authenticate->handle()
#45 /home/redacted/vendor/laravel/sanctum/src/Http/Middleware/EnsureFrontendRequestsAreStateful.php(33): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#46 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Laravel\\Sanctum\\Http\\Middleware\\EnsureFrontendRequestsAreStateful->Laravel\\Sanctum\\Http\\Middleware\\{closure}()
#47 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(78): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#48 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\VerifyCsrfToken->handle()
#49 /home/redacted/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(121): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#50 /home/redacted/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(64): Illuminate\\Session\\Middleware\\StartSession->handleStatefulRequest()
#51 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Session\\Middleware\\StartSession->handle()
#52 /home/redacted/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#53 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse->handle()
#54 /home/redacted/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(67): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#55 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Cookie\\Middleware\\EncryptCookies->handle()
#56 /home/redacted/vendor/laravel/sanctum/src/Http/Middleware/EnsureFrontendRequestsAreStateful.php(26): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#57 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(149): Laravel\\Sanctum\\Http\\Middleware\\EnsureFrontendRequestsAreStateful->Laravel\\Sanctum\\Http\\Middleware\\{closure}()
#58 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#59 /home/redacted/vendor/laravel/sanctum/src/Http/Middleware/EnsureFrontendRequestsAreStateful.php(34): Illuminate\\Pipeline\\Pipeline->then()
#60 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Laravel\\Sanctum\\Http\\Middleware\\EnsureFrontendRequestsAreStateful->handle()
#61 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#62 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Router.php(697): Illuminate\\Pipeline\\Pipeline->then()
#63 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Router.php(672): Illuminate\\Routing\\Router->runRouteWithinStack()
#64 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Router.php(636): Illuminate\\Routing\\Router->runRoute()
#65 /home/redacted/vendor/laravel/framework/src/Illuminate/Routing/Router.php(625): Illuminate\\Routing\\Router->dispatchToRoute()
#66 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(167): Illuminate\\Routing\\Router->dispatch()
#67 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}()
#68 /home/redacted/vendor/sentry/sentry-laravel/src/Sentry/Laravel/Http/SetRequestIpMiddleware.php(45): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#69 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Sentry\\Laravel\\Http\\SetRequestIpMiddleware->handle()
#70 /home/redacted/vendor/sentry/sentry-laravel/src/Sentry/Laravel/Http/SetRequestMiddleware.php(42): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#71 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Sentry\\Laravel\\Http\\SetRequestMiddleware->handle()
#72 /home/redacted/vendor/barryvdh/laravel-debugbar/src/Middleware/InjectDebugbar.php(60): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#73 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Barryvdh\\Debugbar\\Middleware\\InjectDebugbar->handle()
#74 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#75 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#76 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle()
#77 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#78 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#79 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle()
#80 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#81 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle()
#82 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(86): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#83 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle()
#84 /home/redacted/vendor/fruitcake/laravel-cors/src/HandleCors.php(52): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#85 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Fruitcake\\Cors\\HandleCors->handle()
#86 /home/redacted/vendor/fideloper/proxy/src/TrustProxies.php(57): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#87 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Fideloper\\Proxy\\TrustProxies->handle()
#88 /home/redacted/vendor/laravel-enso/core/src/Http/Middleware/AuthorizationCookie.php(16): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#89 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): LaravelEnso\\Core\\Http\\Middleware\\AuthorizationCookie->handle()
#90 /home/redacted/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#91 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(142): Illuminate\\Pipeline\\Pipeline->then()
#92 /home/redacted/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(111): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter()
#93 /home/redacted/public/index.php(52): Illuminate\\Foundation\\Http\\Kernel->handle()
#94 /home/bravo/.config/composer/vendor/cpriego/valet-linux/server.php(232): require('...')
#95 {main}
"}

Metadata

Typsense Version: 0.21.0 (locally hosted)

OS: Linux Mint 19.3, Kernel Linux 5.4.0-90-generic x86_64

Other relevant versions:
laravel/framework v8.74.0
laravel/scout v9.3.2
typesense/laravel-scout-typesense-driver v5.0.1

Hotfix TypeError - Typesense\\Documents::import(): Argument #1 ($documents) must be of type string, array given.

Description

Afrer migrate from (v5.2.8 => v5.2.6) got the type error.
"Typesense\\Documents::import(): Argument #1 ($documents) must be of type string, array given, called in /var/www/artwin.io/vendor/typesense/laravel-scout-typesense-driver/src/Typesense.php on line 191" exception: "TypeError" file: "./vendor/typesense/typesense-php/src/Documents.php"

Metadata

Typesense Version: v5.2.8


Before and after method changes fallow up on the screenshots.

v5 2 6

v5 2 6 --- v5 2 8 v5 2 8

Support "Scoped Search API Keys" to account for multi-tenant applications

Description

When using typesense with laravel-scout-typesense-driver in a tenancy-enabled application, results are not limited to the active tenant.

The Typsense docs propose using "Scoped Search API Keys" in that case, but as far as I can see, laravel-scout-typesense-driver provides no functionality to change the API Key on a per-instance basis, only for the whole app through config/scout.php

I am working on a workaround to supply the Tenant's ID in the data, to limit the results to the current user. But of course, a solution within the driver would be preferable.

Steps to reproduce

$t = TenancyUser::first(); tenancy()->initialize($t); _ModelName_::search('test')->paginate(1, page: 1)->total();

Expected Behavior

Returns number of results for ModelName in the active tenancy

Actual Behavior

Returns number of results for ModelName in ALL tenancy schemas

Metadata

Typesense Version: 0.23.1

OS: Docker image 'typesense/typesense:0.23.1' and 'postgres:13.5'

Error importing document: Field `itemStocks` not found.

if ($this->itemStocks->isNotEmpty()) {
$extraFields['itemStocks'] = $this->itemStocks->map(function ($itemStock) {
return [
'available' => (int)$itemStock->available,
'stock_unit_code' => (string)$itemStock->stock_unit_code,
'source_warehouse_id' => (int)$itemStock->source_warehouse_id
];
})->toArray();
}

When im trying to import object like this i get error Error importing document: Field itemStocks not found.

How to fix it? I declared itemStocks in schema and gived type object[]

Model update resets document auto-generated embeddings

Description

In Typesense version 0.25 it's possible to define an embedding field. But since this field is auto-generated and is not stored locally, any update to the model will cause the embeddings field to reset.

Steps to reproduce

Schema:

    {
      "name": "embedding",
      "type": "float[]",
      "facet": false,
      "optional": false,
      "index": true,
      "sort": false,
      "infix": false,
      "locale": "",
      "embed": {
        "from": [
          "some_field"
        ],
        "model_config": {
          "model_name": "ts/paraphrase-multilingual-mpnet-base-v2"
        }
      },
      "num_dim": 768
    }

After I import the model, Typesense takes some time to generate embeddings. After that process, all documents will have the embedding field with array of 768 floats.

Then, If I call searchable() method on the model, the embedding field becomes empty.

Expected Behavior

embedding field should ether be updated if embed.from fields are changed, or be left unchanged.

Actual Behavior

the embedding field becomes empty

Metadata

Typesense Version: 0.25.0

OS: Ubuntu 20.04

Error from Typesense on Update of Model Instance which is not in index

  • typesense/laravel-scout-typesense-driver: 5.2.2
  • Scout Version: 9.8.1
  • Scout Driver: Typesense
  • Laravel Version: 9.52.4
  • PHP Version: 8.1
  • Database Driver & Version: 10.2.44-MariaDB

Description:

If an instance of a model is created & persisted in DB and should not be searchable yet (e.g. the instance is not public yet) - everything works fine.

When the same instace is beeing saved again, an error occurs and the model cannot be saved.
I guess that the reason is, that on /vendor/laravel/scout/src/ModelObserver.php:102 the Method $model->wasSearchableBeforeUpdate() returns true, even though the instance of the model is not in the index yet.
The problem occurs for all Models in Index, as soon as the shouldBeSearchable() Method returns false (eg if the instance is not public yet).

ModelObserver.php:101ff

... if (! $model->shouldBeSearchable()) { if ($model->wasSearchableBeforeUpdate()) { $model->unsearchable(); } return; } ...

scout1

Updates needed to complete adoption to typesense github org

This repo was adopted from @AbdullahFaqeir's work here: https://github.com/devloopsnet/laravel-scout-typesense-engine.

The following tasks need to be done to complete the adoption:

  • Add LICENSE file (current license MIT)
  • Add credits to Abdullah Al-Faqeir and his company DevLoops for building the original version, in README
  • Change instances of Devloops in class names to Typesense
  • TypesenseServiceProvider.php has references to master_node and read_replica_nodes, which are old configuration parameters no longer supported in the latest version of Typesense Server. Needs to be switched to nodes and nearest_node
  • Publish new version of package under typesense namespace - @jasonbosco
  • Test that new published version works fine
  • Mark devloopsnet/laravel-typesense as abandoned in Packagist and list this package as its replacement
  • Update README.md in devloopsnet/laravel-scout-typesense-engine repo to point people to this repo: devloopsnet/laravel-scout-typesense-engine#17

Cannot index collection with a soft deleted first model when scout.soft_delete is true

Description

When indexing with scout.soft_delete set to true index is not imported if the first model of the Collection is deleted

Steps to reproduce

use scout:import command to import a collection of models where the first one is deleted, when scout.soft_delete is true

Expected Behavior

The collection should be indexed

Actual Behavior

The collection is not indexed

Metadata

Typesense Version: 0.23.1

OS: Linux

We propose the following modification on line 171 of src/Engines/TypesenseEngine (the update method)
From this:
if (!$this->usesSoftDelete($models->first()) || is_null($models->first()?->deleted_at)) {
To this:
if (!$this->usesSoftDelete($models->first()) || is_null($models->first()?->deleted_at) || config('scout.soft_delete', false)) {

Document not updated if 'afterCommit' is true

Description

The TypesenseEngine::update will check if the model is dirty, but if the 'afterCommit' is set to true in the config, the method will always skip.

Steps to reproduce

  • set scout.after_commit to true
  • trigger a save event on a Searchable model

Expected Behavior

TypesenseEngine::update should update the model, ignoring the 'isDirty' if 'afterCommit' is set to true.

Actual Behavior

TypesenseEngine::update skips the update.

Metadata

Typesense Version: 0.24.0

typesense/laravel-scout-typesense-driver: v5.2.0,
laravel/scout: v9.8.0

OS: Ubuntu 22.04

Question : Use filter_by or complex query

Description

Hello everyone,

I would like to know if it is possible (if so, how?) to be able to use the filter_by in order to be able to add filter conditions such as
'price:<=25' ? As given as an example here in the Typesense documentation :
https://typesense.org/docs/0.22.2/api/documents.html#search and
https://typesense.org/docs/0.22.2/api/documents.html#search-parameters

Since it is not possible in Laravel Scout to use complex Where clause

Scout allows you to add simple "where" clauses to your search queries. Currently, these clauses only support basic numeric equality checks and are primarily useful for scoping search queries by an owner ID. Since a search index is not a relational database, more advanced "where" clauses are not currently supported

Thank you in advance for your answer

Pagination not working properly when using query callback

Description

It seems the TypesenseEngine doesn't correctly work with Scout's ->paginate, particularly when results exceeed 250 hits, as the total in the pagination gets set to the default perPage for some reason. It seems the MeiliSearch adapter also had this issue, see: laravel/scout#535

The workaround I have is to implement the pagination manually with:

$searchData = $query->query(fn ($search) => $search->with($relations))->get();
$paginatedSearchResults = new LengthAwarePaginator($searchData, $query->raw()['found'], $perPage);

Steps to reproduce

  1. Have a collection that exceeds 250 documents.
  2. Search against that collection utilising Scout and Typesense of course.
  3. Attempt to paginate that data with a query callback, for example with:
$query->query(fn ($search) => $search->with($relations))
                ->paginate($perPage)
                ->withQueryString();

Expected Behavior

Return a LengthAwarePaginator with 1278 total items.

Actual Behavior

Returns a LengthAwarePaginator with 15 total items.

Metadata

Typsense Version: v5.1.0-rc1

OS: MacOS 12.4 Monterey

Raw results from Typesense:

array:8 [โ–ผ
  "facet_counts" => []
  "found" => 1278
  "hits" => array:15 [โ–ถ]
  "out_of" => 171361
  "page" => 1
  "request_params" => array:3 [โ–ถ]
  "search_cutoff" => false
  "search_time_ms" => 1
]

Pagination without query callback works as expected:

Illuminate\Pagination\LengthAwarePaginator {#2096 โ–ผ
  #items: Illuminate\Database\Eloquent\Collection {#2102 โ–ถ}
  #perPage: 15
  #currentPage: 1
  #query: array:1 [โ–ถ]
  #fragment: null
  #pageName: "page"
  +onEachSide: 3
  #options: array:2 [โ–ถ]
  #total: 1278
  #lastPage: 86
}

Pagination with query callback does NOT work as expected:

Illuminate\Pagination\LengthAwarePaginator {#2431 โ–ผ
  #items: Illuminate\Database\Eloquent\Collection {#2164 โ–ถ}
  #perPage: 15
  #currentPage: 1
  #query: array:1 [โ–ถ]
  #fragment: null
  #pageName: "page"
  +onEachSide: 3
  #options: array:2 [โ–ถ]
  #total: 15
  #lastPage: 1
}

Undefined property: Laravel\Scout\Builder::$whereIns

Getting this error while running a simple query. There is already a issue logged with this same. But its closed without any answer. That's why i'm creating this issue.
If anyone can suggest any answer ASAP would be helpful

Undefined property: Laravel\Scout\Builder::$whereIns

Here is my query

$result = Todo::search('test')->get();

Typesense scout import fails in Laravel 11

Description

My php artisan scout:import fails for large datasets in Laravel11. This issue is not seen with smaller dataset. May be less than 500.

Steps to reproduce

There is no issue when I was on Laravel 10. When I migrated to Laravel 11.X, this issue is seen.
Steps to reproduce:

  1. Create a table with larage data. May be 5000 or 10000.
  2. Do follow the steps of creating toSearchableArray() and Collection as per documentation.
  3. Use "php artisan scout:import "App\Models\YourModel"
  4. You will see error saying record already exist.
  5. On typesense server you will find first batch of initial 500 records already indexed.
  6. So while attempting next batch of index, it is not able to recognise the difference and hence considers documents alre
    scout-error

Expected Behavior

I should get my documents in batches. Atleast in my case it has been always in a batch of 500 documents.

Actual Behavior

I see only first 500 documents indexed but in next batch index it fails saying collection already exists. In laravel10, this was perfectly fine.

Metadata

Typesense Version: 0.26

OS: Ubuntu 22.04

Duplicate array keys in TypesenseEngine::buildSearchParams

Description

Keys are repeated: q, query_by, filter_by, per_page, page, highlight_start_tag, highlight_end_tag, exhaustive_search
in TypesenseEngine::buildSearchParams on line 230

    private function buildSearchParams(Builder $builder, int $page, int|null $perPage): array
    {
        $params = [
            'q' => $builder->query,
            'query_by' => implode(',', $builder->model->typesenseQueryBy()),
            'filter_by' => $this->filters($builder),
            'per_page' => $perPage,
            'page' => $page,
            'highlight_start_tag' => $this->startTag,
            'highlight_end_tag' => $this->endTag,
            'exhaustive_search' => $this->exhaustiveSearch,
            'q'                          => $builder->query,
            'query_by'                   => implode(',', $builder->model->typesenseQueryBy()),
            'filter_by'                  => $this->filters($builder),
            'per_page'                   => $perPage,
            'page'                       => $page,
            'highlight_start_tag'        => $this->startTag,
            'highlight_end_tag'          => $this->endTag,
            'snippet_threshold'          => $this->snippetThreshold,
            'exhaustive_search'          => $this->exhaustiveSearch,
            'use_cache'                  => $this->useCache,
            'cache_ttl'                  => $this->cacheTtl,
            'prioritize_exact_match'     => $this->prioritizeExactMatch,
            'enable_overrides'           => $this->enableOverrides,
            'highlight_affix_num_tokens' => $this->highlightAffixNumTokens,
        ];

       // etc...

}

Metadata

Typesense Version:0.25.2
laravel-scout-typesense-driver: 5.2.8

OS: Docker

Passing "empty" values to search query results in "Parameter `q` is required"

Description

When using certain values for query string, "Parameter q is required" is thrown. More exactly, any value that returns true from empty(...) does not work.

Steps to reproduce

For example, App\Models\Taxon::search("0")->get() throws an error.

Expected Behavior

Found values should be returned.

Actual Behavior

"Parameter q is required" is thrown.

Metadata

Typesense Version:
0.25.1

OS:

Culprit

https://github.com/typesense/laravel-scout-typesense-driver/blob/master/src/Engines/TypesenseEngine.php#L208 and https://github.com/typesense/laravel-scout-typesense-driver/blob/master/src/Engines/TypesenseEngine.php#L208 runs array_filter on parameters array, therefore, removing the "0" query param, and typesense request is then missing the q parameter.

Possible solution

Change array_filter to a stricter function, such as rejecting only null values.

MultiSearch Example(s)

Wondering if anyone has any examples of utilizing MultiSearch functionality or whether or not it is even possible?

Suggestion: reduce unnecessary API calls

Typesense driver makes a lot of unnecessary requests to the server. Real life example: I have a collection with over 10000 documents in it. Each document is updated hourly. Each update changes one model attribute, which triggers 2 request to typesense server. That's 5.5 completely unnecessary requests per second.

20k documents will cause 11 requests per second, 100k documents - 55 requests per second.

I know that typesense server is fast and performant, but still, that requests congest the bandwidth, tcp connections and other resources.

1. Unnecessary updates

At the moment, typesence-driver updates the document on any model update, even if the changed field is not in the schema. This can cause performance problems if there are a lot of updates.

For example:

// MyModel.php

public function toSearchableArray(): array {
    return [
        'id' => (string) $this->id,
        'name' => $this->name,
    ];
}

public function getCollectionSchema(): array {
    return [
        'fields' => [[
            'name' => 'id',
            'type' => 'string',
        ],[
            'name' => 'name',
            'type' => 'string',
        ]],
    ];
}

Now, when I update any field in my model, typesense-driver will try to update the remote index:

MyModel::first()->update([
    'some_field' => 'test',
]);

// typesense-driver sends GET to /collections/my-model
// typesense-driver sends POST to /collections/my-model/documents/import

MyModel::first()->update([
    'some_date' => now(),
]);

// typesense-driver sends GET to /collections/my-model
// typesense-driver sends POST to /collections/my-model/documents/import

My suggestion is to check if the changed fields are in the schema before updating the document. For example, using the getChanges() model method.

The problem that I see here is how to handle computed attributes (ones that use appends array and getter function).

2. Unnecessary collection info retrieval

Upon every update, typesense driver makes a GET request to retrieve a collection info.

I suggest getting that info from schema (getCollectionSchema method).

Metadata

Typsense Version: 5.0

OS: Ubuntu 20.04

Unable to sort

Description

At present it doesn't seem possible to sort, or if it is it's not documented anywhere.

Steps to reproduce

Setup on a model in this example Post and try the following:

Post::search(['q' => $searchTerm, 'sort_by' => 'created_at:asc'])

Expected Behavior

I would expect the results to be ordered by the created_at attribute in ascending order

Actual Behavior

Instead the following error message is show:

Request Malformed: Parameter q is required.

Metadata

Typesense Driver Version: 5.2.4

Import not working with array data.

When importing data that have array value, then throw this error -

Typesense\Documents::import(): Argument #1 ($documents) must be of type string, array given

But, After changing this line manually, it's working -

Paht: vendor/typesense/laravel-scout-typesense-driver/src/Typesense.php
Line: 189

$importedDocuments = $collectionIndex->getDocuments()
                                             ->import($documents, ['action' => $action]);

to

$importedDocuments = $collectionIndex->getDocuments()
                                             ->createMany($documents);

Typsense import errors fail silently

Description

Attempts to import a model, with an incorrect schema, does not return any errors to the user, despite the underlying JSON API call returning an error along with succcess field set to false. This occurs during batch import via php artisan scout:import and single model updates. No errors are returned to the user, and no indication is made of the import failing.

Steps to reproduce

Define a schema on the model like so:

public function getCollectionSchema(): array
    {
        return [
            'name' => $this->searchableAs(),
            'fields' => [
                [
                    'name' => 'id',
                    'type' => 'string',
                ]
            ],
            'default_sorting_field' => 'id',
        ];
    }

public function toSearchableArray()
{
  return [
        'id' => $this->id,
  ];
}

Bulk import the model via php artisan scout:import "App\SomeModel\Job"

Expected Behavior

The command should fail with an error indicating the failed result, along with the error reason.

Actual Behavior

Console reports success:

Imported [App\SomeModel\Job] models up to ID: 552

Yet, no models are actually imported in Typesense. Logging the underlying JSON response within this library, shows the following JSON being returned:

{"error":"Document's 'id' field should be a string.","success":false}

Metadata

Typsense Version: 0.21.0

OS: Ubuntu 20.04.3

Please add possibility to use Laravel Scout Builder ->options() method

Description

Currently we cannot use the Model::search('query')->options([]) feature, since the typesense engine class does not merge the builder options with the initial options. For example the Laravel scout algolia driver does enable this like this:

protected function performSearch(Builder $builder, array $options = [])
    {
        $algolia = $this->algolia->initIndex(
            $builder->index ?: $builder->model->searchableAs()
        );

        $options = array_merge($builder->options, $options);

        if ($builder->callback) {
            return call_user_func(
                $builder->callback,
                $algolia,
                $builder->query,
                $options
            );
        }

        return $algolia->search($builder->query, $options);
    }

Steps to reproduce

add options to the Builder->options() method and see that these options are not applied in the search query

Expected Behavior

The given options array in the options method on the Laravel Scout Builder class must be applied in the search query to typesense

Suggested Solution

Rewrite the current performSearch method of the Typesense engine class from:

protected function performSearch(Builder $builder, array $options = []): mixed
    {
        $documents = $this->typesense->getCollectionIndex($builder->model)
            ->getDocuments();
        if ($builder->callback) {
            return call_user_func($builder->callback, $documents, $builder->query, $options);
        }
        if(!$this->optionsMulti)
        {
            $documents = $this->typesense->getCollectionIndex($builder->model)
                ->getDocuments();
            if ($builder->callback) {
                return call_user_func($builder->callback, $documents, $builder->query, $options);
            }

            return $documents->search($options);
        } else {
            return $this->typesense->multiSearch(["searches" => $this->optionsMulti], $options);
        }
    }
TO:
protected function performSearch(Builder $builder, array $options = []): mixed
    {
        $options = array_merge($options, $builder->options); // newly added line

        $documents = $this->typesense->getCollectionIndex($builder->model)
            ->getDocuments();
        if ($builder->callback) {
            return call_user_func($builder->callback, $documents, $builder->query, $options);
        }
        if(!$this->optionsMulti)
        {
            $documents = $this->typesense->getCollectionIndex($builder->model)
                ->getDocuments();
            if ($builder->callback) {
                return call_user_func($builder->callback, $documents, $builder->query, $options);
            }

            return $documents->search($options);
        } else {
            return $this->typesense->multiSearch(["searches" => $this->optionsMulti], $options);
        }
    }

Using Multi Search after merging to Laravel Scout

No idea if this is the right place to ask, but;

This driver has been merged upstream to the Laravel Scout repository, however it seems the Laravel team has no intention of implementing multi search for any of the drivers, including Typesense. How would you recommend one to do federated/multi search? Use the driver from this repository or?

Thanks in advance

Error in tinker when trying to access `\Typesense\LaravelTypesense\Typesense` singleton

Description

There's an error when trying to create/access the Typesense instance in the tinker console.

Although the library is working fine through the web app, it just fails here in the console.

Steps to reproduce

$ php artisan tinker
Psy Shell v0.10.8 (PHP 8.0.2 โ€” cli) by Justin Hileman
>>> app()->make(\Typesense\LaravelTypesense\Typesense::class);
TypeError: Typesense\LaravelTypesense\TypesenseServiceProvider::Typesense\LaravelTypesense\{closure}(): Argument #1 ($client) must be of type Typesense\Client, Illuminate\Foundation\Application given, called in /home/vagrant/app/vendor/laravel/framework/src/Illuminate/Container/Container.php on line 869

Expected Behavior

I'd expect the Typesense class/singleton to be returned

Actual Behavior

I get the error returned above.

Metadata

Typsense Version: 0.21.0

OS: Ubuntu 18.04.5 LTS

Detail regarding composer installation

HI.. Try to install this package locally

through command:
composer require typesense/laravel-scout-typesense-driver

Error:

[InvalidArgumentException]
  Package typesense/laravel-scout-typesense-driver has a PHP requirement incompatible with your PHP version, PHP extensions and Composer version

Any information about what and which version I should fix?

Laravel 10 Support

Description

Currently the package doesn't support Laravel 10

Steps to reproduce

Install typesense driver package in laravel 10 and I can't run composer update

Expected Behavior

Should be able to work in Laravel 10

Metadata

Typesense Version: 4

OS: MacOS

Grouping isn't supported by the map result function

Description

When utilising the groupBy macro, the map functin from the TypesenseEngine.php fails as the Typesense API returns grouped_hits and not a hits index.

My workaround was to implement a custom macro:

Builder::macro('getGroup', function () {
            $results = $this->engine()->search($this);

            $objectIds = collect($results['grouped_hits'] ?? $results['hits'])
                ->pluck(isset($results['grouped_hits']) ? 'group_key.0' : 'document.id')
                ->values()
                ->all();

            if (isset($results['grouped_hits'])) {
                return collect(array_values($objectIds));
            }

            return null;
        });

I then call this conditionally based on if I need a grouping to be applide with:

if ($groupBy) {
      return $query->getGroup();
}

This can be integrated to the map function quite easily but perhaps a seperate mapByGroup may be better.

Steps to reproduce

Perform a search with the ->groupBy() macro.

Expected Behavior

The TypesenseEngine should be able to map the grouped_hits index.

Actual Behavior

The TypesenseEngine fails to map the grouped_hits as it expects a hits index and therefore fails.

Metadata

Typsense Version: v5.1.0

OS: MacOS 12.4 Monterey

$model->unsearchable(); does not work

Description

When I upload new document document content is extracted and indexed on typesense which is very good.
The issue is when I delete the document I'm falling into the observer delete event, doing :
$model->unsearchable(); and it seems it not unsearchable it from typesense

Steps to reproduce

Upload the document it will be indexed
Delete the document and try to search by some text which was in that text, you will see that that result would be the deleted document

Expected Behavior

When the document is deleted in goes to $model->unsearchable(), and it should be unindexed from typesense maybe delete from there. And when I search I should gen no results since the document was deleted

Latest release doesn't have most recent fixes: Could not parse the filter query: unbalanced `&&` operands.

Description

I'm just doing what I think is the most basic usage of the library.

Model::search('myterm');

But i'm getting the following error:

"Could not parse the filter query: unbalanced && operands."

It looks like this was fixed a month ago: 7a9edba#diff-30cbc8fb802094edb92485725f009b63a895bcc546b84381b550883a17441129R266

However there hasn't been a new release since July so composer is still picking up the old version.

How can I get the latest version?

As it stands right now it seems like this package doesn't work out of the box.

I am on 5.1.0 right now.

Not able to skip an object from being indexed

Description

Problems when trying to skip an object so it DOES NOT become part of the index. There's no apparent way to do it using the function toSearchableArray

Steps to reproduce

The problem seems to be the function toSearchableArray

if I want to bypass an object on that function (ie: I don't want it into the index):

1- Returning an empty array results in a malformed JASON error when syncing:

 Typesense\Exceptions\TypesenseClientError 

  Error importing document: Bad JSON: not a properly formed document.
  
  at vendor/typesense/laravel-scout-typesense-driver/src/Typesense.php:165

2- Returning FALSE results in an error of unexpected response type:

  TypeError 

  Typesense\Documents::Typesense\{closure}(): Argument #1 ($document) must be of type array, bool given

  at vendor/typesense/typesense-php/src/Documents.php:139

3- if I use the Scout method 'unsearchable()' function results in an ObjectNotFound exception:

 Typesense\Exceptions\ObjectNotFound 

  Could not find a document with id: 1105

  at vendor/typesense/typesense-php/src/ApiCall.php:347

Expected Behavior

Ideally, if we would like this to work like Algolia does with Laravel Scout, by using the unsearchable method and returning either an empty array or a false flag, we would be able to skip the object from being added to the index.

Actual Behavior

The import is interrupted with one of those exceptions and the collection is being indexed only up to the point of the first failure.

Metadata

Typsense Version: 0.2.22

OS: Ubuntu 22.04

Don't update Typesense document if the model is soft-deleted

Description

Typesense driver does not check if the model is soft deleted and always updates the data when the model changes. This causes the soft-deleted document to be created in the collection.

Steps to reproduce

For example, in my case I have some searchable model with SoftDeletes trait and observer that clears some meta fields after the model is deleted. Something like that:

class MyModelObserver
{
    public function deleted(MyModel $model)
    {
        $model->update(['worker_id' => null, 'support_id' => null]);
    }
}

// ...

MyModel::delete(id); // The model is still in Typesense

I presume that if I do something like MyModel::withTrashed()->find(id)->update(['name' => 'test']), the model will be also pushed to the Typesense.

Expected Behavior

The Typesense driver should not update the remote document if the model is in soft-deleted state.

Actual Behavior

Typesense deletes the remote document and immediately restores it (because MyModelObserver calls the update method on the model.

Also note that worker_id and support_id is not even in the model schema (see #23).

Metadata

Typsense Version: 5.0

OS: Ubuntu 20.04

"Undefined property: Laravel\\Scout\\Builder::$whereIns",

Description

After uploading the file to Typesense, try to search over it using
MediaSource::search($request->input('search'))
->paginateRaw()
->getCollection()

Got error
"message": "Undefined property: Laravel\Scout\Builder::$whereIns",
vendor/typesense/laravel-scout-typesense-driver/src/Engines/TypesenseEngine.php"
"line": 396,

Exception when searching large datasets for common query

Description

When searching for a string that is relatively common throughout a collection of indexed data, the results can not be fetched or displayed because the search with the Scout driver results in the following exception:

Typesense\\Exceptions\\ObjectUnprocessable(code: 0): Only upto 250 hits can be fetched per page

Steps to reproduce

  1. Start a fresh Laravel project, install Scout and the Typesense Scout driver according to documentation
  2. Add a model and make it searchable
  3. Add a large dataset (~ 2000+ records with "lorem ipsum" text)
  4. Query the dataset and try to display the results in a paginated way:
$items = Item::search($request->input('query'))
    ->options(['query_by' => 'content'])->latest()
    ->paginate(perPage: 10, pageName: 'items')
    ->appends($request->only('query'));

Expected Behavior

I expect the search to succeed, even when there are more then 250 hits, because that's why I use pagination on my frontend.

Actual Behavior

Following exception is thrown:

Typesense\\Exceptions\\ObjectUnprocessable(code: 0): Only upto 250 hits can be fetched per page

Metadata

Laravel Version: 10.43.0
PHP Version: 8.2.12
Scout Version: 10.8.2
Typesense PHP SDK Version: 4.9.1
Typesense Version: 0.25.2
OS: Docker @ Apple Silicon

Error with `??` in `Typesense.php`

Description

Because of how the ?? check works, it will never actually try and load the document/collection in these places:

https://github.com/typesense/laravel-scout-typesense-driver/blob/master/src/Typesense.php#L89
https://github.com/typesense/laravel-scout-typesense-driver/blob/master/src/Typesense.php#L118
https://github.com/typesense/laravel-scout-typesense-driver/blob/master/src/Typesense.php#L163

The typesense library will always return the object, so the check for existence and then the null fallback are pointless, and actually breaks it.

I'll submit a PR for this tomorrow, unless someone beats me to it (@hi019?)

Error with deleting multiple documents

Description

Bulk deleting isn't working for me, I get an error like:

Typesense\Exceptions\ObjectNotFound: Could not find a filter field named `feedback.id` in the schema

My collection schema is:

[
  {
    "created_at":1602963950,
    "default_sorting_field":"last_reply_time",
    "fields":[
      {
        "facet":false,
        "index":true,
        "name":"application_id",
        "optional":false,
        "type":"int64"
      },
      {
        "facet":false,
        "index":true,
        "name":"message",
        "optional":true,
        "type":"string"
      },
      {
        "facet":false,
        "index":true,
        "name":"email",
        "optional":true,
        "type":"string"
      },
      {
        "facet":false,
        "index":true,
        "name":"last_reply_time",
        "optional":false,
        "type":"int64"
      },
      {
        "facet":false,
        "index":true,
        "name":"properties",
        "optional":true,
        "type":"string[]"
      },
      {
        "facet":false,
        "index":true,
        "name":"member_name",
        "optional":true,
        "type":"string"
      },
      {
        "facet":false,
        "index":true,
        "name":"member_email",
        "optional":true,
        "type":"string"
      },
      {
        "facet":false,
        "index":true,
        "name":"reply_messages",
        "optional":true,
        "type":"string[]"
      },
      {
        "facet":false,
        "index":true,
        "name":"reply_emails",
        "optional":true,
        "type":"string[]"
      }
    ],
    "name":"feedback",
    "num_documents":2459169,
    "num_memory_shards":4
  }
]

The issue might be that I haven't explicitly defined id as a field?

For what it's worth, even the example in the readme doesn't indicate that we should have id as a field.

Metadata

Typsense Version: 0.17.0 and 0.21.0 both tested

OS: Ubuntu 20.04.2 LTS

Connection refused when doing simple search

Description

Hi! im getting networkexception, but im not sure if this has something to do with me setting up from docker or something else.

Http\Client\Exception\NetworkException with message 'Failed to connect to localhost port 8108: Connection refused for "http://localhost:8108/collections/todo".'

Steps to reproduce

  1. added in my existing docker-compose.yml
...
  typsense:
    image: typesense/typesense:0.22.2
    container_name: typesense
    environment:
      TYPESENSE_DATA_DIR: /data
      TYPESENSE_API_KEY: xyz
    volumes:
      - /tmp/typesense-server-data:/data
    ports:
      - "8108:8108"
    restart: "no"
    networks:
      laravel:
...
  1. docker compose up -d, then curl http://localhost:8108/health which returned {"ok":true}

  2. Followed the readme instructions section:

  • dependencies (guzzle7 adapter and laravel-scout-typesense-driver)
  • service provider
  • artisan vendor:publish ...
  • config/scout.php
  • Searchable trait as well as TypesenseDocument interface
  • setting SCOUT_DRIVER value to typesense
  • php artisan scout:import App\\Models\\Todo

Expected Behavior

returning the searched results

Actual Behavior

After the installation and syncing, I tried this inside tinker: Todo::search('Test')->get(); then i get this

>>> Images::search('female')->get()
Http\Client\Exception\NetworkException with message 'Failed to connect to localhost port 8108: Connection refused for "http://localhost:8108/collections/todo".'

Metadata

Typsense Version: 0.22.2

OS: macos 12.4

Laravel v8

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.