laravel / scout Goto Github PK
View Code? Open in Web Editor NEWLaravel Scout provides a driver based solution to searching your Eloquent models.
Home Page: https://laravel.com/docs/scout
License: MIT License
Laravel Scout provides a driver based solution to searching your Eloquent models.
Home Page: https://laravel.com/docs/scout
License: MIT License
Let's say you index a model with some relations to Algolia or Elastic. Nested relations instead of separate indices.
If you search directly using Algolia/Elastic, your response will contain records with the entire structure (nested and all).
But Scout will only map the raw results to the main model class and not include its relations in the response.
I am using ElasticSearch. It is great that Laravel scout support ElasticSearch natively. However, I hope more customised features/options will be added. For example, we can set min_score for the search result, mapping for datatype and the language analysers for other languages.
Many Asian developers use ElasticSearch with language analyser plugins to support Asian languages full-text search.
Thank you.
I think Scout needs a function to select which entries are indexed. The implementation of Algolia's Laravel Plugin has an indexOnly()
function.
https://github.com/algolia/algoliasearch-laravel#restrict-indexing-to-a-subset-of-your-data
When i´m trying to create a user using socialite, the user don´t wan´t get pushed to the index.
For now nex user's don't know how to implement this in laravel app's.
I think we need to document this in a readme
I followed each step, where am I going wrong.
If i run
Artisan::call('scout:import')
in an Controller or route closure, following error came:
CommandNotFoundException in Application.php line 490: There are no commands defined in the "scout" namespace.
The command works great in the CLI, but not in an controller.
Hi, im trying to do a wildcard search with elasticsearch. Is it even possible? Ive looked at the driver for ES but cant really figure out if its even possible. I tried to do something like Users::search(['wildcard' => ['name' => 'amb*']])
but thats not working. I didnt see anything in the documentation either so it might not be possible. I just wanted it confirmed before i looked into other solutions :)
This is my test query from es-head and it works like intended.
{"query":{"bool":{"must":[{"wildcard":{"name":"amb*"}}],"must_not":[],"should":[]}},"from":0,"size":50,"sort":[],"aggs":{},"version":true}:
Hi Guys,
This is not so much an issue, as a feature request. I've been using Algoia with Laravel for a while now and one of my big complaints is the inability to send user activity back to the Algolia algorithm for each search term.
I record clicks, purchases etc and use this to rank the results however this applies across the board and not to individual search terms.
If there is anyone from Algolia working on this tool or reading this thread please consider allowing user activity for each search term to be fed back into the algorithm :)
Thanks
Anthony
It would be nice to be able to queue indexing for only specific models. For example, say I have a Person model and I have a Document model.
For the Person model, I don't want to or need to queue indexing because the data should be indexed right way and shouldn't cause any slowdowns.
However, for the Document model, if there is a rather large PDF or RichText document, and I need to dump the contents of that document and then index it, I would definitely want to use queuing.
Currently, Scout only allows for a Switch On / Switch Off for ALL models queuing option.
What I want to do is to use language analyzers (https://www.elastic.co/guide/en/elasticsearch/guide/current/using-language-analyzers.html), but seems like scout doesn't support custom mapping between models and documents.
Is there any way I can accomplish this?
Looks like this repo needs the StyleCI service hook 👍
/cc @GrahamCampbell
When trying to set the scout engine to the "null" implementation (e.g. say as a fallback for when no env variable exists: 'driver' => env('SCOUT_DRIVER', 'null')
or for tests) the string "null"
is parsed into a php null
value instead resulting in an instantiation error.
Suggest either renaming the null driver so something else like nullsearcher unless there is a way in phpunit.xml and .env it can be set as a string.
Current Implementation: Laravel\Scout\Engines\AlgoliaEngine
stores Algolia client as a protected $algolia
property. There is no getter for the property.
Requested feature: Allow access to the Algolia client using a getClient()
method (or similar). So, you could do something like this:
$algolia = app(EngineManager::class)->engine('algolia')->getClient();
Use case: I keep seeing requests for Algolia-specific features both here, and elsewhere on the web. Common requests involve setting up master/slave indices for sorting, configure/sync index settings, etc. All of these requests are Algolia-specific and not generic enough for Scout. If we had access to the Algolia client, we could build decorator traits (or whatever) for people that use Algolia and want functionality outside of the scope of Scout (similar to how the Laravel Filesystem allows access to the Amazon client for custom Amazon S3 functionality).
Currently, I am having to create a redundant Algolia client, and use that for anything custom.
Hi there,
I came across an error while I was working with laravel/scout. Here is a sample pseudo code:
public function toSearchableArray(){
$array = null;
if($this->isVerified)
{
$array = [
'id' => $this->id,
'firstName' => $this->firstname,
'imagePath' => \App::make('MyImageService')->GetFilePath($this->fileID)];
}
return $array;
}
This code works just fine in all scenarios but when I run "php artisan db:seed" I encounter below error:
[Symfony\Component\Debug\Exception\FatalErrorException]
syntax error, unexpected '?'
I couldn't really figure out what's happening, my work around for now is comment out the "toSearchableArray" method whenever I need to run "php artisan db:seed"
PHP Fatal error: Class 'Elasticsearch\ClientBuilder' not found in /media/web/imagesearch.dev/vendor/laravel/scout/src/EngineManager.php on line 45
I can't find that file also.
I've just implemented Scout on an API I've been building. I'm using Fractal by the PHP League which has baked in support for Laravel pagination using \Illuminate\Pagination\LengthAwarePaginator
.
The documentation for Scout states the following (https://laravel.com/docs/5.3/scout#pagination):
In addition to retrieving a collection of models, you may paginate your search results using the paginate method. This method will return a Paginator instance just as if you had paginated a traditional Eloquent query.
However, Scout returns an instance of Illuminate\Pagination\Paginator
where as Eloquent returns an instance of \Illuminate\Pagination\LengthAwarePaginator
. Before I spend too much effort on this, is there any reason why Scout returns a different type of paginator to Eloquent? This seems a bit like an issue since the documentation suggests that it's pagination should be the same as Eloquent's.
See here
https://laracasts.com/discuss/channels/laravel/laravel-scout-paginate-not-working
"Undefined variable: elements"
I have been trying to index different resources into a single index to simplify search.
By customizing the toSearchableArray()
method, I did find a way to change the ObjectID stored in Algolia, but the unsearchable()
method is broken now.
Is there a way to explicitly tell Scout to use a different ObjectID?
This is only happening when the searchable model's $with
array contains relations and a load()
is performed with additional relations on the search results. The relations in the $with
array are loaded twice.
Model Product
has
protected $with = ['sale'];
Then in the search controller
$products = Product::search($term)->paginate();
$products->load(['categories', 'image']);
The sale
relation query is run twice.
If I put everything into $with
or move everything into load()
then it only runs once.
Interestingly the two sale
query statements have the product_id
in completely different orders.
hi @taylorotwell, I've been using Elasticsearch over the past year with Laravel, and i'm halfway building it's engine for scout, just given a heads up in case you don't want Elastic support direct here. that case I'm gonna build it as a separate package based on scout instead.
Thanks
Hi there,
I like custom data before import/update into elasticsearch with laravel/scout
Ex: I only want import some column from table into elasticsearch
please help me!
When I run the search method on a model, I got the following error:
FatalErrorException in SearchableScope.php line 7:
Cannot use Illuminate\Database\Eloquent\Builder as Builder because the name is already in use
The SearchableScope trait wants to use Illuminate\Database\Eloquent\Builder but in the same namespace of SearchableScope is already a Builder class.
The only solution I found was aliassing the Illuminate\Database\Eloquent\Builder class as 'SearchBuilder' in SearchableScope.
After updating Scout to 1.1.5 from 1.1.4 ElasticSearch won't return any results with:
Model::search('argument')->get().
I am using the official Elasticsearch driver. After I rolled back to 1.1.4 it works.
Hello, how can i implement an advanced search?
Like
Search all books that have "javascript" in their title and the author contains "john".
I came up with something like :
\App\Model::search('javascript')->where('author','LIKE','%john%')->get()
but there were no results....
For example,I wanna to search the field “name” in the type "users" with the default Elasticsearch,what should I do?I found nothing about the Elasticsearch engine in the document of laravel-scout
Hey there,
I'm batch importing bulk data (30,000+ model entries) via cli and an elasticsearch driver and have set a custom primary key (a random char of 32 length) on the model. When I trigger the scout import via cli the console output shows the following:
Imported [App\Foo] models up to ID: 0
Imported [App\Foo] models up to ID: 183
Imported [App\Foo] models up to ID: 2542
Imported [App\Foo] models up to ID: 2
Imported [App\Foo] models up to ID: 3000000000
Imported [App\Foo] models up to ID: 499
Imported [App\Foo] models up to ID: 57
Imported [App\Foo] models up to ID: 6312
Imported [App\Foo] models up to ID: 6
Imported [App\Foo] models up to ID: 7
Imported [App\Foo] models up to ID: 87
Besides that I noticed that barely a fifth of all entries are really getting indexed. While sometimes there are passed 10,000+ models to the elasticsearch drivers handler, sometimes there are just a few in a collection. A lot of the models must therefore be queried multiple times for indexing, while others are never recognized. My current guess is that scout or another db service is trying to convert the primary keys to integers and therefore produces inconsistent collections of models to be indexed.
If there's anything I can do to help testing, please let me know.
Thanx!
hi
how i use that with slimframework and Eloquent ?
Hi.
There is a way to index Eloquent relations with Scout?
I saw that the Algolia laravel package is capable for this.
https://www.algolia.com/doc/api-client/laravel/relationships
Thanks for the answer. (awesome package btw 👍 )
If anyone approves me, I may send a PR on this. When I modify toSearchableArray
method in model classes, if I do not put id to returned array it returns me an exception: undefined index id at AlgoliaEngine.php line 134
Can I use query builder like ordering and grouping ?, I find in documentation but there is only where clauses and I try to use orderBy but instead an error Call to undefined method Laravel\Scout\Builder::orderBy()
.
Thank you!
As Algolia doesn't allow sort on query time, we need set up duplicates for different sorting strategy. So multiple indice with different setting required. Thanks.
Hello,
I'm trying to implement scout but paginate method not working properly, am i doing it wrong or it's a bug?
here's my implementations:
driver : tried tntsearch and now using algolia (same issue)
my model: only use Searchable trait (searchableAs() and toSearchableArray() not implemented)
indexing : using "php artisan scout:import "App\Models\Blog\Blog" and output:
Imported [App\Models\Blog\Blog] models up to ID: 100
All [App\Models\Blog\Blog] records have been imported.
if in my controller:
$posts = Blog::search($request->get('q'))
->paginate(10)->appends(["only" => "q"]);
dump($posts);
output blog/search?q=ipsum&page=2:
if i change controller to:
$posts = Blog::search($request->get('q'))
->get();
dump($posts);
thanks
When retrieving results with pagination, the following error occurs..
1/1ErrorException in Builder.php line 142:Undefined index: nbPages in Builder.php line 142 at HandleExceptions->handleError('8', 'Undefined index: nbPages', '/Users/michaelevans/workspace/jargonbuster/jb/vendor/laravel/scout/src/Builder.php', '142', array('perPage' => '10', 'pageName' => 'page', 'page' => '1', 'engine' => object(TNTSearchEngine), 'results' => object(Collection), 'rawResults' => array('ids' => array(), 'execution time' => '7.0291 ms'), 'this' => object(Builder), 'paginator' => object(Paginator))) in Builder.php line 142 at Builder->paginate('10') in PagesController.php line 36 at PagesController->search(object(Request)) at call_user_func_array(array(object(PagesController), 'search'), array(object(Request))) in Controller.php line 52 at Controller->callAction('search', array(object(Request))) in ControllerDispatcher.php line 44 at ControllerDispatcher->dispatch(object(Route), object(PagesController), 'search') in Route.php line 189 at Route->runController() in Route.php line 144 at Route->run(object(Request)) in Router.php line 642 at Router->Illuminate\Routing\{closure}(object(Request)) in Pipeline.php line 53 at Pipeline->Illuminate\Routing\{closure}(object(Request)) in SubstituteBindings.php line 41 at SubstituteBindings->handle(object(Request), object(Closure)) in Pipeline.php line 137 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in Pipeline.php line 33 at Pipeline->Illuminate\Routing\{closure}(object(Request)) in VerifyCsrfToken.php line 64 at VerifyCsrfToken->handle(object(Request), object(Closure)) in Pipeline.php line 137 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in Pipeline.php line 33 at Pipeline->Illuminate\Routing\{closure}(object(Request)) in ShareErrorsFromSession.php line 49 at ShareErrorsFromSession->handle(object(Request), object(Closure)) in Pipeline.php line 137 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in Pipeline.php line 33 at Pipeline->Illuminate\Routing\{closure}(object(Request)) in StartSession.php line 64 at StartSession->handle(object(Request),
Using postgresql, when I import some model, the order they are imported is totally messed up :
php artisan scout:import "App\Domain\Method"
Imported [App\Domain\Method] models up to ID: 101
Imported [App\Domain\Method] models up to ID: 234
Imported [App\Domain\Method] models up to ID: 369
Imported [App\Domain\Method] models up to ID: 500
Imported [App\Domain\Method] models up to ID: 630
Imported [App\Domain\Method] models up to ID: 761
Imported [App\Domain\Method] models up to ID: 895
Imported [App\Domain\Method] models up to ID: 1024
Imported [App\Domain\Method] models up to ID: 1155
Imported [App\Domain\Method] models up to ID: 1284
Imported [App\Domain\Method] models up to ID: 489
Imported [App\Domain\Method] models up to ID: 953
Imported [App\Domain\Method] models up to ID: 1
This results in some models not being imported.
To work this around I have to overwrite the makeAllSearchable function, which is not cool :
public static function makeAllSearchable()
{
$model = new static;
$model->newQuery()->orderBy($model->getKeyName(), 'ASC')->searchable();
}
In Builder::paginate
$results = Collection::make($engine->map(
$rawResults = $engine->paginate($this, $perPage, $page), $this->model
));
$results
is always the same count as $perPage
therefore
->hasMorePagesWhen(($results->count() / $perPage) > $page)
always returns false
.
Potentially why this is happening?
Edit:
$engine->paginate($this, $perPage, $page)
does return the total number of hits in the array. Perhaps we should use this to determine whether there's more?
Edit2:
This works fine:
->hasMorePagesWhen(($rawResults['nbHits'] / $perPage) > $page);
Since we have access to the total number of hits we could easily add a LengthAwarePaginator
which people seem to be wanting. Or we could expose the number of hits and other related stuff so we can build it ourselves.
I was wondering if people would be interested in possibly a trait to say setup this entry to be able to be geo-searched on. As Algolia and Elasticsearch are separate in the way they do this, so it would require some setup in terms of how to get things setup for the array. If there is interest I can create a PR for this functionality.
I want to implement global search in Scout, where I'm able to search in ALL models and get results ranked by relevancy, which Sphinx allows you to do, but I'm facing a few problems. One such problem is I don't know where to add the global search method. I tried creating a class that extends the engine that I'm using and overriding the search method, but when I try to search it complains that a Builder object was not given but instead a string was given, altho I type-hinted the Builder dependency.
Does anyone have an idea what the best approach is to implementing this feature?
$user = factory(App\User::class)->create();
$user->profile()->update([
'display_name' => 'findme',
'bio' => 'user bio',
'type' => 'band'
]);
Not Shown: Profile is created by the User Observer upon user creation
I have tried this in a test ran by phpunit and in php artisan tinker
Database updates as expected.
Algolia entry is created for profile but not updated
In the paginate
function of Laravel\Scout\Builder
, the current search query is added to the paginator's URLs with the hardcoded key of 'query'
:
return $paginator->appends('query', $this->query)
It would be great to be able to customize this key. For example, this would allow paginating search results with ...?q=terms
or ...?search=terms
.
One way to do this is by adding a fluent withQueryKey
method to set a $query_key
property on the Builder
, allowing code such as
$search_results = $model->search($search_query)
->withQueryKey('q')
->paginate(15);
I can create a PR implementing that, if there is interest.
When trying to perform a batch import (e.g. php artisan scout:import "App\Post"
) the following error is thrown
Symfony\Component\Debug\Exception\FatalThrowableError: Type error: Argument 1 passed to Laravel\Scout\SearchableScope::Laravel\Scout{closure}() must be an instance of Laravel\Scout\Builder, instance of Illuminate\Database\Eloquent\Builder given in /home/vagrant/samarkand-web/vendor/laravel/scout/src/SearchableScope.php:32
I'm running unit tests on my code and I've noticed that setting the following to SCOUT_DRIVER=null
doesn't work. I get this error message in my log file 'ErrorException' with message 'Missing argument 1 for Illuminate\Support\Manager::createDriver()
. Setting it to SCOUT_DRIVER='"null"'
works however.
The SCOUT_PREFIX
property also doesn't seem to apply (when I'm not testing code and using the algolia
driver). I've resorted to this method as a workaround:
public function searchableAs()
{
return env('SCOUT_PREFIX', '') . 'mytable';
}
Anyone know what's going on? If it's a bug with the package or my code?
please help. how to solve this error?
another error
{
"error": {
"root_cause": [{
"type": "parsing_exception",
"reason": "no [query] registered for [filtered]",
"line": 1,
"col": 22
}],
"type": "parsing_exception",
"reason": "no [query] registered for [filtered]",
"line": 1,
"col": 22
},
"status": 400
}
It seems scout:flush
doesn't seem to do anything. Deleting a record from the database manually then doing scout:flush
and/or scout:import
doesn't remove the old record.
after updating to 1.1.6 immediately (at php artisan optimize) get this error:
PHP Parse error: syntax error, unexpected '::' (T_PAAMAYIM_NEKUDOTAYIM) in /vendor/laravel/scout/src/Console/FlushCommand.php on line 32
PHP 5.6
Can anyone please tell me how to implement something like this
I have 2 tables
Movies (hasManyActors)
Actors (belongsToMovies)
now i want to search movies
by actor name. Before scout I was doing something like this.
Movies::whereHas('actors', function ($q, $searchterm) {
$q->where('name', 'like', $searchterm);
})->with('actors')->get();
Symfony\Component\Debug\Exception\FatalErrorException · vendor/laravel/scout/src/Console/FlushCommand.php:32
syntax error, unexpected '::' (T_PAAMAYIM_NEKUDOTAYIM)
Stage
local
Severity
error
Stacktrace summary
vendor/laravel/scout/src/Console/FlushCommand.php:32 · [main]
Currently Scout allows for getting a collection of eloquent models as search results. However, what if I want the raw results from the search engine? How would I retrieve those?
I got indexed the model with yours relationships, the elasticsearch show me all data (relationship included) but on search by model, I got only data from main body, I dont got the data from relationship indexed too.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.