GithubHelp home page GithubHelp logo

floriansemm / solrbundle Goto Github PK

View Code? Open in Web Editor NEW
123.0 9.0 72.0 1.29 MB

Solr-Integration into Symfony and Doctrine2

Home Page: http://floriansemm.github.io/SolrBundle

License: MIT License

PHP 97.05% HTML 2.95%

solrbundle's Introduction

SolrBundle

Build Status Latest Stable Version Total Downloads

Introduction

This Bundle provides a simple API to index and query a Solr Index.

Installation

Installation is a quick (I promise!) 3 step process:

  1. Download SolrBundle
  2. Enable the Bundle
  3. Configure the SolrBundle
  4. configure your entity

Step 1: Download SolrBundle

This bundle is available on Packagist. You can install it using Composer:

$ composer require floriansemm/solr-bundle

Step 2: Enable the bundle

Next, enable the bundle in the kernel:

<?php
// app/AppKernel.php

public function registerBundles()
{
    $bundles = array(
        // ...
        new FS\SolrBundle\FSSolrBundle(),
    );
}

Step 3: Configure the SolrBundle

Finally, configure the bundle:

# app/config/config.yml
fs_solr:
    endpoints:
        core0:
            schema: http
            host: host
            port: 8983
            path: /solr/core0
            core: corename
            timeout: 5

Default values will be used for any option left out.

With DSN

# app/config/config.yml
fs_solr:
    endpoints:
        core0:
            dsn: http://host:8983/solr
            core: core0
            timeout: 5

Any values in schema, host, port and path option, will be ignored if you use the dsn option.

Step 4: Configure your entities

To make an entity indexed, you must add some annotations to your entity. Basic configuration requires two annotations: @Solr\Document(), @Solr\Id(). To index data add @Solr\Field() to your properties.

If you want to index documents without any database, then you have to use the same annotations. Make sure you have set a Id or set @Solr\Id(generateId=true).

// ....
use FS\SolrBundle\Doctrine\Annotation as Solr;
    
/**
* @Solr\Document()
* @ORM\Table()
*/
class Post
{
    /**
     * @Solr\Id
     *
     * @ORM\Column(name="id", type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    private $id;
    
    /**
     * @Solr\Field(type="string")
     *
     * @ORM\Column(name="title", type="string", length=255)
     */
    private $title = '';

    /**
     * @Solr\Field(type="string")
     *
     * @ORM\Column(name="text", type="text")
     */
    private $text = '';

   /**
    * @Solr\Field(type="date", getter="format('Y-m-d\TH:i:s.z\Z')")
    *
    * @ORM\Column(name="created_at", type="datetime")
    */
    private $created_at = null;
}

The bundle handles now updates / inserts / deletions for your configured entity.

Annotation reference

@Solr\Document annotation

This annotation denotes that an entity should be indexed as a document. It has several optional properties:

  • repository
  • index
  • indexHandler

Setting custom repository class with repository option

If you specify your own repository, the repository must extend the FS\SolrBundle\Repository\Repository class.

/**
 * @Solr\Document(repository="My/Custom/Repository")
 */
class SomeEntity
{
    // ...
}

index property

It is possible to specify a core the document will be indexed in:

/**
 * @Solr\Document(index="core0")
 */
class SomeEntity
{
    // ...
}

indexHandler property

By default, all documents will be indexed in the core core0. If your entities/documents have different languages, then you can setup a callback method, which should return the core the entity will be indexed in.

/**
 * @Solr\Document(indexHandler="indexHandler")
 */
class SomeEntity
{
    public function indexHandler()
    {
        if ($this->language == 'en') {
            return 'core0';
        }
    }
}

Each core must be set up in config.yml under endpoints. If you leave the index or indexHandler property empty, then the default core will be used (first one in the endpoints list). To index a document in all cores, use * as index value.

@Solr\Id annotation

This annotation is required to index an entity. The annotation has no properties. You should add this annotation to the field that will be used as the primary identifier for the entity/document.

class Post
{
    /**
     * @Solr\Id
     *
     * @ORM\Column(name="id", type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */

    private $id;
}

generateId option

Set this option to true and a the bundle will generate a Id for you. Use this option if you have no underlying DB which generates incremental Ids for you.

@Solr\Field annotation

This annotation should be added to properties that should be indexed. You should specify the type option for the annotation.

type property

Currently, a basic set of types is implemented:

  • string(s)
  • text(s)
  • date(s)
  • integer(s)
  • float(s)
  • double(s)
  • long(s)
  • boolean(s)

If you have a customized schema.xml than you don't need to setup a field-type.

fieldModifier property

Solr supports partial updates of fields in an existing document. Supported values are:

  • set
  • add (multivalue field only, adds a value(s) to a existing list)
  • remove (multivalue field only, removes a value(s) from existing list)
  • inc (integer field only)

nestedClass property

Set this property if you want to index collections with nested Objects.

Object relations

For more information read the more detailed "How to index relation" guide

@Solr\SynchronizationFilter(callback="shouldBeIndexed") annotation

In some cases, an entity should not be indexed. For this, you have the SynchronizationFilter annotation to run a filter-callback.

/**
 * // ....
 * @Solr\SynchronizationFilter(callback="shouldBeIndexed")
 */
class SomeEntity
{
    /**
     * @return boolean
    */
    public function shouldBeIndexed()
    {
        // put your logic here
    }
}

The callback property specifies an callable function, which should return a boolean value, specifying whether a concrete entity should be indexed.

Queries

Query a field of a document

Querying the index is done via the solr.client service:

$query = $this->get('solr.client')->createQuery('AcmeDemoBundle:Post');
$query->addSearchTerm('title', 'my title');
$query->addSearchTerm('collection_field', array('value1', 'value2'));

$result = $query->getResult();

or

$posts = $this->get('solr.client')->getRepository('AcmeDemoBundle:Post')->findOneBy(array(
    'title' => 'my title',
    'collection_field' => array('value1', 'value2')
));

Query all fields of a document

The previous examples were only querying the title field. You can also query all fields with a string.

$query = $this->get('solr.client')->createQuery('AcmeDemoBundle:Post');
$query->queryAllFields('my title');

$result = $query->getResult();

Define a custom query string

If you need more flexiblity in your queries you can define your own query strings:

$query = $this->get('solr.client')->createQuery('AcmeDemoBundle:Post');
$query->setCustomQuery('id:post_* AND (author_s:Name1 OR author_s:Name2)');

$result = $query->getResult();

The QueryBuilder

The query-builder based on https://github.com/minimalcode-org/search Criteria API.

$queryBuilder = $this->get('solr.client')->getQueryBuilder('AcmeDemoBundle:Post');
$result = $queryBuilder
    ->where('author')
        ->is('Name1')
    ->orWhere('author')
        ->is('Name2')
    ->getQuery()
    ->getResult();

To keep your code clean you should move the select-criteria in a repository-class:

class YourRepository extends Repository
{
    public function findAuthor($name1, $name2)
    {
        return $this->getQueryBuilder()
            ->where('author')
                ->is($name1)
            ->orWhere('author')
                ->is($name2)
            ->getQuery()
            ->getResult();
    }
}

Configure HydrationModes

HydrationMode tells the bundle how to create an entity from a document.

  1. FS\SolrBundle\Doctrine\Hydration\HydrationModes::HYDRATE_INDEX - use only the data from solr
  2. FS\SolrBundle\Doctrine\Hydration\HydrationModes::HYDRATE_DOCTRINE - merge the data from solr with the entire doctrine-entity

With a custom query:

$query = $this->get('solr.client')->createQuery('AcmeDemoBundle:Post');
$query->setHydrationMode($mode)

With a custom document-repository you have to set the property $hydrationMode itself:

public function find($id)
{
    $this->hydrationMode = HydrationModes::HYDRATE_INDEX;
    
    return parent::find($id);
}

Repositories

Your should define your own repository-class to make your custom queries reuseable. How to configure a repository for a document have a look at the annotation section

namespace AppBundle\Search;

use FS\SolrBundle\Repository\Repository;

class ProviderRepository extends Repository
{
    public function findPost($what)
    {
        $query = $this->solr->createQuery('AcmeDemoBundle:Post');
        // some query-magic here

        return $query->getResult();
    }
}

In your repository you have full access to the querybuilder.

Commands

Here's all the commands provided by this bundle:

  • solr:index:clear - delete all documents in the index
  • solr:index:populate - synchronize the db with the index
  • solr:schema:show - shows your configured documents

Indexing huge sets of entities

The solr:index:populate command works well for sets up to 300k entities, everthing large makes the command very slow. You can find here some solution how to sync your DB with Solr.

Extend Solarium

To extend Solarium with your own plugins, create a tagged service:

<tag name="solarium.client.plugin" plugin-name="yourPluginName"/>

To hook into the Solarium events create a common Symfony event-listener:

<tag name="kernel.event_listener" event="solarium.core.preExecuteRequest" method="preExecuteRequest" />

Document helper

Retrieve the last insert entity-id

$helper = $this->get('solr.client')->getDocumentHelper();
$id = $helper->getLastInsertDocumentId();

solrbundle's People

Contributors

alcaeus avatar arcanis avatar bb-fuf avatar chellem avatar dcaillibaud avatar floriansemm avatar frne avatar gigo6000 avatar gitter-badger avatar intel352 avatar jakesylvestre avatar jkwakman avatar josephwoodward avatar kix avatar marfillaster avatar nathankot avatar patrikorup avatar royopa avatar zquintana avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

solrbundle's Issues

Dependency to the whole zendframework?

The bundle has a composer dependency to zendframework/zendframework which comes from doctrine/doctrine-module. In which case is this needed? Do i have to install the whole Zend Framework into my Symfony 2 app via Composer?

php composer.phar update
Loading composer repositories with package information
Updating dependencies
  - Installing doctrine/mongodb (dev-master ec10f4d)
    Cloning ec10f4d32500efe8b4115a78974e7cc089294ddf

  - Installing doctrine/mongodb-odm (dev-sf2.0 942bfd6)
    Cloning 942bfd6f54dc83e4d692bd5651ff9ed395191609

  - Installing zendframework/zendframework (dev-develop d883690)
    Cloning d8836909f3772160ca8830ebeaa52de25f710899

    Skipped installation of bin/classmap_generator.php for package zendframework/zendframework: name conflicts with an existing file
  - Installing doctrine/migrations (v1.0-ALPHA1)
    Loading from cache

  - Installing doctrine/doctrine-module (dev-master 5224fac)
    Cloning 5224fac4f11cbdfbf59a59b36727d7f4b3458211

  - Installing doctrine/doctrine-orm-module (0.5.6)
    Downloading: 100%         

  - Installing floriansemm/solr-bundle (0.6)
    Downloading: 100%         

THX for your feedback.
Frank

Fatal error: Class 'SolrClient' not found in SolrBundle/SolrConnection.php on line 22

I have an error like in the topic, I have this problem and I don't know how to solve it. I can't find class SolrClient in SolrBundle. I found SolrClientFake class but it is used just for tests. I'm using Symfony 2,1, but SolrBundle is not managed by composer. I have installed it like in Symfony 2.0.

Any help will be appreciated. If you need more info, I'll provide it as soon as possible.

no declaration for document found in entity

I'm getting this error:

no declaration for document found in entity

throw from MetaInformationFactory:

    if (!$this->annotationReader->hasDocumentDeclaration($entity)) {
        throw new \RuntimeException(sprintf('no declaration for document found in entity %s', $className));
    }

when persisting an entity that I do not have a @solr\Document annotation for. Is this expected? How do I exempt entities from being in Solr?

How to use Solr without annotation?

I am not using annotation for my bundle. So, how can I use Solr for my entity class? I mean, is there any way to define the same thing in entity's orm.xml file?

Improve Console-Output of `SynchronizeIndexCommand`

actual output after synchronisation:

Synchronization successful
Synchronized Documents: 58
Not Synchronized Documents: 0

should something like this:

Synchronization successful
Synchronized Documents: 58/58
Not Synchronized Documents:
    Entity\Post
    Entity\Author

[Feature request] deferred commit

Could you add deferred commit task.

For specific use, this feature is very powerful to accelerate processing.

When we use Doctrine Fixture or import from files.

ThanX

Entities are not managed by Doctrine

The main problem is that the fields which are not indexed will not be populated (including non-indexed relations). Of course, it is possible to get 'real' entities after, using their ID, but maybe it would be nice to have a way to do this automagically.

Ping solr server before indexing

there should exist a check and see if the server is online. Some obscure error came back from the solr-server, when a connection cannot be established.

Class 'SolrClient' not found

I've installed everything including this sudo pecl install -n solr-beta

but still when i run the search i get this error Fatal error: Class 'SolrClient' not found in /var/www/solrsearch/vendor/floriansemm/solr-bundle/FS/SolrBundle/SolrConnection.php on line 22

can some one help me plz !!

can u give me a tutorial how to link the solr to symfony2 project cause i'm knowing that there is some thing wrong with my work

thanks

Installation instruction

Just a suggestion, in case it could be added to the README

I'n not so aware of autoloading logic of symfony, because all my others used bundles are declared in the main composer.json, and I spend some time to understand how to install the bundle and its autoload declaration.

With symfony 2.1.4, in app/autoload.php, a working syntax is

// already here
AnnotationRegistry::registerLoader(array($loader, 'loadClass'));

// then, to add SolrBundle
// cd vendor
// mkdir -p floriansemm/solr-bundle/FS
// cd floriansemm/solr-bundle/FS
// git clone git://github.com/floriansemm/SolrBundle.git
$loader->add('FS\\SolrBundle', array(__DIR__.'/../vendor/floriansemm/solr-bundle'));

Not compatible with Symfony 2.3?

Current composer requirement: "symfony/framework-bundle": ">=2.1.0,<=2.2.x-dev"

Wondering if this can updated to work with Symfony 2.3?

Move metainformations to MetainformationManager

Informations like 'classname', 'document-identifier', 'document-name' should managed by a concrete class, which holds this informations.

The manager generates from an entity this informations.

see Doctrine\ORM\Mapping\ClassMetadataFactory and Doctrine\ORM\Mapping\ClassMetadataInfo

repo name breaks naming convention

You'll want to change this repo name to FsSolrBundle instead of SolrBundle. You're taking up the top level namespace which could cause conflicts when forking.

My Custom doctrine.event_subscriber postLoad is not called

I have a custom postLoad event

<service id="my.listener" class="%my.listener.class%">
            <tag name="doctrine.event_subscriber" />
</service>
class MyListener implements EventSubscriber {
    public function postLoad(LifecycleEventArgs $args) { /*...*/}
    public function getSubscribedEvents()
    {
        return [      
            'postLoad'
        ];
    }
}

Before use SolrBundle when i load my entities the previous is correctly called but now is never called. how can solve this?

Compatibility with Solr 4.x

Is this SolrBundle compatible with useage of solr 4.4?

I try it, but I get some errors on solr side. It shows, that this bundle is build for solr 3.x.

Unknown attribute id in add:allowDups
org.apache.solr.common.SolrException: Unknown commit parameter 'waitFlush'

Is there any way to use this bundle with new solr 4.4?

best regards

entityMapper pb : toEntity return the same entity

You need to clone the source Entity in file FS/SolrBundle/Doctrine/Mapper/EntityMapper.php

...
public function toEntity(\ArrayAccess $document, $sourceTargetEntity) {
if (null === $sourceTargetEntity) {
throw new \InvalidArgumentException('$targetEntity should not be null');
}
$targetEntity = clone $sourceTargetEntity;
...

suffix remains in query

Hi,

I guess this is not a really issue, probably a misunderstood usage of SolrBundle, but I don't see how to solve this pb.

I have an Entity

namespace Sesamath\BibliBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use FS\SolrBundle\Doctrine\Annotation as Solr;

/**
 * @Solr\Document(repository="Sesamath\BibliBundle\Entity\SolrRessourceRepository")
 * @ORM\Table()
 */
class SolrRessource
{
    /**
     * @Solr\Id
     * @ORM\Column(name="id", type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    private $id;

    /**
     * @Solr\Field(type="text")
     * @ORM\Column(name="titre", type="text")
     */
    private $titre;

and (for now) an empty repository

namespace Sesamath\BibliBundle\Entity;

class SolrRessourceRepository extends \FS\SolrBundle\Repository\Repository
{
}

And in a controller I do

    public function solrListAction($by, $search)
    {
        /* @var $solr \FS\SolrBundle\SolrFacade */
        $solr = $this->get('solr')->core('testindex');
        /* @var $query \FS\SolrBundle\Query\SolrQuery */
        $query = $solr->createQuery('SesamathBibliBundle:SolrRessource');
        $query->addSearchTerm($by, $search);
        $entities = $query->getResult();
        $debug = $query->getQueryString();
        return array('entities' => $entities, 'debug' => $debug);
    } // solrListAction

but with $by = 'titre'; $search = 'foo'; I have queryString : titre_t:*foo* so Solr response is org.apache.solr.common.SolrException: undefined field titre_t

What I'm doing wrong ?

Thanks a lot

Add more events

Add events for all actions (add, update, delete, clearindex)

Loading fixtures does not work

I have Symfony 2.1.x configured with Doctrine2

My solr configuration

fs_solr:
    solr:
        hostname: localhost
        port: 8983
        path:
            core0: /admin/solr-test
    auto_index: true
    entity_manager: default

When I'm trying to load my fixtures there is error:

Catchable Fatal Error: Argument 1 passed to FS\SolrBundle\Doctrine\MongoDB\Listener\AddDocumentListener::postPersist() 
must be an instance of Doctrine\ODM\MongoDB\Event\LifecycleEventArgs, 
instance of Doctrine\ORM\Event\LifecycleEventArgs given, called in /../vendor/symfony/symfony/src/Symfony/Bridge/Doctrine/ContainerAwareEventManager.php on line 61 
and defined in /.../vendor/floriansemm/solr-bundle/FS/SolrBundle/Doctrine/MongoDB/Listener/AddDocumentListener.php line 26 

I think when my ORM is configured to use Doctrine2 then bundles should only add listeners for Doctrine2 not Doctrine Mongo

Rewrite documentation

Move documentation to Resources/doc

Chapters:

  • configuration
    • multiple indexies
  • usage
    • entity annotations (field types, mongoDB)
    • searching
    • filter results
    • commands

[Feature Request] Relations

Can You answer this question asked by Christian here http://knpbundles.com/floriansemm/SolrBundle?

"I'd like to know if it works with relationships between entities. For example, if your Post entity had an $author property of the class Author and I wish to index some of the author's information along with the post's. Is it possible?

An example with a many to many relationship would be if the Posts had a $tags property consisting of an array of Tag entities, is it possible to map this with annotations?"

thx

Integration with Symfony2.1

Hi, I actually want to integrate the SolrBundle with my Symfony2.1 project but I did not find a composer version for it. Do I have any other alternative here. Please let me know.

configuration mongodb not working

Here is my configuration:
doctrine_mongodb:
connections:
default:
server: mongodb://%database_host%:%database_port%
options: {}
default_database: %database_name%
document_managers:
default:
auto_mapping: true

I get this error:
The service "solr" has a dependency on a non-existent service "doctrine.orm.default_configuration".

thank you for an answer from you.

multiple mongo_db configurations produce exception

exception:

ServiceNotFoundException: The service "solr" has a dependency on a non-existent service "doctrine_mongodb.odm.default_configuration".

the extension must do some service-magic: it must read all configured mongodb-conntections. see DoctrineMongoDBExtension

PHP Fatal error: Call to undefined method Doctrine\Common\Annotations\AnnotationReader::setIgnoreNotImportedAnnotations() in .../Symfony/vendor/bundles/FS/SolrBundle/Doctrine/Annotation/AnnotationReader.php on line 20

Hello,

I use Symfony 2.0 and Doctrine 2.2.1. I have this error message

If i comment the line, i can't use doctrine without put Solr annotation on all my entities.
...
public function __construct() {
$this->reader = new Reader();
//$this->reader->setIgnoreNotImportedAnnotations(true);
}
...

AnnotationReader trying to access parent class of entity.

Trying to perform a query or even run the app/console solr:synchronize command gives me this error:

 Call to a member function getProperties() on a non-object in [...]/SolrBundle/Doctrine/Annotation/AnnotationReader.php on line 33

My entity has no parent class, and the example in the documentation doesn't either.

What am I doing wrong?

Schema is not updated

This is the first time I use Solr and bundle, follow the guide and everything seems properly but when creating an entity, I get an error

org.apache.solr.common.SolrException: ERROR: [doc=1] unknown field 'document_name_s'

I understand Solr needs a schema that can be updated dynamically. and the bundle does for me when going to store Entity into the index but this is not happening.

My entity class

/** 
 * @ORM\Entity
 * @ORM\Table()
 * @Solr\Document 
 */
class Entity {
    /** orm suff...
     *  @Solr\Id
     *  @ORM\Id
     *  @ORM\GeneratedValue(strategy="AUTO")
     */
    private $id

    /**
     * @ORM\Column(name="product_name", type="string")
     * @Solr\Field(type="string")
     */
    private $productName;

    //...
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.