search index & accent insensitive search about django-watson

[etianen]

Hi,

I'm struggling a bit with your index notation. What database backend are you using?

On 28 Jul 2013, at 10:25, Michal Novotný [email protected] wrote:

There is an interesting difference in produced indexes in a situation where I would expect them to be the same.

Unicode function of my class Crag returns attribute name.

def unicode(self):
return self.name
watson.register(Crag) produces index 'ceus':3 'céüse':1A,2

Now If I register Crag just with the field name I would expect the index to be the same. but it is not.

watson.register(Crag, ('name',)) produces index 'céüse':1A,2

I'd like always the first variant to apply because it provides provides case insensitive search.

—
Reply to this email directly or view it on GitHub.

[clime]

It is postgresql.

[etianen]

Hmm. That's really odd.

So, in your case, each watson search entry has three indexed fields: "title", "description" and "content. These are indexed with the following weightings:

title (defaults to unicode(obj)) - A
description (defaults to "") - B
content (defaults to " ".join(fields)) - C

If you register that model with no extra fields, then the index should contain:

"céüse" - A
"" - B
"" - C

If you register that model with a name field, then the index should contain:

"céüse" - A
"" - B
"céüse" - C

So I really have no idea how you're getting the index results you are. Raw postgres index notations don't mean a lot to me, I'm afraid.

On 30 Jul 2013, at 11:41, Michal Novotný [email protected] wrote:

It is postgresql.

—
Reply to this email directly or view it on GitHub.

[clime]

I am sorry for the confusion. To explain:

There is also field normalized_name on my Crag model which contains unaccented lower-cased name. That is where 'ceus' is coming from in 'ceus':3 'céüse':1A,2. For some reason there is "e" being cut, not sure why but that is probably not important. These 1,2,3 numbers are probably order and A is a weight I guess. So it makes sense. I didn't realize that watson.register(Crag) includes also the normalized_name column on its own.

So I guess the way to make search accent-insensitive is to register these columns with already unaccented text. Fair enough.

[etianen]

Aha, yes! django-watson's default behaviour is to index all text and char fields.

Mystery solved. :D

On 1 Aug 2013, at 22:41, Michal Novotný [email protected] wrote:

I am sorry for the confusion. To explain:

There is also field normalized_name on my Crag model which contains unaccented lower-cased name. That is where 'ceus' is coming from in 'ceus':3 'céüse':1A,2. For some reason there is "e" being cut, not sure why but that is probably not important. These 1,2,3 numbers are probably order and A is a weight I guess. So it make sense. I didn't realize that watson.register(Crag) includes also the normalized_name column on its own.

So I guess the way to make search accent-insensitive is to register these columns with already unaccented text. Fair enough.

—
Reply to this email directly or view it on GitHub.