[bt-devel] Fwd: Re: clucene crash when searching

[Eeli Kaikkonen]

Joachim Ansorg wrote:
> I don't remember exactly why we switched but here's what I think, in random 
> order :)
>
>   
Thanks! Now we can move ahead in this discussion...

> We always had troubles with the Sword search engines. I don't know how the 
> current implementation is, though.
>
>   
The Sword development has been quite good recently and I think today we 
all want to commit ourselves to the goal of making the library better. I 
understand very well why you wanted more independence but I would rather 
work together for common goals if possible. If we have problems, we have 
to work with the core library developers. We know and they know that 
it's not always been the quick and easy way, but it's better nowadays. 
Now when even GnomeSword is reaching for Windows we have double the 
reason to make the common library rock solid. With that we (all 
frontends) can create the best software for all platforms.

> -It's  slow. Really slow. Using modules with many tags (like the KJV) the 
> search has taken a long time to finish. Especially on non high-end computers 
> it takes a long time. More than 5  seconds is not an acceptable response time. 
> Often it had been worse, I think. And that's just for seaches in one module at 
> a time.
>   
That's true, though machines are getting better. But this is not 
necessarily a problem for two reasons:
1) We have the quick clucene search. Most users could (and probably 
would) use that.
2) I have already implemented threaded behaviour (in installer) and I 
think it can be done with the search, too. That way the GUI would not be 
blocked by the slow search.

> -No complex searches are possible. and/or/grouped combinations are not easily 
> done with the sword engine.
>   
Again, we now already have the lucene search which can handle that.

> -We had no control where the index files are stored (using the clucene 
> implementation of Sword).
> -clucene was sometimes available in the sword lib, sometimes not. That 
> depended on how the user configured and installed his libsword.
>   

I'm not for dropping our own clucene support away. It's quite reasonable 
to have our own implementation. I wouldn't support the Sword clucene 
implementation at all, but (some of) the other engines. We wouldn't 
loose anything but gain everything.

> -It's a mess to work with different types of search engines which all have 
> different feature sets.
>
>   
What kind of work that means? Coding? Or end user experience?

For coding, I would keep the search engine types separate from each 
other. That would make the amount of code larger but the implementation 
would be clean and easy to work with. Each type could be worked with 
separately.

For end users, I wouldn't add complexity to the current interface. We 
have very simple solution available: add tabs to the search dialog. That 
way there would be only tab widget and tab texts added to the dialog. 
Everything else would be hidden until the user wanted to switch the 
engine. After switching the tab the new interface would also be as 
simple as possible for that search type.

I would probably take the Phrase and Regexp search engines. The Words 
engine we have already. But clucene is not good for fixed phrases (all 
words in certain order). Also it doesn't support complex wildcards. Even 
the Regexp engine alone would make everything possible for those who 
really need something else than lucene.

> The problem is that CLucene is almost unmaintained and crashes on certain 
> kinds of systems (got reports about crashes on BSD for example). Java Lucene 
> is much, much better. We've hoped that CLucene developes like JLucene, but 
> sadly that didn't work that way...
>   

Ok. That's one more reason to support many engines.


--Eeli Kaikkonen