[bt-devel] Fwd: Re: clucene crash when searching

[Martin Gruner]

Hi Eeli.

Unfortunately I do not agree with you, as you may expect.

Regarding Sword, there is no work going on for the search engine(s), and the 
cluene implementation of Sword has (at least) the same problems our 
implementation has. I second your attitude of collaboration in Sword, but am 
still pessimistic about the speed of its development.

I do not see that we would gain much by adding support of Sword's non-indexed 
search engines, except for the ability to search for phrases.

Searching in BT should be simple and consistent. That means that we should 
not, in my opinion, offer different search syntaxes to the user. Maybe one 
exception: a regexp-based search for power users, but all "normal" users 
should have one single search to work with (from a user's point of view).

Using a search engine that works with modern, index-based technologies is the 
proper way to go.

> > The problem is that CLucene is almost unmaintained and crashes on certain
> > kinds of systems (got reports about crashes on BSD for example). Java
> > Lucene is much, much better. We've hoped that CLucene developes like
> > JLucene, but sadly that didn't work that way...
>
> Ok. That's one more reason to support many engines.

No. It is one reason to choose the search engine wisely.

My suggestion would be to talk about the search engine we do use, clucene. I 
just checked - they released a bugfix 0.9.21 version recently, and 0.9.23, 
which is a beta-quality preview release of their next development branch, 
which is supposed to improve Lucene compatibility/feature coverage. Ben also 
told me that he was going to implement the wildcard operator in the beginning 
of words (like "*minded").
But nobody can say how long this will take. So we may want to use another open 
source search engine which suits our needs better.

We could start a wiki page listing the specific problems that we see with 
clucene, and investigate if they can be solved. At the same time we can 
collect information about other search engines in a matrix of 
features/properties that we do need. Maybe we come up with something better, 
more stable and feature-rich than clucene?

A major problem that I see: What about our release roadmap? We should not 
start changing the search engine in the 1.7.x branch/release cycle. I'm 
unhappy with the status quo, we cannot stay in beta state for a long time and 
continue changing the internals of our software. We should release FIRST, and 
THEN start making major changes.

What do you think, Eeli, and all others?

God bless,

mg