[sword-devel] lucene indexing failing on some modules
dmsmith555 at yahoo.com
Fri May 9 13:12:13 MST 2008
Peter von Kaehne wrote:
> As an aside,
> lucene indeces are stored by sword in the module file, but by jsword in
> a separate place inside .jsword
> Is there a reason behind this?
JSword used Lucene first. Our search engine had lots to be desired. We
had to put it somewhere. Joe decided on this location. Later Sword added
cLucene and a different location.
Sometimes Sword modules are installed in /usr/share/sword (or similar)
with permissions that don't permit a user to write to the folder. This
allows indexing of all modules.
We were planning to have the indexes be something that a user could
download (For a whole myriad of reasons, this is not a good idea) and
figured that these would be stored separately from the modules in an
area that JSword managed.
Lucene is many releases ahead of clucene and JSword uses the full
capabilities of the most recent Lucene. (From the early 1.4.x days to
now with 2.3.2, indexing time has reduced from 40+ min to less than 1.)
As a result, JSword's lucene indexes are incompatible with Sword's. We
don't have our versioning system worked out, but we need a way to
indicate that an index was generated by a particular level of Lucene (or
cLucene) and what kind of analyzer was used and potentially what kind of
options the user has chosen. (JSword does not currently give the user a
choice as I thought it would be too confusing.) Putting our indexes in
the module's directory might confuse Sword applications.
More information about the sword-devel