[sword-devel] StripText

Daniel Glassey sword-devel@crosswire.org
Fri, 15 Feb 2002 01:42:01 +0000

Troy or anyone else. Is it really absolutely necessary to apply filters in 
StripText. Is there anything that is removed by the filters that really isn't 
wanted in the search? The search is speeded up by about 33% on a normal 
optimised build just by not applying the optionfilters and stripfilters.
The operation of applying a filter is much more expensive than searching the 
extra markup.

If they are still needed, one option could be to do a full unfiltered search 
then a filtered search on the results. 
Or alternatively, during the Search function do a search on unfiltered text, 
then repeat the search on filtered text if it matches.

I haven't tried this so I don't know how badly this scales.
Naively I would assume that since about 5% of the search is in the Search 
function, ~33% in filter ~33% in operator char* and ~33% in operator ++ at 
the worst case scenario the search will be 5% slower if it going to return 
all verses, very roughly 15% faster if returning 50% verses.

If anyone else could try this out and report on it that would be great, 
otherwise I'll try and get time tomorrow sometime to try it and see how times 
go. Unless of course the filters can be ignored.


Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com