[sword-devel] Infinite loop bug in BosworthToller

Thu Jan 14 13:24:07 MST 2016

I can take care of imp2ld and tei2mod. The latter needs the same changes. I need the strongsPad method to be made visible, but I don’t have RW to that part of the repo.

> On Jan 14, 2016, at 3:21 PM, Костя Маслюк <kostyamaslyuk at gmail.com> wrote:
> 
> The patches attached there are merged upstream (https://github.com/greg-hellings/sword/commit/227b71aa5aa3931dc0f8ce998ebc07ea1f55419a <https://github.com/greg-hellings/sword/commit/227b71aa5aa3931dc0f8ce998ebc07ea1f55419a>) so bug is fixed on the library side. Counterpart we have some modules in our repositories that we have to rebuild with latest tools to update *.idx files. Until then our software (except Xiphos that have workaround) will freeze trying to load those modules. It is known and i do not know who is responsible for this work, i cant prove it any further.
> 
> 
> 
> 2016-01-14 22:43 GMT+04:00 DM Smith <dmsmith at crosswire.org <mailto:dmsmith at crosswire.org>>:
> Funny that you mentioned MOD-61. I just attached a patch to it that fixes the bugs that I found. Yes the problem is that strongsPadding is true for all dictionaries.
> 
> The first bug was that any module that has Strong’s Numbers that has keys that are not padded has to have StrongsPadding=false. Strong’s Numbers are indicated by the conf entry Feature=GreekDef and/or Feature=HebrewDef. Adding this to AbbottSmith made it work.
> 
> The second bug is that any module that doesn’t have Strong’s Numbers (i.e. doesn’t have Feature=GreekDef and/or Feature=HebrewDef) was having its numerical keys (keys that have a leading number or are a number) padded with leading 0. So a key like 3DogNight was padding to 00003DogNight. Adding StrongsPadding=false to such module (like BosworthToller) would prevent the problem.
> 
> The solution was to modify the construction of an lexdict (RawLD, RawLD4 and zLD) to make the default of StrongsPadding to depend on whether the module’s conf has Feature=GreekDef and/or Feature=HebrewDef or not. When it is present, it defaults to true. When it isn’t present the value of StrongsPadding is ignored and set to false.
> 
> I think your other patches, that report the consistency of the keys when making the module is good. It is bad to have some padded keys and some not. For example, AbbottSmith has two kinds of keys in it.
> 
> I don’t know if this solves all the dictionary problems surrounding Strong’s numbers. But it solves a couple of bugs.
> 
> In Him,
> 	DM
> 
>> On Jan 14, 2016, at 1:20 PM, Костя Маслюк <kostyamaslyuk at gmail.com <mailto:kostyamaslyuk at gmail.com>> wrote:
>> 
>> As far as i remember strongsPadding is true for dictionaries by default, so your $$$1 is actually $$$00001
>> 
>> Good to know sword version for software and for imp2mod used to build module. Rebuilding module and using latest sword have to fix this issue.
>> 
>> p.s. diggin into http://www.crosswire.org/tracker/browse/MOD-61 <http://www.crosswire.org/tracker/browse/MOD-61> would clear up the essence of this issue.
>> 
>> 
>> 2016-01-14 21:26 GMT+04:00 DM Smith <dmsmith at crosswire.org <mailto:dmsmith at crosswire.org>>:
>> It was reported yesterday by Karl, that BosworthToller has an infinite loop bug. And that mod2imp hits this bug.
>> 
>> Here is where mod2imp starts to loop:
>> $$$-ǼWED
>> <entryFree n="-ǽwed"><p><form><orth>-ǽwed</orth></form>. v. ge-ǽ.</p></entryFree>
>> $$$-ǼWIRDLIAN
>> <entryFree n="-ǽwirdlian"><p><form><orth>-ǽwirdlian</orth></form>. v. ge-ǽ</p></entryFree>
>> $$$-ǼÞE
>> <entryFree n="-ǽþe"><p><form><orth>-ǽþe</orth></form>. v. cyning-ǽ.</p></entryFree>
>> $$$1
>> <entryFree n="-ǽþe"><p><form><orth>-ǽþe</orth></form>. v. cyning-ǽ.</p></entryFree>
>> $$$1
>> <entryFree n="-ǽþe"><p><form><orth>-ǽþe</orth></form>. v. cyning-ǽ.</p></entryFree>
>> This last entry is stuck in a loop.
>> 
>> From JSword, I’ve dumped the module just fine. Here is the section that is problematic:
>> Legend:
>> first column: index of the entry in the module
>> Second column: offset
>> Third column: size in bytes
>> Fourth column: the key
>> Fifth column: block number
>> Sixth column: block offset
>> (didn’t include columns for the index and offset in the uncompressed block.)
>> Last column: the first part of the entry, manually truncated for readability.
>> 860	-ǼWED	<entryFree n="-ǽwed"><p><form><orth>-ǽwed</orth></form>. ….
>> 861	-ǼWIRDLIAN	<entryFree n="-ǽwirdlian"><p><form><orth>-ǽwirdlian</orth></form>. ….
>> 862	-ǼÞE	<entryFree n="-ǽþe"><p><form><orth>-ǽþe</orth></form>. ….
>> 863	1	<entryFree n="1"><p><form><orth>1</orth></form>. ….
>> 864	3	<entryFree n="3"><p><form><orth>3</orth></form>. ….
>> 865	A	<entryFree n="a"><p><form><orth>a</orth></form>; ….
>> 866	A	<entryFree n="A"><p><form><orth>A</orth></form>. ….
>> 867	A-BACAN	<entryFree n="a-bacan"><p><form><orth>a-bacan</orth></form>, ….
>> 868	A-BANNAN	<entryFree n="a-bannan"><p><form><orth>a-bannan</orth></form>; ….
>> 870	A-BEALH	<entryFree n="a-bealh"><p><form><orth>a-bealh</orth></form> ….
>> 
>> Note: there are duplicate keys. This might be merely “a” problem, pun intended.
>> 
>> Just a wild guess, 1 is padded as if it were a Strong’s Number to become 00001 and that is used for a lookup. The standard behavior on lookup is to return the nearest match.
>> 
>> Padding to Strong’s Numbers should only happen if the module has Feature=GreekDef or Feature=HebrewDef. I don’t think it cares and pads the number anyway.
>> 
>> If I add StrongsPadding=false to the conf, the module works.
>> 
>> DM
>> 
>> 
>> 
>> 
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org <mailto:sword-devel at crosswire.org>
>> http://www.crosswire.org/mailman/listinfo/sword-devel <http://www.crosswire.org/mailman/listinfo/sword-devel>
>> Instructions to unsubscribe/change your settings at above page
>> 
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org <mailto:sword-devel at crosswire.org>
>> http://www.crosswire.org/mailman/listinfo/sword-devel <http://www.crosswire.org/mailman/listinfo/sword-devel>
>> Instructions to unsubscribe/change your settings at above page
> 
> 
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org <mailto:sword-devel at crosswire.org>
> http://www.crosswire.org/mailman/listinfo/sword-devel <http://www.crosswire.org/mailman/listinfo/sword-devel>
> Instructions to unsubscribe/change your settings at above page
> 
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20160114/c1905c58/attachment-0001.html>