Modules
  1. Modules
  2. MOD-61

ZhEnglish has a bug when acquiring the keys

    Details

    • Type: Bug Bug
    • Status: Reopened (View Workflow)
    • Priority: Major Major
    • Resolution: Unresolved
    • Component/s: None
    • Labels:
      None

      Description

      I don't know the details but it looks like ZhEnglish creates an endless loop when the keys are acquired one after another. See the bug report in BibleTime: https://sourceforge.net/tracker/index.php?func=detail&aid=1818522&group_id=954&atid=100954

      1. fix1.patch
        1 kB
        Kostya Maslyuk
      2. fix-lds.patch
        6 kB
        Kostya Maslyuk
      3. imp2ld.patch
        0.5 kB
        Kostya Maslyuk
      4. lds-symmetry.diff
        6 kB
        Kostya Maslyuk

        Activity

        Hide
        Kostya Maslyuk added a comment -

        sorry, update

        Show
        Kostya Maslyuk added a comment - sorry, update
        Hide
        Kostya Maslyuk added a comment -

        it would be more convenient to use personal clone to track changes:

        https://gitorious.org/sword-svn-mirrors/kalemas_at_mail_ru-trunk/ , branch fix-lds-clear

        Show
        Kostya Maslyuk added a comment - it would be more convenient to use personal clone to track changes: https://gitorious.org/sword-svn-mirrors/kalemas_at_mail_ru-trunk/ , branch fix-lds-clear
        Hide
        Troy A Griffitts added a comment - - edited

        OK, documenting #sword chat with Kostya this morning:

        The use case which fails here is:
        for (module = TOP; !module.popError(); module++);

        The problem scenario is demonstrated by a module created with keys: 0001, 0002, 0003, 4, 0005, 0006

        This will be reordered in the .idx file to 0001, 0002, 0003, 0005, 0006, 4, which is programmatically fine, though probably not what the author desires-- but that is beside the point.

        SWLD descendants' increment() method is called in the for loop incrementor in the problem use case. Looking at the increment() implementation, it calls getEntry() which strongsPads and then first does a binary search to find the current key, and then ++ the index offset to move forward to the next record. This is all fine (though not the most optimized, but that a different issue).

        The problem is that we do not have symmetry setting and getting the key. "0006"++ returns "4". Fine. "4"++ returns "0006". Not fine. If we setKey("4") we would return "0005" because we first pad to "0004" and then snap to the closest entry which is "0005". So the loop is:

        0001, 0002, 0003, 0005, 0006, 4, 0006, 4, 0006, ...

        This is because "0006"++ == "4"
        "4"++ == "0006" because the padding first takes us to "0005"

        Anyway, all the normalization for a key is scattered everywhere in the code. This all sucks and needs a major cleanup.
        a) We current have 2 normalizations we might do for LD keys: toupper and strongsPad.
        b) All normalizations should be isolated and done symmetrically. Keys must be round-trip. This must be true. setKey(getKey()) must not change the position of a module. This is not true for "4" in the problem module scenario because padding logic is not symmetrical.

        Candidates for where to isolate the normalization: setEntry/getEntry/deleteEntry, or maybe better, simply do it in SWLD::setKey

        Show
        Troy A Griffitts added a comment - - edited OK, documenting #sword chat with Kostya this morning: The use case which fails here is: for (module = TOP; !module.popError(); module++); The problem scenario is demonstrated by a module created with keys: 0001, 0002, 0003, 4, 0005, 0006 This will be reordered in the .idx file to 0001, 0002, 0003, 0005, 0006, 4, which is programmatically fine, though probably not what the author desires-- but that is beside the point. SWLD descendants' increment() method is called in the for loop incrementor in the problem use case. Looking at the increment() implementation, it calls getEntry() which strongsPads and then first does a binary search to find the current key, and then ++ the index offset to move forward to the next record. This is all fine (though not the most optimized, but that a different issue). The problem is that we do not have symmetry setting and getting the key. "0006"++ returns "4". Fine. "4"++ returns "0006". Not fine. If we setKey("4") we would return "0005" because we first pad to "0004" and then snap to the closest entry which is "0005". So the loop is: 0001, 0002, 0003, 0005, 0006, 4, 0006, 4, 0006, ... This is because "0006"++ == "4" "4"++ == "0006" because the padding first takes us to "0005" Anyway, all the normalization for a key is scattered everywhere in the code. This all sucks and needs a major cleanup. a) We current have 2 normalizations we might do for LD keys: toupper and strongsPad. b) All normalizations should be isolated and done symmetrically. Keys must be round-trip. This must be true. setKey(getKey()) must not change the position of a module. This is not true for "4" in the problem module scenario because padding logic is not symmetrical. Candidates for where to isolate the normalization: setEntry/getEntry/deleteEntry, or maybe better, simply do it in SWLD::setKey
        Show
        Kostya Maslyuk added a comment - - edited https://gitorious.org/sword-svn-mirrors/kalemas_at_mail_ru-trunk/commits/f7da5a780ca000e9d22e953a7737a5fc60267128 fixed symmetry
        Hide
        Mac User added a comment - - edited

        Hi guys, this bug has plagued me for years. I don't really know anything about the API, but it's a problem with the module, I believe. Or with the way Chinese characters are handled. Something to do with encodings? Here's some more data that might help:

        1. Attempting to open ZhEnglish in Eloquent hangs the app (still, with 2.4.10). You know this. BibleDesktop opens it OK, but when you try to view particular entries, you get errors and they can't be rendered. Other entries work fine. E.g. starting from the first dictionary entries, the "-IAN" entry throws up the error "An error has occurred: Error reading". Likewise with entries "-IST", "-IZE", "10 OF A PECK", "1ST EARTHLY BRANCH", and so on. It appears to be the same entries throwing up errors each time, and in no discernable pattern.

        2. In Eloquent and BibleDesktop on Mac, there are no problems with ZhPinyin, and ZhHanzi loads OK, but with ZhHanzi in Eloquent, looking up by typing the headword doesn't work for some characters; while it does for others. Indexing the module and searching does work (if any were missing, I didn't notice). This may or may not be related. I can't find a way to look up a lexicon by typing in BibleDesktop at all.

        The good news is that on newer Mac operating systems (and iOS), it's very easy just to look up words using the built-in Dictionary (enable Chinese dictionaries first), and it uses much nicer, newer dictionaries than this 2002 version of CEDICT! Doesn't help anyone on other platforms though.

        Another idea might be to just start again with a better starting dictionary. There are much newer versions of CEDICT (now CC-CEDICT), but these don't have Bible terms. WM-Dict is much better for Bible users, as it includes lots of Bible terms: https://sites.google.com/site/wmdict/ . Also, aizhu.com has a separate and quite helpful dictionary built in, I don't know if the developer might be interested in sharing? Could see if the problem persists.

        Show
        Mac User added a comment - - edited Hi guys, this bug has plagued me for years. I don't really know anything about the API, but it's a problem with the module, I believe. Or with the way Chinese characters are handled. Something to do with encodings? Here's some more data that might help: 1. Attempting to open ZhEnglish in Eloquent hangs the app (still, with 2.4.10). You know this. BibleDesktop opens it OK, but when you try to view particular entries, you get errors and they can't be rendered. Other entries work fine. E.g. starting from the first dictionary entries, the "-IAN" entry throws up the error "An error has occurred: Error reading". Likewise with entries "-IST", "-IZE", "10 OF A PECK", "1ST EARTHLY BRANCH", and so on. It appears to be the same entries throwing up errors each time, and in no discernable pattern. 2. In Eloquent and BibleDesktop on Mac, there are no problems with ZhPinyin, and ZhHanzi loads OK, but with ZhHanzi in Eloquent, looking up by typing the headword doesn't work for some characters; while it does for others. Indexing the module and searching does work (if any were missing, I didn't notice). This may or may not be related. I can't find a way to look up a lexicon by typing in BibleDesktop at all. The good news is that on newer Mac operating systems (and iOS), it's very easy just to look up words using the built-in Dictionary (enable Chinese dictionaries first), and it uses much nicer, newer dictionaries than this 2002 version of CEDICT! Doesn't help anyone on other platforms though. Another idea might be to just start again with a better starting dictionary. There are much newer versions of CEDICT (now CC-CEDICT), but these don't have Bible terms. WM-Dict is much better for Bible users, as it includes lots of Bible terms: https://sites.google.com/site/wmdict/ . Also, aizhu.com has a separate and quite helpful dictionary built in, I don't know if the developer might be interested in sharing? Could see if the problem persists.

          People

          • Assignee:
            Chris Little
            Reporter:
            Eeli Kaikkonen
          • Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated: