osis2mod: reversified material not being written to files... sometimes

Super strange bug.

I can't get it to occur while running in the VC++ debugger or with the debug build. With all optimizations turned off, the bug still presents in the release build.

Using the release build: Running in Win7/x64 causes the bug to appear, but running in XP mode (a Win32 VM) does not cause the bug to appear.

Bug does not affect RawText modules, only zText. Thus, it probably isn't an osis2mod bug but a zText bug.

Using $Rev: 2562 $ osis2mod on 32bit Ubuntu Linux i have the bug when producing a zmod, not when producing a normal mod

I wonder if this is similar to the problem that Troy found in another place? There was a close that needed to happen on the compressed block.
In C++ on normal exit all open files are to be closed. This would give the right behavior. But it is not good for the program to assume that it will exit correctly. Many years ago I saw a problem at work where a program would die during the exit statement and files were not being closed.

IMHO - Solving this issue is long overdue. Is anyone working on it?

This bug is not solved in osis2mod.exe $Rev: 2562 $.

I'm working on making a module for the 2005 Falam Version (from Myanmar Bibles),
and learning a lot in the process, even though this is not the first time I've made a module.

The compressed module has text overwrites. The rawtext module does not.

The three non-KJV verses are 1Chr.19.20 3John.1.15 Rev.12.18
The text for previous verses 1Chr.19.19 3John.1.14 Rev.12.17 gets overwritten when I make the ztext module.

My source text file is valid to the OSIS 2.1.1 schema, even though there are other problems that I'm working on.

David Haslam

Diatheke output for Rev.12.17

Diatheke output for Rev.12.17

Here is the snippet of XML for the above example.

<verse sID="Rev.12.17" n="17" osisID="Rev.12.17"/>
Rulhreng cu nunau nu parah a thin a heng tuk ih nunau tefa pawl Pathian thukham a thluntu le Jesuh ih phuanmi thutak a rinsantu hmuahhmuah do dingah a pok lala.
<verse eID="Rev.12.17"/>
<verse sID="Rev.12.18" n="18" osisID="Rev.12.18"/>
<note type="x-appended">(18)</note>Cutiin tifinriat kapah rulhreng cu a ding.
<verse eID="Rev.12.18"/>

This is for making the ztext module (Falam2):

INFO(V11N): I Chronicles 19:20 is not in the KJV versification.
INFO(V11N): 1Chr.19.20 is not in the KJV versification. Appending content to 1Chr.19.19
INFO(V11N): III John 1:15 is not in the KJV versification.
INFO(V11N): 3John.1.15 is not in the KJV versification. Appending content to 3John.1.14
INFO(V11N): Revelation of John 12:18 is not in the KJV versification.
INFO(V11N): Rev.12.18 is not in the KJV versification. Appending content to Rev.12.17

This is for making the rawtext module (Falam2):

INFO(V11N): I Chronicles 19:20 is not in the KJV versification.
INFO(V11N): 1Chr.19.20 is not in the KJV versification. Appending content to 1Chr.19.19
INFO(WRITE): Appending entry: 1Chr.19.19: <note type="x-appended">(20)</note>Cuih hnu cun Siria mi in Ammon mi cu an bawm duh nawn lo. <div eID="gen6999" type="paragraph"/> <chapter eID="gen6991" osisID="1Chr.19"/>
INFO(WRITE): Appending entry: Mark.16.9: <title subType="x-preverse">MARKA THUTHANGṬHA NETNAK TAWI</title> [ Nunau pawl cu Piter le a rualpi pawl hnenah an feh ih an theihmi thuhla cu a tawinak in an sim.
INFO(WRITE): Appending entry: Mark.16.10: Cuih hnuah Jesuh amah rori in kumkhua rundamnak thu, thu nung le thu thianghlim cu nisuahnak ihsin nitlaknak tiang a dungthluntu pawl hmangin a than a si.] <div eID="gen50429" type="paragraph"/> <chapter eID="gen50421" osisID="Mark.16"/> <div eID="gen50013" osisID="Mark" type="book"/>
INFO(V11N): III John 1:15 is not in the KJV versification.
INFO(V11N): 3John.1.15 is not in the KJV versification. Appending content to 3John.1.14
INFO(WRITE): Appending entry: 3John.1.14: <note type="x-appended">(15)</note>Daihnak na parah um hramseh. <div eID="gen53997" type="paragraph"/><div sID="gen53998" type="paragraph"/> Na rualpi pawl in cibai an lo buk. Kan rualpi pawl kha pumpak ciar in cibai in rak buksak aw. <div eID="gen53998" type="paragraph"/> <chapter eID="gen53989" osisID="3John.1"/> <div eID="gen53986" osisID="3John" type="book"/>
INFO(V11N): Revelation of John 12:18 is not in the KJV versification.
INFO(V11N): Rev.12.18 is not in the KJV versification. Appending content to Rev.12.17
INFO(WRITE): Appending entry: Rev.12.17: <note type="x-appended">(18)</note>Cutiin tifinriat kapah rulhreng cu a ding. <div eID="gen54226" type="paragraph"/> <chapter eID="gen54189" osisID="Rev.12"/>

NB. I have yet to revisit what is the problem with Mark.16.9 & 10
This can probably be resolved by further edits to the source text on my part.

The interesting point is that these two verses were not even reported as having a problem when I made the ztext module.

Whether this sheds any further light on the root cause, I've no idea.

David

Note to self:

1. Tackle the alternative shorter ending for Mark.16.9-10 that appears after Mark.16.20

This comment is FIO.

I thought it was fixed. As I'm doing some other things in osis2mod today, I'll give a shot at this.

I've added a print statement (to standard error) at the end of the program immediately before exit. If this is not seen, then an error has occurred that prevented it's output and the module should be suspect.

I found two errors in the re-versification.

1. The more significant bug was that the program could segfault when it encountered a verse not in the versification. This didn't happen for all verses not in the v11n. The problem was in the SWORD engine where setIndex was being passed a -1 triggered by a call to verseList.increment(1) on a verse (with autonormalize(0)) not in the v11n, causing infinite recursion. It now properly returns an error.
2. The second problem happened for verses in a chapter not in the v11n. It would correctly reversify to the book's last chapter in the v11n. But it would pick -1 for the verse. This has been fixed.

I noted that one of the messages for re-v11n was redundant and eliminated it. Also when -d 64 was used, it would split one of the messages. This too was fixed.

Now, when re-v11n happens there should be two messages. Even when debugging is off.

1. One indicating that the verse is not in the v11n and has been reversified, giving both the original and the next verse.
2. The second should give what is being written.
   Example:
   INFO(V11N): Rev.12.18 is not in the KJV versification. Appending content to Rev.12.17
   INFO(WRITE): Appending entry: Rev.12.17: <note type="x-appended">(18)</note>Cutiin tifinriat kapah rulhreng cu a ding. <div eID="gen54226" type="paragraph"/> <chapter eID="gen54189" osisID="Rev.12"/>

I'm still trying to reproduce the zText problem, having not spent much time on it. This is a problem in the SWORD engine, not osis2mod. From Troy's analysis, the flush and close of the compressor is properly called in the destructor chain of a module. I'm wondering whether the optimizer is re-ordering the code and the destructor chain is not calling flush/close properly.

Maybe the Heisenberg Uncertainty Principle really applies here: Measuring (aka debugging) changes what's measured.

(Chris, try turning off optimization and see if that fixes the problem.)

Has there been any further progress towards solving this issue?

I'm not able to fix it, as I cannot find it.