[sword-devel] Locale and text retrieval code

[Manfred Bergmann]

Am 30.01.2009 um 19:50 schrieb Manfred Bergmann:

> Hi.
>
> I'm currently having some problems with locales and can't really  
> figure out what the problem is.
> First of all in the initialization code after some checking for  
> system language and such a call to:
>
> sword::LocaleMgr *lManager = sword::LocaleMgr::getSystemLocaleMgr();
> lManager->setDefaultLocaleName("de");
>
> is done.
>
> On the first use of VerseKey:
> VerseKey vk;
>
> I get this output on the console:
> ------------
> VerseKey::Book: 1. Könige does not have a matching toupper abbrevs  
> entry! book number returned was: -1(10). Required entry should be:
> 1. KöNIGE=11
> VerseKey::Book: 2. Könige does not have a matching toupper abbrevs  
> entry! book number returned was: -1(11). Required entry should be:
> 2. KöNIGE=12
> VerseKey::Book: Sprüche does not have a matching toupper abbrevs  
> entry! book number returned was: -1(19). Required entry should be:
> SPRüCHE=20
> VerseKey::Book: Matthäus does not have a matching toupper abbrevs  
> entry! book number returned was: -1(0). Required entry should be:
> MATTHäUS=40
> VerseKey::Book: Römer does not have a matching toupper abbrevs  
> entry! book number returned was: -1(5). Required entry should be:
> RöMER=45
> VerseKey::Book: Hebräer does not have a matching toupper abbrevs  
> entry! book number returned was: -1(18). Required entry should be:
> HEBRäER=58
> ------------
>
> All this results in that no reference lookups for any book name with  
> Umlauts can be done.
> We use a subclass of StringMgr which is initialized and the  
> "supportsUnicode()" method is called (but not the upperUTF8()  
> interestingly).
> Any ideas what the problem could be?

I managed to compile the sword library with ICU and that fixed the  
issue.

> The second thing I don't really understand is how to correctly pull  
> out text for a verse key.
> Some code is following, sorry for that.
>
> In MacSword I have implemented the code like this:
>
> -----------------------
>    // needed to check for UTF8 string
>    MSStringMgr *strMgr = new MSStringMgr();
>
>    // incoming reference
>    const char *cref = [reference UTF8String];
>    sword::VerseKey	vk;
>    sword::ListKey listkey = vk.ParseVerseList(cref, vk, true);
>    // for the duration of this query be want the key to persist
>    listkey.Persist(true);
>    swModule->setKey(listkey);
>
>    // iterate through keys
>    for ((*swModule) = sword::TOP; !swModule->Error(); (*swModule)++) {
>        const char *keyCStr = swModule->getKeyText();
>        const char *txtCStr = swModule->RenderText();
>        NSMutableDictionary *dict = [NSMutableDictionary  
> dictionaryWithCapacity:2];
>        NSString *key = @"";
>        NSString *txt = @"";
>        if(strMgr->isUtf8(txtCStr)) {
>            txt = [NSString stringWithUTF8String:txtCStr];
>        } else {
>            txt = [NSString stringWithCString:txtCStr  
> encoding:NSISOLatin1StringEncoding];
>        }
>
>        if([self isUnicode]) {
>            key = [NSString stringWithUTF8String:keyCStr];
>        } else {
>            key = [NSString stringWithCString:keyCStr  
> encoding:NSISOLatin1StringEncoding];
>        }
>
>        // add to dict
>        [dict setObject:txt forKey:SW_OUTPUT_TEXT_KEY];
>        [dict setObject:key forKey:SW_OUTPUT_REF_KEY];
>        // add to array
>        [ret addObject:dict];
>    }
>    // remove persitent key
>    swModule->setKey("gen.1.1");
> ------------------------

The above code still crashes for references that doesn't exist while  
the below code works everywhere and always.

> This actually works but due to the problem with the locale and  
> Umlauts it crashes somewhere in the library in the head of the for  
> loop ((*swModule) = sword::TOP).
>
> I have some older code lying around (which actually is used in  
> MacSword 1.4.3 done by William) which I to be honest don't understand:
>
> --------------------------
> 	sword::VerseKey vk;
>    	int lastIndex;
> 	((sword::VerseKey*)(swModule->getKey()))->Headings(1);	
> 	sword::ListKey listkey = vk.ParseVerseList(toUTF8(reference),  
> "Gen1", true);	
> 	for (int i = 0; i < listkey.Count(); i++) {
> 		sword::VerseKey *element = My_SWDYNAMIC_CAST(VerseKey,  
> listkey.GetElement(i));
> 		
> 		// is it a chapter or book - not atomic
> 		if(element) {
> 			swModule->Key(element->LowerBound());
> 			// find the upper bound
> 			vk = element->UpperBound();
> 			vk.Headings(true);
> 		} else {
> 			// set it up
> 			swModule->Key(*listkey.GetElement(i));
> 		}
>
> 		// while not past the upper bound
> 		do {			
> 			//add verse index to dictionary
> 			char *ctxt = (char *)swModule->RenderText();
> 			int clen = strlen(ctxt);
>            NSString *text = fromUTF8(ctxt);
>            NSString *verse = fromUTF8(swModule->Key().getText());
>
>            // add to dict
>            NSMutableDictionary *dict = [NSMutableDictionary  
> dictionaryWithCapacity:2];
>            [dict setObject:text forKey:SW_OUTPUT_TEXT_KEY];
>            [dict setObject:verse forKey:SW_OUTPUT_REF_KEY];
>            // add to array
>            [ret addObject:dict];
>
> 			lastIndex = (swModule->Key()).Index();
> 			(*swModule)++;
> 			if(lastIndex == (swModule->Key()).Index())
> 				break;
> 		}while (element && swModule->Key() <= vk);
> 	}
> ---------------------
> This code also has the locale problem but it doesn't crash.
> The problem is however that I don't know the right way of pulling  
> out text and I don't really understand some portions of the above  
> code. These two loops really confuse me. I would be happy if someone  
> could tell me line by line what happens here.
> I had the impression that actually the code above, which crashes, is  
> correct. But it seems, it is not.
>
>
>
> Regards,
> Manfred
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page