<div dir="ltr">Hi Sijo<div><br></div><div>STEP has its own datasources (not sword modules), in the form of Lucene indexes. I'm not sure if the functionality is there already, but it would be a nice plus I were able to version those indexes together with the Sword modules. (But it's a plus, and so shouldn't drive design.)</div>
<div><br></div><div>Chris</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 6 April 2014 22:17, Sijo Cherian <span dir="ltr"><<a href="mailto:sijo.cherian@gmail.com" target="_blank">sijo.cherian@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Thanks Chris.<br><br>Is the only usecase for "external Lucene indexes" to install a downloadable index?<br>
<br>I am assuming any "external Lucene indexes" usage is going through IndexManager interface (in our default case LuceneIndexManager).<br>
</div>In that case I will add version management through IndexManager api.<br><div><br>So the Installed index versions will go in the folder returned by<br>LuceneIndexManager.getStorageArea(Book book)<br><br><br></div><div>
cheers back<span class="HOEnZb"><font color="#888888"><br>sijo<br></font></span></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Apr 5, 2014 at 5:24 PM, Chris Burrell <span dir="ltr"><<a href="mailto:chris@burrell.me.uk" target="_blank">chris@burrell.me.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Sorry about the radio-silence. I agree with the thread so far. STEP also uses that folder to deposit its own set of Lucene indexes. So that would be a nice central place.<div>
'</div><div>'twould be a nice feature if we could register external Lucene indexes with JSword such that an external app could use Sijo's work. In other words, for other indexes, external to JSword, it would be nice to be able to ask whether indexes could be upgraded, etc.<br>
</div><div><br></div><div>Cheers,</div><div>Chris</div><div><br></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On 2 April 2014 23:34, DM Smith <span dir="ltr"><<a href="mailto:dmsmith@crosswire.org" target="_blank">dmsmith@crosswire.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><div><div>On Apr 2, 2014, at 5:23 PM, Sijo Cherian <<a href="mailto:sijo.cherian@gmail.com" target="_blank">sijo.cherian@gmail.com</a>> wrote:</div>
<br><blockquote type="cite"><div dir="ltr"><blockquote type="cite" style="font-family:arial,sans-serif;font-size:13px"><p><span style="font-family:arial;font-size:small">I will appreciate feedback or concern from jsword users (And,BD..) on having a property file to store current index version (currently indexed default version + per-book's index version, if different).</span><br>
</p><p><span style="font-family:arial;font-size:small">After that, any recommendation on suitable location for this file (I am thinking {IndexFolder}/JSword/lucene/</span><span style="font-family:arial;font-size:small">js.index.metadata.prop file). I am not sure if all platforms have same folder structure (like </span><span style="font-family:arial;font-size:small">JSword/lucene)</span><br>
</p></blockquote></div></blockquote><div><br></div></div>I think it should be in the folder that holds the lucene index files. I'm not sure if AndBible has done anything special to the code that locates the lucene indexes, but the JSword code that determines the state of the index presumes the location. It differs by platform, but that is transparent within JSword. I'm going to guess that AndBible has handled pathing so that it also is transparent. So I think that if you can put it there all front-ends should work with it.<div>
<br><blockquote type="cite"><div dir="ltr"><blockquote type="cite" style="font-family:arial,sans-serif;font-size:13px"><p>
</p></blockquote><blockquote type="cite" style="font-family:arial,sans-serif;font-size:13px"><p><span style="font-family:arial;font-size:small">note: In code comments, I am referring to this file as </span><span style="font-size:small;font-family:arial">InstalledIndex.prop.</span><span style="font-family:arial;font-size:small"><br>
</span></p></blockquote></div></blockquote><div><br></div></div>Regarding the writing and retrieval of values it is up to you. You might be able to add the API to IndexManager.</div><div><br></div><div>It seems that IndexManager should have a IndexStatus getIndexStatus(). (search for setIndexStatus and you'll see why). Ultimately a user asks getIndexStatus from Book, which delegates the question to BookMetaData. The status INVALID is defined as the index is no longer valid. Nothing sets this. Nothing really uses it either.<div>
<div><br><blockquote type="cite"><div dir="ltr"><blockquote type="cite" style="font-family:arial,sans-serif;font-size:13px"><p><span style="font-family:arial;font-size:small">
</span></p><p><span style="font-size:small;font-family:arial"><br></span></p><p><span style="font-size:small;font-family:arial">Many thanks,</span></p><p><span style="font-size:small;font-family:arial">sijo</span></p></blockquote>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sun, Mar 30, 2014 at 4:27 PM, DM Smith <span dir="ltr"><<a href="mailto:dmsmith@crosswire.org" target="_blank">dmsmith@crosswire.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word">Sijo asked this via github. Thought it'd be good to get feedback here.<div>
<br></div>
<div>-- DM<br><div><br><div>Begin forwarded message:</div><br><blockquote type="cite"><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:Helvetica"><b>From: </b></span><span style="font-family:'Helvetica'">Sijo Cherian <<a href="mailto:notifications@github.com" target="_blank">notifications@github.com</a>><br>
</span></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:Helvetica"><b>Subject: </b></span><span style="font-family:'Helvetica'"><b>[jsword] JS-98: IndexManager.needsReindexing(Book book); as the API to detect if per-book index need to be recreated (#74)</b><br>
</span></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:Helvetica"><b>Date: </b></span><span style="font-family:'Helvetica'">March 29, 2014 at 7:31:52 PM EDT<br>
</span></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:Helvetica"><b>To: </b></span><span style="font-family:'Helvetica'">crosswire/jsword <<a href="mailto:jsword@noreply.github.com" target="_blank">jsword@noreply.github.com</a>><br>
</span></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px"><span style="font-family:Helvetica"><b>Reply-To: </b></span><span style="font-family:'Helvetica'">crosswire/jsword <<a href="mailto:reply+i-30455246-2897689b705296a200fa088d2af792bed6bc6a4a-1645068@reply.github.com" target="_blank">reply+i-30455246-2897689b705296a200fa088d2af792bed6bc6a4a-1645068@reply.github.com</a>><br>
</span></div><br><div><p>Following is the upgrade criteria implemented by comparing IndexMetadata from prop files: needsReindexing returns true, if Latest.Index.Version.xxx > Installed.Index.Version.xxx OR if {index folder} is not found at all.</p>
<ul>
<li>For now uses two prop file: 1. IndexMetadata.prop file (can be updated with each release of JSword) 2. InstalledIndex.prop file for "Installed.Index.Version.xx" values that should stay persistent on clients computer between upgrades</li>
<li>Support for PerBookVersion property: Since index creation api is per Book based, added support for (optional) PerBookVersion properties to allow selective upgrade of one book's index, if needed.</li>
<li>This pull request, still assumes index version as 1.2, so should not break anything</li>
</ul><p>Notes: Looked into storing version values in the lucene index using FlexibleIndexing in lucene 4.0, but that seem very dependent on lucene api. Prop file gives human readable configuration and easy to debug. </p>
<p>ToDo/Questions: <br>
1. Managing the add/update of values in InstalledIndex.prop programmatically<br>
2. Need recommendation on suitable location for InstalledIndex.prop file (perhaps as {IndexFolder}/JSword/lucene/js.index.metadata.prop file) <br>
3. Need to add reindexAllBooksIfNeeded() API, that folks (AndBible, BD, Alkitab and?) can use to upgrade all index as bulk, for the installed books , only then the DefaultInstalledVersion can be updated. o.c.j.bridge.BookIndexer.java shows a sample API in the comments. <br>
4. How do the clients want the DownloadIndex option to work with index versioning?</p>
<hr>
<h4>You can merge this Pull Request by running</h4>
<pre> git pull <a href="https://github.com/sijocherian/jsword" target="_blank">https://github.com/sijocherian/jsword</a> master</pre><p>Or view, comment on, or merge it at:</p><p> <a href="https://github.com/crosswire/jsword/pull/74" target="_blank">https://github.com/crosswire/jsword/pull/74</a></p>
<h4>Commit Summary</h4>
<ul>
<li>JS-98 : mechanism to detect if index need to be recreated by comparing IndexMetadata from prop file for now Started partially with the patch from Tonny Kohar dated 15/May/09</li>
<li>- Added InstalledIndex.prop file (location of this file TBD)</li>
</ul>
<h4>File Changes</h4>
<ul>
<li>
<strong>M</strong>
<a href="https://github.com/crosswire/jsword/pull/74/files#diff-0" target="_blank">src/main/java/org/crosswire/jsword/bridge/BookIndexer.java</a>
(20)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/crosswire/jsword/pull/74/files#diff-1" target="_blank">src/main/java/org/crosswire/jsword/index/IndexManager.java</a>
(16)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/crosswire/jsword/pull/74/files#diff-2" target="_blank">src/main/java/org/crosswire/jsword/index/lucene/IndexMetadata.java</a>
(29)
</li>
<li>
<strong>A</strong>
<a href="https://github.com/crosswire/jsword/pull/74/files#diff-3" target="_blank">src/main/java/org/crosswire/jsword/index/lucene/InstalledIndex.java</a>
(67)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/crosswire/jsword/pull/74/files#diff-4" target="_blank">src/main/java/org/crosswire/jsword/index/lucene/LuceneIndexManager.java</a>
(53)
</li>
<li>
<strong>M</strong>
<a href="https://github.com/crosswire/jsword/pull/74/files#diff-5" target="_blank">src/main/resources/IndexMetadata.properties</a>
(18)
</li>
<li>
<strong>A</strong>
<a href="https://github.com/crosswire/jsword/pull/74/files#diff-6" target="_blank">src/main/resources/InstalledIndex.properties</a>
(39)
</li>
<li>
<strong>A</strong>
<a href="https://github.com/crosswire/jsword/pull/74/files#diff-7" target="_blank">src/test/java/org/crosswire/jsword/index/lucene/LuceneIndexManagerTest.java</a>
(131)
</li>
</ul>
<h4>Patch Links:</h4>
<ul>
<li><a href="https://github.com/crosswire/jsword/pull/74.patch" target="_blank">https://github.com/crosswire/jsword/pull/74.patch</a></li>
<li><a href="https://github.com/crosswire/jsword/pull/74.diff" target="_blank">https://github.com/crosswire/jsword/pull/74.diff</a></li>
</ul><p style="font-size:small;color:#666">—<br>Reply to this email directly or <a href="https://github.com/crosswire/jsword/pull/74" target="_blank">view it on GitHub</a>.<img height="1" width="1"></p>
</div></blockquote></div><br></div></div><br>_______________________________________________<br>
jsword-devel mailing list<br>
<a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Regards,<br>Sijo
</div>
_______________________________________________<br>jsword-devel mailing list<br><a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a><br><a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a><br>
</blockquote></div></div></div><br></div><br>_______________________________________________<br>
jsword-devel mailing list<br>
<a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
jsword-devel mailing list<br>
<a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Regards,<br>Sijo
</div>
</div></div></blockquote></div><br></div>