<div dir="ltr">Hi,<div><br></div><div>Could somebody check the script that automatically creates the AV .zip files is still running?</div><div><br></div><div>I had problems downloading RusSynodalLIO and OSHB via JSword and found the zips were missing. I have just forced the creation of RusSynodalLIO by selecting the jar from ModDisp.jsp and that now exists. I have not forced creation of OSHB yet in case somebody wanted to verify the OSHB.zip file is not there first.</div>
<div><br></div><div>Here is where I am looking: <a href="http://www.crosswire.org/ftpmirror/pub/sword/avpackages/rawzip/">http://www.crosswire.org/ftpmirror/pub/sword/avpackages/rawzip/</a></div><div>The RusSynodalLIO zip now exists because I just triggered creation:<br>
RusSynodalLIO.zip 19-Oct-2013 01:513.0M<br><br>but there is no OSHB.zip</div><div><br></div><div>Thanks</div><div>Martin</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On 30 July 2013 22:43, DM Smith <span dir="ltr"><<a href="mailto:dmsmith@crosswire.org" target="_blank">dmsmith@crosswire.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Thanks for everyone's input.<div><br></div><div>Attached is a shell script which is modelled after <a href="http://crosswire.org/svn/crosswire-java/trunk/src/org/crosswire/sword/web/PackageMod.java" target="_blank">http://crosswire.org/svn/crosswire-java/trunk/src/org/crosswire/sword/web/PackageMod.java</a>.<div>
PackageMod.java does the following:</div><div>Zipping is only done if at least one of the module's files or conf has a more recent modification time than the zip.</div><div>The name of the zip is taken from the [NAME] in the conf.</div>
<div>The zip overwrites any existing zip.</div><div><br></div><div>My script differs slightly:</div><div>It zips to a temporary file in the same directory and then copies it over top of the existing zip.</div><div>It manages the mods.d.tar.gz file when a zip is created, again by creating a temporary and moving that over top of the existing.</div>
<div>It assumes that there are no spaces in any path and that the repository is on UNIX.</div><div><br></div><div>I've tried this on all the repositories I could find.<br><div><br></div><div>I did find that there were a few conf files that did not have module files. Should these confs be renamed, deleted, or otherwise moved out of the way.</div>
<div><br></div><div>I did not update/manage InstallSize, which could be easily added.</div><div><br></div><div>Please review and comment.</div><div><br></div><div>BTW, it is very verbose.</div><div><br></div><div>In Him,</div>
<div><span style="white-space:pre-wrap"> </span>DM</div><div><br></div><div></div></div></div></div><br><div style="word-wrap:break-word"><div><div><div></div><div><br><div><div>On Jul 21, 2013, at 5:00 AM, "Troy A. Griffitts" <<a href="mailto:scribe@crosswire.org" target="_blank">scribe@crosswire.org</a>> wrote:</div>
<br><blockquote type="cite"><div>A few quick comments.<br>
<br>
If you're using http to grab the zips, we already have an http URL which will be sure to give you the latest zip-- the packager.<br>
<br>
If I implement the cache mechanism in the engine, I will use the same logic as what is in the packager to determine if any found zip is current.<br>
<br>
The code for the packager is in the crosswire-java svn repo. You should be able to find the servlet with a grep. Old sucky code I'm not proud of but works.<br><br><div class="gmail_quote">Chris Little <<a href="mailto:chrislit@crosswire.org" target="_blank">chrislit@crosswire.org</a>> wrote:<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<pre>On 07/20/2013 03:05 PM, DM Smith wrote:<br><br><blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #729fcf;padding-left:1ex">On Jul 20, 2013, at 5:42 PM, Chris Little <<a href="mailto:chrislit@crosswire.org" target="_blank">chrislit@crosswire.org</a>> wrote:<br>
<br><blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #ad7fa8;padding-left:1ex">On 07/20/2013 07:14 AM, Martin Denham wrote:<br><blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #8ae234;padding-left:1ex">
They should be there again now. And Bible depends on the av repository.<br><br>See this thread for more info:<br><a href="http://svn.crosswire.org/pipermail/jsword-devel/2013-April/004639.html" target="_blank">http://svn.crosswire.org/pipermail/jsword-devel/2013-April/004639.html</a><br>
<br>It is a real shame the av zips are not created automatically.</blockquote><br><br>Or...<br>It's a real shame that JSword doesn't use the repositories correctly, according to their intended usage.</blockquote><br>
Yes it is a shame. I'm almost done w the code to use the expanded repository via ftp. This will be with the release after next.</blockquote><br>Awesome.<br><br><blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #729fcf;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #ad7fa8;padding-left:1ex">Or...<br>It's a real shame you don't write a cron job that can update the ZIPs nightly, if necessary. (Maybe Karl can help you, if he has a script to update ZIPs in his repository.)</blockquote>
<br>Can anyone point me to the code that does the zipping? While coding it up is trivial, there's no point to re-inventing the wheel. Just port the code to a loop.<br><br><blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #ad7fa8;padding-left:1ex">
The latter is probably more useful at this point and more urgent since I am going to write a cron job to delete the ZIPs on occasion.</blockquote><br>Please don't do this until we have had a chance to write the script.</blockquote>
<br>Yeah, it's not urgent. It can wait a while. And I only intend to have it <br>run once a week or so.<br><br><blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #729fcf;padding-left:1ex">
What would the purpose of doing the mass deletes? Currently JSword uses the file time to determine whether the module has changed. Since our frontends do not advertise updated modules, this is no big deal. We've got the mechanism written to use the conf version field, but it is not in the frontends yet.</blockquote>
<br>There are a couple of reasons to clear out the cache on occasion. The <br>web download applet will generate ZIPs for updates and new modules, but <br>don't remove pulled modules (and sometimes I forget to do it manually). <br>
We also get people who think they're really smart and start feeding the <br>download applet things like NIV and NKJV. And the applet will happily <br>package those for them (except that they are empty of content). So it's <br>
nice to clear out those sorts of files from the server.<br><br><blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #729fcf;padding-left:1ex">If the purpose is to ensure that old stuff is appropriately deleted, then a script comparing module conf to zips deleting those that don't match would be better. This is even simpler to write than the zipper program. This could be added to the zipper program.</blockquote>
<br>Sure, there are other ways of handling the issues.<br><br><blockquote class="gmail_quote" style="margin:0pt 0pt 1ex 0.8ex;border-left:1px solid #729fcf;padding-left:1ex">BTW, Troy's position has been that it is OK to go to the zip cache first and failing that get the module by parts. But that there should be no requirement of a frontend to have the zips. Many have noted that getting the zip is faster and more reliable than getting by parts.</blockquote>
<br>At present, I believe that is a bad position. Shortly after updates, <br>ZIPs and the module parts can get out of sync so that, e.g., you think <br>you're downloading version 2.0 of a module, but you're actually <br>
downloading 1.0 since that's what the ZIP is because no one has <br>downloaded through the applet, causing the ZIP to update. And no amount <br>to trying to update from the ZIP is going to update to 2.0 until someone <br>
else hits the download applet.<br><br>A cron job running an update script would at least mean that both the <br>mods.d.tar.gz and ZIPs get updated at approximately the same time.<br><br>--Chris<br><br><br><br><hr><br>sword-devel mailing list: <a href="mailto:sword-devel@crosswire.org" target="_blank">sword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/sword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/sword-devel</a><br>Instructions to unsubscribe/change your settings at above page<br></pre></blockquote>
</div><br>
-- <br>
Sent from my Android phone with K-9 Mail. Please excuse my brevity.</div>_______________________________________________<br>sword-devel mailing list: <a href="mailto:sword-devel@crosswire.org" target="_blank">sword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/sword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/sword-devel</a><br>Instructions to unsubscribe/change your settings at above page</blockquote></div>
<br></div></div></div></div><br>_______________________________________________<br>
sword-devel mailing list: <a href="mailto:sword-devel@crosswire.org">sword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/sword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/sword-devel</a><br>
Instructions to unsubscribe/change your settings at above page<br></blockquote></div><br></div>