[sword-devel] Missing ZIP files

Martin Denham mjdenham at gmail.com
Sat Oct 19 07:40:17 MST 2013


Hi Stephan,

That is the way it used to work but a few months ago DM created a script to
automatically build the AV zip files because if a zip file doesn't exist
then JSword apps throw an error when a user tries to download the module.

Cheers
Martin


On 19 October 2013 12:43, Stephan <info at tetzels.de> wrote:

> The zip files are only created if you use the ModDisp.jsp script/applet. (
> http://www.crosswire.org/**sword/modules/ModDisp.jsp?**
> modType=Bibles&av=true<http://www.crosswire.org/sword/modules/ModDisp.jsp?modType=Bibles&av=true>
> )
>
> Best Regards,
> Stephan
>
> Am 19.10.2013 15:58, schrieb Martin Denham:
>
>> Hi,
>>
>> Could somebody check the script that automatically creates the AV .zip
>> files is still running?
>>
>> I had problems downloading RusSynodalLIO and OSHB via JSword and found
>> the zips were missing.  I have just forced the creation of RusSynodalLIO
>> by selecting the jar from ModDisp.jsp and that now exists.  I have not
>> forced creation of OSHB yet in case somebody wanted to verify the
>> OSHB.zip file is not there first.
>>
>> Here is where I am looking:
>> http://www.crosswire.org/**ftpmirror/pub/sword/**avpackages/rawzip/<http://www.crosswire.org/ftpmirror/pub/sword/avpackages/rawzip/>
>> The RusSynodalLIO zip now exists because I just triggered creation:
>>      RusSynodalLIO.zip  19-Oct-2013   01:513.0M
>>
>> but there is no OSHB.zip
>>
>> Thanks
>> Martin
>>
>>
>>
>>
>> On 30 July 2013 22:43, DM Smith <dmsmith at crosswire.org
>> <mailto:dmsmith at crosswire.org>**> wrote:
>>
>>     Thanks for everyone's input.
>>
>>     Attached is a shell script which is modelled after
>>     http://crosswire.org/svn/**crosswire-java/trunk/src/org/**
>> crosswire/sword/web/**PackageMod.java<http://crosswire.org/svn/crosswire-java/trunk/src/org/crosswire/sword/web/PackageMod.java>
>> .
>>     PackageMod.java does the following:
>>     Zipping is only done if at least one of the module's files or conf
>>     has a more recent modification time than the zip.
>>     The name of the zip is taken from the [NAME] in the conf.
>>     The zip overwrites any existing zip.
>>
>>     My script differs slightly:
>>     It zips to a temporary file in the same directory and then copies it
>>     over top of the existing zip.
>>     It manages the mods.d.tar.gz file when a zip is created, again by
>>     creating a temporary and moving that over top of the existing.
>>     It assumes that there are no spaces in any path and that the
>>     repository is on UNIX.
>>
>>     I've tried this on all the repositories I could find.
>>
>>     I did find that there were a few conf files that did not have module
>>     files. Should these confs be renamed, deleted, or otherwise moved
>>     out of the way.
>>
>>     I did not update/manage InstallSize, which could be easily added.
>>
>>     Please review and comment.
>>
>>     BTW, it is very verbose.
>>
>>     In Him,
>>     DM
>>
>>
>>
>>     On Jul 21, 2013, at 5:00 AM, "Troy A. Griffitts"
>>     <scribe at crosswire.org <mailto:scribe at crosswire.org>> wrote:
>>
>>      A few quick comments.
>>>
>>>     If you're using http to grab the zips, we already have an http URL
>>>     which will be sure to give you the latest zip-- the packager.
>>>
>>>     If I implement the cache mechanism in the engine, I will use the
>>>     same logic as what is in the packager to determine if any found
>>>     zip is current.
>>>
>>>     The code for the packager is in the crosswire-java svn repo. You
>>>     should be able to find the servlet with a grep. Old sucky code I'm
>>>     not proud of but works.
>>>
>>>     Chris Little <chrislit at crosswire.org
>>>     <mailto:chrislit at crosswire.org**>> wrote:
>>>
>>>         On 07/20/2013 03:05 PM, DM Smith wrote:
>>>
>>>             On Jul 20, 2013, at 5:42 PM, Chris Little
>>>             <chrislit at crosswire.org <mailto:chrislit at crosswire.org**>>
>>>
>>>             wrote:
>>>
>>>                 On 07/20/2013 07:14 AM, Martin Denham wrote:
>>>
>>>                     They should be there again now. And Bible depends
>>>                     on the av repository.
>>>
>>>                     See this thread for more info:
>>>                     http://svn.crosswire.org/**
>>> pipermail/jsword-devel/2013-**April/004639.html<http://svn.crosswire.org/pipermail/jsword-devel/2013-April/004639.html>
>>>
>>>                     It is a real shame the av zips are not created
>>>                     automatically.
>>>
>>>
>>>
>>>                 Or...
>>>                 It's a real shame that JSword doesn't use the
>>>                 repositories correctly, according to their intended
>>> usage.
>>>
>>>
>>>             Yes it is a shame. I'm almost done w the code to use the
>>>             expanded repository via ftp. This will be with the release
>>>             after next.
>>>
>>>
>>>         Awesome.
>>>
>>>                 Or...
>>>                 It's a real shame you don't write a cron job that can
>>>                 update the ZIPs nightly, if necessary. (Maybe Karl can
>>>                 help you, if he has a script to update ZIPs in his
>>>                 repository.)
>>>
>>>
>>>             Can anyone point me to the code that does the zipping?
>>>             While coding it up is trivial, there's no point to
>>>             re-inventing the wheel. Just port the code to a loop.
>>>
>>>                 The latter is probably more useful at this point and
>>>                 more urgent since I am going to write a cron job to
>>>                 delete the ZIPs on occasion.
>>>
>>>
>>>             Please don't do this until we have had a chance to write
>>>             the script.
>>>
>>>
>>>
>>>         Yeah, it's not urgent. It can wait a while. And I only intend to
>>> have it
>>>         run once a week or so.
>>>
>>>             What would the purpose of doing the mass deletes?
>>>             Currently JSword uses the file time to determine whether
>>>             the module has changed. Since our frontends do not
>>>             advertise updated modules, this is no big deal. We've got
>>>             the mechanism written to use the conf version field, but
>>>             it is not in the frontends yet.
>>>
>>>
>>>
>>>         There are a couple of reasons to clear out the cache on
>>> occasion. The
>>>         web download applet will generate ZIPs for updates and new
>>> modules, but
>>>         don't remove pulled modules (and sometimes I forget to do it
>>> manually).
>>>
>>>         We also get people who think they're really smart and start
>>> feeding the
>>>         download applet things like NIV and NKJV. And the applet will
>>> happily
>>>         package those for them (except that they are empty of content).
>>> So it's
>>>
>>>         nice to clear out those sorts of files from the server.
>>>
>>>             If the purpose is to ensure that old stuff is
>>>             appropriately deleted, then a script comparing module conf
>>>             to zips deleting those that don't match would be better.
>>>             This is even simpler to write than the zipper program.
>>>             This could be added to the zipper program.
>>>
>>>
>>>
>>>         Sure, there are other ways of handling the issues.
>>>
>>>             BTW, Troy's position has been that it is OK to go to the
>>>             zip cache first and failing that get the module by parts.
>>>             But that there should be no requirement of a frontend to
>>>             have the zips. Many have noted that getting the zip is
>>>             faster and more reliable than getting by parts.
>>>
>>>
>>>
>>>         At present, I believe that is a bad position. Shortly after
>>> updates,
>>>         ZIPs and the module parts can get out of sync so that, e.g., you
>>> think
>>>         you're downloading version 2.0 of a module, but you're actually
>>>
>>>         downloading 1.0 since that's what the ZIP is because no one has
>>>         downloaded through the applet, causing the ZIP to update. And no
>>> amount
>>>         to trying to update from the ZIP is going to update to 2.0 until
>>> someone
>>>
>>>         else hits the download applet.
>>>
>>>         A cron job running an update script would at least mean that
>>> both the
>>>         mods.d.tar.gz and ZIPs get updated at approximately the same
>>> time.
>>>
>>>         --Chris
>>>
>>>
>>>
>>>         ------------------------------**------------------------------**
>>> ------------
>>>
>>>         sword-devel mailing list:sword-devel at crosswire.org  <mailto:
>>> sword-devel at crosswire.**org <sword-devel at crosswire.org>>
>>>
>>>
>>>         http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>>>         Instructions to unsubscribe/change your settings at above page
>>>
>>>
>>>     --
>>>     Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>>     ______________________________**_________________
>>>     sword-devel mailing list: sword-devel at crosswire.org
>>>     <mailto:sword-devel at crosswire.**org <sword-devel at crosswire.org>>
>>>
>>>     http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>>>     Instructions to unsubscribe/change your settings at above page
>>>
>>
>>
>>     ______________________________**_________________
>>     sword-devel mailing list: sword-devel at crosswire.org
>>     <mailto:sword-devel at crosswire.**org <sword-devel at crosswire.org>>
>>
>>     http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>>     Instructions to unsubscribe/change your settings at above page
>>
>>
>>
>>
>> ______________________________**_________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>> Instructions to unsubscribe/change your settings at above page
>>
>>
> ______________________________**_________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
> Instructions to unsubscribe/change your settings at above page
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20131019/c2f2cf79/attachment-0001.html>


More information about the sword-devel mailing list