[sword-devel] Missing ZIP files
DM Smith
dmsmith at crosswire.org
Sat Oct 19 08:02:53 MST 2013
I've not gotten it into place. I think it needs to be reviewed first. I'll run it to make them current.
-- DM
On Oct 19, 2013, at 10:40 AM, Martin Denham <mjdenham at gmail.com> wrote:
> Hi Stephan,
>
> That is the way it used to work but a few months ago DM created a script to automatically build the AV zip files because if a zip file doesn't exist then JSword apps throw an error when a user tries to download the module.
>
> Cheers
> Martin
>
>
> On 19 October 2013 12:43, Stephan <info at tetzels.de> wrote:
> The zip files are only created if you use the ModDisp.jsp script/applet. (http://www.crosswire.org/sword/modules/ModDisp.jsp?modType=Bibles&av=true)
>
> Best Regards,
> Stephan
>
> Am 19.10.2013 15:58, schrieb Martin Denham:
> Hi,
>
> Could somebody check the script that automatically creates the AV .zip
> files is still running?
>
> I had problems downloading RusSynodalLIO and OSHB via JSword and found
> the zips were missing. I have just forced the creation of RusSynodalLIO
> by selecting the jar from ModDisp.jsp and that now exists. I have not
> forced creation of OSHB yet in case somebody wanted to verify the
> OSHB.zip file is not there first.
>
> Here is where I am looking:
> http://www.crosswire.org/ftpmirror/pub/sword/avpackages/rawzip/
> The RusSynodalLIO zip now exists because I just triggered creation:
> RusSynodalLIO.zip 19-Oct-2013 01:513.0M
>
> but there is no OSHB.zip
>
> Thanks
> Martin
>
>
>
>
> On 30 July 2013 22:43, DM Smith <dmsmith at crosswire.org
> <mailto:dmsmith at crosswire.org>> wrote:
>
> Thanks for everyone's input.
>
> Attached is a shell script which is modelled after
> http://crosswire.org/svn/crosswire-java/trunk/src/org/crosswire/sword/web/PackageMod.java.
> PackageMod.java does the following:
> Zipping is only done if at least one of the module's files or conf
> has a more recent modification time than the zip.
> The name of the zip is taken from the [NAME] in the conf.
> The zip overwrites any existing zip.
>
> My script differs slightly:
> It zips to a temporary file in the same directory and then copies it
> over top of the existing zip.
> It manages the mods.d.tar.gz file when a zip is created, again by
> creating a temporary and moving that over top of the existing.
> It assumes that there are no spaces in any path and that the
> repository is on UNIX.
>
> I've tried this on all the repositories I could find.
>
> I did find that there were a few conf files that did not have module
> files. Should these confs be renamed, deleted, or otherwise moved
> out of the way.
>
> I did not update/manage InstallSize, which could be easily added.
>
> Please review and comment.
>
> BTW, it is very verbose.
>
> In Him,
> DM
>
>
>
> On Jul 21, 2013, at 5:00 AM, "Troy A. Griffitts"
> <scribe at crosswire.org <mailto:scribe at crosswire.org>> wrote:
>
> A few quick comments.
>
> If you're using http to grab the zips, we already have an http URL
> which will be sure to give you the latest zip-- the packager.
>
> If I implement the cache mechanism in the engine, I will use the
> same logic as what is in the packager to determine if any found
> zip is current.
>
> The code for the packager is in the crosswire-java svn repo. You
> should be able to find the servlet with a grep. Old sucky code I'm
> not proud of but works.
>
> Chris Little <chrislit at crosswire.org
> <mailto:chrislit at crosswire.org>> wrote:
>
> On 07/20/2013 03:05 PM, DM Smith wrote:
>
> On Jul 20, 2013, at 5:42 PM, Chris Little
> <chrislit at crosswire.org <mailto:chrislit at crosswire.org>>
>
> wrote:
>
> On 07/20/2013 07:14 AM, Martin Denham wrote:
>
> They should be there again now. And Bible depends
> on the av repository.
>
> See this thread for more info:
> http://svn.crosswire.org/pipermail/jsword-devel/2013-April/004639.html
>
> It is a real shame the av zips are not created
> automatically.
>
>
>
> Or...
> It's a real shame that JSword doesn't use the
> repositories correctly, according to their intended usage.
>
>
> Yes it is a shame. I'm almost done w the code to use the
> expanded repository via ftp. This will be with the release
> after next.
>
>
> Awesome.
>
> Or...
> It's a real shame you don't write a cron job that can
> update the ZIPs nightly, if necessary. (Maybe Karl can
> help you, if he has a script to update ZIPs in his
> repository.)
>
>
> Can anyone point me to the code that does the zipping?
> While coding it up is trivial, there's no point to
> re-inventing the wheel. Just port the code to a loop.
>
> The latter is probably more useful at this point and
> more urgent since I am going to write a cron job to
> delete the ZIPs on occasion.
>
>
> Please don't do this until we have had a chance to write
> the script.
>
>
>
> Yeah, it's not urgent. It can wait a while. And I only intend to have it
> run once a week or so.
>
> What would the purpose of doing the mass deletes?
> Currently JSword uses the file time to determine whether
> the module has changed. Since our frontends do not
> advertise updated modules, this is no big deal. We've got
> the mechanism written to use the conf version field, but
> it is not in the frontends yet.
>
>
>
> There are a couple of reasons to clear out the cache on occasion. The
> web download applet will generate ZIPs for updates and new modules, but
> don't remove pulled modules (and sometimes I forget to do it manually).
>
> We also get people who think they're really smart and start feeding the
> download applet things like NIV and NKJV. And the applet will happily
> package those for them (except that they are empty of content). So it's
>
> nice to clear out those sorts of files from the server.
>
> If the purpose is to ensure that old stuff is
> appropriately deleted, then a script comparing module conf
> to zips deleting those that don't match would be better.
> This is even simpler to write than the zipper program.
> This could be added to the zipper program.
>
>
>
> Sure, there are other ways of handling the issues.
>
> BTW, Troy's position has been that it is OK to go to the
> zip cache first and failing that get the module by parts.
> But that there should be no requirement of a frontend to
> have the zips. Many have noted that getting the zip is
> faster and more reliable than getting by parts.
>
>
>
> At present, I believe that is a bad position. Shortly after updates,
> ZIPs and the module parts can get out of sync so that, e.g., you think
> you're downloading version 2.0 of a module, but you're actually
>
> downloading 1.0 since that's what the ZIP is because no one has
> downloaded through the applet, causing the ZIP to update. And no amount
> to trying to update from the ZIP is going to update to 2.0 until someone
>
> else hits the download applet.
>
> A cron job running an update script would at least mean that both the
> mods.d.tar.gz and ZIPs get updated at approximately the same time.
>
> --Chris
>
>
>
> ------------------------------------------------------------------------
>
> sword-devel mailing list:sword-devel at crosswire.org <mailto:sword-devel at crosswire.org>
>
>
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> <mailto:sword-devel at crosswire.org>
>
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> <mailto:sword-devel at crosswire.org>
>
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
>
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20131019/faf3aec5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4145 bytes
Desc: not available
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20131019/faf3aec5/attachment-0001.p7s>
More information about the sword-devel
mailing list