greg.hellings at gmail.com
Thu May 14 13:47:50 MST 2009
On May 14, 2009, at 15:39, DM Smith <dmsmith at crosswire.org> wrote:
> Chris Little wrote:
>> Troy A. Griffitts wrote:
>>> I would like to add new officially supported methods, like adding:
>>> o Installation of modules from another existing SWORD library
>>> installation via HTTP ( e.g., http://crosswire.org/ftpmirror/pub/sword/raw
>>> A skeleton class for this method was already added in 1.6.0 so we
>>> can shoot to implement this in the 1.6.x branch without changing
>>> the API interface.
>>> Aside from the officially supported methods of installation, there
>>> are caching methods we implement to assist in optimizing
>>> installation, e.g. if we find a mods.d.tar.gz file at the root of
>>> the 'other' SWORD library installation from where we're
>>> installing, we'll copy and untargz this to discover which modules
>>> are available, instead of copying each mods.d/*.conf file one at a
>>> **** BUT THE IMPORTANT THING HERE in my mind, is that, while we
>>> have caching mechanisms in place, and while we might even add more
>>> (hypothetically, a zips/ folder at the root of the 'other' SWORD
>>> library installation where we can check for a pre-zipped module
>>> archive so we can avoid downloading module data files and images
>>> one at a time, and other methods)...
>>> .... THE VERY IMPORTANT THING NOT TO MISS HERE.... (have I
>>> focused the attention of skimmers yet? :) )
>>> ... is that these are EXACTLY THAT: CACHING/OPTIMIZATION
>>> features. *** NOT *** required features to make a valid
>>> repository SWORD module repository.
>>> A frontend should NOT depend on these features being available.
>>> If they exist, use them-- great! If not, please support the
>>> minimum common installation features outlined above.
>> Specifically with reference to the way JSword deals with module
>> downloading: I really look forward to seeing a shift from use of
>> the cached ZIPs to using the FTP site (via HTTP if necessary).
> Maybe someone can help a bit. The problem we ran into was that there
> was no reliable way to download a folder via http and there was no
> way to reliably know the content of the folder.
> If there were a manifest of the files in a module that would work
> and then http of all the parts would be just fine. I have a script
> in my bin directory (~dmsmith/bin/listing.pl) on the server that
> builds such a manifest. I would love it if we can make it a part of
> the creation of mods.d.tar.gz by building it daily into the mods.d
> folder on the server.
I've seen sites that had a file named ls-lR that's a recursive long
file list of all the directories under it. Might that be used in a
case like this?
>> Greater than 99% of the time, the ZIPs will be fine, but there will
>> be occasional problems. One is simply that their use requires an
>> extra step, which very occasionally fails. We had some problems
>> after the server upgrade, where new ZIPs couldn't be created, which
>> would have affected JSword users and went unnoticed until someone
>> downloading one of the affected ZIPs from the webpage noticed and
>> alerted us.
>> The (to my mind) greater issue is simply that it this system
>> requires someone else to have already downloaded the most recent
>> version of a module. The ZIPs are only created by the download
>> servlet. Assuming our hope that people migrate to InstallMgrs comes
>> true, fewer and fewer people will download ZIPs from the webpage,
>> making it progressively less likely that the correct ZIP exists.
> Yes that is exactly the problem that JSword has.
> One of the things that Troy suggested is that FTP of the zip, can
> generate the zip if it does not exist. Such a mechanism would be
> useful to SWORD as it would increase the reliability of transfer. If
> you look at the logs you'll see that sometimes a part fails to
> download probably leaving a partial or corrupt module.
> Such a mechanism would leave the problem the same as it is today,
> but it would be minimized.
> sword-devel mailing list: sword-devel at crosswire.org
> Instructions to unsubscribe/change your settings at above page
More information about the sword-devel