[sword-devel] InstallMgr
DM Smith
dmsmith at crosswire.org
Thu May 14 21:07:53 MST 2009
Just a bit of history regarding JSword. We have an FTP mechanism. We
moved it to limbo, but can put it back at any time, with little
effort. It worked just fine on Windows XP until a particular patch
came out. After that, FTP was blocked by Windows XP. But we already
had complaints that FTP did not work through proxies and was blocked
by some firewalls.
At this point we added HTTP as a second option, with FTP enabled by
default. This did not reduce the number of problem reports regarding
downloading modules, except perhaps a little.
When we first implemented HTTP, we tried to go after the directory
listing, but during developing it some server upgrade changed it. So
we went after the zips instead.
Occasionally, we get a problem report that some module is not
available that is listed and this is always due to JSword not doing
things in a proper way. But the number of problem reports is far fewer
with this mechanism than with FTP. In fact it is almost none.
BTW, I'm not against FTP nor adding it back to JSword, if we can work
around the problems. Right now the SWORD answer to proxy and firewall
problems is to download the zip and manually install it. I don't think
this is a good answer.
Troy, I can put the FTP package back and I can see if Apache's
implementation is any better. But we would still want to offer http as
a solution to those problems which cannot be solved. And we would not
want to increase the number of reported problems, so we would have to
be smarter about it than we were before.
I understand the rational for the current mechanism that any
installation of modules can be used as a repository. Today a module
consists of a conf file that gives the path to where the constituent
files can be found and it consists of those files.
I'd like to suggest that we add another file or modify the conf to
give a manifest of that content. Then there would be no need for an
http parse. The ftp parse would continue to work just as well as it
always did, but it would work just as well for it to use the manifest.
The perl script I mentioned creates a single manifest file much as the
old conf listed all the modules, but it could be made to create
separate info or insert it to a conf file. My script produces far more
info in it than is necessary for a manifest.
This would be in keeping with the notion of a installation of modules
without cache can be used as a repository.
Working together in Christ,
DM
On May 14, 2009, at 11:13 PM, Daniel Owens wrote:
> Thanks for your assurances, Matthew and Troy. Just to make sure I've
> understood, FTP was chosen initially because you can download
> multiple files more easily, which is why JSword requires zipped
> modules when downloading over HTTP. Adding the ability to download
> from a working SWORD library over HTTP requires a bit more work in
> the engine so that all front ends support it. Once that is added to
> the engine and front ends support it, then a repository via HTTP
> will likely just need the mods.d and modules folders with modules
> installed just like an FTP server, right? Thanks for bearing with
> me. If I understand you correctly, this will solve the anonymous FTP
> hangup I ran into while not adding to the list of things content
> providers must do. Sounds good to me.
>
> Daniel
>
> Troy A. Griffitts wrote:
>> Thank Matthew. Yes Daniel, we have had an established standard for
>> installing modules for quite some time which uses FTP for remote
>> module installation. I would like all of our client applications
>> to support all of our officially supported install methods.
>>
>> The reason for this is so we can assure a content provider that
>> their content will be available to 'SWORD applications' if they:
>>
>> x y z
>>
>> Currently, we can say this, but we have to add a disclaimer:
>> "JSword based apps will not work unless you also zip up all of your
>> modules and make these zips available as HTTP downloads."
>>
>> I wish for this disclaimer to shrink, not grow, so I am trying to
>> herd in the development efforts and ask all frontend developers to
>> ensure that their applications at least support the already
>> longstanding installation policies in place.
>>
>>
>> Now concerning new methods, like HTTP access. Matthew is correct
>> that we hope to add support for this in the 1.6.x branch of the
>> engine. At that time, we will ask all frontend developers to
>> expose this functionality as well so we can then give content
>> creators one more option (CD, FTP, ... + HTTP) to distribute their
>> content.
>>
>>
>> The details of how this will be implemented are being discussed,
>> but whether we SHOULD implement HTTP access, I think is unanimous.
>>
>> I would like for this be implemented in a which doesn't depend on
>> scripts for minimum HTTP support for a repository. Following the
>> same theme as other methods: "Just expose your working SWORD module
>> library via HTTP."
>>
>> This is difficult to implement for us, but we can do it once in C++
>> and have all frontends get the new functionality, and then all
>> content providers have an easy means to expose content.
>>
>> On top of that, we may add what I've been labeling as CACHING /
>> OPTIMIZATION mechanisms to make downloading quicker, but these will
>> require the content providers be more savvy, possibly implement
>> cron jobs to zip up folders, etc. But these mechanisms will be
>> optional for the content providers. They will not need to do these
>> things to have a valid repository.
>>
>> Imagine using your favourite SWORD application installer to install
>> a library of all the materials your ministry uses to reach your
>> people group. Everything is just how you want it on your computer.
>> Now you can copy it to a CD as-is, and it is a valid installation
>> source to any other SWORD frontend. You can pass that CD to a
>> church with a website and they can copy it as-is to their website
>> and serve the entire library with SWORDweb. They can point an FTP
>> server to it as-is, and it becomes a valid remote installation
>> source for their congregation or other people working in the same
>> domain and running any SWORD client,
>> ...
>> and in the future, they hopefully will be able to allow HTTP access
>> to the same location as-is and it becomes a valid remote
>> installation source.
>>
>> The zip scripts and ls-LR files are all fine to help improve speed
>> and reduce processor power, but I am willing to bite the bullet and
>> do the work to make things functional without them, if it keeps
>> this easy to propagate scenario available.
>>
>> Am I helping explain my reasoning?
>>
>> -Troy.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Matthew Talbert wrote:
>>>> I think I caught your drift with the caps in your previous
>>>> message, but it's
>>>> a bit discouraging and doesn't yet answer the problem I raised
>>>> awhile back.
>>>>
>>>> To recap, because of security concerns with having anonymous FTP
>>>> access, one
>>>> group I work with effectively denied us the ability to set up a
>>>> repository.
>>>> If HTTP is not required of front-ends, then you MUST have an FTP
>>>> repository
>>>> to get consistent coverage among the front-ends. I'm not hearing
>>>> a clear
>>>> argument for why HTTP shouldn't become the standard except that
>>>> "Our scripts
>>>> for generating zipped modules doesn't work reliably" and "it's
>>>> hard to
>>>> create a list of modules from different formats." Are both of these
>>>> insurmountable?
>>>>
>>>> I'm trying to understand why FTP is the preferred method when
>>>> HTTP is a
>>>> lower common denominator between providers. Could you explain
>>>> that to me?
>>>>
>>>> My goal isn't to push an agenda but find a way to set up a
>>>> repository that
>>>> will work with all front-ends but does not use anonymous FTP.
>>>>
>>>> Mildly discouraged,
>>>> Daniel
>>>
>>> I don't think there's any need for discouragement Daniel. I believe
>>> what Troy is saying is that *frontends* should support FTP. He's
>>> also
>>> saying that he intends to add support for HTTP in the engine. At
>>> that
>>> point, both FTP and HTTP repositories will be possible and should be
>>> supported by all frontends. So the requirement isn't on the
>>> repositories being a certain way, but frontends being able to handle
>>> both types as a common denominator. Troy can correct me if I'm
>>> misunderstanding him.
>>>
>>> I will be happy to see HTTP support for installmgr.
>>>
>>> One additional request that I have would be to allow installing a
>>> single zip from somewhere on the file system. Will this be possible
>>> with the proposed implementation? I'm willing to help with this,
>>> as I
>>> want it in Xiphos and was intending to implement it there, but it
>>> would be better to have in the library.
>>>
>>> Matthew
>>>
>>> _______________________________________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>>
>>
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
More information about the sword-devel
mailing list