[sword-devel] CrossWire website showing all known modules available (was: Remote Module Repository Wiki)

[Jonathan Morgan]

On Sat, Nov 6, 2010 at 2:06 AM, Troy A. Griffitts <scribe at crosswire.org>wrote:

> On 11/05/2010 02:02 PM, Jonathan Morgan wrote:
> > 2. Offering a list of downloads at CrossWire tends to suggest that they
> > are the *only* books available.  While I can download zip files from
> > Xiphos FTP directly (for example), it's not easy to find out about, and
> > I'm sure some people evaluate the software purely on the basis of the
> > books available without even trying the software.
>
> Since those pages are already generated dynamically, I suppose we could
> simply use our own InstallMgr class to browse all known module
> repositories for all known modules and display them as well.
>

That was indeed my thought.

That invokes the idea of offering RawZips of those modules, since those
> are also currently generated on the fly by those pages as well, which I
> think might be a political issue.  But at least the mention of the
> module would be nice to see.  Thanks for the great idea.  Wanna
> implement it? :)
>

I might think about it at some point, but for the moment I really can't take
on any new responsibilities until BPBible 0.5 is completed (I hope I haven't
said that too many times, but if I have just be aware that I'm probably much
more sick of saying it than you are of hearing it).  However, I will list a
few "problems" below.

Offering links to Raw Zips (if the particular repository offered them) would
be nice though not necessarily essential.

Well, I supposed it's also a political issue for us.  We might not want
> to list all modules of all known repositories on our page. :)
>

I agree with this to some extent, but I'm not entirely sure how much
difference there is between showing it on your webpage and showing it in the
InstallMgr of an application (sure, more people will see it, but not
necessarily that many more).  Surely you are already using some selection to
determine which repositories can be included?

Anyway, now more on the multiple repository problem.

I understand the vision of multiple repositories and its promise (I think).
However, I think there are a number of problems that can be caused by having
multiple repositories:
1. Automatic updating of installed modules becomes harder: There are more
repositories to download mods.d.tar.gz (or equivalent) from.  This
potentially means taking longer, using more resources, etc., and there must
be a point at which you decide this really isn't something we should be
doing.

2. Showing users a view of all books available becomes harder: I think all
the Install Manager implementations I have used end up having one tab per
repository (or something similar) and only showing all the books from that
repository.  This makes it much harder for the user to determine whether a
resource is present (for example, why would I click on the Xiphos tab when
looking for StrongsRealGreek?  Or for Dore Woodcuts?)  I would prefer a
combined view of all of the available modules, and that hits the same
problem with update I mentioned earlier: you potentially need to fetch your
list of available modules from lots of places.

3. Following on from this is the duplicates problem DM mentioned earlier.
At a trivial level it comes up with the NETFree module, which bible.org host
in their own repository, but also asked CrossWire to duplicate it to
increase exposure.  If we had a good multi-repository solution then maybe it
would be exposed enough and would just be left to bible.org to host.  More
fundamental is the case where the two repositories for whatever reason have
different versions of the same module.  DM suggested using KJV (CrossWire)
and KJV (Other Publisher).  I personally think there should only be able to
be one installed at a time, in the same way as currently one module with the
same name would be able to be installed in a directory.  The reason I think
this is because I get the feeling users don't particularly care which
publisher a book came from: they care what the book is.  They ask "What
books are available for your software?", and I think would like one
authoritative list, not "Here is a KJV from publisher X, and here is another
one from publisher Y".

I wonder whether a good solution to the "What books are available in the n
known repositories?" question would be to change the central InstallMgr list
of repositories to not just include a list of blessed repositories, but to
provide a combined mods.d.tar.gz equivalent, which is automatically updated
periodically from all the repositories and ensures applications only have to
check it rather than a potentially unlimited number of repositories.  Any
thoughts on this approach? [Ben notes that size of mods.d.tar.gz itself
could end up a problem for responsiveness on slower network connections with
this approach - the CrossWire one is currently 95 KB, and could increase].

These problems are probably almost non-issues with the current number of
repositories.  However, if the goal really is lots of publishers hosting
their own repositories I think they are something we might have to think
about.  As indicated above, if this is decided to be a good idea I might be
able to help implement it at some point - but probably not this year.

Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20101108/6366fb84/attachment.html>