how to best organise a large community "important" books library?

Zenaan Harkness zen at freedbms.net
Wed May 17 00:08:35 PDT 2017


So you're a doomsday prepper and you intend to throw a few dollars at
a multi TiB "restart civilization" digital book library with a few
million books, a few laptops and some solar cells to power them.

How best to hierarchically organise the group/ community doomsday
digital book library?

Say 5 million books, each in its own directory (for any cover art,
PDF and or text versions of the book, perhaps audio, code, etc).

Debian has ~40K packages, and divides these into {a..z}* and
lib{a..z}*, for roughly 60 "top level" directories. Each package has
its subdirectory in one of these. The largest directories are at
about 2K to 3K entries (subdirectories).

However, when prepping for a few million books, we have around 1000
times as many "packages"/books to store, which could mean e.g.
50,000 or more in the popular s directory for example - probably a
little more than is sane.

"Formal" library software (e.g. Evergreen) may be the answer for repo
wide searchability of meta data, but when others share interest, they
may only be able to store, and or may be only particularly interested
in a sub-category or two such as health and agriculture/gardening, so
it might be best anyway to use a handful of top level "general
categories" to reduce our maximums down from 50K books per dir, to 10
fold fewer at least.

Has anyone done anything like this, and if so, how did you solve it?

(Medium term, the problem begs for a git-like addressed, git-like P2P
addressable/verifiable "content management" solution; e.g. if I have
a random collection of say 10K books, it would be nice to be able to
say something like:
   git library init

   # add my books:
   git add .
   git commit

   git library sort categories
   git library add index1 # create an "indexing" branch
   git commit

   # add some upstream/p2p libraries for indexing/search/federation:
   git library index add upstream:uni.berkely
   git library p2p   add i2p.tunnel:myfriend.SHA
   git library full-index pull myfriend.SHA
   git library math-index pull myfriend.SHA
   git library book:SHA   pull myfriend.SHA
)

Perhaps this has not been really done before?


More information about the cypherpunks mailing list