software - multiple version installs (any distro developers here?)

[Zenaan Harkness]

On Tue, Aug 16, 2016 at 08:45:16PM -0400, grarpamp wrote:
> On 8/16/16, Zenaan Harkness <zen at freedbms.net> wrote:
> > [symlinks, versioning and stuff]
> 
> > I'm sure this has been thunked before
> 
> Many times, over many decades, by many people... and been done.
> 
> Those users and shops have their entire /usr/local or /opt
> (bin sbin lib include share doc etc and so on) as nothing but script
> managed symlink trees as means of tagging the active default version.
> And keep the multiple program-M.m.r versions on disk therein, which
> are manually callable as needed for new testing and legacy use.
> With scripts to build, install, and symlink it all.

Awesome! That's exactly what I want.

Except I want Debian to do all that for me.


> Dependencies,
> namespace collisions, runtime issues with ld rpath and friends, and
> the whims of program authors... all tend to buzzkill many efforts
> there.

A major distro heading that way (e.g. Debian) might encourage developers
to increase the configurability of their own software perhaps?

Gotta start the improvement somewhere...


> Github might have something for you.

I want to establish, re doing this "at the distro level":
- whether it's a better way
- are their any insurmountable runtime issues

And therefore if it all makes sense, to see if there'd be any buy in
with developers of a particular distro.

Is it generally a better way, say for the large mainstream desktop and
server focused distros?

In what scenarios is installation direct to /usr/[bin|lib|share|...]
actually a distinctly better way?


> There are also variant symlinks:
> https://www.dragonflybsd.org/cgi/web-man?command=varsym&section=1
> http://leaf.dragonflybsd.org/cgi/web-man?command=varsym.conf&section=5
> http://leaf.dragonflybsd.org/cgi/web-man?command=varsym&section=2
> https://wiki.freebsd.org/200808DevSummit?action=AttachFile&do=view&target=variant-symlinks-for-freebsd.pdf
> https://github.com/drq883/varsym
> https://lwn.net/Articles/680705/
> https://github.com/onslauth/varsymfs
> 
> The "FHS" of a base OS, be it Linux or BSD, typically exclusively
> reserve /usr/local for their default package manager, and reserve

Ahah! Yes so we need a new default packaging/ installation format to be
supported by RPM and DPKG, to support the better way, e.g.:
/usr/{a..z} has the package install files, and
/usr/[bin|lib|share|..] are symlink farms.

How hard can it be to support this more rational approach?


> /opt for site use. Some sites prefer /opt first in PATH once it's
> compiled and installed by competant system admin.

These fundamentals have no reason to change. The "/opt" example was
simply how I am locally using /opt for custom software installs beyond
what debian provides.

Also convenient to use .../{a..z} for my offline "setups" (.tar.gz
downloads), git clones, documentation (books, URLs I save) etc - each of
these {a..z} hierarchies has been inspired primarily by:
 - Debian's package pools
 - Daniel Berstein's "universal program namespace" intention which
   never really took off.


> Single letter trees for package installs serve no purpose and are
> anti human.

Disagree - not an absolute. Check this:

   > > On Aug 15, 2016 9:15 PM, "Zenaan Harkness" <zen at freedbms.net>
   > > wrote:
   > > > With just a smattering of /etc/.. based meta files, .deb meta
   > > > files and
   > > > tool awareness, Debian could install all software in this form
   > > > - say
   > > > into /usr/a/.. /usr/b/.. etc.
   > >⋅
   > > There are over 48000 packages in debian. What's just a smattering
   > > times
   > > 48000?
   >⋅
   > Hah! Check this out for anyone on a Debian system:
   >⋅
   > /var/lib/dpkg/info/
   >⋅
   > In fact, you might prefer one of the following :)
   >⋅
   > ls -l /var/lib/dpkg/info/ | head -n 30
   > ls /var/lib/dpkg/info/ | wc
   >⋅
   > to get a feel for the existing situation in Debian.
   >⋅
   > Up to 6 (count 'em: 1, 2, 3, 4, 5 and ... 6 YEAH), six files, get
   > this,
   > per package!
   >⋅
   > I find that quite amazing.
   >⋅
   > That means that on a test install where all ~48K packages are
   > installed,
   > this one directory would contain roughly 250K files.
   >⋅
   > Quite impressive really...


So look into /var/lib/dpkg/info - that's heading for 9K files on my
system - and this is a relatively fresh install (<12months)!

That's not human friendly. Debian's package pools are definitely more
human friendly (says the almighty humble I).

But sure, a flat hierarchy may be more human friendly, still with each
package in its own /usr/program-v0.7.2/... directory, and still having
the usual symlink farms in /usr/bin and /usr/lib etc.

It's probably a bike shed issue - my preference is, given the number of
packages that get installed in "an average desktop installation", to
have an {a..z} layer, as in /usr/{a..z}/program-v0.7.2/...


> There's no performance need to turn the fs into a binary
> tree (todays kernels and fs can handle flat dirs many times the
> needs of any installed package set). Nor need to turn the fs into
> a database, that's what package managers are for.

In agreement on this.

The point is just multiple versions parallel installs, that's all.

This should make a lot of things easier in various ways - Debian
"backports" for example, not only custom end-user installs, besides the
obvious developer test installs - think:

   $ git checkout tag-v-3.7.14
   $ ./configure DEST=/usr/m/my-program-v-3.7.14
   $ make
   $ make install
   $ make symlink-as-primary


Wouldn't you like it to be "GNU/Linux, the hegemonic OS"?


> > I'd like to see the libre unix software ecosystem move forward
> > on this.
> 
> Nobody wants such a symlink mess in the base system, ever.

Have you never been frustrated fighting the base package system with
multiple versions of something you want installed?

Tried to install a backports only to end up in a dependency hell where
some backport lib must also be installed, and this stuffs up other
packages from your base install?

Given my experience, your "nobody wants" is too broad by at least 1 :)


> And things like pivot root, snapshots, portable processes, rolling
> release, vm, jail, layered fs, and patchable kernel are all happening.
>
> Leaving just handling /usr/local...

Symlink farms to actual per-version install folders would make "alt
'environments' almost trivial to implement with nothing but a change in
PATH and one or two other env vars.


> Most people usually don't have need for multiple versions, just a
> current version that works. Even packaging just that is hard and a
> lot of work by a lot of people. Thus port / package tools naturally
> remain rather simple.
> 
> Sure there are issues with limited accomodations for versioning /
> site in package managers. So see /opt above. Or roll your own distro,
> and packager and reach wide adoption with it.

:)

I suspect could be possible to have as a simple plugin/set of rules, for
dpkg, which would not conflict with the existing layout, just supplement
for those who want a "better backports installation ability".

That's the itch that gets me on this one - failed attempts to install
backports packages - along with some successful attempts too - but my
experience is inconsistent, and it is evident the process is more
complex than it needs to be.


> > arcane per distro package incantations
>
> Until then, excepting some tool and port sharing among Linux
> and BSD's, things are exactly this, and wishful thunking.

I still think the different distros would not share, in general, the
binary packages due to compile flags and preferred versions.

Multi version program installs would apply to any distro.


> If troubled on Linux, at least look at the BSD's... their hier's
> are cleaner and enforced, partly from not going the "distro" model
> of random thunkers.

Yes, a bucket list item for me - my attempt at Red Hat doesn't bode well
though - we are creatures of habit.

Thanks heaps for the feedback.