minion-design.pdf / low bandwidth high latency long term connections

[Zenaan Harkness]

On Mon, Oct 28, 2019 at 10:17:55AM +1000, jamesd at echeque.com wrote:
> On 2019-10-22 10:03, Zenaan Harkness wrote:
> > On Mon, Oct 21, 2019 at 06:58:57PM -0300, Punk - Stasi 2.0 wrote:
> > > 
> > > 	courtesy of tor scum dingledine and matheson and a GCHQ guy, danezis who now works for facebook-NSA as well. Wait, facebook, nsa, gchq, university of london, etc so many aliases for the same mafia.
> > > 
> > > 
> > > 	anyway :  https://www.mixminion.net/minion-design.pdf
> > 
> > Have not begun to read that yet, but one of the stream covfefe will
> > "recommend as a default for all users" is a really low bandwidth
> > ("bw") and high latency, permanent connection.
> > 
> > 1KiB/s is much too fat, and at 80MiB/day (2.4GiB per month) folks
> > will complain.
> > 
> > But let's consider just 512 bytes every 10 minutes (a single small
> > UDP packet):
> > 
> >    1 * 6 * 24 * 30 ~= up to 4320 separate "message sends" per month
> > 
> >    each message send can contain up to 512 / 128 = 4 "std tweets"
> >    therefore 17,280 "std tweets" at 10 minute latency
> > 
> >    at 512 * 6 * 24 * 30 ~= 2.1 MiB bandwidth per month
> 
> But you want to tweet to everyone, or everyone who is subscribing
> to your tweets, or to everyone a group.

You're right of course, but I mislead the thinking by using the word
"tweet".

"Tweeting to followers" is actually a different, and equally valid,
use case, one which we must consider separately.

The present use case is private short text messages sent to a very
small number of recipients, which are highly valuable/important, and
can tolerate relatively high latency between pressing the send
button, and the recipient(s) receiving the message - this is a niche
use case, but one for which we must cater, along with all other use
cases.


> So assume every participant is subscribed to various groups, and
> flood fills the data around throughout all members of the group,
> and all members subscribed to that particular poster.
> 
> Well, most people will not be posting every ten minutes, but a
> typical message will contain the hashes summarizing the previous
> state of the groups

That's an arbitrary design imposition - that may or may not be
useful, depending on your app.


> at the previous multiple of 2^15 seconds, plus a bloom filter for
> the messages since then.  And if you have a bunch of groups, and a
> bunch of people you are subscribed to, it is going to add up.

Aye. Indeed.