IQNets: stream splitting, ala RAID stripe, with P XOR Rand; + massive micro routing

[Zenaan Harkness]

On Sat, Oct 26, 2019 at 09:49:55AM +1100, Zenaan Harkness wrote:
> On Fri, Oct 25, 2019 at 09:44:23PM +0000, jim bell wrote:
> >     On Monday, October 21, 2019, 04:10:23 AM PDT, grarpamp <grarpamp at gmail.com> wrote:  
> > >See the related...
> > >OFFSystem
> > 
> > One application of using this XOR principle is to avoid the problem
> > of a anonymization output node (TOR or otherwise) containing openly
> > suspicious or incriminating information.  If all data through the
> > network splits, before it exits, converted to two (or more???) 
> > seemingly-random data steams, outputted by two (or more???)
> > distinct nodes, it can be recombined to regenerate the desired
> > source data.   
> >
> > An individual node's output is simply random data.

Not fully:

  - YES for individual nodes in the stream (since it's multiple split
    streams, not a single stream), but

  - NO for the origin node which must split/scatter the original
    stream and send the parts across the separate routes, and also

  - NO for whichever destination node does the unsplit/gather - it
    has to receive all parts in order to reassemble the stream

I.e. the 2 end nodes (origin and destination), must, necessarily,
either emit or receive, all of the parts of the stream - and if the
XORing is a known algorithm (say we are XORing against a known
Wikimedia zip file, or against previous packets), then our GPA
stalker can also do these same operations - we're back to having to
properly encrypt, not merely split, the entire stream.

And we note that in Tor's implementation, it is known that the end
point nodes are the primary nodes of interest to GPAs anyway (for
statistical traffic analysis).

So we're back to having to properly hide (encrypt+decrypt) the multi
streams.


If we wish to use XOR to "properly hide" the parts, once again, we
must begin with either an actual (physically generated pure random)
OTP (which must therefore be somehow pre-shared), or agree on a
crypto PRNG (e.g. AES) and corresponding session key for XORing each
stream - and now we're back to traditional crypto.


----------------------------------------

Stream splitting/ massive micro routing

Setting aside OTPs and XORing for the moment, and considering stream
splitting, like "RAID striping" of a network stream (or even just a
packet):

Yes, IQNets intends to maximize multi-streaming or micro-streaming,
to attempt to:

  - maximize usage of many end-user, often low-bandwidth, peer nodes
    (and similarly route nodes), in order to achieve high bandwidth
    (compare to Tor which relies on beefy CIA funded "desirable"
    middle-of-route nodes, since with traditional TCP, your maximum
    stream speed is the speed of your slowest mid-route node)

  - provide effectively transparent (to the end user) "node in my
    route just dropped out" handling

  - multi-home websites, especially with appropriate content
    addressing model apps, downloading from multiple servers or
    sources simultaneously, bittorrent style ("every peer is a
    server", with incentivization to serve what you dl)

  - other benefits?


Issues to handle include:

  - micro streams are a scatter gather
    - the end user's node can do the scatter/ splitting
    - some node must do the gathering/ joining

  - server (comms) lib are prime targets for the gather/join step
    - but at least initially, TCP should work to seamlessly gather,
      as that is TCP's core design intention - turn a series of
      packets into a stable stream - UDP packets arriving at
      unpredictable times and therefore out of sequence, and
      sometimes not arriving at all (e.g. getting dropped by
      overloaded routers in the middle) and checksumming to verify
      any flipped bits, and sending requests to resend any dropped
      packets, is all handled by TCP, and if you're using UDP in your
      APP, you're aware of the limitations.

  - overhead issues for massive micro streaming need to be looked at/
    thought about and analyzed in aggregate - e.g.:
    - my TCP socket is auto-split into many micro routes
    - the likelihood of UDP packets arriving out of order, shall
      increase, therefore e.g.  a TCP socket on top of this network,
      shall on average, require a larger 'UDP packet assembling'
      buffer
    - what is the marginal impack upon a node, and upon network
      bandwidth, and latency, for each additional micro route?

    - UDP protocols which want low latency, can utilize multiple
      micro routes, and study the latency per route, and drop routes
      which are unsatisfactory - real time latency/route optimization