Tsunami UDP Protocol - Re: UDP based data xfer protocols

[Zenaan Harkness]

Tsunami UDP Protocol
https://en.wikipedia.org/wiki/Tsunami_UDP_Protocol

  The Tsunami UDP Protocol is a UDP-based protocol that was developed
  for high-speed file transfer over network paths that have a high
  bandwidth-delay product. Such protocols are needed because standard
  TCP does not perform well over paths with high bandwidth-delay
  products.[1]

  Tsunami effects a file transfer by chunking the file into numbered
  blocks of 32 KB. Communication between the client and server
  applications flows over a low bandwidth TCP connection, and the
  bulk data is transferred over UDP.


Bandwidth-delay product
https://en.wikipedia.org/wiki/Bandwidth-delay_product

  In data communications, the bandwidth-delay product is the product
  of a data link's capacity (in bits per second) and its round-trip
  delay time (in seconds).[1] The result, an amount of data measured
  in bits (or bytes), is equivalent to the maximum amount of data on
  the network circuit at any given time, i.e., data that has been
  transmitted but not yet acknowledged. The bandwidth-delay product
  was originally proposed[2] as a rule of thumb for sizing router
  buffers in conjunction with congestion avoidance algorithm Random
  Early Detection (RED).

  A network with a large bandwidth-delay product is commonly known as
  a long fat network (shortened to LFN). As defined in RFC 1072, a
  network is considered an LFN if its bandwidth-delay product is
  significantly larger than 105 bits (12,500 bytes).

  Ultra-high speed local area networks (LANs) may fall into this
  category, where protocol tuning is critical for achieving peak
  throughput, on account of their extremely high bandwidth, even
  though their delay is not great. While a connection with 1 Gbit/s
  and a round-trip time below 100 μs is no LFN, a connection with 100
  Gbit/s would need to stay below 1 μs RTT to not be considered an
  LFN.

  An important example of a system where the bandwidth-delay product
  is large is that of geostationary satellite connections, where
  end-to-end delivery time is very high and link throughput may also
  be high. The high end-to-end delivery time makes life difficult for
  stop-and-wait protocols and applications that assume rapid
  end-to-end response.

  A high bandwidth-delay product is an important problem case in the
  design of protocols such as Transmission Control Protocol (TCP) in
  respect of TCP tuning, because the protocol can only achieve
  optimum throughput if a sender sends a sufficiently large quantity
  of data before being required to stop and wait until a confirming
  message is received from the receiver, acknowledging successful
  receipt of that data. If the quantity of data sent is insufficient
  compared with the bandwidth-delay product, then the link is not
  being kept busy and the protocol is operating below peak efficiency
  for the link. Protocols that hope to succeed in this respect need
  carefully designed self-monitoring, self-tuning algorithms.[3] The
  TCP window scale option may be used to solve this problem caused by
  insufficient window size, which is limited to 65,535 bytes without
  scaling.

  [Examples omitted]



Tsunami - how it works, from
https://sourceforge.net/projects/tsunami-udp/files/tsunami-udp%20docs/tsunami-docs-v1.1-cvsbuild11/tsunami-docs-v1.1-cvsbuild11.tar.gz/download

  ...
  How It Works

  Tsunami performs a file transfer by sectioning the file into
  numbered blocks of usually 32kB size.  Communication between the
  client and server applications flows over a low bandwidth TCP
  connection.  The bulk data is transferred over UDP.

  Most of the protocol intelligence is worked into the client code -
  the server simply sends out all blocks, and resends blocks that the
  client requests. The client specifies nearly all parameters of the
  transfer, such as the requested file name, target data rate,
  blocksize, target port, congestion behaviour, etc, and controls
  which blocks are requested from the server and when these requests
  are sent.

  The client starts file transfers with a get-file request. At the
  first stage of a transfer the client passes all its transfer
  parameters to the server inside the request. The server reports
  back the length of the requested file in bytes, so that the client
  can calculate how many blocks it needs to receive. 

  Immediately after a get-file request the server begins to send out
  file blocks on its own, starting from the first block. It flags
  these blocks as "original blocks". The client can request blocks to
  be sent again. These blocks are flagged as "retransmitted blocks"
  by the server.

  When sending out blocks, to throttle the transmission rate to the
  rate specified by the client, the server pauses for the correct
  amount of time after each block before sending the next.

  The client regularly sends error rate information to the server.
  The server uses this information to adjust the transmission rate;
  the server can gradually slow down the transmission when the client
  reports it is too slow in receiving and processing the UDP packets.
  This, too, is controlled by the cient. In the settings passed from
  client to server at the start of a transfer, the client configures
  the server's speed of slowdown and recovery/"speed-up", and
  specifies an acceptable packet loss percentage (for example 7%).

  The client keeps track of which of the numbered blocks it has
  already received and which blocks are still pending. This is done
  by noting down the received blocks into a simple bitfield. When a
  block has been received, in the bitfield the bit corresponding to
  the received block is set to '1'.

  If the block number of a block that the client receives is larger
  than what would be the correct and expected consecutive block, the
  missing intervening blocks are queued up for a pending
  retransmission. The retransmission "queue" is a simple sorted list
  of the missing block numbers. The list size is allowed to grow
  dynamically, to a limit. At regular intervals, the retransmission
  list is processed - blocks that have been received in the meantime
  are removed from the list, after which the list of really missing
  blocks is sent as a normal block transmission request to the
  server.

  When adding a new pending retransmission to the client's list makes
  the list exceed a hard-coded limit, the entire transfer is
  reinitiated to start at the first block in the list i.e. the
  earliest block in the entire file that has not been successfully
  transferred yet. This is done by sending a special
  restart-transmission request to the server.

  When all blocks of the file have been successfully received, the
  client sends a terminate-transmission request to the server.