anonget 1.0 (formerly uinmyn)

18 Sep 2011

      -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

NEW IN THIS RELEASE

anonget (1.0)

  * Name change from "uinmyn" (last version was 0.07)
  * Optional send confirmation now available, providing a progress
    report of the URL request and to help diagnose failures.
  * Reformat anonget-list-gotten-urls.py output for easier reading.
  * Change "is-not-my.name" to "mixnym.net".
  * Rewrite the documentation.
                                          _     __   ___
     _______ __  ____  _ __  _____ ____ _| |_  /  | / _ \
    /  _  | '_ \/  _ \| '_ \/  _ //  _ \_   _|  | || | | |
    | |_| | | | | |_| | | | | |_| | |/ / | |    | || |_| |
    \___/_|_| |_\____/|_| |_\  __/ \___/ |__|  /___O\___/
                            / /__
			    \____|
           __________________________________________
          |                                          |
          |                                          |
          |   IS THIS ANONYMOUS BROWSING, OR WHAT?!  |
          |                                          |
          |__________________________________________|

N.B.: Below, replace all occurrences of "<AT>" with "@".

WHAT IS IT?

anonget facilitates browsing the Internet with strong anonymity via
url<AT>mixnym.net [1].

HOW IT WORKS

Given a list of URLs, anonget encrypts the list and a fresh random key
and formats it for mailing by mixmaster [2] to url<AT>mixnym.net
through a chain of anonymizing remailers.  The request contains
nothing to identify you or your site, and the outgoing mail is
indistinguishable from other remailer traffic that you may generate.

url<AT>mixnym.net then gets the requested information from the
Internet, encrypts it to the fresh key, and broadcasts it worldwide on
Usenet newsgroup alt.anonymous.messages.

The fresh key (saved by anonget) is used to recognize and decrypt the
reply.

No one, not even the mixnym.net administrator, is able to link content
being fetched with any particular user, or with other requests.

EXAMPLE 1

The command

   echo http://www.banana.mixmin.net | anonget-fetch-body.py db \
    | mixmaster url<AT>mixnym.net

generates a fresh random key for the encryption and hsub of the reply,
adds the key to db (which need not have existed beforehand), formats
and encrypts key and list of URLs (just http://www.banana.mixmin.net
in this case) as required by url<AT>mixnym.net, and sends it.  db is
plain ASCII, so you can look at it to better see what's happening.

Later, in a directory full of fresh a.a.m articles, the command

   for f in *; do anonget-do-aam-article.py db < $f >> mbox; done

will append the requested content to mbox if it has arrived.

ADDING SEND CONFIRMATION

Alternatively, a request can be formulated to not only get the wanted
information as above, but to send confirmation that it was mailed from
the last remailer in the chain.  This provides a progress report of
the request and can help to diagnose failures.  A copy of the request
is sent to a mail-to-news gateway with Subject: another hsub generated
with the same fresh key.

EXAMPLE 2

   (echo 'Chain: *,*,kulin'; echo http://www.banana.mixmin.net \
    | anonget-fetch-whole.py db) | mixmaster

Unlike anonget-fetch-body.py of example 1, anonget-fetch-whole.py
generates a whole mail, including To:, Cc:, and Subject: headers,
instead of relying on mixmaster to provide them.  (Yet more headers,
like the Chain: in example 2, can be added as shown.  Note the
parentheses in the command.)

When anonget-do-aam-article.py processes the article in example 1, it
emits the inner reply mail and deletes the db entry.  But when it
senses that the article is not the returned data, but the request
itself, it constructs a confirmation message and leaves the db entry
intact.  (Note that if the fetched data get processed before the
request confirmation (due to net lag), the request confirmation will
not be noticed because the database entry will already have been
deleted.  But in this case it doesn't matter.)

CACHING OR ARCHIVING THE REPLY, AND RENDERING IT

A convenient way to save and render an anonget reply is to insert it
into the browser-accessible cache provided by wwwoffle [3].  The
script anonget-to-wwwoffle-cache.py is provided for this purpose.

To just list the URLs contained in an "inner mail", the script
anonget-list-gotten-urls.py can be applied.

AUTOMATIC DISPOSITION OF THE REPLY

Instead of the explicit anonget-do-aam-article.py command above,
procmail [4] can be used to forward the reply as local configuration
may require.  Return codes from anonget-do-aam-article.py are useful
for procmail, which might be checking the same message for other hits.
The following .procmailrc clause sequence will send the decrypted
"inner posting" to "local-recipient".

   :0fW
   | anonget-do-aam-article.py /path/to/anonget_pending_requests_db

   :0a
   ! local-recipient

RESPECTING YOUR PRIVACY

Out of respect for your privacy, anonget doesn't require you to access
any web site for its installation or use (though you may if you wish:
mailto:stealthsuite<AT>nym.mixmin.net?subject=send%20index.html).
Everything is right here in this posting.

ABOUT THE CODE

Succinctness and readability are given priority here over efficiency.
If efficiency becomes an issue, there are lots of opportunities here
for speedup.

These scripts are meant mainly as hints for you to adapt to the needs
of your particular case.  But they could be used verbatim if you like.

INSTALLATION

   Have python, gnupg, and mixmaster installed.
   Have public key '94F204C28BF00937EFC85D1AFF4DB66014D0C447' on your
      gnupg keyring.
   Copy the scripts from below to somewhere on your executables path.

Of course, you are taking a full feed of a.a.m at all times without
interruption, separating wheat from chaff only after it's all behind
closed doors.  Otherwise, the world is informed about which articles
you find interesting.

Remember, long random latency is part of the price of anonymity.  It
can't be done with TOR or any other low-latency method.

FOOTNOTES

[1] http://www.mixnym.net
[2] http://sourceforge.net/projects/mixmaster/
[3] WWW OFFLine Explorer http://www.gedanken.demon.co.uk/wwwoffle/
[4] http://www.procmail.org/

CHANGE LOG

   0.07

   Use long keyid, not send<AT>mixnym.net, to identify encryption
   key, foiling fake key postings.  Contributed by Steve Crook.

   Use Python's internal encode('base64') instead of imported Base64
   to encode the random key.  Contributed by Steve Crook.

   Use Python "%"-style string substitution for type conversion and
   tidier code (yet to be finished).  Contributed by Steve Crook.

   Support hsub's ranging from 48 to 80 chars in length.  Contributed
   by Steve Crook.

   Systematically code all email addresses to avert Google corruption
   of its archived version of this posting.

   0.06

   Add time-stamps to database entries to facilitate removal of old,
   stale entries.

   Changed slogan from "Is this anonymous surfing, or what?".

   Documentation improvements.

   0.05

   Quote URL in uinmyn-do-aam-article.py against shell interpretation.

   0.04

   Did TODO item:
   Use PIPE for the gpg stdin in uinmyn-do-aam-article.py.
   (eliminating previous stupid size limit).

   0.03

   Add "decode=True" in uinmyn-to-wwwoffle-cache.py to capture coded
   files, such as .pdfs.

   Change the www.bananasplit.info reference to www.banana.mixmin.net.

   0.02

   Add a script to enter gotten pages into a local browser-accessible
   cache.

   Improvements to the documentation.

Here are the scripts.

8<--------8<------- anonget-fetch-body.py --------8<--------8<--------
#! /usr/bin/python

# SYNOPSIS
#   anonget-fetch-body.py <pending-requests-dict>

# DESCRIPTION

#   Reads URLs from standard input, one per line, generates a fresh
#   random key to encrypt the reply and for its hsub, adds the key and
#   the requested URLs to pending-requests-dict (creating it anew if
#   it doesn't exist), constructs and encrypts a request for
#   url<AT>mixnym.net, and writes it to standard output.

from os import urandom
import sys, os, time
import pprint
from subprocess import Popen, PIPE

ANONGET_PENDING_REQUESTS = sys.argv[1]
NYMSERV_KEY = '94F204C28BF00937EFC85D1AFF4DB66014D0C447'

# same key for both encryption and hsub:
key  = urandom(15).encode('base64').rstrip()

gpgcmd = 'gpg --armor --encrypt --recipient %s ' % NYMSERV_KEY
gpgopts = '--trust-model always --no-emit-version --batch'
gpgproc = Popen(gpgcmd + gpgopts, shell=True, stdin=PIPE)
gpgproc.stdin.write("KEY %s\n" % key)

urls = [time.time()] # time-tag precedes the urls in the db entry
for line in sys.stdin.readlines():
    urls.append(line.rstrip())
    gpgproc.stdin.write('SOURCE %s\n' % line.rstrip())

gpgproc.stdin.close()

# Not dealt with here: Mutually exclusive access to the
# ANONGET_PENDING_REQUESTS database, as would be required if request
# issuance might be concurrent with a.a.m. processing.

if os.path.isfile(ANONGET_PENDING_REQUESTS):
    execfile(ANONGET_PENDING_REQUESTS)
else:
    anonget_pending_requests = {}
anonget_pending_requests[key] = urls

new_upr_fd = open(ANONGET_PENDING_REQUESTS, 'w')
new_upr_fd.write('anonget_pending_requests =\\\n')
pprint.pprint(anonget_pending_requests,stream=new_upr_fd)
8<--------8<------- anonget-do-aam-article.py ----8<--------8<--------
#! /usr/bin/python

# SYNOPSIS
#   anonget-do-aam-article.py <pending-requests-dict>

# DESCRIPTION

#   Reads an article from standard input.  If its Subject: might be an
#   hsub, check it against each key in pending-requests-dict in turn.
#   If a hit is found, decrypt the inner posting and write it to
#   standard output.

import sys, os
from hashlib import sha256
import email
import pprint
from subprocess import Popen, PIPE

ANONGET_PENDING_REQUESTS = sys.argv[1]

posting = email.message_from_file(sys.stdin)

# Abort if Subject doesn't exist, otherwise get its length
if 'Subject' in posting:
    sublen = len(posting['Subject'])
else:
    # No Subject, no hSub.
    sys.exit(1)

# hSubs must fall within the length criteria
if sublen < 48 or sublen > 80:
    sys.exit(1)

try:
    iv = posting['subject'][:16].decode('hex')
except TypeError:
    sys.exit(1)

# Not dealt with here: Mutually exclusive access to the
# ANONGET_PENDING_REQUESTS database, as would be required if request
# issuance might be concurrent with a.a.m. processing.

if os.path.isfile(ANONGET_PENDING_REQUESTS):
    execfile(ANONGET_PENDING_REQUESTS)
else:
    anonget_pending_requests = {}

for key, urls in anonget_pending_requests.iteritems():
    hsub = (iv + sha256(iv + key).digest()).encode('hex')[:sublen]
    if posting['Subject'] == hsub:
        # Doing part of the job in shell:
        gpgproc = Popen('mkfifo $HOME/passwdf ; (echo ' + key + ' > $HOME/passwdf &); gpg --passphrase-fd 3 3<$HOME/passwdf --trust-model always --batch ; rm $HOME/passwdf', stdin=PIPE, stderr=PIPE, shell=True)
        gpgstderr = gpgproc.communicate(input=posting.get_payload())[1]
        if gpgstderr.find('decryption failed') < 0:
            # Other processing that could be done here:
            #  Deal with discrepancies between requested and obtained URL lists.
            #  Save the used key in an archive, rather than losing it forever.
            del anonget_pending_requests[key]
            new_upr_fd = open(ANONGET_PENDING_REQUESTS, 'w')
            new_upr_fd.write('anonget_pending_requests =\\\n')
            pprint.pprint(anonget_pending_requests,stream=new_upr_fd)
            sys.exit(0)
        else:
            # gpg says 'decryption failed'; assume it's a send confirmation.
            print 'Subject: URL request send confirmation ' + key
            print ''
            print 'Key: ' + key
            print 'Request-time: ' + str(urls[0])
            print 'URLs: ' + str(urls[1:])
            print 'gpg-stderr:'
            print gpgstderr
            sys.exit(0)
sys.exit(1)
8<--------8<--------8<------- anonget-fetch-whole.py -------8<--------
#! /usr/bin/python

# SYNOPSIS
#   anonget-fetch-whole.py <pending-requests-dict>

# DESCRIPTION

#   Alternative to anonget-fetch-body.py includes send confirmation.

#   Reads URLs from standard input, one per line, generates a fresh
#   random key to encrypt the reply and for its hsub, adds the key and
#   the requested URLs to pending-requests-dict (creating it anew if
#   it doesn't exist), constructs and encrypts a request for
#   url<AT>mixnym.net, and writes it to standard output, prepended
#   with mail headers for send confirmation.

from os import urandom
from hashlib import sha256
import sys, os, time
import pprint
from subprocess import Popen, PIPE
from datetime import datetime

ANONGET_PENDING_REQUESTS = sys.argv[1]
NYMSERV_KEY = '94F204C28BF00937EFC85D1AFF4DB66014D0C447'
HSUBLEN = 48

# same key for both encryption and hsub:
key  = urandom(15).encode('base64').rstrip()

# Mail headers
print 'To: url<AT>mixnym.net'
datestring = datetime.utcnow().strftime("%Y%m%d")
print 'Cc: mail2news-' + datestring + '-alt.anonymous.messages<AT>m2n.mixmin.net'
iv = urandom(8)                 # hsub code adapted from that of Steve Crook
hsub = (iv + sha256(iv + key).digest()).encode('hex')[:HSUBLEN]
print 'Subject: ' + hsub
print ''
sys.stdout.flush()              # so headers definitely precede body

gpgcmd = 'gpg --armor --encrypt --recipient %s ' % NYMSERV_KEY
gpgopts = '--trust-model always --no-emit-version --batch'
gpgproc = Popen(gpgcmd + gpgopts, shell=True, stdin=PIPE)
gpgproc.stdin.write("KEY %s\n" % key)

urls = [time.time()] # time-tag precedes the urls in the db entry
for line in sys.stdin.readlines():
    urls.append(line.rstrip())
    gpgproc.stdin.write('SOURCE %s\n' % line.rstrip())

gpgproc.stdin.close()

# Not dealt with here: Mutually exclusive access to the
# ANONGET_PENDING_REQUESTS database, as would be required if request
# issuance might be concurrent with a.a.m. processing.

if os.path.isfile(ANONGET_PENDING_REQUESTS):
    execfile(ANONGET_PENDING_REQUESTS)
else:
    anonget_pending_requests = {}
anonget_pending_requests[key] = urls

new_upr_fd = open(ANONGET_PENDING_REQUESTS, 'w')
new_upr_fd.write('anonget_pending_requests =\\\n')
pprint.pprint(anonget_pending_requests,stream=new_upr_fd)

gpgproc.wait()
8<--------8<--------8<------- anonget-list-gotten-urls.py --8<--------
#! /usr/bin/python

# SYNOPSIS
#   anonget-list-gotten-urls.py

# DESCRIPTION

#   List the URLs in the url<AT>mixnym.net "inner mail" on standard
#   input together with the sizes of the fetched pages.

import sys, email

pages = email.message_from_file(sys.stdin)
for page in pages.get_payload():
    print repr(len(page.get_payload())).rjust(8),page.get('Content-Description')
8<--------8<--------8<------- anonget-to-wwwoffle-cache.py -8<--------
#! /usr/bin/python

# SYNOPSIS
#   anonget-to-wwwoffle-cache.py

# DESCRIPTION

#   Given a (decrypted) url<AT>mixnym.net "inner posting", enter each
#   page that it contains into the wwwoffle cache, overlaying any
#   previous content having matching URLs.

import sys, os, email
from subprocess import Popen, PIPE

pages = email.message_from_file(sys.stdin)
for page in pages.get_payload():
    p = Popen('wwwoffle-write \'' + page.get('Content-Description') + '\'', stdin=PIPE, shell=True)
    p.stdin.write('HTTP/1.0 200 OK\n')
    p.stdin.write('Content-Type: ' + page.get('Content-Type') + '\n\n')
    p.stdin.write(page.get_payload(decode=True))
    p.stdin.close()
    p.wait()
8<--------8<--------8<--------8<--------8<--------8<--------8<--------

 -- StealthMonger <StealthMonger<AT>nym.mixmin.net>
    Long, random latency is part of the price of Internet anonymity.

Key: mailto:stealthsuite<AT>nym.mixmin.net?subject=send%20stealthmonger-key

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8+ <http://mailcrypt.sourceforge.net/>

iEYEARECAAYFAk5cSdcACgkQDkU5rhlDCl4QKwCeL4TMx0mf3zx8Qx647PVfR0cm
Vb0An0Ca73PPHc5qBddPUwG0UrPZPHnq
=H/dc
-----END PGP SIGNATURE-----