Building Tor from Source

[Karl]

@zen i see the value of putting it in a git repo; i made some
misleading typos.  should be fill-in-able though.

On 10/11/20, Karl <gmkarl at gmail.com> wrote:
> I'm too busy [s/reverse-engineering microchips from live
> bombs/doddering around confused/] to know how to compile anything from
> its source.
>
> s/A/B/ is sed script substitution notation.  It means replace A with
> B.  I tell jokes because I'm in pain, and I'd rather share nice things
> than harm, especially if those nice things could have some important
> small part.
>
> A lot of people are working on the Tor project.  You can see what
> they're working on at https://gitweb.torproject.org/ .  I have no clue
> how to use gitweb to actually get source code, but I'm pretty good at
> guessing people's [s/social security numbers/blindingly obvious facial
> expressions three hours later/] , so I tried typing `git clone
> https://git.torproject.org/tor.git` into a terminal and what do you
> know, it worked.
>
> [Note to zfs:  I ended up using a raid1 array and git-annex
> repositories for now, now that i have an indoor place to put
> harddrives in.]
>
>
> ```
> $ ls
> acinclude.m4  ChangeLog  CODE_OF_CONDUCT  configure.ac  CONTRIBUTING
> Doxyfile.in  LICENSE  Makefile.am     README        scripts
> warning_flags.in
> autogen.sh    changes    config.rust.in   contrib       doc
> INSTALL      m4       Makefile.nmake  ReleaseNotes  src
> ```
>
> Tell me if I'm skipping too many steps and I can include a full log,
> but I won't unless asked directly.  Yes, it's boring, you're expected
> to only pretend to have read it.
>
> This is an "autotools" project.  Autotools is an ancient linux C thing
> for setting up projects.  Autotools looks like dead animal vomit on
> the inside (this means any modern coder can tell you that nobody
> should have designed it how it is obviously designed), but it is still
> very widely used because it just keeps working.  (the reason it keeps
> working is because some bleary-eyed people with beards have memorized
> all the dead vomit inside it and keep shoving it around whenever
> needed, to keep it working).
>
> We're lucky here: the tor people included an `autogen.sh` script,
> which does the autotools voodoo automatically for you.  If that script
> isn't there, often you can get away with just running `autoreconf`,
> which is basically all that autogen.sh script does.
>
> If autogen.sh doesn't work for you, you're probably missing important
> system development packages like automake, autoconf, libtool, etc.
> Please ask if you need more help.  Oh, junior, you think you're cool
> by not asking?  Junior, you're going to suffer if you don't learn what
> to do.  You won't go to college and then you'll [s/get kidnapped by a
> drug ring and forced to collect rape victims after you run away to the
> streets to pursue your dream of being a musician/disappoint the pretty
> girl we all care about/] [s/disappoint the pretty girl we all care
> about/get caught up with and protected by a bunch of genius hackers
> who never needed to go to college, and learn to take down the
> system/].  Just please ask if you don't understand and need to.  You
> can even ask privately!  You can even ask the ugly girl who wishes she
> had some way to please you!  There's one out there.
>
> Here's what it looks like:
>
> ```
> $ ./autogen.sh
> /bin/autoreconf
> configure.ac:387: installing './ar-lib'
> configure.ac:426: installing './compile'
> configure.ac:36: installing './config.guess'
> configure.ac:36: installing './config.sub'
> configure.ac:25: installing './install-sh'
> configure.ac:25: installing './missing'
> Makefile.am: installing './depcomp'
> parallel-tests: installing './test-driver'
> ```
>
> This generates a 'configure' script, the hallmark of autotools.  It's
> a computer program generated by another computer program, that
> generates parts of actual computer program we're building, in such a
> way that it has the right structures for the computer you're building
> it on.  That's a lot of automatic moving parts that build other
> automatic moving parts, and this process is often guided by further
> automatic moving parts in centralized build systems, but no it won't
> [s/mind control your employees, turn into a cloud of nanites, and
> research more wormholes/confuse you more than we can handle/] this time.
>
> Run the `configure` script it put in the folder, to set up the project
> to build.  You only need to do this once.
>
> ```
> $ ./configure
> checking for a BSD-compatible install... /bin/install -c
> checking whether build environment is sane... yes
> checking for a thread-safe mkdir -p... /bin/mkdir -p
> checking for gawk... gawk
> ...
> ```
>
> At the end, `configure` generates a makefile, which is the guts of
> building a project.  It lists all the files that will be worked with,
> what will be done to them, and what will be produced by doing that, in
> many layers of steps needed to reach the final goal of building Tor.
> It's just a bunch of computer mumbo jumbo that makes Tor exist from
> its source code.
>
> Whoops!  When you run `configure`, it looks to find out all the stuff
> you need to have on your system to make it run.  On my system, the
> dependency I was missing related to the documentation.  I like to be
> able to work with any part of the system, so I'll probably resolve
> this dependency so I know I can build the documentation:
>
> ```
> ==================================
>
> Building Tor has failed since manpages cannot be built.
>
> You need asciidoc installed to be able to build the manpages.
> To build without manpages, use the --disable-asciidoc argument
> when calling configure.
>
> ==================================
> $ # oh no!  what do I do!
> ```
>
> Unfortunately, every system installs packages differently.  But if I
> got to this point, I know how to install packages:
>
> ```
> $ sudo yum search asciidoc
> # I don't really prefer enterprise rpm-based systems, but this
> # one a confused family member purchased has my favorite
> # [s/evolved digital familiar made of malware and trojan
> # communities/system config that i never backed up/]
> [sudo] password for user:
> Loaded plugins: langpacks, product-id, search-disabled-repos,
> subscription-manager
> This system is registered with an entitlement server, but is not
> receiving updates. You can use subscription-manager to assign
> subscriptions.
> ===================================================================================
> N/S matched: asciidoc
> ====================================================================================
> asciidoc-doc.noarch : Additional documentation and examples for asciidoc
> asciidoc-latex.noarch : Support for asciidoc latex output
> rubygem-asciidoctor.noarch : A fast, open source AsciiDoc implementation in
> Ruby
> rubygem-asciidoctor-doc.noarch : Documentation for rubygem-asciidoctor
> asciidoc.noarch : Text based document generation
> $ sudo yum install asciidoc
> ```
>
> ```
> $ ./configure
> ...
>
> Tor Version: Tor 0.4.5.0-alpha-dev
>
> Build Features
>   Compiler:                                                      gcc
> -std=gnu99
>   Host OS:                                                       linux-gnu
> ...
> ```
>
> The makefile is generated!  To build the source, we type `make`, which
> loads the makefile and compiles it.  Nowadays everybody [s/expects to
> be able to teleport their brain across the ocean whenever they need to
> buy some milk at the grocery store (for real, see amazon.com)/enjoys
> things more when they are faster/], so we can add `-j4` to the make
> command, which basically speeds it up by asking the computer to get
> four times more confused while doing it (multitasking on its parts).
>
> ```
> $ make
> make  all-am
> make[1]: Entering directory `/shared/src/tor'
>   CC
> src/ext/ed25519/ref10/src_ext_ed25519_ref10_libed25519_ref10_a-fe_0.o
> ...
> ```
>
> Each line that goes by is a sourcefile that's being compiled.  A long
> list of very precise words handtyped by a dedicated worker with a
> beard or glasses or likely both, likely working in a confined indoor
> environment almost like chickens in an industrial chicken farm, but we
> love to do it!  Really!  Ask any one of us!  It gives us nastalgia of
> our earlier years.
>
> Ohhh....  My source code didn't compile.  This is pretty common.
> Nowadays [s/the corporate thugs have killed so many of our
> friends/people are so much more interested in adding features instead
> of testing things/] that everybody has to use some mysterious wizardry
> called 'docker' to copy the systems of the devs so as to make anything
> actually compile.  I'm not a fan of docker, because it ignores all the
> problems that the devs don't see because of the specifics of their
> systems, [s/producing widespread bugs that are discovered in the form
> of hacked government systems a decade later/and makes it harder for
> people to work on the software on low-end systems/].  Here's the
> arcane compilation error I ran into:
>
> ```
>   CC       src/test/src_test_test_slow-test_slow.o
> In file included from src/test/test_tortls_openssl.c:42:0:
> ./src/lib/tls/tortls_internal.h:55:8: error: conflicting types for
> ‘SSL_SESSION_get_master_key’
>  size_t SSL_SESSION_get_master_key(struct ssl_session_st *s,
>         ^
> In file included from src/test/test_tortls_openssl.c:23:0:
> /usr/local/include/openssl/ssl.h:2007:15: note: previous declaration
> of ‘SSL_SESSION_get_master_key’ was here
>  __owur size_t SSL_SESSION_get_master_key(const SSL_SESSION *sess,
>                ^
> src/test/test_tortls_openssl.c: In function ‘get_cipher_by_name’:
> src/test/test_tortls_openssl.c:541:19: error: dereferencing pointer to
> incomplete type
>    int num = method->num_ciphers();
>                    ^
> src/test/test_tortls_openssl.c:544:38: error: dereferencing pointer to
> incomplete type
>      const SSL_CIPHER *cipher = method->get_cipher(i);
>                                       ^
> src/test/test_tortls_openssl.c: In function
> ‘test_tortls_client_is_using_v2_ciphers’:
> src/test/test_tortls_openssl.c:710:3: warning: ‘TLSv1_method’ is
> deprecated (declared at /usr/local/include/openssl/ssl.h:1877)
> [-Wdeprecated-declarations]
>    ctx = SSL_CTX_new(TLSv1_method());
>    ^
> src/test/test_tortls_openssl.c:717:6: error: dereferencing pointer to
> incomplete type
>    ssl->session = sess;
>       ^
> src/test/test_tortls_openssl.c:724:6: error: dereferencing pointer to
> incomplete type
>    one->id = 0x00ff;
>       ^
> src/test/test_tortls_openssl.c:726:7: error: dereferencing pointer to
> incomplete type
>    sess->ciphers = ciphers;
> make[1]: *** [src/test/src_test_test-test_tortls_openssl.o] Error 1
> make[1]: *** Waiting for unfinished jobs....
> make[1]: Leaving directory `/shared/src/tor'
> make: *** [all] Error 2
> ```
>
> It looks like my `openssl` header file differs from the one used by
> the Tor project.
>
> There are some basic ways to approach this, the simplest of which is
> of course to use docker.  Other things one might try might include
> compiling a version of the software known to be stable, changing the
> version of my local library, or searching for the detail of the issue
> to identify its cause and give it a relevant fix.
>
> In open source, especially among the gnu crowd (which is the only
> crowd), the idea is that everybody benefits from everyone's work.
> When you do something, you want to contribute back to the community,
> so that your work has value.  For this, you get appreciation,
> friendship, collaboration and teaching, and often even lots of money
> if you know the ropes and need it!
>
> So, the right solution to my build error, is likely to provide a patch
> to the Tor project, that changes the source so that either it builds
> on people's systems with my openssl version, or the configure script
> gives an error and informs the user of what they need to change.
> Thaaaat would be in a possible further e-mail.
>
> Tata!
>