Building Tor from Source

11 Oct 2020

      I'm too busy [s/reverse-engineering microchips from live
bombs/doddering around confused/] to know how to compile anything from
its source.

s/A/B/ is sed script substitution notation.  It means replace A with
B.  I tell jokes because I'm in pain, and I'd rather share nice things
than harm, especially if those nice things could have some important
small part.

A lot of people are working on the Tor project.  You can see what
they're working on at https://gitweb.torproject.org/ .  I have no clue
how to use gitweb to actually get source code, but I'm pretty good at
guessing people's [s/social security numbers/blindingly obvious facial
expressions three hours later/] , so I tried typing `git clone
https://git.torproject.org/tor.git` into a terminal and what do you
know, it worked.

[Note to zfs:  I ended up using a raid1 array and git-annex
repositories for now, now that i have an indoor place to put
harddrives in.]

```
$ ls
acinclude.m4  ChangeLog  CODE_OF_CONDUCT  configure.ac  CONTRIBUTING
Doxyfile.in  LICENSE  Makefile.am     README        scripts warning_flags.in
autogen.sh    changes    config.rust.in   contrib       doc
INSTALL      m4       Makefile.nmake  ReleaseNotes  src
```

Tell me if I'm skipping too many steps and I can include a full log,
but I won't unless asked directly.  Yes, it's boring, you're expected
to only pretend to have read it.

This is an "autotools" project.  Autotools is an ancient linux C thing
for setting up projects.  Autotools looks like dead animal vomit on
the inside (this means any modern coder can tell you that nobody
should have designed it how it is obviously designed), but it is still
very widely used because it just keeps working.  (the reason it keeps
working is because some bleary-eyed people with beards have memorized
all the dead vomit inside it and keep shoving it around whenever
needed, to keep it working).

We're lucky here: the tor people included an `autogen.sh` script,
which does the autotools voodoo automatically for you.  If that script
isn't there, often you can get away with just running `autoreconf`,
which is basically all that autogen.sh script does.

If autogen.sh doesn't work for you, you're probably missing important
system development packages like automake, autoconf, libtool, etc.
Please ask if you need more help.  Oh, junior, you think you're cool
by not asking?  Junior, you're going to suffer if you don't learn what
to do.  You won't go to college and then you'll [s/get kidnapped by a
drug ring and forced to collect rape victims after you run away to the
streets to pursue your dream of being a musician/disappoint the pretty
girl we all care about/] [s/disappoint the pretty girl we all care
about/get caught up with and protected by a bunch of genius hackers
who never needed to go to college, and learn to take down the
system/].  Just please ask if you don't understand and need to.  You
can even ask privately!  You can even ask the ugly girl who wishes she
had some way to please you!  There's one out there.

Here's what it looks like:

```
$ ./autogen.sh
/bin/autoreconf
configure.ac:387: installing './ar-lib'
configure.ac:426: installing './compile'
configure.ac:36: installing './config.guess'
configure.ac:36: installing './config.sub'
configure.ac:25: installing './install-sh'
configure.ac:25: installing './missing'
Makefile.am: installing './depcomp'
parallel-tests: installing './test-driver'
```

This generates a 'configure' script, the hallmark of autotools.  It's
a computer program generated by another computer program, that
generates parts of actual computer program we're building, in such a
way that it has the right structures for the computer you're building
it on.  That's a lot of automatic moving parts that build other
automatic moving parts, and this process is often guided by further
automatic moving parts in centralized build systems, but no it won't
[s/mind control your employees, turn into a cloud of nanites, and
research more wormholes/confuse you more than we can handle/] this time.

Run the `configure` script it put in the folder, to set up the project
to build.  You only need to do this once.

```
$ ./configure
checking for a BSD-compatible install... /bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
...
```

At the end, `configure` generates a makefile, which is the guts of
building a project.  It lists all the files that will be worked with,
what will be done to them, and what will be produced by doing that, in
many layers of steps needed to reach the final goal of building Tor.
It's just a bunch of computer mumbo jumbo that makes Tor exist from
its source code.

Whoops!  When you run `configure`, it looks to find out all the stuff
you need to have on your system to make it run.  On my system, the
dependency I was missing related to the documentation.  I like to be
able to work with any part of the system, so I'll probably resolve
this dependency so I know I can build the documentation:

```
==================================

Building Tor has failed since manpages cannot be built.

You need asciidoc installed to be able to build the manpages.
To build without manpages, use the --disable-asciidoc argument
when calling configure.

==================================
$ # oh no!  what do I do!
```

Unfortunately, every system installs packages differently.  But if I
got to this point, I know how to install packages:

```
$ sudo yum search asciidoc
# I don't really prefer enterprise rpm-based systems, but this
# one a confused family member purchased has my favorite
# [s/evolved digital familiar made of malware and trojan
# communities/system config that i never backed up/]
[sudo] password for user:
Loaded plugins: langpacks, product-id, search-disabled-repos,
subscription-manager
This system is registered with an entitlement server, but is not
receiving updates. You can use subscription-manager to assign
subscriptions.
===================================================================================
N/S matched: asciidoc
====================================================================================
asciidoc-doc.noarch : Additional documentation and examples for asciidoc
asciidoc-latex.noarch : Support for asciidoc latex output
rubygem-asciidoctor.noarch : A fast, open source AsciiDoc implementation in Ruby
rubygem-asciidoctor-doc.noarch : Documentation for rubygem-asciidoctor
asciidoc.noarch : Text based document generation
$ sudo yum install asciidoc
```

```
$ ./configure
...

Tor Version: Tor 0.4.5.0-alpha-dev

Build Features
  Compiler:                                                      gcc -std=gnu99
  Host OS:                                                       linux-gnu
...
```

The makefile is generated!  To build the source, we type `make`, which
loads the makefile and compiles it.  Nowadays everybody [s/expects to
be able to teleport their brain across the ocean whenever they need to
buy some milk at the grocery store (for real, see amazon.com)/enjoys
things more when they are faster/], so we can add `-j4` to the make
command, which basically speeds it up by asking the computer to get
four times more confused while doing it (multitasking on its parts).

```
$ make
make  all-am
make[1]: Entering directory `/shared/src/tor'
  CC       src/ext/ed25519/ref10/src_ext_ed25519_ref10_libed25519_ref10_a-fe_0.o
...
```

Each line that goes by is a sourcefile that's being compiled.  A long
list of very precise words handtyped by a dedicated worker with a
beard or glasses or likely both, likely working in a confined indoor
environment almost like chickens in an industrial chicken farm, but we
love to do it!  Really!  Ask any one of us!  It gives us nastalgia of
our earlier years.

Ohhh....  My source code didn't compile.  This is pretty common.
Nowadays [s/the corporate thugs have killed so many of our
friends/people are so much more interested in adding features instead
of testing things/] that everybody has to use some mysterious wizardry
called 'docker' to copy the systems of the devs so as to make anything
actually compile.  I'm not a fan of docker, because it ignores all the
problems that the devs don't see because of the specifics of their
systems, [s/producing widespread bugs that are discovered in the form
of hacked government systems a decade later/and makes it harder for
people to work on the software on low-end systems/].  Here's the
arcane compilation error I ran into:

```
  CC       src/test/src_test_test_slow-test_slow.o
In file included from src/test/test_tortls_openssl.c:42:0:
./src/lib/tls/tortls_internal.h:55:8: error: conflicting types for
‘SSL_SESSION_get_master_key’
 size_t SSL_SESSION_get_master_key(struct ssl_session_st *s,
        ^
In file included from src/test/test_tortls_openssl.c:23:0:
/usr/local/include/openssl/ssl.h:2007:15: note: previous declaration
of ‘SSL_SESSION_get_master_key’ was here
 __owur size_t SSL_SESSION_get_master_key(const SSL_SESSION *sess,
               ^
src/test/test_tortls_openssl.c: In function ‘get_cipher_by_name’:
src/test/test_tortls_openssl.c:541:19: error: dereferencing pointer to
incomplete type
   int num = method->num_ciphers();
                   ^
src/test/test_tortls_openssl.c:544:38: error: dereferencing pointer to
incomplete type
     const SSL_CIPHER *cipher = method->get_cipher(i);
                                      ^
src/test/test_tortls_openssl.c: In function
‘test_tortls_client_is_using_v2_ciphers’:
src/test/test_tortls_openssl.c:710:3: warning: ‘TLSv1_method’ is
deprecated (declared at /usr/local/include/openssl/ssl.h:1877)
[-Wdeprecated-declarations]
   ctx = SSL_CTX_new(TLSv1_method());
   ^
src/test/test_tortls_openssl.c:717:6: error: dereferencing pointer to
incomplete type
   ssl->session = sess;
      ^
src/test/test_tortls_openssl.c:724:6: error: dereferencing pointer to
incomplete type
   one->id = 0x00ff;
      ^
src/test/test_tortls_openssl.c:726:7: error: dereferencing pointer to
incomplete type
   sess->ciphers = ciphers;
make[1]: *** [src/test/src_test_test-test_tortls_openssl.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory `/shared/src/tor'
make: *** [all] Error 2
```

It looks like my `openssl` header file differs from the one used by
the Tor project.

There are some basic ways to approach this, the simplest of which is
of course to use docker.  Other things one might try might include
compiling a version of the software known to be stable, changing the
version of my local library, or searching for the detail of the issue
to identify its cause and give it a relevant fix.

In open source, especially among the gnu crowd (which is the only
crowd), the idea is that everybody benefits from everyone's work.
When you do something, you want to contribute back to the community,
so that your work has value.  For this, you get appreciation,
friendship, collaboration and teaching, and often even lots of money
if you know the ropes and need it!

So, the right solution to my build error, is likely to provide a patch
to the Tor project, that changes the source so that either it builds
on people's systems with my openssl version, or the configure script
gives an error and informs the user of what they need to change.
Thaaaat would be in a possible further e-mail.

Tata!

Karl

Karl

coderman

Karl

coderman

Karl

grarpamp

Karl

tags

participants (3)