Beepers can also be used to track you down!

Basically, they'd send pings to your beeper throughout the city they expected you in, then they'd find out which cell you were in. After that, they can use a small radar-like gun to actually find your beeper, also by pings.

There's probably a way to disable the transmitter in the beeper, but I

All you need is a faraday cage, an all metal enclosure with metal to metal contacts between all its surfaces. A Copper brillo type scrub pad, expanded out to make a pocket big enough for the pager.