Re: Curious RNG stalemate [was: use of cpunks]
At 09:56 AM 10/17/2013, grarpamp wrote:
I'd guess that with good sources, today's prng code is sufficiently strong and at least some unix systems do save state across reboot.
Now if someone would just sell a completely open discrete logic serial port hw entropy source for under $50... that would end a lot of the talk. Even with a more costly radiation source rather than other phenomena you'd still likely make good profit in quantity from China at that price.
First of all, lots of important hardware doesn't have ports on it, particularly virtual machines, which have a whole raft of issues even if you're running them on a server you physically control rather than somebody else's cloud service. The server has some ports, but you need to make sure your hypervisor and clients have drivers that will let the client access the hypervisor's /dev/random or equivalent. VMware will have to do their own; you might contribute to OpenStack. Another important kind of hardware where that doesn't work are home routers, because the market price of $29-99 can't support much extra money for randomness hardware; if it's not in the ARM core or whatever other low-power cheap CPU, then it's only going to be able to extract entropy from timing and network traffic, and there's unlikely to be a high-precision clock chip. Maybe you can get the manufacturer to burn a pseudo-random number into the box along with the ethernet MAC or something, but otherwise it's going to have to be software. (So maybe you can augment Tomato/WRT-11/etc to listen for traffic for a while before starting, and write an app for your PC that beacons some entropy for the router to listen to?) As far as your entropy dongle goes, the only way to get it cheap is to make large volumes, which means you need a device that's intended for some other application, like a $20 TV tuner/audio frob or a webcam in a dark can getting CCD noise, or a webcam you wave at. If you want speed, you need USB, not serial, but that's fine, because almost nobody's including real serial ports these days. If you want slow, you can get a geiger counter from Sparkfun/etc. for ~$99, but you're not going to get anything intentionally radioactive shipped cheap from China.
If you want slow, you can get a geiger counter from Sparkfun/etc. for ~$99, but you're not going to get anything intentionally radioactive shipped cheap from China.
Could you hack a smoke detector? They use (IIRC) Americanium decay to ionise an air path to a detector. You might be able to use the voltage variation in that circuit with very little modification, without even opening the case perhaps (which is good, because Americanium is supposedly somewhat toxic)? Of course, then you'd have to watch out that an attacker doesn't smoke in the same room as your entropy source... ;) On Thu, 17 Oct 2013 13:45:10 -0700 Bill Stewart <bill.stewart@pobox.com> wrote:
At 09:56 AM 10/17/2013, grarpamp wrote:
I'd guess that with good sources, today's prng code is sufficiently strong and at least some unix systems do save state across reboot.
Now if someone would just sell a completely open discrete logic serial port hw entropy source for under $50... that would end a lot of the talk. Even with a more costly radiation source rather than other phenomena you'd still likely make good profit in quantity from China at that price.
First of all, lots of important hardware doesn't have ports on it, particularly virtual machines, which have a whole raft of issues even if you're running them on a server you physically control rather than somebody else's cloud service. The server has some ports, but you need to make sure your hypervisor and clients have drivers that will let the client access the hypervisor's /dev/random or equivalent. VMware will have to do their own; you might contribute to OpenStack.
Another important kind of hardware where that doesn't work are home routers, because the market price of $29-99 can't support much extra money for randomness hardware; if it's not in the ARM core or whatever other low-power cheap CPU, then it's only going to be able to extract entropy from timing and network traffic, and there's unlikely to be a high-precision clock chip. Maybe you can get the manufacturer to burn a pseudo-random number into the box along with the ethernet MAC or something, but otherwise it's going to have to be software. (So maybe you can augment Tomato/WRT-11/etc to listen for traffic for a while before starting, and write an app for your PC that beacons some entropy for the router to listen to?)
As far as your entropy dongle goes, the only way to get it cheap is to make large volumes, which means you need a device that's intended for some other application, like a $20 TV tuner/audio frob or a webcam in a dark can getting CCD noise, or a webcam you wave at. If you want speed, you need USB, not serial, but that's fine, because almost nobody's including real serial ports these days. If you want slow, you can get a geiger counter from Sparkfun/etc. for ~$99, but you're not going to get anything intentionally radioactive shipped cheap from China.
Be aware in all of this of the Heisenberg-Schödinger Credulity Effect. That effect is that the word "quantum" sucks people's brains out, and otherwise sensible people suffer from impaired reasoning. It is certainly true that radioactivity is a random effect, and is quantum in nature. That does not mean that in order for a random sampling to be quantum, it must be based on radioactivity; there are other quantum sources of randomness. Noisy diodes, resister noise, CCD noise, etc. are all quantum. If you want to get picky, *all* physical effects are quantum, even ones that aren't usefully random. There is nothing magic about one physical source or other that makes it more suited for crypto. Thinking that a hardware source should be radioactive is affirming the consequence, as well. Not does it mean that a radioactive (or other) source is suitable for cryptography without some sort of conditioning. Hardware sources are often biased in distribution, or have other numeric flaws that can be fixed with a whitening function. In short, radioactivity is neither necessary nor sufficient for cryptographic use. If you want to use a source for crypto, you want to run it through a system like /dev/random or at the very least a DRBG to give clean outputs. Furthermore, what we really want in crypto is what I call "unguessability." This is both weaker than true randomness and stronger. It's stronger in that the numbers have to remain secret. A completely random process that everyone knows is completely unsuitable for crypto, but a weakly entropic input can be jiggered into suitability. To sum up -- don't get wrapped around the axle about radioactivity. It's not the only random process in the universe, and you have to do a lot of work once you have it. The sort of work that you need to do is precisely what a well-done OSRNG does. Jon
Accepted, entirely, but if "noisy diodes" are all you need for quantum entropy, why are designs for OSHW entropy generators so scarce? I suggested smoke alarms not through radioactivity-fetishism but because of ubiquity and low cost, likely low difficulty to adapt. Jon Callas <jon@callas.org> wrote:
Be aware in all of this of the Heisenberg-Schödinger Credulity Effect. That effect is that the word "quantum" sucks people's brains out, and otherwise sensible people suffer from impaired reasoning.
It is certainly true that radioactivity is a random effect, and is quantum in nature. That does not mean that in order for a random sampling to be quantum, it must be based on radioactivity; there are other quantum sources of randomness. Noisy diodes, resister noise, CCD noise, etc. are all quantum. If you want to get picky, *all* physical effects are quantum, even ones that aren't usefully random. There is nothing magic about one physical source or other that makes it more suited for crypto. Thinking that a hardware source should be radioactive is affirming the consequence, as well.
Not does it mean that a radioactive (or other) source is suitable for cryptography without some sort of conditioning. Hardware sources are often biased in distribution, or have other numeric flaws that can be fixed with a whitening function.
In short, radioactivity is neither necessary nor sufficient for cryptographic use. If you want to use a source for crypto, you want to run it through a system like /dev/random or at the very least a DRBG to give clean outputs.
Furthermore, what we really want in crypto is what I call "unguessability." This is both weaker than true randomness and stronger. It's stronger in that the numbers have to remain secret. A completely random process that everyone knows is completely unsuitable for crypto, but a weakly entropic input can be jiggered into suitability.
To sum up -- don't get wrapped around the axle about radioactivity. It's not the only random process in the universe, and you have to do a lot of work once you have it. The sort of work that you need to do is precisely what a well-done OSRNG does.
Jon
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On Fri, Oct 18, 2013 at 08:16:51AM +0100, Cathal Garvey (Phone) wrote:
Accepted, entirely, but if "noisy diodes" are all you need for quantum entropy, why are designs for OSHW entropy generators so scarce?
Are they? http://www.maximintegrated.com/app-notes/index.mvp/id/3469 This is analog electronics 101. All you have is to sample that at sufficient rate on the cheap. That used to be a problem, but no longer is http://www.rtl-sdr.com/
I suggested smoke alarms not through radioactivity-fetishism but because of ubiquity and low cost, likely low difficulty to adapt.
We do not want a dinky little entropy drip. We want a regular firehose. The USB RTL samples at 1.4 MSamples/s. Total part costs is probably 20 USD, in bulk. Why is nobody selling a kit like that? Because worrying about sufficient entropy in crypto settings is a terribly niche thing. Sadly. Now try for a decent clock. (Hint: time-nuts. And did you know they use CSACs for IED trigger jamming?).
On Fri, Oct 18, 2013 at 12:54 AM, Eugen Leitl <eugen@leitl.org> wrote:
... We do not want a dinky little entropy drip. We want a regular firehose.
entropy firehose: 4 x 1.0Ghz Padlock engines with 2 physical noise sources per core (8 total) http://store.viatech.com/protected/product/frontProductDetail.action?id=9680 depending on MSR configuration, number of sources contributing, type of rngdaemon processing, you can achieve 8 to 400Mbps of high quality, high assurance, high density entropy. plenty of random for your off-line key generation needs, local network entropy distribution, virtio supply for guest VMs, unlimited source for full disk encryption initialization, high volume session establishment and network services, and any number of other concurrent uses. i have yet to max out a source this prolific in any reasonable (non-bench mark) scenario. this is just my favorite. there are lots of options if you look around far enough, and expend enough effort to interface with. and of course you can always build you own. there are plenty of interesting options discussed in various Cryptographic Hardware and Embedded Systems proceedings and other published texts...
On Fri, Oct 18, 2013 at 01:17:13AM -0700, coderman wrote:
On Fri, Oct 18, 2013 at 12:54 AM, Eugen Leitl <eugen@leitl.org> wrote:
... We do not want a dinky little entropy drip. We want a regular firehose.
entropy firehose: 4 x 1.0Ghz Padlock engines with 2 physical noise sources per core (8 total) http://store.viatech.com/protected/product/frontProductDetail.action?id=9680
I have a couple older VIA C3 with hardware RNG, bought long ago for just that purpose. What kind of motherboard is in there, do you know?
depending on MSR configuration, number of sources contributing, type of rngdaemon processing, you can achieve 8 to 400Mbps of high quality, high assurance, high density entropy.
It is unauditable, just as any integrated RNG sources. Which is not that big of problem, if you mix in enough external entropy from a trusted source. The trusted source need to be sufficiently simple to be validated by inspection. You can source RTL-SDRs from many sources. All you need is to match impedance and output power from your analog white noise circuit to goldilocks level. In case anyone is interested, http://www.reddit.com/r/amateurradio/comments/1mro3q/wideband_white_noise_ge...
plenty of random for your off-line key generation needs, local network entropy distribution, virtio supply for guest VMs, unlimited source for full disk encryption initialization, high volume session establishment and network services, and any number of other concurrent uses.
i have yet to max out a source this prolific in any reasonable (non-bench mark) scenario.
this is just my favorite. there are lots of options if you look around far enough, and expend enough effort to interface with.
and of course you can always build you own. there are plenty of interesting options discussed in various Cryptographic Hardware and Embedded Systems proceedings and other published texts...
But there is still no simple kit you could directly plug into your coax socket. That is a threshold of entry too high for people who can't tell which part of the soldering iron is the hot one.
On Fri, Oct 18, 2013 at 1:37 AM, Eugen Leitl <eugen@leitl.org> wrote:
... I have a couple older VIA C3 with hardware RNG, bought long ago for just that purpose.
the same XSTORE enabled kernel (/dev/hw_random|/dev/hwrandom), rngd (rng-tools pkg) will work on the models linked.
What kind of motherboard is in there, do you know?
E-series P910 Pico-ITX: http://www.viaembedded.com/en/products/boards/1950/1/EPIA-P910.html http://www.viaembedded.com/en/products/boards/1830/1/VIA_QuadCore_E-Series.h...
It is unauditable, just as any integrated RNG sources.
the XSTORE instruction can be put in truly raw sample mode, at elevated sample rate. do much more thorough userspace entropy daemon processing on the raw sources, mix to obscure state and compress for desired entropy density adjustments (some consumers are picky) among other entropy pool and CSPRNG hygiene, and you've solved your entropy contention problems! determining the proper state of a hardware raw source like this is a longer discussion, and sadly very implementation/architecture specific.
In case anyone is interested, http://www.reddit.com/r/amateurradio/comments/1mro3q/wideband_white_noise_ge...
yes, they work great. but they're not readily available, and as an entropy source they're still low bit rate. you've got at best ~8Mhz of sample with a low entropy density; effective consumer entropy production from this setup will be a fraction of this rate. the RTL2832U and Elonics E4000 are both limited this way...
But there is still no simple kit you could directly plug into your coax socket. That is a threshold of entry too high for people who can't tell which part of the soldering iron is the hot one.
no argument here. some people like the ARM TrustZone entropy sources, which are very accessible in recent gear spanning consumer space... i still don't like any TrustZone for entropy though. YMMV ah well, they keep getting smaller / faster / better ... just be patient?
On Fri, Oct 18, 2013 at 1:59 AM, coderman <coderman@gmail.com> wrote:
... the same XSTORE enabled kernel (/dev/hw_random|/dev/hwrandom), rngd (rng-tools pkg) will work on the models linked.
FYI: for best results with rng-tools you'll want to use the unofficial debian srcs: https://github.com/cernekee/rng-tools/blob/master/ChangeLog
I've found a few since, too; some I could build right away. Leading to a n00b question: if you have a custom-built HWRNG, and you *don't need the full output* but just a complementary source of entropy for /dev/random, how would one seed /dev/random with the HWRNG without washing out the good entropy already in /dev/random? That is, I gather some CSPRNGs can consider relative weights of RNG inputs, seeding the pool more often from some than others, or sanitising some inputs more than other. So my custom-built HWRNG, with possibly not-trustworthy output for crypto usages, would be a nice ancillary input to /dev/random if I could be sure it would be only used to supplement, never to replace, more proven and trustworthy sources. Bonus question; if I take the direct output of my HWRNG, and use it with a hash function and a long, random seed that is invariant, that should even out the bits of output and help account for fluctuations in true entropy, right? That is: mypassphrase = SHA512(b'some long string of high-entropy seed data') entropy = HWRNG_READ(64) entropy = SHA512(entropy, mypassphrase) seed_dev_random(entropy) On Fri, 18 Oct 2013 09:54:46 +0200 Eugen Leitl <eugen@leitl.org> wrote:
On Fri, Oct 18, 2013 at 08:16:51AM +0100, Cathal Garvey (Phone) wrote:
Accepted, entirely, but if "noisy diodes" are all you need for quantum entropy, why are designs for OSHW entropy generators so scarce?
Are they?
http://www.maximintegrated.com/app-notes/index.mvp/id/3469
This is analog electronics 101. All you have is to sample that at sufficient rate on the cheap. That used to be a problem, but no longer is
I suggested smoke alarms not through radioactivity-fetishism but because of ubiquity and low cost, likely low difficulty to adapt.
We do not want a dinky little entropy drip. We want a regular firehose. The USB RTL samples at 1.4 MSamples/s. Total part costs is probably 20 USD, in bulk.
Why is nobody selling a kit like that? Because worrying about sufficient entropy in crypto settings is a terribly niche thing. Sadly.
Now try for a decent clock. (Hint: time-nuts. And did you know they use CSACs for IED trigger jamming?).
On Fri, Oct 18, 2013 at 2:56 AM, Cathal Garvey <cathalgarvey@cathalgarvey.me> wrote:
... Leading to a n00b question: if you have a custom-built HWRNG, and you *don't need the full output* but just a complementary source of entropy for /dev/random, how would one seed /dev/random with the HWRNG without washing out the good entropy already in /dev/random?
That is, [...] would be a nice ancillary input to /dev/random if I could be sure it would be only used to supplement, never to replace, more proven and trustworthy sources.
to run in this mode your entropy daemon would read some bytes from the kernel pool via /dev/random, mix those bytes with the collected entropy, then feed back into the pool with the same entropy estimate. in this way you're not "feeding" entropy from your HWRNG but instead using it to improve the entropy collected via other sources (like high res timers, etc.) to be honest i've never considered running in this mode - the main benefit of a HWRNG is the improvement in entropy generation and this approach does not provide any such improvement.
Bonus question; if I take the direct output of my HWRNG, and use it with a hash function and a long, random seed that is invariant, that should even out the bits of output and help account for fluctuations in true entropy, right?
this is what the "mixing" or "blinding" step of an entropy daemon does: obscure the state of the HWRNG and compress the sampled entropy via cryptographically secure digests or ciphers. (there are a few different constructions for this, you should pursue the literature :) best regards,
On Oct 18, 2013, at 12:16 AM, Cathal Garvey (Phone) <cathalgarvey@cathalgarvey.me> wrote:
Accepted, entirely, but if "noisy diodes" are all you need for quantum entropy, why are designs for OSHW entropy generators so scarce? I suggested smoke alarms not through radioactivity-fetishism but because of ubiquity and low cost, likely low difficulty to adapt.
Because people think that over-the-top is necessary. Perhaps more to the point, people start gilding the lily, and then worrying about how pure the gold is on the lily, and then deciding that the gilt on the lily needs to be mono-atomic and to form a single crystal. Even more to the point, they start thinking in their heads that they will be criticized for not having a single-crystal structure on the gilt on their lily, and give up. After that, they criticize other people who grow lilies because -- heck, anyone can do that, and years ago, they gave up on lilies because of how hard it is to get mono-crystalline gilt. Go look it up in the cypherpunks archives, for pete's sake. Nicholas Bourbaki discussed it to death there back in '92. Building a good RNG is both simpler than you think and harder. You need: * An unguessability source. It doesn't have to be as good as you think it does. If it's crap, you just need more. It just has to be unguessable. The deterministic process going on on my LAN might be good enough. It might not. What matters is the work factor of guessing. Here's an example of a source I have seen that is plenty good enough, but not what most people think: - Take an array of unsigned ints; 16 or 32 in length is fine. - On every interrupt, read the "cpu clock." Lots of CPUs have something good enough. Tick counters, high-speed uptime, etc. It almost doesn't matter. - XOR that into the current array element. - Rotate that element by an odd number of bits. Mathematically, all that's required is that you pick an odd number that's relatively prime with the word size so you get maximum mixing over time intra-element. Most people can pick a suitable odd number. - Increment/wrap your element pointer. Poof, you're done. Incidentally, this is *also* a quantum process, it's just quantized fatter than other quantum processes. That's why you want to do it on every interrupt. When I first saw this, my jaw dropped at its elegance. * A pool and distiller. Hash functions are great here, as are other things. The thing to remember is only that you might get a megabyte of input that has only a single bit of unguessability in it, and you need to cope. That's why hash functions are great. Entropy estimation is highly desirable, but not necessary. This is a long discussion. It's possible to build one with no estimator that works well, and one with an estimator that works poorly. * An output function. Ciphers and hash functions are your friends. The SP 800-90 DRBGs are all designed by committee, but work (with the obvious exception). Far more important is to have your output function stir back into the pool. Something as simple as feeding back the length of the request is fine. Even better is to feed in something like a cpu tick counter. If you do something mildly reasonable here in stir-back, then you can completely forget about entropy depletion. This is another long discussion, and that's why I'm simply asserting it. There are gentlepersons who disagree with me. That's it. Yeah, the devil's in the details, and my sketch is kinda like saying oh, all you have to do to land on the moon is get some rockets and life support. But this is a lot easier than landing on the moon, despite many people thinking it's harder. And yes, yes, there are other other considerations like rebooting, suspend, hibernate, restoring VMs, and of course initial boot. Jon
On Thu, Oct 17, 2013 at 02:39:01PM -0700, Jon Callas wrote:
It is certainly true that radioactivity is a random effect, and is quantum in nature. That does not mean that in order for a random sampling to be quantum, it must be based on radioactivity; there are other quantum sources of randomness. Noisy diodes, resister noise, CCD noise, etc. are all quantum. If you want to get picky, *all* physical effects are quantum, even ones that aren't usefully random. There is nothing magic about one physical source or other that makes it more suited for crypto. Thinking that a hardware source should be radioactive is affirming the consequence, as well.
Radioactivity is almost uniquely insensitive to tampering through environmental influences, though, owing to the large energy scale of nuclear processes [1]. Unfortunately, it does not automatically follow that the circuit used to detect it is also similarly robust, and NSA would probably be able to easily develop the capability to eavesdrop on Geiger-counter based RNGs if they become widespread. A high DC voltage, and abrupt current pulses - this is sounding rather similar to a spark-gap transmitter. [1] A handful of exceptions exist involving low-energy beta decays, such as Dy-163 and Re-187. -- Andrea Shepard <andrea@persephoneslair.org> PGP fingerprint (ECC): 2D7F 0064 F6B6 7321 0844 A96D E928 4A60 4B20 2EF3 PGP fingerprint (RSA): 7895 9F53 C6D1 2AFD 6344 AF6D 35F3 6FFA CBEC CA80
On Wed, Oct 23, 2013 at 03:01:31AM -0700, Andrea Shepard wrote:
On Thu, Oct 17, 2013 at 02:39:01PM -0700, Jon Callas wrote:
It is certainly true that radioactivity is a random effect, and is quantum in nature. That does not mean that in order for a random sampling to be quantum, it must be based on radioactivity; there are other quantum sources of randomness. Noisy diodes, resister noise, CCD noise, etc. are all quantum. If you want to get picky, *all* physical effects are quantum, even ones that aren't usefully random. There is nothing magic about one physical source or other that makes it more suited for crypto. Thinking that a hardware source should be radioactive is affirming the consequence, as well.
Radioactivity is almost uniquely insensitive to tampering through environmental influences, though, owing to the large energy scale of nuclear processes [1].
I'm not at all sure "uniquely insensitive to tampering" is true against an attacker who can influence the RNGs physical environment. Suppose you're timing alpha particles, using a clock accurate to microseconds, and the attacker puts a microgram of 210-Polonium a few centimeters from your detector; you'll have an event to measure every microsecond and your detector saturates, resulting in an unending stream of 1s. A similar attack (saturating a detector which is supposed to be secure based on a "physical principle") defeats some "quantum key distribution" systems (which seem to be snake oil for the most part); for example, https://events.ccc.de/congress/2009/Fahrplan/events/3576.en.html Certainly it's possible to add complexity to the system to ensure that "everything is as it should be" and "nothing odd is going on"; this complexity negates the putatively "simple" nature of systems that are "uniquely immune" or whatever. -andy
On 2013-10-18 06:45, Bill Stewart wrote:
Another important kind of hardware where that doesn't work are home routers, because the market price of $29-99 can't support much extra money for randomness hardware; if it's not in the ARM core or whatever other low-power cheap CPU, then it's only going to be able to extract entropy from timing and network traffic,
If each router gets a secret unique 128 bit random number at software install time, this, plus the boot up time, suffices. After the router has been running a while, it gathers more randomness from network events, but a secret plus the boot up time will suffice at first. And if the router is too cheap to have a clock, so does not know the boot up time, well, pretty early in its interactions with its environment, it will be asking the time from some system that does have a clock, at which point it does have enough randomness.
participants (9)
-
Andrea Shepard
-
Andy Isaacson
-
Bill Stewart
-
Cathal Garvey
-
Cathal Garvey (Phone)
-
coderman
-
Eugen Leitl
-
James A. Donald
-
Jon Callas