I wonder what a good assumption is about level-of-expertise. I should think rather low, since a more experienced person will not be in as much need of the doc in the first place. Think moderately-informed user:
Matt: Thanks, and good work. My comments are indicated by [your text] in brackets, my comments >>> set off by >>>'s. To help separate, look for "-----------------------------" --------------------------- snake-oil-faq ---------------------------- Snake-Oil Warning Signs Encryption Software to Avoid $Id: snake-oil-faq.html,v 0.4 1996/09/16 13:52:26 cmcurtin Exp $ Distribution Please do not distribute this beyond the circles of cryptographic competence yet. This is an incomplete work-in-progress. Feedback is greatly appreciated. The Snake Oil FAQ is (to be) posted monthly to cypherpunks, sci.crypt, alt.security, comp.security, and comp.infosystems. We're targeting those who have influence over or direct involvement in the purchasing decisions of computer security software and equipment in the corporate and academic worlds, as well as individual users who wish to assert their privacy through the use of good cryptography. ----------------------------- like the readers of InfoWorld, eh? Given that, there are several places where knowledge of the subject is assumed that the real consumer of the FAQ doesn't have. ----------------------------- Disclaimer All contributors' employers will no doubt disown any statements herein. We're not speaking for anyone but ourselves, based on our own experiences, etc., etc., etc. This is a general guideline, and as such, cannot be the sole metric by which a security product is rated, since there can be exceptions to any of these rules. -----------------------------
Actually, I think there are some rules in here that there are no exceptions to. Check; nothing comes immediately to mind tho.
----------------------------- [(But if you're looking at something that sounds familiar on several of the 'things to watch out for,' you're probably dealing with snake oil. ]
But if many of the items on the "Things to look out for" list seem to apply to a product, the product is very likely weak.
-----------------------------
From time to time, a reputable and decent vendor will produce something that is actually quite good, but will use some [braindead] marketing technique, so be aware of exceptions.
"Braindead", eh, hmmmmm. Too dignified ;)-----------------------------
Every effort has been made to produce an accurate and useful document, but the information contained herein is completely without warranty. If you find any errors, or wish to otherwise contribute, please contact the document keeper, C Matthew Curtin <cmcurtin@research.megasoft.com> Introduction Good cryptography is an excellent and necessary tool for almost anyone. However, there is a multitude of products around. Many good cryptographic products are available, both commercial (including shareware) and free. However, there are also some extremely bad cryptographic products (known in the field as "Snake Oil"), which not only fail do their job of providing security, but are based on, and add to, the many misconceptions and misunderstandings surrounding cryptography and security. -----------------------------
They also prey on the inexperience of the consumer, rely on the mystery and mystique of mathmatical-sounding jargon, to make poorly engineered products seem to be something they are not.
Superficially, it is difficult for someone to distinguish the output of a secure encryption utility from snake oil: both look garbled. The purpose of this document is >>> to <<< present some obvious "red flags" [so that] >>> which <<< people unfamiliar with the nuts and bolts of cryptography can use as a guideline for determining whether they're dealing with snake oil or the Real Thing. For a variety of reasons, this document is general in scope and does not mention specific products or algorithms as being "good" or "Snake Oil". When evaluating any product, be sure to understand what your needs are. For data security products, what do you need protected? Do you want an archiver that [supports strong encryption? ] -----------------------------
Problem: what is "Strong Encryption" ??? From a user's point of view this term is too fuzzy. Try: "that will keep data secure from your kid sister? A rogue government? For 5 minutes? Etc. etc. "
----------------------------- [An E-mail client? Something that will encrypt on-line communications? Do you want to encrypt an entire disk or partition, or selectively some files? Do you need on-the-fly (automatic) encryption and decryption, or are you willing to select when and which files you want encrypted? ]
I'd leave that out: not pertinent to snake-oil vs. Good Stuff, but is about the kind of application user requires.
How secure is "secure enough?" Does the data need to be unreadable by third parties for 5 minutes? One year? 50 years? 100 years? >>> see above. ----------------------------- ----------------------------- [Different products will serve different needs, and it's rare that a product will serve every need. (Sometimes a product won't be needed: it may be better to use a utility to encrypt files, transmit them over a network using standard file transfer tools, and decrypt them at the other end than to use a separate encrypted utility in some cases.)]
I don't understand: "sometimes a product won't be needed?" I think this paragraph could be left out. After all, OS utility or Snoop-Dooper-Doggy-Doo-Ware product, ya still gotta know what you're doing, right? So you everthing in the FAQ still applies; or maybe I'm missing the point here.
Some basics The cryptography-faq (found at ftp://rtfm.mit.edu/pub/usenet/cryptography-faq/) is a more general tutorial of cryptography, and should also be consulted. In an effort to make this FAQ more complete, some very basic topics are included below. Conventional vs. Public Key Cryptography ----------------------------- There are two basic types of cryptosystems: symmetric (also known as "conventional," [sometimes also called] >>> or <<< "private key") and asymmetric (public key). Symmetric ciphers require both the sender and the recipient to have the same key. That key is [applied]
used by the cryptographic algorithm <<<to encrypt the data >>> originated <<< by the sender, and again by the recipient to decrypt the data. Asymmetric ciphers are much more flexible, from a key management perspective. Each user has a pair of keys: a public key and a private key. The public key is shared widely, given to everyone, while the private key is kept secret. If Alice wishes to mail Bob some secrets, she simply gets Bob's public key, encrypts her message with it, and sends it off to Bob. When Bob gets the message, he uses is private key to decrypt the message.
Asymmetric [cryptosystems] >>>algorithms<<< are much slower than [their symmetric counterparts.]
symmetric algorithms, and are almost exclusively used to encrypt short "session keys," which are then used to encrypt a message using the speedier symmetric algorithms. This use of public key cryptography is called "key exchange."
----------------------------- [Also, key sizes must be much larger.]
I agree with one comment that discouraged comparing the 2 algo types.-----------------------------
See the cryptography FAQ for a more detailed discussion of [the topic.] >>>these topics.<<< Key Sizes ----------------------------- Some ciphers, while currently secure against most attacks, are not considered viable in the next few years because of relatively small keysizes and increasing processor speeds [(making a brute-force attacks feasible).]
Again, I maintain that the audience for this stuff can't be relied upon to even know what that means. Try: "which makes the cipher vulnerable to breaking by trying every possible key combination (called a brute-force attack)." --- or something like that.
The tables below should give some general guidelines for making intelligent decisions about the key length you need. If the key is too short, the system will be easily broken, even if the cipher is a good one. In [1] and [2], we're presented with some guidelines for deciding appropriate key length. (It is important to note that this is based on the ability to predict computing power 40, 65, and 100 years from now. Major breakthroughs in computing power 30 years from now might render everything on this chart kiddieplay.) Security Requirements for Different Information Type of Traffic Lifetime Minimum [Symmetric] Key Length Tactical military information minutes/hours 56-64 bits Product announcements, mergers, interest rates days/weeks 64 bits Long-term business plans years 64 bits Trade secrets (e.g., recipe for Coca-Cola) decades 112 bits H-bomb secrets >40 years 128 bits Identities of spies >50 years 128 bits Personal affairs >50 years 128 bits Diplomatic embarrassments >65 years at least 128 bits U.S. Census data 100 years at least 128 bits -----------------------------
Where is the attribution for the table?-----------------------------
As mentioned earlier, asymmetric ciphers require significantly longer keys to provide the same level of security as their symmetric cipher counterparts. Here is a comparison table, again, from Applied Cryptography, second edition. Symmetric and Public-Key Lengths With Similar Resistance to Brute-Force Attacks Symmetric Key Length Public-key Key Length 56 bits 384 bits 64 bits 512 bits 80 bits 768 bits 112 bits 1792 bits 128 bits 2304 bits -----------------------------
BEWARE, Danger: comparing public/private key cryptosystems again. I think if you really want to do so, then the comparison should really be explained.
Some Common Snake-Oil Warning Signs The following are some of the "red flags" one should watch for when examining an encryption product * Technobabble The vendor's description of the product may contain a lot of hard-to-follow use of technical terms to describe how the product works. If this appears to be confusing nonsense, it may very well be (even to someone familiar with the terminology). Technobabble is a good means of confusing a potential user and masking the fact that the vendor doesn't understand anything either. A sign of technobabble is a descrption which drops a lot of technical terms for how the system works without actually explaining how it works. Often specifically coined terms are used to describe the scheme which are not found in the literature. * New Type of Cryptography? Beware of any vendor who claims to have invented a "new type of cryptography" or a "revolutionary breakthrough". Truly "new break-throughs" are likely to show up in the >>> scientific <<< literature, and [many in the field] >>> professionals won't <<< [are unlikely to] trust them until after years of analysis, by which time they are not so new anymore. Avoid software which claims to use 'new paradigms' of computing such as cellular automata, neural nets, genetic algorithms, chaos theory, etc. Just because software uses to different method of computation doesn't make it more secure. ----------------------------- >>> As a matter of fact, these techniques are the subject of ongoing cryptographic research and nobody has published successful results based on their use yet. ----------------------------- Anything that claims to have invented a new [public key] cryptosystem without publishing the details or underlying mathematical principles is highly suspect. Modern cryptography, especially public key systems, is grounded in mathematical theory. The security is based on problems that are believed, if not known to be hard to solve. -----------------------------
There are some other comments in cpunks on this last bit. I defer.-----------------------------
The strength of any encryption scheme is only proven by the test of time, >>> involving exhaustive analysis by cryptographers<<<. New crypto is like new pharmaceuticals, not new cars. * Proprietary Algorithms Avoid software which uses "proprietary" or "secret" algorithms. Security through obscurity is not considered a safe means of protecting your data. If the vendor does not feel confident that the method used can withstand years of scrutiny by the [academic] >>> professional and academic cryptographic <<< community, then you should be wary of trusting it. (Note that a vendor who specializes in the cryptography may have a proprietary algorithm which they'll show to others if they sign a non-disclosure agreement. If the vendor is well-reputed in the field, this can be an exception.) -----------------------------
How can you tell a well-reputed vendor? I am thinging of one co. that promises to release their algo. details upon NDA, but at least in my case the details never showed up! This is slippery here!
Beware of specially modified versions of well-known algorithms. This may intentionally or unintentionally weaken the cipher. The use of a trusted algorithm, >>> availability of <<< [if not with] technical notes explaining the implementation ([if not availability of] >>> and preferably <<< the source code for the product) are a sign of good faith on the part of the vendor that you can take apart and test the implementation yourself. A common excuse for not disclosing how a program works is that "hackers might try to crack the program's security." While this may be a valid concern, it should be noted that such 'hackers' can reverse engineer the program to see how it works anyway. If the program is implemented properly and the algorithm is secure, this is not a problem. (If a hypothetical 'hacker' was able to get access you your system, access to encrypted data might be the least of your problems.) -----------------------------
Add: The strength of a cryptosystem should depend ONLY on the security of the keys involved, and not the security of the algorithm.
* Experienced Security Experts and Rave Reviews Beware of any product claiming that "experienced security experts" have analyzed it, but it won't say who (especially if the scheme has not been published in a reputable journal). Don't rely on reviews in newspapers, magazines or television shows, since they generally don't have cryptologists (celebrity hackers who know about telephone systems don't count) take the software apart for them. Just because the vendor is a well known company or the algorithm is patented doesn't make it secure either. * Unbreakability Some vendors will claim their software is "unbreakable". This is marketing hype, and a common sign of snake-oil. Avoid any vendor that makes unrealistic claims. -----------------------------
The reader is not qualified to evaluate realistic/unrealistic.-----------------------------
No algorithm is unbreakable. Even the best algorithms are breakable using "brute force" (trying every possible key), but if the key size is large enough, this is impractical even with vast amounts of computing power. One-time pads are unbreakable, but they must be implemented perfectly, which is, at best, very difficult. See the next section for a more detailed discussion. -----------------------------
Add: Avoid products that use huge numbers to impress you that it would take massive amounts of time to break them. This is ONLY true under the assumption that the only way to break the system is by exhaustively trying every possible key, and this assumption hass to be proved before the claim is valid. A cryptosystem using a keylength of 50,000 bits theoretically would take 2 raised to the 50,000th power to break (a ridiculously large number) if, AND ONLY IF the algorithm had no weaknesses. The hard part of cryptosystem design is making an algorithm with no weaknesses, such that exhaustive brute-force search is the only method of breaking it, not using long keys.
* One-Time-Pads A vendor might claim the system uses a one-time-pad (OTP), which is theoretically unbreakable. That is, snake-oil sellers will try to capitalize on the known strength of a OTP. It is important to understand that any variation in the implementation means that it is not an OTP, and has nowhere near the security of an OTP. A OTP system is not an algorithm. It works by having a "pad" of random bits in the possession of both the sender and recipient. -----------------------------
Explain what you mean by a "pad" --- using a term which to the newbie may not usually be associated with crypto. Origin being the pads of paper that they used to use etc.etc. ???
----------------------------- The message is encrypted using [the next n bits in the pad as they key, where n is the number of bits in the message]
as many bits from the key as there are bits in the message. That is, for each bit in the message, there is a random bit from the one-time-pad.<<<
After the bits are used from the pad, they're destroyed, and can never again be used. The bits in the pad must be truly random, generated using a real random source, such as specialized hardware, radioactive decay timings, etc., and not from an algorithm or cipher. Anything else is not a one-time-pad. The vendor may confuse random session keys or initialization vectors with OTPs. * Algorithm or product XXX is insecure Be wary of anything that makes claims that particular algorithms or other products are insecure without backing up those claims (or at least citing references to them). Sometimes attacks are theoretical or impractical (requiring special circumstances or massive computing power running for many years), and it's easy to confuse a layman by mentioning these. * Keys and Passwords The "key" and the "password" are often not the same thing. The "key" generally refers to the actual data used by the cipher, while the "password" refers to the word or phrase the user types in, which the software converts into the key (usually through a process called "hashing" or "key initialization").
Other comments addressed this paragraph. I defer.
The reason this is done is because the characters a user is likely to type in do not cover the full range of possible characters. (Such keys would be more redundant and easier for an attacker to guess.) By hashing a key can be made from an arbitrary password that covers the full range of possible keys. It also allows one to use longer words, or phrases and whole sentences as a "passphrase", which is more secure. Anything that restricts users' passwords to something like 10 or 16 or even 32 characters is foolish. If the actual "password" is the cipher's key (rather than hashing it into a key, as explained above), avoid it. If the vendor confuses the distinctions between bits, bytes and characters when discussing the key, avoid this product. Convenience is nice, but be wary of anything that sounds too easy to use. -----------------------------
Instead, try: be wary of any product that overly emphasizes ease-of-use without due attention to its cryptographic strength.
Avoid anything that lets anyone with your copy of the software to access files, data, etc. without having to use some sort of key or passphrase. Avoid anything that doesn't let you generate your own keys (ie, the vendor sends you a key in the mail, or it's embedded in the copy of the software you buy). Avoid anything by a vendor who does not seem to understand the difference between public-key (asymmetric) cryptography and private-key (symmetric) cryptography. * Lost keys and passwords If there's a third-party utility that can crack the software, avoid it.
Which - the utility or the crypto?
If the vendor claims it can recover lost passwords (without using a key-backup or escrow feature), avoid it. If there is a key-backup or escrow feature, are you in control of the backup, or does the vendor or someone else hold a copy of the key? * Exportable from the USA If the software is made in North America, can it be exported? If the answer is yes, chances are it's not very strong. Strong cryptography is considered munitions in terms of export from the United States, and requires approval from the State Department. Chances are if the software is exportable, the algorithm is weak or it is crackable (hence it was approved for export). If the vendor is unaware of export restrictions, avoid the software: the vendor is not familiar with the state of the art. Because of export restrictions, some legitimate (not-Snake Oil) products may have a freely exportable version for outside of the USA, which is different from a separate US/Canada-only distribution. Also note that just because software has made it outside of North America does not mean that it is exportable: sometimes a utility will be illegally exported and posted on an overseas site. Other Considerations Interface isn't everything: user-friendliness is an important factor, but if the product isn't secure then you're better off with something that is secure (if not as easy to use). No product is secure if it's not used properly. You can be the weakest link in the chain if you use a product carelessly. Do not trust any product to be foolproof, and be wary any product that claims it is. Contributors The following folks have contributed to this FAQ. Jeremey Barrett <jeremey@forequest.com> -----------------------------
OK, you can use my RealName:Gregg Weissman <geeman@best.com> <<<
Jim Ray <liberty@gate.net> Robert Rothenburg Walking-Owl <wlkngowl@unix.asb.com> References 1. B. Schneier, Applied Cryptography, second edition, John Wiley & Sons, 1996 2. M. Blaze, W. Diffie, R. L. Rivest, B. Schneier, T. Shimomura, E. Thompson, M. Wiener, "Minimal Key Lengths for Symmetric Ciphers to Provide Adequate Commercial Security," available via ftp://ftp.research.att.com/dist/mab/keylength.ps ---------------------------------------------------------------------------- C Matthew Curtin Last modified: Mon Sep 16 09:51:41 EDT ---------------------------------------------------------------------- -- C Matthew Curtin MEGASOFT, INC Chief Scientist I speak only for myself. Don't whine to anyone but me about anything I say. Hacker Security Firewall Crypto PGP Privacy Unix Perl Java Internet Intranet cmcurtin@research.megasoft.com http://research.megasoft.com/people/cmcurtin/