Pretty Curved Privacy.. ECC Curve p25519 util(Bernstein approved curve)
From the README...
DESCRIPTION Pretty Curved Privacy (pcp1) is a commandline utility which can be used to encrypt files. pcp1 uses eliptc curve cryptography for encryption (CURVE25519 by Dan J. Bernstein). While CURVE25519 is no worldwide accepted standard it hasn't been compromised by the NSA - which might be better, depending on your point of view. Caution: since CURVE25519 is no accepted standard, pcp1 has to be considered as experimental software. In fact, I wrote it just to learn about the curve and see how it works. Beside some differences it works like GNUPG. So, if you already know how to use gpg, you'll feel almost home. QUICKSTART Lets say, Alicia and Bobby want to exchange encrypted messages. Here's what the've got to do. First, both have create a secret key: Alicia Bobby pcp1 -k pcp1 -k After entering their name, email address and a passphrase to protect the key, it will be stored in their vault file (by default ~/.pcpvault). Now, both of them have to export the public key, which has to be imported by the other one. With pcp you can export the public part of your primary key, but the better solution is to export a derived public key especially for the recipient: Alicia Bobby pcp1 -p -r Bobby -O alicia.pub pcp1 -p -r Alicia -O bobby.pub They've to exchange the public key somehow (which is not my problem at the moment, use ssh, encrypted mail, whatever). Once exchanged, they have to import it: Alicia Bobby pcp1 -P -I bobby.pub pcp1 -P -I alicia.pub They will see a response as this when done: key 0x29A323A2C295D391 added to .pcpvault. Now, Alicia finally writes the secret message, encrypts it and sends it to Bobby, who in turn decrypts it: Alicia Bobby echo "Love you, honey" > letter pcp1 -e -i 0x29A323A2C295D391 -I letter -O letter.z85 cat letter.z85 | mail bobby@foo.bar pcp1 -d -I letter.z85 | less And that's it. Please note the big difference to GPG though: both Alicia AND Bobby have to enter the passphrase for their secret key! That's the way CURVE25519 works: you encrypt a message using your secret key and the recipients public key and the recipient does the opposite, he uses his secret key and your public key to actually decrypt the message. Oh - and if you're wondering why I named them Alicia and Bobby: I was just sick of Alice and Bob. We're running NSA-free, so we're using other sample names as well. INSTALLATION There are currently no packages available, so pcp has to be compiled from source. Follow these steps: First, you will need libsodium: git clone git://github.com/jedisct1/libsodium.git cd libsodium ./autogen.sh ./configure && make check sudo make install sudo ldconfig cd .. Next, pcp: git clone git://github.com/tlinden/pcp.git cd pcp ./configure sudo make install cd .. Optionally, you might run the unit tests: make test DOCUMENTATION To learn how to use pcp, read the manpage: man pcp1 7. Licensed under the GNU GENERAL PUBLIC LICENSE version 3. HOME The homepage of Pretty Curved Privacy can be found on http://www.daemon.de/PrettyCurvedPrivacy. The source is on Github: https://github.com/TLINDEN/pcp -- Tentacle #99 ecc public key curve p25519(pcp 0.15) 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 Governments are instituted among men, deriving their just powers from the consent of the governed, that whenever any form of government becomes destructive of these ends, it is the right of the people to alter or abolish it, and to institute new government, laying its foundation on such principles, and organizing its powers in such form, as to them shall seem most likely to effect their safety and happiness.’ https://github.com/TLINDEN/pcp.git to get pcp(curve25519 cli)
On Fri, Jan 10, 2014 at 04:13:29AM -0800, gwen hastings wrote:
From the README...
DESCRIPTION
Pretty Curved Privacy (pcp1) is a commandline utility which can be used to encrypt files. pcp1 uses eliptc curve cryptography for encryption (CURVE25519 by Dan J. Bernstein). While CURVE25519 is no worldwide accepted standard it hasn't been compromised by the NSA - which might be better, depending on your point of view.
sounds like my pbp: https://github.com/stef/pbp except with a 'c' and less features, from the readme: v0.2 - experimental PBP[0] is a simple python wrapper and a command line interface around libsodium, to provide basic functionality resembling PGP. It uses scrypt for a KDF and a much simpler packet format, which should be much harder to fingerprint, pbp also provides an experimental forward secrecy mode and a multi-party DH mode. Installation pip install pbp you possibly need to run (or an equivalent command) sudo apt-get install git python-virtualenv gcc python-dev libffi-dev to satisfy all basic dependencies. Design goals: 1. use modern crypto 2. provide similar functionality to PGP 3. be extensible 4. difficult to identify based on fingerprinting 5. provide extensive testing 6. strive for security Crypto Cryptographic primitives are based on the NaCl library from http://nacl.cr.yp.to. The KDF used is scrypt. PGP-like Provides basic public key encrypt/decrypt, sign/verify and secret key encrypt/decrypt modes, as well as the ability to sign, verify, list, generate, export and import keys. Extensibility using pbp and the underlying pysodium[1] library it's easy to extend pbp. Some examples are the experimental forward secrecy mode (see description in docs/chaining-dh.txt), the support for ECDH key exchanges from the command-line and generation of arbitrarily large random byte streams. [1] https://github.com/stef/pysodium also available on https://pypi.python.org/pypi/pysodium Fingerprinting pbp tries to avoid to store any sensitive plaintext info, the encrypted files all should look like random noise. for a description of the packet formats see docs/fileformats.txt. Testing All py files come with their internal tests, unit tests are in tests.py, and commandline functionality is tested in test.sh. Security pbp locks the process memory, so it cannot be swapped to disk. Also pbp uses SecureString[2] to overwrite sensitive key material after usage in memory, so keys have a short window of opportunity to leak. [2] https://github.com/dnet/pysecstr Usage Generate a key pbp -g -n alice sending howdy.txt using public key encryption from alice to bob pbp -c -S alice -r bob -i howdy.txt decrypt an encrypted file using public key crypto pbp -d -S bob -i howdy.txt.pbp sending howdy.txt using secret key encryption pbp -c -i howdy.txt decrypt an encrypted file using secret key crypto pbp -d -i howdy.txt.pbp sign howdy.txt pbp -s -S alice -i /howdy.txt verify howdy.txt pbp -v -i howdy.txt.sig sign bobs key pbp -m -S alice -n bob check sigs on carols key pbp -C -n carol alice encrypts howdy.txt to bob using experimental forward secret mode pbp -e -S alice -r bob -i howdy.txt -o ./secret-message bob decrypts howdy.txt from alice using experimental forward secret mode pbp -E -S bob -r alice -i ./secret-message initiate ECDH key exchange pbp -D1 respond to ECDH key exchange pbp -D2 -Dp 'public component from D1' finish ECDH key exchange pbp -D3 -Dp 'public component from D2' -De 'secret exponent from D1' random streaming 23GByte of cryptographic randomness pbp -R -Rs 23G -o /mnt/huge_fs/random_data participate in a 4-way DH exchange, 1st message pbp -Ds -Dp 4 -S alice -n 'friends001' -i oldkeychain -o newkeychain participate in a 4-way DH exchange, 2nd message pbp -De -S alice -n 'friends001' -i oldkeychain -o newkeychain this is one big pipe that creates a 3-way ECDH secret between alice, bob and carol: pbp -Ds -S alice -Dp 3 -n 'test-dh' -i /dev/null | pbp -Ds -S bob -Dp 3 -n 'test-dh' | pbp -Ds -S carol -Dp 3 -n 'test-dh' | pbp -De -S alice -Dp 3 -n 'test-dh' | pbp -De -S bob -Dp 3 -n 'test-dh' of course instead of a pipe you could use any kind of transport mechanism (c) 2013, stf <s@ctrlc.hu>, dnet vsza@vsza.hu, AGPLv3.0+ [0] also it's very funny to say pbp with a mouth full of dry cookies. don't try this in company! -- pgp: https://www.ctrlc.hu/~stef/stef.gpg pgp fp: FD52 DABD 5224 7F9C 63C6 3C12 FC97 D29F CA05 57EF otr fp: https://www.ctrlc.hu/~stef/otr.txt
Thanks Stef!! Great we have another alternative for those that prefer curve p25519 implementations in python. And for now I will be testing pcp as I prefer conventionally compiled and linked code for crypto I use in C/C++. I will have to see about static linking for pcp :) More Contenders for curve 25519 replacements to gnupg and openpgp crypto? All the best and Thanx for making us aware of PBP GH ps can you see about compatibility testing PBP with enigmail/thunderbird? On 1/10/14 4:21 AM, stef wrote:
On Fri, Jan 10, 2014 at 04:13:29AM -0800, gwen hastings wrote:
From the README...
DESCRIPTION
Pretty Curved Privacy (pcp1) is a commandline utility which can be used to encrypt files. pcp1 uses eliptc curve cryptography for encryption (CURVE25519 by Dan J. Bernstein). While CURVE25519 is no worldwide accepted standard it hasn't been compromised by the NSA - which might be better, depending on your point of view.
sounds like my pbp: https://github.com/stef/pbp except with a 'c' and less features, from the readme:
v0.2 - experimental
PBP[0] is a simple python wrapper and a command line interface around libsodium, to provide basic functionality resembling PGP. It uses scrypt for a KDF and a much simpler packet format, which should be much harder to fingerprint, pbp also provides an experimental forward secrecy mode and a multi-party DH mode.
Installation
pip install pbp
you possibly need to run (or an equivalent command) sudo apt-get install git python-virtualenv gcc python-dev libffi-dev to satisfy all basic dependencies.
Design goals:
1. use modern crypto 2. provide similar functionality to PGP 3. be extensible 4. difficult to identify based on fingerprinting 5. provide extensive testing 6. strive for security
Crypto
Cryptographic primitives are based on the NaCl library from http://nacl.cr.yp.to. The KDF used is scrypt.
PGP-like
Provides basic public key encrypt/decrypt, sign/verify and secret key encrypt/decrypt modes, as well as the ability to sign, verify, list, generate, export and import keys.
Extensibility
using pbp and the underlying pysodium[1] library it's easy to extend pbp. Some examples are the experimental forward secrecy mode (see description in docs/chaining-dh.txt), the support for ECDH key exchanges from the command-line and generation of arbitrarily large random byte streams.
[1] https://github.com/stef/pysodium also available on https://pypi.python.org/pypi/pysodium
Fingerprinting
pbp tries to avoid to store any sensitive plaintext info, the encrypted files all should look like random noise. for a description of the packet formats see docs/fileformats.txt.
Testing
All py files come with their internal tests, unit tests are in tests.py, and commandline functionality is tested in test.sh.
Security
pbp locks the process memory, so it cannot be swapped to disk. Also pbp uses SecureString[2] to overwrite sensitive key material after usage in memory, so keys have a short window of opportunity to leak.
[2] https://github.com/dnet/pysecstr
Usage
Generate a key
pbp -g -n alice
sending howdy.txt using public key encryption from alice to bob
pbp -c -S alice -r bob -i howdy.txt
decrypt an encrypted file using public key crypto
pbp -d -S bob -i howdy.txt.pbp
sending howdy.txt using secret key encryption
pbp -c -i howdy.txt
decrypt an encrypted file using secret key crypto
pbp -d -i howdy.txt.pbp
sign howdy.txt
pbp -s -S alice -i /howdy.txt
verify howdy.txt
pbp -v -i howdy.txt.sig
sign bobs key
pbp -m -S alice -n bob
check sigs on carols key
pbp -C -n carol
alice encrypts howdy.txt to bob using experimental forward secret mode
pbp -e -S alice -r bob -i howdy.txt -o ./secret-message
bob decrypts howdy.txt from alice using experimental forward secret mode
pbp -E -S bob -r alice -i ./secret-message
initiate ECDH key exchange
pbp -D1
respond to ECDH key exchange
pbp -D2 -Dp 'public component from D1'
finish ECDH key exchange
pbp -D3 -Dp 'public component from D2' -De 'secret exponent from D1'
random streaming 23GByte of cryptographic randomness
pbp -R -Rs 23G -o /mnt/huge_fs/random_data
participate in a 4-way DH exchange, 1st message
pbp -Ds -Dp 4 -S alice -n 'friends001' -i oldkeychain -o newkeychain
participate in a 4-way DH exchange, 2nd message
pbp -De -S alice -n 'friends001' -i oldkeychain -o newkeychain
this is one big pipe that creates a 3-way ECDH secret between alice, bob and carol:
pbp -Ds -S alice -Dp 3 -n 'test-dh' -i /dev/null | pbp -Ds -S bob -Dp 3 -n 'test-dh' | pbp -Ds -S carol -Dp 3 -n 'test-dh' | pbp -De -S alice -Dp 3 -n 'test-dh' | pbp -De -S bob -Dp 3 -n 'test-dh'
of course instead of a pipe you could use any kind of transport mechanism
(c) 2013, stf <s@ctrlc.hu>, dnet vsza@vsza.hu, AGPLv3.0+
[0] also it's very funny to say pbp with a mouth full of dry cookies. don't try this in company!
-- Tentacle #99 ecc public key curve p25519(pcp 0.15) 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 Governments are instituted among men, deriving their just powers from the consent of the governed, that whenever any form of government becomes destructive of these ends, it is the right of the people to alter or abolish it, and to institute new government, laying its foundation on such principles, and organizing its powers in such form, as to them shall seem most likely to effect their safety and happiness.’ https://github.com/TLINDEN/pcp.git to get pcp(curve25519 cli)
On Fri, Jan 10, 2014 at 04:49:43AM -0800, gwen hastings wrote:
Thanks Stef!! And for now I will be testing pcp as I prefer conventionally compiled and linked code for crypto I use in C/C++.
sure, i prefer python as it precludes all the language class vulnerabilities and leaves us only with application class vulns. but in hw i do this in c of course. expect an accompanying hw device doing pbp stuff soon.
ps can you see about compatibility testing PBP with enigmail/thunderbird?
never tried, on the commandline it's a bit incompatible yet with pgp commandline args, but can be easily adapted. -- pgp: https://www.ctrlc.hu/~stef/stef.gpg pgp fp: FD52 DABD 5224 7F9C 63C6 3C12 FC97 D29F CA05 57EF otr fp: https://www.ctrlc.hu/~stef/otr.txt
Hi Stef, it seems its gnupg command line options you would want to emulate.. pgp left the command line abandoned sometime ago..:( GH On 1/10/14 4:52 AM, stef wrote:
On Fri, Jan 10, 2014 at 04:49:43AM -0800, gwen hastings wrote:
Thanks Stef!! And for now I will be testing pcp as I prefer conventionally compiled and linked code for crypto I use in C/C++.
sure, i prefer python as it precludes all the language class vulnerabilities and leaves us only with application class vulns. but in hw i do this in c of course. expect an accompanying hw device doing pbp stuff soon.
ps can you see about compatibility testing PBP with enigmail/thunderbird?
never tried, on the commandline it's a bit incompatible yet with pgp commandline args, but can be easily adapted.
-- Tentacle #99 ecc public key curve p25519(pcp 0.15) 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 Governments are instituted among men, deriving their just powers from the consent of the governed, that whenever any form of government becomes destructive of these ends, it is the right of the people to alter or abolish it, and to institute new government, laying its foundation on such principles, and organizing its powers in such form, as to them shall seem most likely to effect their safety and happiness.’ https://github.com/TLINDEN/pcp.git to get pcp(curve25519 cli)
Hi, Dnia piątek, 10 stycznia 2014 04:49:43 gwen hastings pisze:
Great we have another alternative for those that prefer curve p25519 implementations in python.
Indeed. But my question is: are these implementations compatible? As in, can you guys encrypt and sign messages to each other using your respective implementations, and still be able to communicate? Please let it be a "yes". -- Pozdr rysiek
That seems to be a no for now.. I have been testing both and after some initial brouhaha about a __pycache__ for pysodium have both working I will have some observations later for the authors... GH On 1/10/14 6:12 AM, rysiek wrote:
Hi,
Dnia piątek, 10 stycznia 2014 04:49:43 gwen hastings pisze:
Great we have another alternative for those that prefer curve p25519 implementations in python.
Indeed. But my question is: are these implementations compatible? As in, can you guys encrypt and sign messages to each other using your respective implementations, and still be able to communicate?
Please let it be a "yes".
-- Tentacle #99 ecc public key curve p25519(pcp 0.15) 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 Governments are instituted among men, deriving their just powers from the consent of the governed, that whenever any form of government becomes destructive of these ends, it is the right of the people to alter or abolish it, and to institute new government, laying its foundation on such principles, and organizing its powers in such form, as to them shall seem most likely to effect their safety and happiness.’ https://github.com/TLINDEN/pcp.git to get pcp(curve25519 cli)
On Fri, Jan 10, 2014 at 06:43:48AM -0800, gwen hastings wrote:
That seems to be a no for now..
most of the basic stuff should be pretty easily convertible. like your public key needs binary conversion and can be fed into any nacl/sodium crypto_box right? -- pgp: https://www.ctrlc.hu/~stef/stef.gpg pgp fp: FD52 DABD 5224 7F9C 63C6 3C12 FC97 D29F CA05 57EF otr fp: https://www.ctrlc.hu/~stef/otr.txt
Hi Stef, Consider this an invitation to you and Tlinden to converge on file formats and key representations in both implementation(s) if this is technically possible. It would give folks a choice at least. regards gh ps tlindens github loc at the end of sig On 1/10/14 7:02 AM, stef wrote:
On Fri, Jan 10, 2014 at 06:43:48AM -0800, gwen hastings wrote:
That seems to be a no for now..
most of the basic stuff should be pretty easily convertible. like your public key needs binary conversion and can be fed into any nacl/sodium crypto_box right?
-- Tentacle #99 ecc public key curve p25519(pcp 0.15) 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 Governments are instituted among men, deriving their just powers from the consent of the governed, that whenever any form of government becomes destructive of these ends, it is the right of the people to alter or abolish it, and to institute new government, laying its foundation on such principles, and organizing its powers in such form, as to them shall seem most likely to effect their safety and happiness.’ https://github.com/TLINDEN/pcp.git to get pcp(curve25519 cli)
Dnia piątek, 10 stycznia 2014 09:35:32 gwen hastings pisze:
Hi Stef, Consider this an invitation to you and Tlinden to converge on file formats and key representations in both implementation(s) if this is technically possible.
It would give folks a choice at least.
More importantly, it would remove the "so which incompatible implementation should I choose? naah, I'll stick to whatever I am using now" problem. Which is always underestimated, always important. -- Pozdr rysiek
On Fri, Jan 10, 2014 at 08:46:17PM +0100, rysiek wrote:
Dnia piątek, 10 stycznia 2014 09:35:32 gwen hastings pisze:
Hi Stef, Consider this an invitation to you and Tlinden to converge on file formats and key representations in both implementation(s) if this is technically possible.
It would give folks a choice at least.
More importantly, it would remove the "so which incompatible implementation should I choose? naah, I'll stick to whatever I am using now" problem. Which is always underestimated, always important.
given the fact that pbp is based on python the entry barrier should not be very high to go from consulting to contributing ;) -- pgp: https://www.ctrlc.hu/~stef/stef.gpg pgp fp: FD52 DABD 5224 7F9C 63C6 3C12 FC97 D29F CA05 57EF otr fp: https://www.ctrlc.hu/~stef/otr.txt
consistent key formats are critical, need to converge on: - endianness - coordinate representation x, x&y, x and sign … or bits to show which of these …. perhaps borrow ANSI method - hint / indication of cipher suite / curve - text encoding of binary format (ascii) - text encoding of binary format (utf8) - human readable format > ecc public key curve p25519(pcp 0.15) leaking crypto suite key should be usable in other contexts besides pcp 0.15 > 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 one of several possible text encodings Others might include: - base 29 - base 59 - base 4096 (for UTF8 channels)
On 2014-01-11 16:27, nymble wrote:
- base 4096 (for UTF8 channels)
How should base 4096 be expressed in UTF8, what does it look like to humans, and what happens to it when one copies and pastes it? How efficient is it compared to base 60 (upper case, lower case, and numbers, with 0/O and 1/l canonicalized)
On Jan 10, 2014, at 11:29 PM, James A. Donald <jamesd@echeque.com> wrote:
On 2014-01-11 16:27, nymble wrote:
- base 4096 (for UTF8 channels)
How should base 4096 be expressed in UTF8, what does it look like to humans, and what happens to it when one copies and pastes it? Works really well in Twitter. Likely well in Facebook and on web pages.
Looks like Hangul or Chinese depending on starting point in code space. 갇됰꼊꽦덐묻렐딉뇧뎶궄뚴뚮덺묝꿋륻긺경
How efficient is it compared to base 60 (upper case, lower case, and numbers, with 0/O and 1/l canonicalized) Base 60 is better for ASCII channels … plain email and human transcription.
On Fri, Jan 10, 2014 at 10:27:41PM -0800, nymble wrote: > > consistent key formats are critical, need to converge on: > - endianness then pls also make hw converge on endianness. > - coordinate representation x, x&y, x and sign … > or bits to show which of these …. perhaps borrow ANSI method makes no sense in case of nacl/sodium. it is const. > - hint / indication of cipher suite / curve makes no sense in case of nacl/sodium. it is const. > - text encoding of binary format (ascii) > - text encoding of binary format (utf8) > - human readable format easily convertible. > > ecc public key curve p25519(pcp 0.15) > leaking crypto suite > key should be usable in other contexts besides pcp 0.15 > > > > 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 > one of several possible text encodings > Others might include: > - base 29 > - base 59 > - base 4096 (for UTF8 channels) i like base85. ;) diversity! -- pgp: https://www.ctrlc.hu/~stef/stef.gpg pgp fp: FD52 DABD 5224 7F9C 63C6 3C12 FC97 D29F CA05 57EF otr fp: https://www.ctrlc.hu/~stef/otr.txt
Bitcoin base58 seemed a to have some minor unfortunate side effects to me, the intent is good to avoid transcription error, but surely one could find 64-chars. it could have easily been base 60 to start with (dont delete both 0 and O, and 1 and l just make the equivalent!). Then you have URL encoding ambiguity, C/python/bash programming string quoting that rules out some more non alphanum chars. (base 64 includes +/). Just seems some ugly code mess and implications for vanity address etc to deal with non-power-of-2 encoding. Adam On Sat, Jan 11, 2014 at 10:58:48AM +0100, stef wrote: >> > 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 >> one of several possible text encodings >> Others might include: >> - base 29 >> - base 59 >> - base 4096 (for UTF8 channels) > >i like base85. ;) > >diversity!
Base58 presumably included the "not mangleable by intermediate servers that only speak ascii" design constraint. Adam Back <adam@cypherspace.org> wrote: >Bitcoin base58 seemed a to have some minor unfortunate side effects to >me, >the intent is good to avoid transcription error, but surely one could >find >64-chars. it could have easily been base 60 to start with (dont delete >both >0 and O, and 1 and l just make the equivalent!). Then you have URL >encoding >ambiguity, C/python/bash programming string quoting that rules out some >more >non alphanum chars. (base 64 includes +/). Just seems some ugly code >mess >and implications for vanity address etc to deal with non-power-of-2 >encoding. > >Adam > >On Sat, Jan 11, 2014 at 10:58:48AM +0100, stef wrote: >>> > 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 >>> one of several possible text encodings >>> Others might include: >>> - base 29 >>> - base 59 >>> - base 4096 (for UTF8 channels) >> >>i like base85. ;) >> >>diversity! -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On Jan 11, 2014, at 5:47 AM, Adam Back <adam@cypherspace.org> wrote:
> Bitcoin base58 seemed a to have some minor unfortunate side effects to me,
> the intent is good to avoid transcription error, but surely one could find
> 64-chars. it could have easily been base 60 to start with (dont delete both
> 0 and O, and 1 and l just make the equivalent!).
Possible, but breaks string compare. Also adds human confusion in interpretation/typing.
> Then you have URL encoding
> ambiguity,
oh … yes. same point
> C/python/bash programming string quoting that rules out some more
> non alphanum chars. (base 64 includes +/). Just seems some ugly code mess
> and implications for vanity address etc to deal with non-power-of-2
> encoding.
Yes … code is very ugly, Human usability is more important ...
>
> Adam
>
> On Sat, Jan 11, 2014 at 10:58:48AM +0100, stef wrote:
>>> > 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910
>>> one of several possible text encodings
>>> Others might include:
>>> - base 29
>>> - base 59
oops intended base 58
>>> - base 4096 (for UTF8 channels)
base64: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/“
compact, nice power of 2, human transcription errors likely
base58: “123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz”
fairly readable, relatively compact
base29: “ABCDEFGHJKMNPQRTUVWXYZ2346789"
base 29 assumes lower upper case equivalence, always converts to upper for decode
(removes 5,S 0,O i,1,i,I )
base 29 has the best human usability/readability and is not mangled
very good for license keys and short sequences
… 1/2 of efficiency (losing lower versus upper)
>> i like base85. ;)
oh - thanks, assume you mean RFC1924
base85: “0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz!#$%&()*+-;<=>?@^_`{|}~”
nice shorter encoding. Not URL or human friendly, works well for email cut/paste ‘<‘ might be problematic for web usage.
Paul
>>
>> diversity!
> - base 4096 (for UTF8 channels) I may reveal some crippling ignorance, but: UTF8 is an encoding system to allow for effectively infinite character extensions in binary text data. The original forms of binary encoding for text were merely to assign characters to a large segment of the 264 possibilities of a single byte; 2^8. To account for the fact that in early data transports many of these codepoints were considered instructions and could therefore inject transport-specific commands etc. (if I understand the problem correctly)*, base64/base32 were intended to allow arbitrary binary to be encoded into a transport that accepted text without including codepoints likely to have control significance. Amusingly, the base32/64 alphabets are restricted further to remove characters that might, if accidentally rearranged, cause people to see naughty words in binary. When it comes down to it, there are only 2^8 possibilities in binary. UTF8's extensions are indicated by additional byte sequences that indicate "the following bytes should be viewed as an extension". I'm not sure how many "ensuing" bytes can be regarded as an extended encoding at a time, but I think it's only in the range of 1-3. If we assume it's 3, and further assume that, after declaring that the following 3 bytes are an extension, that any arbitrary binary sequence will be interpreted as a visible, copy/pasteable character, then you're looking at a length penalty to encode arbitrary data of 33%. For every three bytes, you're escaping them to random characters by prefixing with another byte. Yes, it's more nuanced than that; you can factor in the ascii set and use that where possible, only escaping binary values outside the ascii set, but one way or another you're adding length to the binary string by messing with it, with the aim being a character-representable set of binary data that can be copy/pasted safely and passed through diverse transports. So the question is what's more important; ability to transport strings of data without a significant length penalty, ability to transport strings of arbitrary data without affecting the transport, or ability to copy/paste (a subset of "transport" I guess). Given these, my personal feeling is that if your concern is transport-related, which implies that you can't control the transport, then stick with base64. If your concern is length, then I don't feel UTF8 will offer a significant advantage, and you're much better off using something like length-prefixing like bencoding does it. If your concern is copy-pasteability, then base58 works and probably is no worse than base-utf8, while being significantly easier to implement in code. Spurious rant over. * Take for example the way early email was sent, where headers were specified and then the server awaited the body of the message, the end of which was indicated by what amounts to a string of characters; a newline, a period, and another newline. Easily injected by accident or design, along with other commands. On 11/01/14 06:27, nymble wrote: > > consistent key formats are critical, need to converge on: > - endianness > - coordinate representation x, x&y, x and sign … > or bits to show which of these …. perhaps borrow ANSI method > - hint / indication of cipher suite / curve > - text encoding of binary format (ascii) > - text encoding of binary format (utf8) > - human readable format > >> ecc public key curve p25519(pcp 0.15) > leaking crypto suite > key should be usable in other contexts besides pcp 0.15 > > >> 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 > one of several possible text encodings > Others might include: > - base 29 > - base 59 > - base 4096 (for UTF8 channels) > > >
--On Saturday, January 11, 2014 4:02 PM +0000 Cathal Garvey <cathalgarvey@cathalgarvey.me> wrote:
segment of the 264 possibilities of a single byte; 2^8.
How did you manage to come up with that bug/typo? =P
_<
On 11/01/14 20:32, Juan Garofalo wrote:
--On Saturday, January 11, 2014 4:02 PM +0000 Cathal Garvey <cathalgarvey@cathalgarvey.me> wrote:
segment of the 264 possibilities of a single byte; 2^8.
How did you manage to come up with that bug/typo? =P
On Jan 11, 2014, at 8:02 AM, Cathal Garvey <cathalgarvey@cathalgarvey.me> wrote: >> - base 4096 (for UTF8 channels) > > I may reveal some crippling ignorance, but: No … but it depends whats being counted. ... > If we assume it's 3, and further assume that, after declaring that the > following 3 bytes are an extension, that any arbitrary binary sequence > will be interpreted as a visible, copy/pasteable character, then you're > looking at a length penalty to encode arbitrary data of 33%. For every > three bytes, you're escaping them to random characters by prefixing with > another byte. Twitter limits transmissions to 140 UTF8 characters. More than 140 binary characters can be encoded into 140 UTF 8 characters. In any UTF8 capabile browser/mail/whatever - the apparent length of a binary string will appear shorter even though the underlying bytes are longer. That said … such encoding tricks (base4096 or the like) might be better saved for encrypted content in Twiiter or other UTF8 limited channels. Public key representation would need to be consistent over other ASCII limited channels. > > Yes, it's more nuanced than that; you can factor in the ascii set and > use that where possible, only escaping binary values outside the ascii > set, but one way or another you're adding length to the binary string by > messing with it, with the aim being a character-representable set of > binary data that can be copy/pasted safely and passed through diverse > transports. > > So the question is what's more important; ability to transport strings > of data without a significant length penalty, ability to transport > strings of arbitrary data without affecting the transport, or ability to > copy/paste (a subset of "transport" I guess). > > Given these, my personal feeling is that if your concern is > transport-related, which implies that you can't control the transport, > then stick with base64. If your concern is length, then I don't feel > UTF8 will offer a significant advantage, and you're much better off > using something like length-prefixing like bencoding does it. If your > concern is copy-pasteability, then base58 works and probably is no worse > than base-utf8, while being significantly easier to implement in code. > > Spurious rant over. > > * Take for example the way early email was sent, where headers were > specified and then the server awaited the body of the message, the end > of which was indicated by what amounts to a string of characters; a > newline, a period, and another newline. Easily injected by accident or > design, along with other commands. > > On 11/01/14 06:27, nymble wrote: >> >> consistent key formats are critical, need to converge on: >> - endianness >> - coordinate representation x, x&y, x and sign … >> or bits to show which of these …. perhaps borrow ANSI method >> - hint / indication of cipher suite / curve >> - text encoding of binary format (ascii) >> - text encoding of binary format (utf8) >> - human readable format >> >>> ecc public key curve p25519(pcp 0.15) >> leaking crypto suite >> key should be usable in other contexts besides pcp 0.15 >> >> >>> 1l0$WoM5C8z=yeZG7?$]f^Uu8.g>4rf#t^6mfW9(rr910 >> one of several possible text encodings >> Others might include: >> - base 29 >> - base 59 >> - base 4096 (for UTF8 channels) >> >> >> > <0x988B9099.asc>
At 10:27 PM 1/10/2014, nymble wrote:
one of several possible text encodings Others might include: - base 29 - base 59 - base 4096 (for UTF8 channels)
The primary reasons for text encoding were that people wanted to transmit data through channels that might modify content or had limitations on the size and type of content, such as 7-bit ASCII, special interpretations of control characters, especially \r, \n, \0, \t, conversion to/from EBCDIC or other character sets, line length limitations, case-folding, multiple space compaction, parity bits, etc. A secondary goal is to support transcription by humans or optical character readers that are likely to make mistakes on some similar-looking characters, but that's much less common. A tertiary goal is that some programmers like to "improve" programs or make them "more efficient" by twiddling bits in ways that lead to software bugs, security holes, and the wrong kinds of chaos and anarchy, and yes I'm particularly including Phil Zimmerman and the standards committees who designed ASN.1 and DNS. To give those guys some slack, most of us started programming before the 8-bit byte was really universal and saving bytes here and there was *really* *important*.* The most common encodings out there encode most of the characters in base-16 (or octal, for old DEC applications) or base-64 (uuencode and MIME), with various wrappers around them to handle line-length limitations and sometimes checksums. Sadly, base-85 didn't catch on - it used 5 characters to hold 4 bytes, vs. base-64's 6 characters for 4 bytes, but it was late to the game and required doing multiplication and division instead of just bit-shifting. I've never seen base-29 or base-59 encodings - is base-29 some attempt to fit into 5-level Baudot coding now that the deaf community have pretty much all moved off Model-28 TTY emulators to ASCII or mobile phone texting? Base-4096 in UTF-8 would be silly - it gets you 12 bits per variable-width character, requiring at least two bytes, so you could just as well use two bytes of base-64 and not risk munging by systems that don't understand UTF-8. (* My first programming environment had a printer with 132 48-character type bars and Model 026 keypunches doing Hollerith cards, which could print 56 different characters; I don't think we did any hacks using non-printer-supported punchcard fields and the card sorter, but it was possible.)
On Sat, Jan 11, 2014 at 01:11:20PM -0800, Bill Stewart wrote:
bits, etc. A secondary goal is to support transcription by humans or optical character readers that are likely to make mistakes on some similar-looking characters, but that's much less common. A
it might make sense to accommodate both camps by supporting the terse base85 and a format for transcription - number-station style, decimal digits in fiver-groups. seems to be state of the art until today apparently ;) -- pgp: https://www.ctrlc.hu/~stef/stef.gpg pgp fp: FD52 DABD 5224 7F9C 63C6 3C12 FC97 D29F CA05 57EF otr fp: https://www.ctrlc.hu/~stef/otr.txt
On Jan 11, 2014, at 2:02 PM, stef <s@ctrlc.hu> wrote:
On Sat, Jan 11, 2014 at 01:11:20PM -0800, Bill Stewart wrote:
bits, etc. A secondary goal is to support transcription by humans or optical character readers that are likely to make mistakes on some similar-looking characters, but that's much less common. A
it might make sense to accommodate both camps by supporting the terse base85 and a format for transcription - number-station style, decimal digits in fiver-groups. seems to be state of the art until today apparently ;)
Yes…. transcription needs to be readable … perhaps base 27 or 58. Also .. noticed that UTF8 works as a continuos map to base 52643 Base 64 might be best for QR codes ... --- x-coordinate of public key Q = d*G for 'curve25519 of len = 32 --- xQ = 37664895557658108626454777585526504736961071659807064241611508404557428053916 xQ hex = 5345934eacb5b9adb3ffb4dee98d8942539cbdc23242a4d9c65e758fc7fc579c xQ base27 = FEXME 8R64N GDX72 6EXH4 XGEPZ E3NFK CWYJE 7P2J6 249JR XNX9T GNZT xQ base58 = 6c4NZ dLBJJ XdBCD V4ivS 6eovD gEFbP euMtQ uLDkW UXTR xQ base64 = U0WTTqy1ua2z/7Te6Y2JQlOcvcIyQqTZxl51j8f8V5w= xQ base85 = Q$>?bthKqVv;VZ->5Yj(Q=GlRGD4);#$I)g$NX2E xQ base52643 = 쀋퐟쥎흇곝항렷샔썿곊삃쥠챲떨쓹봪쁍촚멆
-- pgp: https://www.ctrlc.hu/~stef/stef.gpg pgp fp: FD52 DABD 5224 7F9C 63C6 3C12 FC97 D29F CA05 57EF otr fp: https://www.ctrlc.hu/~stef/otr.txt
participants (10)
-
Adam Back -
Bill Stewart -
Cathal Garvey -
Cathal Garvey (Phone)
-
gwen hastings -
James A. Donald -
Juan Garofalo -
nymble -
rysiek -
stef