Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of osfda.org
	designates 217.23.13.216 as permitted sender)
	client-ip=217.23.13.216; envelope-from=mw@osfda.org;
	helo=smtp.osfda.org; 
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\))
From: Michael Wozniak <mw@osfda.org>
In-Reply-To: <lq39p1$gff$1@ger.gmane.org>
Date: Tue, 15 Jul 2014 09:32:01 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <365F4709-54A6-4E14-BA6F-BCFB36DBAA96@osfda.org>
References: <CANEZrP3ZzCBohXWZmZxE=ofP74Df4Hd-hCLH6jYn=JKbiqNQXA@mail.gmail.com>
	<CAObn+gfbH61kyv_ttT4vsQuNFRWLB5H3xaux7GQ0co82ucO_eA@mail.gmail.com>
	<lq39p1$gff$1@ger.gmane.org>
To: Andreas Schildbach <andreas@schildbach.de>
Cc: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] BIP 38 NFC normalisation issue
Precedence: list

I have a python implementation that seems to pass this test vector:

https://github.com/wozz/electrum/blob/bip38_import/lib/bip38.py#L299


On Jul 15, 2014, at 9:19 AM, Andreas Schildbach <andreas@schildbach.de> =
wrote:

> I think generally control-characters (such as \u0000) should be
> disallowed in passphrases. (Even the use of whitespaces is very
> questionable.)
>=20
> I'm ok with allowing pile-of-poo's. On mobile phones there is =
keyboards
> just containing emoticons -- why not allow those? Assuming NFC works =
of
> course.
>=20
>=20
> On 07/15/2014 03:07 PM, Eric Winer wrote:
>> I don't know for sure if the test vector is correct NFC form.  But =
for
>> what it's worth, the Pile of Poo character is pretty easily =
accessible
>> on the iPhone and Android keyboards, and in this string it's already =
in
>> NFC form (f09f92a9 in the test result).  I've certainly seen it in
>> usernames around the internet, and wouldn't be surprised to see it in
>> passphrases entered on smartphones, especially if the author of a
>> BIP38-compatible app includes a (possibly ill-advised) suggestion to
>> have your passphrase "include special characters".
>>=20
>> I haven't seen the NULL character on any smartphone keyboards, though =
-
>> I assume the iOS and Android developers had the foresight to know how
>> much havoc that would wreak on systems assuming null-terminated =
strings.
>> It seems unlikely that NULL would be in a real-world passphrase =
entered
>> by a sane user.
>>=20
>>=20
>> On Tue, Jul 15, 2014 at 8:03 AM, Mike Hearn <mike@plan99.net
>> <mailto:mike@plan99.net>> wrote:
>>=20
>>    [+cc aaron]
>>=20
>>    We recently added an implementation of BIP 38 (password protected
>>    private keys) to bitcoinj. It came to my attention that the third
>>    test vector may be broken. It gives a hex version of what the NFC
>>    normalised version of the input string should be, but this does =
not
>>    match the results of the Java unicode normaliser, and in fact I
>>    can't even get Python to print the names of the characters past =
the
>>    embedded null. I'm curious where this normalised version came =
from.
>>=20
>>    Given that "pile of poo" is not a character I think any sane user
>>    would put into a passphrase, I question the value of this test
>>    vector. NFC form is intended to collapse things like umlaut =
control
>>    characters onto their prior code point, but here we're feeding the
>>    algorithm what is basically garbage so I'm not totally surprised
>>    that different implementations appear to disagree on the outcome.
>>=20
>>    Proposed action: we remove this test vector as it does not =
represent
>>    any real world usage of the spec, or if we desperately need to
>>    verify NFC normalisation I suggest using a different, more =
realistic
>>    test string, like Z=C3=BCrich, or something written in Thai.
>>=20
>>=20
>>=20
>>    Test 3:
>>=20
>>      * Passphrase =CF=92=CC=81=E2=90=80=F0=90=90=80=F0=9F=92=A9 =
(\u03D2\u0301\u0000\U00010400\U0001F4A9; GREEK
>>        UPSILON WITH HOOK <http://codepoints.net/U+03D2>, COMBINING
>>        ACUTE ACCENT <http://codepoints.net/U+0301>, NULL
>>        <http://codepoints.net/U+0000>, DESERET CAPITAL LETTER LONG I
>>        <http://codepoints.net/U+10400>, PILE OF POO
>>        <http://codepoints.net/U+1F4A9>)
>>      * Encrypted key:
>>        6PRW5o9FLp4gJDDVqJQKJFTpMvdsSGJxMYHtHaQBF3ooa8mwD69bapcDQn
>>      * Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF
>>      * Unencrypted private key (WIF):
>>        5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4
>>      * /Note:/ The non-standard UTF-8 characters in this passphrase
>>        should be NFC normalized to result in a passphrase
>>        of0xcf9300f0909080f09f92a9 before further processing
>>=20
>>=20
>>=20
>>=20
>>    =
--------------------------------------------------------------------------=
----
>>    Want fast and easy access to all the code in your enterprise? =
Index and
>>    search up to 200,000 lines of code with a free copy of Black Duck
>>    Code Sight - the same software that powers the world's largest =
code
>>    search on Ohloh, the Black Duck Open Hub! Try it now.
>>    http://p.sf.net/sfu/bds
>>    _______________________________________________
>>    Bitcoin-development mailing list
>>    Bitcoin-development@lists.sourceforge.net
>>    <mailto:Bitcoin-development@lists.sourceforge.net>
>>    https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>=20
>>=20
>>=20
>>=20
>> =
--------------------------------------------------------------------------=
----
>> Want fast and easy access to all the code in your enterprise? Index =
and
>> search up to 200,000 lines of code with a free copy of Black Duck
>> Code Sight - the same software that powers the world's largest code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.net/sfu/bds
>>=20
>>=20
>>=20
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>=20
>=20
>=20
>=20
> =
--------------------------------------------------------------------------=
----
> Want fast and easy access to all the code in your enterprise? Index =
and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development