path: root/5d/90ef2f9af4fddd2bb759aa3f2af4402f12f1d1

Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
	helo=mx.sourceforge.net)
	by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <jgarzik@bitpay.com>) id 1X753N-0006Nr-8F
	for bitcoin-development@lists.sourceforge.net;
	Tue, 15 Jul 2014 15:54:05 +0000
Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of bitpay.com
	designates 74.125.82.180 as permitted sender)
	client-ip=74.125.82.180; envelope-from=jgarzik@bitpay.com;
	helo=mail-we0-f180.google.com; 
Received: from mail-we0-f180.google.com ([74.125.82.180])
	by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
	(Exim 4.76) id 1X753L-0007fU-AG
	for bitcoin-development@lists.sourceforge.net;
	Tue, 15 Jul 2014 15:54:05 +0000
Received: by mail-we0-f180.google.com with SMTP id w61so26376wes.11
	for <bitcoin-development@lists.sourceforge.net>;
	Tue, 15 Jul 2014 08:53:57 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20130820;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to:cc:content-type:content-transfer-encoding;
	bh=ptj4LXn1notgm/oiJIrroMZFr47bYVNf0X0N6vfQKCk=;
	b=ILgGwJGYbRhR566y7YI9y+vzsTehOecvh1/fWkBKlto+rZHk2dUFcIURGn5ELev1Da
	KnQDZb6bGvH32bMVrED61o8BUi8YrgZAqdXar/jz6CxeHpLeKlKFXoHJ5FMINt0gnx/o
	rn5jByfqOuWVLObx2zL83RhpxV9K+RvL2Q/Lwqrx69iAlUYeITy9jH9teaIHQ9vpeBeR
	o51cOS6tiwnyB7KKXKl/9R4TOMhXP0zBVf9xAltodmvl9Dm3hhNMS6ec1EFXtz+Wwhoe
	9Ud8aP90quJul0u7dC0oLxPcU+AmpoHwAAXaBeR7awFFQZFuejsBBYt1JoGL2loEj/ic
	+/Sg==
X-Gm-Message-State: ALoCoQnRL5DhzptNbRzSlu6A/Krpb2oHO0kw4fWAA95qvzjZf15kVzI4jCOAOqGIYZIhK51xSuhN
X-Received: by 10.194.186.178 with SMTP id fl18mr27839613wjc.83.1405439636933; 
	Tue, 15 Jul 2014 08:53:56 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.194.5.67 with HTTP; Tue, 15 Jul 2014 08:53:36 -0700 (PDT)
In-Reply-To: <lq3hjc$s4a$1@ger.gmane.org>
References: <CANEZrP3ZzCBohXWZmZxE=ofP74Df4Hd-hCLH6jYn=JKbiqNQXA@mail.gmail.com>
	<CAObn+gfbH61kyv_ttT4vsQuNFRWLB5H3xaux7GQ0co82ucO_eA@mail.gmail.com>
	<CAJHLa0MMaQU7Xu=vo0Z2C5h5M-Oa9_a648yDRrw6UR=r81X66w@mail.gmail.com>
	<lq3hjc$s4a$1@ger.gmane.org>
From: Jeff Garzik <jgarzik@bitpay.com>
Date: Tue, 15 Jul 2014 11:53:36 -0400
Message-ID: <CAJHLa0PieYWVC798NeXWG25hhid9iYbNyzm2cK-yEdTb1W39Sw@mail.gmail.com>
To: Andreas Schildbach <andreas@schildbach.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -1.6 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
	sender-domain
	-0.0 SPF_PASS               SPF: sender matches SPF record
	-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
	author's domain
	0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
	not necessarily valid
	-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1X753L-0007fU-AG
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] BIP 38 NFC normalisation issue
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Tue, 15 Jul 2014 15:54:05 -0000

On whitespace:  Security UX testing I've seen shows it is mentally
easier for some users to memorize and use longer passphrases, if they
are permitted spaces.  I've not seen anything written on use of
tabs/NLs/FFs in passphrases.

I can see the logic of some systems, that convert \s+ into ' ' for
purposes of password hashing, even though that might frustrate a
security nerd or two.
http://security.stackexchange.com/questions/32691/why-not-allow-spaces-in-a=
-password

I do think control characters should be filtered.

On Tue, Jul 15, 2014 at 11:32 AM, Andreas Schildbach
<andreas@schildbach.de> wrote:
> Can you provide the rationale for standard practice? For example, why
> should whitespace be allowed? I regularly use trim() on any passphrase
> (or other input ftm).
>
> So what's the action point? Should we amend the spec to filter control
> characters? That would get rid of the \u0000 problem.
>
>
> On 07/15/2014 05:17 PM, Jeff Garzik wrote:
>> Unicode guarantees that null-terminated strings still work.  U+0000
>> terminates a unicode (or C) string.  strlen() gets the string byte
>> count.  mbstowcs() gets the character count.
>>
>> Whitespace can be problematic, but should be allowed.  Control
>> characters should be filtered.  Emoticons probably cannot be filtered
>> without substandard approaches such as character blacklists, a road
>> you do not want to travel.
>>
>> (all this is simply standard practice)
>>
>>
>> On Tue, Jul 15, 2014 at 9:07 AM, Eric Winer <enwiner@gmail.com> wrote:
>>> I don't know for sure if the test vector is correct NFC form.  But for =
what
>>> it's worth, the Pile of Poo character is pretty easily accessible on th=
e
>>> iPhone and Android keyboards, and in this string it's already in NFC fo=
rm
>>> (f09f92a9 in the test result).  I've certainly seen it in usernames aro=
und
>>> the internet, and wouldn't be surprised to see it in passphrases entere=
d on
>>> smartphones, especially if the author of a BIP38-compatible app include=
s a
>>> (possibly ill-advised) suggestion to have your passphrase "include spec=
ial
>>> characters".
>>>
>>> I haven't seen the NULL character on any smartphone keyboards, though -=
 I
>>> assume the iOS and Android developers had the foresight to know how muc=
h
>>> havoc that would wreak on systems assuming null-terminated strings.  It
>>> seems unlikely that NULL would be in a real-world passphrase entered by=
 a
>>> sane user.
>>>
>>>
>>> On Tue, Jul 15, 2014 at 8:03 AM, Mike Hearn <mike@plan99.net> wrote:
>>>>
>>>> [+cc aaron]
>>>>
>>>> We recently added an implementation of BIP 38 (password protected priv=
ate
>>>> keys) to bitcoinj. It came to my attention that the third test vector =
may be
>>>> broken. It gives a hex version of what the NFC normalised version of t=
he
>>>> input string should be, but this does not match the results of the Jav=
a
>>>> unicode normaliser, and in fact I can't even get Python to print the n=
ames
>>>> of the characters past the embedded null. I'm curious where this norma=
lised
>>>> version came from.
>>>>
>>>> Given that "pile of poo" is not a character I think any sane user woul=
d
>>>> put into a passphrase, I question the value of this test vector. NFC f=
orm is
>>>> intended to collapse things like umlaut control characters onto their =
prior
>>>> code point, but here we're feeding the algorithm what is basically gar=
bage
>>>> so I'm not totally surprised that different implementations appear to
>>>> disagree on the outcome.
>>>>
>>>> Proposed action: we remove this test vector as it does not represent a=
ny
>>>> real world usage of the spec, or if we desperately need to verify NFC
>>>> normalisation I suggest using a different, more realistic test string,=
 like
>>>> Z=C3=BCrich, or something written in Thai.
>>>>
>>>>
>>>>
>>>> Test 3:
>>>>
>>>> Passphrase =CF=92=CC=81=E2=90=80=F0=90=90=80=F0=9F=92=A9 (\u03D2\u0301=
\u0000\U00010400\U0001F4A9; GREEK UPSILON
>>>> WITH HOOK, COMBINING ACUTE ACCENT, NULL, DESERET CAPITAL LETTER LONG I=
, PILE
>>>> OF POO)
>>>> Encrypted key: 6PRW5o9FLp4gJDDVqJQKJFTpMvdsSGJxMYHtHaQBF3ooa8mwD69bapc=
DQn
>>>> Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF
>>>> Unencrypted private key (WIF):
>>>> 5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4
>>>> Note: The non-standard UTF-8 characters in this passphrase should be N=
FC
>>>> normalized to result in a passphrase of0xcf9300f0909080f09f92a9 before
>>>> further processing
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ----------------------------------------------------------------------=
--------
>>>> Want fast and easy access to all the code in your enterprise? Index an=
d
>>>> search up to 200,000 lines of code with a free copy of Black Duck
>>>> Code Sight - the same software that powers the world's largest code
>>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>>> http://p.sf.net/sfu/bds
>>>> _______________________________________________
>>>> Bitcoin-development mailing list
>>>> Bitcoin-development@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>
>>>
>>>
>>> -----------------------------------------------------------------------=
-------
>>> Want fast and easy access to all the code in your enterprise? Index and
>>> search up to 200,000 lines of code with a free copy of Black Duck
>>> Code Sight - the same software that powers the world's largest code
>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>> http://p.sf.net/sfu/bds
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>>
>>
>>
>
>
>
> -------------------------------------------------------------------------=
-----
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development



--=20
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/