Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1X7LWd-0004yO-Ii for bitcoin-development@lists.sourceforge.net; Wed, 16 Jul 2014 09:29:23 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.219.54 as permitted sender) client-ip=209.85.219.54; envelope-from=mh.in.england@gmail.com; helo=mail-oa0-f54.google.com; Received: from mail-oa0-f54.google.com ([209.85.219.54]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1X7LWc-0002TX-6f for bitcoin-development@lists.sourceforge.net; Wed, 16 Jul 2014 09:29:23 +0000 Received: by mail-oa0-f54.google.com with SMTP id n16so654846oag.41 for ; Wed, 16 Jul 2014 02:29:16 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.52.5 with SMTP id p5mr19109747oeo.55.1405502956746; Wed, 16 Jul 2014 02:29:16 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.76.35.234 with HTTP; Wed, 16 Jul 2014 02:29:16 -0700 (PDT) In-Reply-To: References: Date: Wed, 16 Jul 2014 11:29:16 +0200 X-Google-Sender-Auth: WMPM1clGTS1cWDtVjnHLJT7JOaY Message-ID: From: Mike Hearn To: Andreas Schildbach Content-Type: multipart/alternative; boundary=001a11332c8e77c22d04fe4c2852 X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1X7LWc-0002TX-6f Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] BIP 38 NFC normalisation issue X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 09:29:23 -0000 --001a11332c8e77c22d04fe4c2852 Content-Type: text/plain; charset=UTF-8 Yes sorry, you're right, the issue starts with the null code point. Python seems to have problems starting there too. It might work if we took that out. On Wed, Jul 16, 2014 at 11:17 AM, Andreas Schildbach wrote: > Guys, you are always talking about the Unicode astral plane, but in fact > its a plain old (ASCII) control character where this problem starts and > likely ends: \u0000. > > Let's ban/filter ISO control characters and be done with it. Most > control characters will never be enterable by any keyboard into a > password field. Of course I assume that Character.isISOControl() works > consistently across platforms. > > > http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#isISOControl%28char%29 > > > On 07/16/2014 12:23 AM, Aaron Voisine wrote: > > If the user creates a password on an iOS device with an astral > > character and then can't enter that password on a JVM wallet, that > > sucks. If JVMs really can't support unicode NFC then that's a strong > > case to limit the spec to the subset of unicode that all popular > > platforms can support, but it sounds like it might just be a JVM > > string library bug that could hopefully be reported and fixed. I get > > the same result as in the test case using apple's > > CFStringNormalize(passphrase, kCFStringNormalizationFormC); > > > > Aaron Voisine > > breadwallet.com > > > > > > On Tue, Jul 15, 2014 at 11:20 AM, Mike Hearn wrote: > >> Yes, we know, Andreas' code is indeed doing normalisation. > >> > >> However it appears the output bytes end up being different. What I get > back > >> is: > >> > >> cf930001303430300166346139 > >> > >> vs > >> > >> cf9300f0909080f09f92a9 > >> > >> from the spec. > >> > >> I'm not sure why. It appears this is due to the character from the > astral > >> planes. Java is old and uses 16 bit characters internally - it wouldn't > >> surprise me if there's some weirdness that means it doesn't/won't > support > >> this kind of thing. > >> > >> I recommend instead that any implementation that wishes to be compatible > >> with JVM based wallets (I suspect Android is the same) just refuse any > >> passphrase that includes characters outside the BMP. At least unless > someone > >> can find a fix. I somehow doubt this will really hurt anyone. > >> > >> > ------------------------------------------------------------------------------ > >> Want fast and easy access to all the code in your enterprise? Index and > >> search up to 200,000 lines of code with a free copy of Black Duck > >> Code Sight - the same software that powers the world's largest code > >> search on Ohloh, the Black Duck Open Hub! Try it now. > >> http://p.sf.net/sfu/bds > >> _______________________________________________ > >> Bitcoin-development mailing list > >> Bitcoin-development@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development > >> > > > > > ------------------------------------------------------------------------------ > > Want fast and easy access to all the code in your enterprise? Index and > > search up to 200,000 lines of code with a free copy of Black Duck > > Code Sight - the same software that powers the world's largest code > > search on Ohloh, the Black Duck Open Hub! Try it now. > > http://p.sf.net/sfu/bds > > > > > > > ------------------------------------------------------------------------------ > Want fast and easy access to all the code in your enterprise? Index and > search up to 200,000 lines of code with a free copy of Black Duck > Code Sight - the same software that powers the world's largest code > search on Ohloh, the Black Duck Open Hub! Try it now. > http://p.sf.net/sfu/bds > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > --001a11332c8e77c22d04fe4c2852 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Yes sorry, you're right, the issue starts with the nul= l code point. Python seems to have problems starting there too. It might wo= rk if we took that out.


On Wed, Jul 16, 2014 at 11:17 AM, Andreas Schildbach <= andreas@schildba= ch.de> wrote:
Guys, you are always talking about the Unicode astral plane, but in fact its a plain old (ASCII) control character where this problem starts and
likely ends: \u0000.

Let's ban/filter ISO control characters and be done with it. Most
control characters will never be enterable by any keyboard into a
password field. Of course I assume that Character.isISOControl() works
consistently across platforms.

http://docs.oracle.com/javase/7= /docs/api/java/lang/Character.html#isISOControl%28char%29


On 07/16/2014 12:23 AM, Aaron Voisine wrote:
> If the user creates a password on an iOS device with an astral
> character and then can't enter that password on a JVM wallet, that=
> sucks. If JVMs really can't support unicode NFC then that's a = strong
> case to limit the spec to the subset of unicode that all popular
> platforms can support, but it sounds like it might just be a JVM
> string library bug that could hopefully be reported and fixed. I get > the same result as in the test case using apple's
> CFStringNormalize(passphrase, kCFStringNormalizationFormC);
>
> Aaron Voisine
> breadwallet.com
>
>
> On Tue, Jul 15, 2014 at 11:20 AM, Mike Hearn <mike@plan99.net> wrote:
>> Yes, we know, Andreas' code is indeed doing normalisation.
>>
>> However it appears the output bytes end up being different. What I= get back
>> is:
>>
>> cf930001303430300166346139
>>
>> vs
>>
>> cf9300f0909080f09f92a9
>>
>> from the spec.
>>
>> I'm not sure why. It appears this is due to the character from= the astral
>> planes. Java is old and uses 16 bit characters internally - it wou= ldn't
>> surprise me if there's some weirdness that means it doesn'= t/won't support
>> this kind of thing.
>>
>> I recommend instead that any implementation that wishes to be comp= atible
>> with JVM based wallets (I suspect Android is the same) just refuse= any
>> passphrase that includes characters outside the BMP. At least unle= ss someone
>> can find a fix. I somehow doubt this will really hurt anyone.
>>
>> ------------------------------------------------------------------= ------------
>> Want fast and easy access to all the code in your enterprise? Inde= x and
>> search up to 200,000 lines of code with a free copy of Black Duck<= br> >> Code Sight - the same software that powers the world's largest= code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.= net/sfu/bds
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitco= in-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/b= itcoin-development
>>
>
> ----------------------------------------------------------------------= --------
> Want fast and easy access to all the code in your enterprise? Index an= d
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest cod= e
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/= sfu/bds
>



---------------------------------------------------------------------------= ---
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/b= ds
_______________________________________________
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment

--001a11332c8e77c22d04fe4c2852--