1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
|
Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
helo=mx.sourceforge.net)
by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <gcbd-bitcoin-development@m.gmane.org>)
id 1X7Xbc-00047e-SO for bitcoin-development@lists.sourceforge.net;
Wed, 16 Jul 2014 22:23:20 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of m.gmane.org
designates 80.91.229.3 as permitted sender)
client-ip=80.91.229.3;
envelope-from=gcbd-bitcoin-development@m.gmane.org;
helo=plane.gmane.org;
Received: from plane.gmane.org ([80.91.229.3])
by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256)
(Exim 4.76) id 1X7Xba-0003FQ-25
for bitcoin-development@lists.sourceforge.net;
Wed, 16 Jul 2014 22:23:20 +0000
Received: from list by plane.gmane.org with local (Exim 4.69)
(envelope-from <gcbd-bitcoin-development@m.gmane.org>)
id 1X7XbQ-00045V-Oq for bitcoin-development@lists.sourceforge.net;
Thu, 17 Jul 2014 00:23:08 +0200
Received: from f052021167.adsl.alicedsl.de ([78.52.21.167])
by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
id 1AlnuQ-0007hv-00 for <bitcoin-development@lists.sourceforge.net>;
Thu, 17 Jul 2014 00:23:08 +0200
Received: from andreas by f052021167.adsl.alicedsl.de with local (Gmexim 0.1
(Debian)) id 1AlnuQ-0007hv-00
for <bitcoin-development@lists.sourceforge.net>;
Thu, 17 Jul 2014 00:23:08 +0200
X-Injected-Via-Gmane: http://gmane.org/
To: bitcoin-development@lists.sourceforge.net
From: Andreas Schildbach <andreas@schildbach.de>
Date: Thu, 17 Jul 2014 00:22:57 +0200
Message-ID: <lq6u01$lfi$1@ger.gmane.org>
References: <CANEZrP3ZzCBohXWZmZxE=ofP74Df4Hd-hCLH6jYn=JKbiqNQXA@mail.gmail.com> <CAObn+gfbH61kyv_ttT4vsQuNFRWLB5H3xaux7GQ0co82ucO_eA@mail.gmail.com> <CANg-TZAe2PO9nwQktmDSJFtaLsg6hogOw6mj0SaROdJJr33vog@mail.gmail.com> <CANEZrP3E2mdvOUHiW9W_hM3Z_kn9um8E6aX5vf-S9tA7KgnpUQ@mail.gmail.com> <CACq0ZD5qTc-2f+puPaXMDFZNWUx8kvOZQMxqkM_e4YafhTW7cA@mail.gmail.com> <lq5fvo$4s6$1@ger.gmane.org> <CANEZrP0x2Ypb063VkcoE+h_OHfRVOusmXB2X1VQx77sZhAuTFw@mail.gmail.com> <lq5l5a$4fl$1@ger.gmane.org> <lq5m78$j23$1@ger.gmane.org> <CACq0ZD6Nib4kU8kAo6jdXamw7NTTd5_JwO4x1fsVOMP2-F1=cA@mail.gmail.com>
<lq6sq5$71h$1@ger.gmane.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Complaints-To: usenet@ger.gmane.org
X-Gmane-NNTP-Posting-Host: f052021167.adsl.alicedsl.de
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
rv:24.0) Gecko/20100101 Thunderbird/24.6.0
In-Reply-To: <lq6sq5$71h$1@ger.gmane.org>
X-Enigmail-Version: 1.5.2
X-Spam-Score: -0.4 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,
no trust [80.91.229.3 listed in list.dnswl.org]
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
1.1 DKIM_ADSP_ALL No valid author signature,
domain signs all mail
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
X-Headers-End: 1X7Xba-0003FQ-25
Subject: Re: [Bitcoin-development] BIP 38 NFC normalisation issue
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Wed, 16 Jul 2014 22:23:21 -0000
Ok, I just fixed the String filtering so that it can handle SMP chars
and my implementation behaves exactly like in your modified testcase
quoted below.
Bitcoinj code available on this branch, in case we decide to change the
spec:
https://github.com/schildbach/bitcoinj/commits/bip38-normalize-control-characters
On 07/17/2014 12:02 AM, Andreas Schildbach wrote:
> Please excuse me. I had a more thorough look at the original problem and
> found that the only problem with the original test case was that you
> cannot specify codepoints from the SMP using \u in Java. I always tried
> \u010400 but that doesn't work.
>
> Here is a fix for bitcoinj. The test now passes.
>
> https://github.com/bitcoinj/bitcoinj/pull/143
>
> We can (and probably should) still need to filter control chars, I'll
> have a look at that now again.
>
>
> On 07/16/2014 11:06 PM, Aaron Voisine wrote:
>> If I first remove \u0000, so the non-normalized passphrase is
>> "\u03D2\u0301\U00010400\U0001F4A9", and then NFC normalize it, it
>> becomes "\u03D3\U00010400\U0001F4A9"
>>
>> UTF-8 encoded this is: 0xcf93f0909080f09f92a9 (not the same as what
>> you got, Andreas!)
>>
>> Encoding private key: 5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4
>> with this passphrase, I get a BIP38 key of:
>> 6PRW5o9FMb4hAYRQPmgcvVDTyDtr6R17VMXGLmvKjKVpGkYhBJ4uYuR9wZ
>>
>> I recommend rather than simply removing control characters from the
>> password that instead the spec require that passwords containing
>> control characters are invalid. We don't want people trying to be
>> clever and putting them in thinking they are adding to the password
>> entropy.
>>
>> Also for UI compatibility across many platforms, I'm also in favor
>> disallowing any character below U+0020 (space)
>>
>> I can submit a PR once we figure out why Andreas's passphrase was
>> different than what I got.
>>
>> Aaron Voisine
>> breadwallet.com
>>
>>
>> On Wed, Jul 16, 2014 at 4:04 AM, Andreas Schildbach
>> <andreas@schildbach.de> wrote:
>>> Damn, I just realized that I implement only the decoding side of BIP38.
>>> So I cannot propose a complete test vector. Here is what I have:
>>>
>>>
>>> Passphrase: ϓ␀𐐀💩 (\u03D2\u0301\u0000\U00010400\U0001F4A9; GREEK
>>> UPSILON WITH HOOK, COMBINING ACUTE ACCENT, NULL, DESERET CAPITAL LETTER
>>> LONG I, PILE OF POO)
>>>
>>> Passphrase bytes after removing ISO control characters and NFC
>>> normalization: 0xcf933034303066346139
>>>
>>> Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF
>>>
>>> Unencrypted private key (WIF):
>>> 5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4
>>>
>>>
>>> Can someone calculate the encrypted key from it (using whatever
>>> implementation) and I will verify it decodes properly in bitcoinj?
>>>
>>>
>>>
>>> On 07/16/2014 12:46 PM, Andreas Schildbach wrote:
>>>> I will change the bitcoinj implementation and propose a new test vector.
>>>>
>>>>
>>>>
>>>> On 07/16/2014 11:29 AM, Mike Hearn wrote:
>>>>> Yes sorry, you're right, the issue starts with the null code point.
>>>>> Python seems to have problems starting there too. It might work if we
>>>>> took that out.
>>>>>
>>>>>
>>>>> On Wed, Jul 16, 2014 at 11:17 AM, Andreas Schildbach
>>>>> <andreas@schildbach.de <mailto:andreas@schildbach.de>> wrote:
>>>>>
>>>>> Guys, you are always talking about the Unicode astral plane, but in fact
>>>>> its a plain old (ASCII) control character where this problem starts and
>>>>> likely ends: \u0000.
>>>>>
>>>>> Let's ban/filter ISO control characters and be done with it. Most
>>>>> control characters will never be enterable by any keyboard into a
>>>>> password field. Of course I assume that Character.isISOControl() works
>>>>> consistently across platforms.
>>>>>
>>>>> http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#isISOControl%28char%29
>>>>>
>>>>>
>>>>> On 07/16/2014 12:23 AM, Aaron Voisine wrote:
>>>>> > If the user creates a password on an iOS device with an astral
>>>>> > character and then can't enter that password on a JVM wallet, that
>>>>> > sucks. If JVMs really can't support unicode NFC then that's a strong
>>>>> > case to limit the spec to the subset of unicode that all popular
>>>>> > platforms can support, but it sounds like it might just be a JVM
>>>>> > string library bug that could hopefully be reported and fixed. I get
>>>>> > the same result as in the test case using apple's
>>>>> > CFStringNormalize(passphrase, kCFStringNormalizationFormC);
>>>>> >
>>>>> > Aaron Voisine
>>>>> > breadwallet.com <http://breadwallet.com>
>>>>> >
>>>>> >
>>>>> > On Tue, Jul 15, 2014 at 11:20 AM, Mike Hearn <mike@plan99.net
>>>>> <mailto:mike@plan99.net>> wrote:
>>>>> >> Yes, we know, Andreas' code is indeed doing normalisation.
>>>>> >>
>>>>> >> However it appears the output bytes end up being different. What
>>>>> I get back
>>>>> >> is:
>>>>> >>
>>>>> >> cf930001303430300166346139
>>>>> >>
>>>>> >> vs
>>>>> >>
>>>>> >> cf9300f0909080f09f92a9
>>>>> >>
>>>>> >> from the spec.
>>>>> >>
>>>>> >> I'm not sure why. It appears this is due to the character from
>>>>> the astral
>>>>> >> planes. Java is old and uses 16 bit characters internally - it
>>>>> wouldn't
>>>>> >> surprise me if there's some weirdness that means it doesn't/won't
>>>>> support
>>>>> >> this kind of thing.
>>>>> >>
>>>>> >> I recommend instead that any implementation that wishes to be
>>>>> compatible
>>>>> >> with JVM based wallets (I suspect Android is the same) just
>>>>> refuse any
>>>>> >> passphrase that includes characters outside the BMP. At least
>>>>> unless someone
>>>>> >> can find a fix. I somehow doubt this will really hurt anyone.
>>>>> >>
>>>>> >>
>>>>> ------------------------------------------------------------------------------
>>>>> >> Want fast and easy access to all the code in your enterprise?
>>>>> Index and
>>>>> >> search up to 200,000 lines of code with a free copy of Black Duck
>>>>> >> Code Sight - the same software that powers the world's largest code
>>>>> >> search on Ohloh, the Black Duck Open Hub! Try it now.
>>>>> >> http://p.sf.net/sfu/bds
>>>>> >> _______________________________________________
>>>>> >> Bitcoin-development mailing list
>>>>> >> Bitcoin-development@lists.sourceforge.net
>>>>> <mailto:Bitcoin-development@lists.sourceforge.net>
>>>>> >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>> >>
>>>>> >
>>>>> >
>>>>> ------------------------------------------------------------------------------
>>>>> > Want fast and easy access to all the code in your enterprise?
>>>>> Index and
>>>>> > search up to 200,000 lines of code with a free copy of Black Duck
>>>>> > Code Sight - the same software that powers the world's largest code
>>>>> > search on Ohloh, the Black Duck Open Hub! Try it now.
>>>>> > http://p.sf.net/sfu/bds
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Want fast and easy access to all the code in your enterprise? Index and
>>>>> search up to 200,000 lines of code with a free copy of Black Duck
>>>>> Code Sight - the same software that powers the world's largest code
>>>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>>>> http://p.sf.net/sfu/bds
>>>>> _______________________________________________
>>>>> Bitcoin-development mailing list
>>>>> Bitcoin-development@lists.sourceforge.net
>>>>> <mailto:Bitcoin-development@lists.sourceforge.net>
>>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Want fast and easy access to all the code in your enterprise? Index and
>>>>> search up to 200,000 lines of code with a free copy of Black Duck
>>>>> Code Sight - the same software that powers the world's largest code
>>>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>>>> http://p.sf.net/sfu/bds
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Bitcoin-development mailing list
>>>>> Bitcoin-development@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Want fast and easy access to all the code in your enterprise? Index and
>>>> search up to 200,000 lines of code with a free copy of Black Duck
>>>> Code Sight - the same software that powers the world's largest code
>>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>>> http://p.sf.net/sfu/bds
>>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Want fast and easy access to all the code in your enterprise? Index and
>>> search up to 200,000 lines of code with a free copy of Black Duck
>>> Code Sight - the same software that powers the world's largest code
>>> search on Ohloh, the Black Duck Open Hub! Try it now.
>>> http://p.sf.net/sfu/bds
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>> ------------------------------------------------------------------------------
>> Want fast and easy access to all the code in your enterprise? Index and
>> search up to 200,000 lines of code with a free copy of Black Duck
>> Code Sight - the same software that powers the world's largest code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.net/sfu/bds
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
|
