1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
|
Return-Path: <james.obeirne@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 1D2BC1AEF
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 8 Oct 2015 00:29:46 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-ig0-f172.google.com (mail-ig0-f172.google.com
[209.85.213.172])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6811879
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 8 Oct 2015 00:29:45 +0000 (UTC)
Received: by igcpe7 with SMTP id pe7so1952465igc.0
for <bitcoin-dev@lists.linuxfoundation.org>;
Wed, 07 Oct 2015 17:29:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:from:date:message-id:subject:to
:cc:content-type;
bh=YYLxkGgHh6j0msYdb0ucHGIksofq0VExMUFxsr+cUp4=;
b=u2aLLUZjdG9qEnP4kjkpM8kbZC1R8Gxxmk1ILegs2HYzesxON8vvz5HZ9Dbgqhr9Yp
XEtunV3u9+JphJ+cl/tb6JkVJWOF4Ne9rAxkT1QweAykmKaxSJANA5i44D7r1hPS3Fm1
nKXId2pay0pSfliE3xGDFjB+7gofN0uOBcxVH9K2Z/0AeyDmItk45r0BcxTTgHfTnjOt
lJyXWLCKp2sxguZrcqbUcRKSx0NLVTjmjUvCcq+VRJhaGo+s09IaKOBmuxOz1IdQ/Xaf
twetYXhrdDbMzJG7sFxRHlYU+zqzdf9+DaVafZFsQROHPIZKAdo3M3zR4c4Oyi/zD09Y
Vyyg==
X-Received: by 10.50.28.113 with SMTP id a17mr495622igh.67.1444264184896; Wed,
07 Oct 2015 17:29:44 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.64.195.131 with HTTP; Wed, 7 Oct 2015 17:29:25 -0700 (PDT)
In-Reply-To: <CAPfvXfLw25J_mXn6b9QO_VDTU3EoVP9zxt7bbN0RF+2coYAdJA@mail.gmail.com>
References: <56155572.5040501@domob.eu>
<CAPfvXfLw25J_mXn6b9QO_VDTU3EoVP9zxt7bbN0RF+2coYAdJA@mail.gmail.com>
From: "James O'Beirne" <james.obeirne@gmail.com>
Date: Wed, 7 Oct 2015 17:29:25 -0700
Message-ID: <CAPfvXfKxKoAO3n0ru4X3mMvtWkwPQyK4n7YziCac75yap7_YRg@mail.gmail.com>
To: Daniel Kraft <d@domob.eu>
Content-Type: multipart/alternative; boundary=089e01538ac8b3e2d205218cf5fa
X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW
autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] The new obfuscation patch & GetStats
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Oct 2015 00:29:46 -0000
--089e01538ac8b3e2d205218cf5fa
Content-Type: text/plain; charset=UTF-8
This has been confirmed as a bug. Thanks again for reporting. I've filed a
fix here (https://github.com/bitcoin/bitcoin/pull/6777), and will be
writing tests to prevent regressions.
On Wed, Oct 7, 2015 at 4:32 PM, James O'Beirne <james.obeirne@gmail.com>
wrote:
> Hey, Daniel.
>
> Patch author here. Thanks for the diligence; I think this indeed may be an
> oversight, though I'm going to need to look into a bit more thoroughly at
> home. Curious that it didn't fail any of the automated tests.
>
> Correct me if I'm wrong, but the only actual invocation of that method is
> here
> <https://github.com/bitcoin/bitcoin/blob/master/src/rpcblockchain.cpp#L448>
> (and even then, proxied through a few layers of CCoinView-machinery). In
> fact, this line
> <https://github.com/bitcoin/bitcoin/blob/master/src/coins.cpp#L48> makes
> me suspect that the implementation of GetStats you reference may be dead
> code.
>
> In any case, you raise a good point: if users of CLevelDBWrapper go
> directly for the iterator, they run the risk of dealing with obfuscated
> data. This should be remedied somehow.
>
> I'll give it more look this evening.
>
> Thanks again for the find,
> James
>
> On Wed, Oct 7, 2015 at 10:25 AM, Daniel Kraft via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Hi!
>>
>> I hope this is not a stupid question, but I thought I'd ask here first
>> instead of opening a Github ticket (in case I'm wrong).
>>
>> With the recently merged "obfuscation" patch, content of the
>> "chainstate" LevelDB is obfuscated by XOR'ing against a random "key".
>> This is handled by CLevelDBWrapper's Read/Write methods, which probably
>> cover most of the usecases.
>>
>> *However*, shouldn't it also be handled when iterating over the
>> database? In particular, I would expect that the obfuscation key is
>> applied before line 119 in txdb.cpp (i. e., while iterating over the
>> coin database in CCoinsViewDB::GetStats).
>>
>> Is there a reason why this need not be done there, or is this an actual
>> oversight?
>>
>> Yours,
>> Daniel
>>
>> --
>> http://www.domob.eu/
>> OpenPGP: 1142 850E 6DFF 65BA 63D6 88A8 B249 2AC4 A733 0737
>> Namecoin: id/domob -> https://nameid.org/?name=domob
>> --
>> Done: Arc-Bar-Cav-Hea-Kni-Ran-Rog-Sam-Tou-Val-Wiz
>> To go: Mon-Pri
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>
--089e01538ac8b3e2d205218cf5fa
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">This has been confirmed as a bug. Thanks again for reporti=
ng. I've filed a fix here (<a href=3D"https://github.com/bitcoin/bitcoi=
n/pull/6777">https://github.com/bitcoin/bitcoin/pull/6777</a>), and will be=
writing tests to prevent regressions.</div><div class=3D"gmail_extra"><br>=
<div class=3D"gmail_quote">On Wed, Oct 7, 2015 at 4:32 PM, James O'Beir=
ne <span dir=3D"ltr"><<a href=3D"mailto:james.obeirne@gmail.com" target=
=3D"_blank">james.obeirne@gmail.com</a>></span> wrote:<br><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;p=
adding-left:1ex"><div dir=3D"ltr">Hey, Daniel.<div><br></div><div>Patch aut=
hor here. Thanks for the diligence; I think this indeed may be an oversight=
, though I'm going to need to look into a bit more thoroughly at home. =
Curious that it didn't fail any of the automated tests.</div><div><br><=
/div><div>Correct me if I'm wrong, but the only actual invocation of th=
at method is <a href=3D"https://github.com/bitcoin/bitcoin/blob/master/src/=
rpcblockchain.cpp#L448" target=3D"_blank">here</a> (and even then, proxied =
through a few layers of CCoinView-machinery). In fact, <a href=3D"https://g=
ithub.com/bitcoin/bitcoin/blob/master/src/coins.cpp#L48" target=3D"_blank">=
this line</a> makes me suspect that the implementation of GetStats you refe=
rence may be dead code.</div><div><br></div><div>In any case, you raise a g=
ood point: if users of CLevelDBWrapper go directly for the iterator, they r=
un the risk of dealing with obfuscated data. This should be remedied someho=
w.</div><div><br></div><div>I'll give it more look this evening.</div><=
div><br></div><div>Thanks again for the find,</div><div>James</div></div><d=
iv class=3D"gmail_extra"><br><div class=3D"gmail_quote"><div><div class=3D"=
h5">On Wed, Oct 7, 2015 at 10:25 AM, Daniel Kraft via bitcoin-dev <span dir=
=3D"ltr"><<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" targe=
t=3D"_blank">bitcoin-dev@lists.linuxfoundation.org</a>></span> wrote:<br=
></div></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;b=
order-left:1px #ccc solid;padding-left:1ex"><div><div class=3D"h5">Hi!<br>
<br>
I hope this is not a stupid question, but I thought I'd ask here first<=
br>
instead of opening a Github ticket (in case I'm wrong).<br>
<br>
With the recently merged "obfuscation" patch, content of the<br>
"chainstate" LevelDB is obfuscated by XOR'ing against a rando=
m "key".<br>
This is handled by CLevelDBWrapper's Read/Write methods, which probably=
<br>
cover most of the usecases.<br>
<br>
*However*, shouldn't it also be handled when iterating over the<br>
database?=C2=A0 In particular, I would expect that the obfuscation key is<b=
r>
applied before line 119 in txdb.cpp (i. e., while iterating over the<br>
coin database in CCoinsViewDB::GetStats).<br>
<br>
Is there a reason why this need not be done there, or is this an actual<br>
oversight?<br>
<br>
Yours,<br>
Daniel<br>
<span><font color=3D"#888888"><br>
--<br>
<a href=3D"http://www.domob.eu/" rel=3D"noreferrer" target=3D"_blank">http:=
//www.domob.eu/</a><br>
OpenPGP: 1142 850E 6DFF 65BA 63D6=C2=A0 88A8 B249 2AC4 A733 0737<br>
Namecoin: id/domob -> <a href=3D"https://nameid.org/?name=3Ddomob" rel=
=3D"noreferrer" target=3D"_blank">https://nameid.org/?name=3Ddomob</a><br>
--<br>
Done:=C2=A0 Arc-Bar-Cav-Hea-Kni-Ran-Rog-Sam-Tou-Val-Wiz<br>
To go: Mon-Pri<br>
<br>
</font></span><br></div></div>_____________________________________________=
__<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>
--089e01538ac8b3e2d205218cf5fa--
|
