1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
|
Return-Path: <gmaxwell@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 4EBDD25A
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 30 Oct 2015 04:28:48 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-ig0-f173.google.com (mail-ig0-f173.google.com
[209.85.213.173])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9EC1212A
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 30 Oct 2015 04:28:47 +0000 (UTC)
Received: by igbhv6 with SMTP id hv6so3604513igb.0
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 29 Oct 2015 21:28:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:cc:content-type:content-transfer-encoding;
bh=72FH0ykTMrej2HRR1BUp919wGQ23XrwJdF9+7PWz7Lw=;
b=lVS8LTNCteUOrialmITRJKZbnzHW99lJ6ZJNhou0rDl5p+2dlgkBrBBRCld4BaKUdI
86/JMg0tw7WW9cwffVD0PzrpmKt9BuNhzJsFrGyUYSHeh+azc3SOzrKjc/gDO6BTEiVm
GwFNv4VaY3FcAfHnVF1b4icE6ULojnfK4ujK+nYB7RTyBRRnDrQ77ysyqjIalcHmpmE6
kK1AavIMsMb4fzGArPHi7V0iPUAegMDMON+wl8Hw3fP04atEr+OFAa5tC13rsK79u25y
tBkQ3j193dCXIc4jh732recbwmhju/8g18IbCivwSOdYqvs1JTfLlSlL1bg06+sMXff4
xhrQ==
MIME-Version: 1.0
X-Received: by 10.50.43.200 with SMTP id y8mr819646igl.48.1446179327142; Thu,
29 Oct 2015 21:28:47 -0700 (PDT)
Received: by 10.107.192.199 with HTTP; Thu, 29 Oct 2015 21:28:47 -0700 (PDT)
In-Reply-To: <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com>
References: <5631C363.5060705@neomailbox.net>
<201510290803.52734.luke@dashjr.org>
<5632DE33.7030600@bitcartel.com>
<CAAS2fgTga_vTfOKrFu_hEzXSfTfg9FRfJ6aL6ginuGFqnbm7=w@mail.gmail.com>
<3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com>
Date: Fri, 30 Oct 2015 04:28:47 +0000
Message-ID: <CAAS2fgRdK4bDr3x_y9UpdH234PQSfD7U539HBLA==+hLQJ_7Fw@mail.gmail.com>
From: Gregory Maxwell <gmaxwell@gmail.com>
To: Peter R <peter_r@gmx.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM,
RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>,
telemaco <telemaco@neomailbox.net>
Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Fri, 30 Oct 2015 04:28:48 -0000
On Fri, Oct 30, 2015 at 4:04 AM, Peter R <peter_r@gmx.com> wrote:
> Can you give a specific example of how nodes that used different database=
technologies might determine different answers to whether a given transact=
ion is valid or invalid? I=E2=80=99m not a database expert, but to me it w=
ould seem that if all the unspent outputs can be found in the database, and=
if the relevant information about each output can be retrieved without cor=
ruption, then that=E2=80=99s all that really matters as far as the database=
is concerned.
If you add to those set of assumptions the handling of write ordering
is the same (e.g. multiple updates in an change end up with the same
entry surviving) and read/write interleave returning the same results
then it wouldn't.
But databases sometimes have errors which cause them to fail to return
records, or to return stale data. And if those exist consistency must
be maintained; and "fixing" the bug can cause a divergence in
consensus state that could open users up to theft.
Case in point, prior to leveldb's use in Bitcoin Core it had a bug
that, under rare conditions, could cause it to consistently return not
found on records that were really there (I'm running from memory so I
don't recall the specific cause). Leveldb fixed this serious bug in a
minor update. But deploying a fix like this in an uncontrolled manner
in the bitcoin network would potentially cause a fork in the consensus
state; so any such fix would need to be rolled out in an orderly
manner.
> I=E2=80=99d like a concrete example to help me understand why more than o=
ne implementation of something like the UTXO database would be unreasonable=
.
It's not unreasonable, but great care is required around the specifics.
Bitcoin consensus implements a mathematical function that defines the
operation of the system and above all else all systems must agree (or
else the state can diverge and permit double-spends); if you could
prove that a component behaves identically under all inputs to another
function then it can be replaced without concern but this is something
that cannot be done generally for all software, and proving
equivalence even in special cases it is an open area of research. The
case where the software itself is identical or nearly so is much
easier to gain confidence in the equivalence of a change through
testing and review.
With that cost in mind one must then consider the other side of the
equation-- utxo database is an opaque compressed representation,
several of the posts here have been about desirability of blockchain
analysis interfaces, and I agree they're sometimes desirable but
access to the consensus utxo database is not helpful for that.
Similarly, other things suggested are so phenomenally slow that it's
unlikely that a node would catch up and stay synced even on powerful
hardware. Regardless, in Bitcoin core the storage engine for this is
fully internally abstracted and so it is relatively straight forward
for someone to drop something else in to experiment with; whatever the
motivation.
I think people are falling into a trap of thinking "It's a <database>,
I know a <black box> for that!"; but the application and needs are
very specialized here; no less than, say-- the table of pre-computed
EC points used for signing in the ECDSA application. It just so
happens that on the back of the very bitcoin specific cryptographic
consensus algorithim there was a slot where a pre-existing high
performance key-value store fit; and so we're using one and saving
ourselves some effort. If, in the future, Bitcoin Core adopts a
merkelized commitment for the UTXO it would probably need to stop
using any off-the-shelf key value store entirely, in order to avoid a
20+ fold write inflation from updating hash tree paths (And Bram Cohen
has been working on just such a thing, in fact).
|
