1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
|
Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
helo=mx.sourceforge.net)
by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <gmaxwell@gmail.com>) id 1UOAem-0007ct-Ep
for bitcoin-development@lists.sourceforge.net;
Fri, 05 Apr 2013 17:42:32 +0000
Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.217.177 as permitted sender)
client-ip=209.85.217.177; envelope-from=gmaxwell@gmail.com;
helo=mail-lb0-f177.google.com;
Received: from mail-lb0-f177.google.com ([209.85.217.177])
by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1UOAej-0006GN-2E
for bitcoin-development@lists.sourceforge.net;
Fri, 05 Apr 2013 17:42:32 +0000
Received: by mail-lb0-f177.google.com with SMTP id r10so3945700lbi.8
for <bitcoin-development@lists.sourceforge.net>;
Fri, 05 Apr 2013 10:42:22 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.112.139.7 with SMTP id qu7mr6631746lbb.13.1365183742355;
Fri, 05 Apr 2013 10:42:22 -0700 (PDT)
Received: by 10.112.134.164 with HTTP; Fri, 5 Apr 2013 10:42:22 -0700 (PDT)
In-Reply-To: <CAKuKjyUFBuPMPRV6R-u0iTa=8DWMN9vqdOxnr8o8kxg9rJtVBA@mail.gmail.com>
References: <CAKuKjyUFBuPMPRV6R-u0iTa=8DWMN9vqdOxnr8o8kxg9rJtVBA@mail.gmail.com>
Date: Fri, 5 Apr 2013 10:42:22 -0700
Message-ID: <CAAS2fgT71s0EgF055wyqcCdXmMtY-=q0pQDh9P8pRgKJcELOQg@mail.gmail.com>
From: Gregory Maxwell <gmaxwell@gmail.com>
To: Adam Ritter <aritter@gmail.com>
Content-Type: text/plain; charset=UTF-8
X-Spam-Score: -1.6 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(gmaxwell[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1UOAej-0006GN-2E
Cc: Bitcoin Development <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Integration testing for BitCoin
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Fri, 05 Apr 2013 17:42:32 -0000
On Fri, Apr 5, 2013 at 10:24 AM, Adam Ritter <aritter@gmail.com> wrote:
> Hey guys,
>
> I just bought some BitCoins after being lazy to do it for the last few
> years, but also looked at the client code and the messages that are
> going on this mailing list.
> I saw that there are quite some unit tests, but I didn't find
> integration test for BitCoin, and I believe that it's quite important
> for the future of BitCoin (making the current code more stable,
> testing attack scenarios, refactoring and extending code).
[...]
> Tests that simulate multiple bitcoin users and can verify that the
> whole network of bitcoin clients work together
> to achieve the goals of Bitcoin. Also maybe [System
> testing](http://en.wikipedia.org/wiki/System_testing)
> would be a better name for the tests, but I'm not sure.
I prefer to call them system tests.
We use a system called blocktester that Matt Corallo wrote,
https://code.google.com/r/bluemattme-bitcoinj/source/browse/core/src/test/java/com/google/bitcoin/core/FullBlockTestGenerator.java?name=fullverif&r=874c5904b12d1fcec5b556429cf208f63cd4e1bc
It's based on BitcoinJ and works by simulating a peer against a
slightly instrumented copy of Bitcoin(d/-qt) (modified to avoid
computationally expensive mining). The tests simulates many
complicated network scenarios and tests the boundaries of many
(hopefully all) the particular rules of the blockchain validation
protocol. We can use these tests to compare different versions of the
reference software to each other and to bitcoinj (or other full node
implementations) as well as comparing them to our abstract
understanding of what we believe the rules of the protocol to be.
These tests are run as part of the automated tests on every proposed
patch to the reference software. Via a robot called pulltester which
comments on github requests and produces logs like this:
http://jenkins.bluematt.me/pull-tester/92a129980fb9b506da6c7f876aa8adb405c88e17/.
Pulltester also performs automatic code coverage measurements.
Additionally, we run a public secondary test bitcoin network called
'testnet', which can be accessed by anyone by starting the reference
software with testnet=1. Testnet operates the same as the production
network except it allows mining low difficulty blocks to prevent it
going for long times without blocks, and some of the protective
relaying rules against "non standard" transaction types are disabled.
Most of this testing work has been centered around validating the
blockchain behavior because thats what has serious systemic risk.
Measuring the json rpc behavior is strictly less interesting, though
interesting too.
|
