path: root/15/2b2cd926c375e4108c18da945f0a22b445d23c

Return-Path: <jeanpaulkogelman@me.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id B421183D
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu,  2 Jul 2015 08:32:45 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from st11p02im-asmtp001.me.com (st11p02im-asmtp001.me.com
	[17.172.220.113])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id EB18F154
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Thu,  2 Jul 2015 08:32:44 +0000 (UTC)
Received: from [10.0.1.7] (216-19-182-8.dyn.novuscom.net [216.19.182.8])
	by st11p02im-asmtp001.me.com
	(Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Mar 31
	2015))
	with ESMTPSA id <0NQU00EI5QEHC230@st11p02im-asmtp001.me.com> for
	bitcoin-dev@lists.linuxfoundation.org;
	Thu, 02 Jul 2015 08:32:43 +0000 (GMT)
X-Proofpoint-Virus-Version: vendor=fsecure
	engine=2.50.10432:5.14.151,1.0.33,0.0.0000
	definitions=2015-07-02_07:2015-06-30, 2015-07-02,
	1970-01-01 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0
	suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam
	adjust=0
	reason=mlx scancount=1 engine=7.0.1-1412110000
	definitions=main-1507020146
MIME-version: 1.0 (Mac OS X Mail 8.2 \(2098\))
Content-type: multipart/signed;
	boundary="Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9";
	protocol="application/pgp-signature"; micalg=pgp-sha512
X-Pgp-Agent: GPGMail 2.5
From: Jean-Paul Kogelman <jeanpaulkogelman@me.com>
In-reply-to: <20150702071814.GA1904@banane.informatik.uni-ulm.de>
Date: Thu, 02 Jul 2015 01:33:08 -0700
Message-id: <7A4D6052-CAA8-4E22-92E4-83828E589570@me.com>
References: <F6C7E867-1CCA-4DFB-8A88-361316A76FC3@me.com>
	<20150702071814.GA1904@banane.informatik.uni-ulm.de>
To: Henning Kopp <henning.kopp@uni-ulm.de>
X-Mailer: Apple Mail (2.2098)
X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, 
	RP_MATCHES_RCVD autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] Defining a min spec
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Thu, 02 Jul 2015 08:32:45 -0000


--Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

In the case of Bitcoin Core, for a starting point, you basically have to =
work backwards from what we have right now. We know some of the bounds =
already. Block size already tells you a lot about your bandwidth =
requirements, and Pieter=E2=80=99s simulations gives you even more =
information when you take orphan rates into account. There=E2=80=99s =
also a hard cap on the number of SigOps if I recall correctly, so =
that=E2=80=99s probably a good starting point for a MIPS metric, etc.

Memory is probably the hardest to pin down since some memory structures =
like (from what I understand, correct me if I=E2=80=99m wrong) the UTXO =
database live fully in memory and are basically unbounded. Perhaps this =
can somehow be capped at a certain size and move all the really old =
UTXO=E2=80=99s that are unlikely to move to disk and just take the CPU / =
disk hit in the unlikely event that they are referenced by a new block. =
Has the address database been capped yet? Mempool? I realize that it=E2=80=
=99s probably debatable whether or not this behaviour should be =
independent of available memory since any bugs here could affect =
consensus (especially the UTXO db).

Ultimately, what comes out of it is a list of numbers. A Mbit network =
I/O, B MIPS, C MB memory, D Disk space, etc. At that point you can =
debate whether or not such a machine can be considered an entrypoint =
bitcoin full node. You round up the numbers that are not really =
available anymore in off the shelf hardware (like disk space) and you =
round down the numbers that seem too high. For all we know we=E2=80=99re =
already over budget on some of the metrics that we decide to track as =
min spec. Network I/O for example. At that point you can start focussed =
research into bringing Bitcoin Core back into budget on those metrics. =
Then the discussion moves from =E2=80=9Cit=E2=80=99s probably too =
high=E2=80=9D to =E2=80=9Cwe=E2=80=99re X% over budget=E2=80=9D.

The most valuable thing that could come out of this is to get some kind =
of formulation how all the different levers in Bitcoin Core affect the =
min spec and ideally have a benchmark tool. For example, we could settle =
on a min spec that would exclude the Raspberry Pi 1 on MIPS, but when =
secp256k1 is enabled for validation, the MIPS requirement could drop =
significantly, allowing us to adjust the min spec downward to include =
the Raspberry Pi 1 again (again, just an example).

Ideally some people would have the actual min spec machine built and =
running. The cost of that shouldn=E2=80=99t be too high (it=E2=80=99s =
the min spec after all) and I=E2=80=99m sure people would be happy to =
chip in a couple bits for this.

Remember, the min spec should be able to handle Bitcoin Core running =
under full load; that=E2=80=99s maxed out blocks with maxed out SigOps, =
etc.


jp


> On Jul 2, 2015, at 12:18 AM, Henning Kopp <henning.kopp@uni-ulm.de> =
wrote:
>=20
> Hi Jean-Paul,
>=20
> that's a very interesting point of view and I have never thought about
> it this way, since I have a totally different background.
>=20
> How would you go on about defining a min spec? Is this done by testing
> the software on different hardware configurations or are you looking
> at the requirements a priori?
>=20
> Best regards
> Henning
>=20
>=20
> On Wed, Jul 01, 2015 at 09:04:19PM -0700, Jean-Paul Kogelman wrote:
>> Hi folks,
>>=20
>> I=E2=80=99m a game developer. I write time critical code for a living =
and have to deal with memory, CPU, GPU and I/O budgets on a daily basis. =
These budgets are based on what we call a minimum specification (of =
hardware); min spec for short. In most cases the min spec is based on =
entry model machines that are available during launch, and will give the =
user an enjoyable experience when playing our games. Obviously, we can =
turn on a number of bells and whistles for people with faster machines, =
but that=E2=80=99s not the point of this mail.
>>=20
>> The point is, can we define a min spec for Bitcoin Core? The number =
one reason for this is: if you know how your changes affect your =
available budgets, then the risk of breaking something due to capacity =
problems is reduced to practically zero.
>>=20
>> One way of doing so is to work backwards from what we have right now: =
Block size (network / disk I/O), SigOps/block (CPU), UTXO size (memory), =
etc. Then there=E2=80=99s Pieter=E2=80=99s analysis of network =
bottlenecks and how it affects orphan rates that could be used to set =
some form of cap on what transfer time + verification time should be to =
keep the orphan rate at an acceptable level.
>>=20
>> So taking all of the above (and more) into account, what =
configuration would be the bare minimum to comfortably run Bitcoin Core =
at maximum load and can it be reasonably expected to still be out there =
in the field running Bitcoin Core? Also, can the parameters that were =
used to determine this min spec be codified in some way so that they can =
later be used if Bitcoin Core is optimized (or extended with new =
functionality) and see how it affects the min spec? Basically, with any =
reasonably big change, one of the first questions could be: =E2=80=9CHow =
does this change affect min spec?"
>>=20
>> For example, currently OpenSSL is used to verify the signatures in =
the transactions. The new secp256k1 implementation is several times =
faster than (depending on CPU architecture, I=E2=80=99m sure) =
OpenSSL=E2=80=99s implementation. So it would result in faster =
verification time. This can then result in the following things; either =
network I/O and CPU requirements are adjusted downward in the min spec =
(you can run the new Bitcoin Core on a cheaper configuration), or other =
parameters can be adjusted upwards (number of SigOps / transaction, =
block size?), through proper rollout obviously. Since we know how min =
spec is affected by these changes, they should be non-controversial by =
default. Nobody running min spec is going to be affected by it, etc.
>>=20
>> Every change that has a positive effect on min spec (do more on the =
same hardware) basically pushes the need to start following any of the =
curve laws (Nielsen, Moore) forward. No need for miners / node operators =
to upgrade.
>>=20
>> Once we hit what we call SOL (Speed Of Light, the fastest something =
can go on a specific platform) it=E2=80=99s time to start looking at =
periodically adjusting min spec upwards, or by that time maybe it=E2=80=99=
s possible to use conservative plots of the curve laws as a basis.
>>=20
>> Lastly, a benchmark test could be developed that can tell everyone =
running Bitcoin Core how their setup compares to the min spec and how =
long they can expect to run on this setup.
>>=20
>> What do you guys think?
>>=20
>>=20
>> jp
>=20
>=20
>=20
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>=20
>=20
> --
> Henning Kopp
> Institute of Distributed Systems
> Ulm University, Germany
>=20
> Office: O27 - 3402
> Phone: +49 731 50-24138
> Web: http://www.uni-ulm.de/in/vs/~kopp


--Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org

iQIcBAEBCgAGBQJVlPdEAAoJEG93Vo4Z7tpFmYgQAMYaSUrped38/ASQnjME/jG7
ojcykAFIAArlHKh4qLhka7JnsZ47i3dYrzEZN0IOUGyOANnaa7zddsfDo29uT89O
kk3t/LSE04Bdk/e2J18cwnqsp8yfdhQyJ+XyOLzwVtaSUuwm5SVZd0RwkScB+Crq
bm8muzut/2IDD+RTCmuXnBBLSp1Hss1tnDuhv1sQh9w1UAfh278Z6XHtLu/Qgm6y
45DTWi8btvST7I68CGY5ftYOQOxNGGLz7ZLRSVwrxBDwGtU0NS4p4OUu/L5Vt/lZ
5f2lDFUOCsyuXVBfmm8eozMECTCm1aCneEwOKt9k0B4q6hv8YYj154qFzPn5mVYp
d5SEHO7riSKNFYcqC/pWYu7rFInTxw2BRWrs1gjDA7AlwFHjd+GTTkX5H/fLMAe1
atyI+4xFVbCAE4HI9K0yAXi9oJ9uo/tC8qaXSSfpOTWhpXG+iACKpA6BHJJg7Q7s
eeu5gNKNSF/AWBYykcuHwBlI9dVOiDtDifah8XpkF7jeqwMN0sFnZdBqQxuSGrFy
tihcJ2yC3e8kciLJ6ls9Y1rZbobi/7RBTATKK7VwatIcR+Wv2FxULJumKwtENYcu
kdZj5njc8gdMpcditWT0h/cpX6kHCbA+E4uqu+LVDCV2QLaIbalQo5ZHjOqC9EP5
D6Uzafx1EHzHdTxOW/9E
=9JCZ
-----END PGP SIGNATURE-----

--Apple-Mail=_A6FCAFAC-1DF1-4296-8850-DF0741FE9AB9--