path: root/16/31631add6a619f2c7f07ef01a284c502ad2162

Delivery-date: Fri, 02 May 2025 15:24:42 -0700
Received: from mail-qv1-f60.google.com ([209.85.219.60])
	by mail.fairlystable.org with esmtps  (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
	(Exim 4.94.2)
	(envelope-from <bitcoindev+bncBC5P5KEHZQLBBIMM2XAAMGQEPBVRMVQ@googlegroups.com>)
	id 1uAyoD-0003RT-PT
	for bitcoindev@gnusha.org; Fri, 02 May 2025 15:24:42 -0700
Received: by mail-qv1-f60.google.com with SMTP id 6a1803df08f44-6e8f9057432sf50681936d6.1
        for <bitcoindev@gnusha.org>; Fri, 02 May 2025 15:24:41 -0700 (PDT)
ARC-Seal: i=2; a=rsa-sha256; t=1746224676; cv=pass;
        d=google.com; s=arc-20240605;
        b=kTLDXuPoRvC2CuuK4S8GQA8/WGBeZO+HshRTvCmSBiWbHBJ7WcEMcWNx9M0A3Lho94
         7Sl4eW8yhokdEOm1jY11gY7VSwRHRz68L6dH7pLkTXO080smEG2suB6KwcPV9kdzPJ3c
         GZg1M5KGwKFx9f81YZce7U57lYLHjX09kNpLbAjDWOCrTdftQ8C7Nv2llWxDCAJFrQ66
         wWR6mY+YX4LPchhjZ+s/FYeImEnc2aMPg9XB9i1n/G6Vt9aMvFEEHW9r0hamrpIIHZ6r
         57p3q02wj118sqP4pDb2j8yzLWWSsjaXbYzWN7YF3TIbqNJB3m4SHHbUmFud0azG9IBt
         H+Vw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605;
        h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post
         :list-id:mailing-list:precedence:thread-index:content-language
         :content-transfer-encoding:mime-version:message-id:date:subject
         :in-reply-to:references:to:from:sender:dkim-signature;
        bh=bslp+BIUJ94txuQVFCB4bEpJaERRDuLwHLGcVc/VKms=;
        fh=mEMpKTToPHo71zUiQs/6palwAcz8VSe6UBH+W7JN3M0=;
        b=MpOzY16TszSXdyEEGO+7BED0JxSVc7Rpx8sZVxsYx0pf8sYt18DMS9zclzHoum13oV
         l+fxd6iqpSG7RmFUSGVfk676Q5OxRCsLgwxonnobBmH6jwJBpw50I55niUMYHvi5PRxy
         6CJYfkckg52muYFCVn955hEteduHwT0qyhigYx1lm4x2rN/U2jEa/2AHuC4Sc/OS2zEy
         7QjhKkkN/bfS4sGuF1Wk50IZeWV5ixv+OS0YNTjpHawd8PV1nC0ryNaXinl033LLlwSt
         cuxR8cSjOvMEIHLplSDRUm2GwHtQiGSR4WeLyzWWFqYbQd4mflf+nTBLheNO3E5j4QnZ
         wQ3g==;
        darn=gnusha.org
ARC-Authentication-Results: i=2; gmr-mx.google.com;
       dkim=pass header.i=@voskuil-org.20230601.gappssmtp.com header.s=20230601 header.b=0YHeZ3g7;
       spf=none (google.com: eric@voskuil.org does not designate permitted sender hosts) smtp.mailfrom=eric@voskuil.org;
       dara=pass header.i=@googlegroups.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=googlegroups.com; s=20230601; t=1746224676; x=1746829476; darn=gnusha.org;
        h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post
         :list-id:mailing-list:precedence:x-original-authentication-results
         :x-original-sender:thread-index:content-language
         :content-transfer-encoding:mime-version:message-id:date:subject
         :in-reply-to:references:to:from:sender:from:to:cc:subject:date
         :message-id:reply-to;
        bh=bslp+BIUJ94txuQVFCB4bEpJaERRDuLwHLGcVc/VKms=;
        b=G+r5Ryim9fudKFCTL0PlTWsveLwh2ZJ8eJrMomJ6vDJoSqzJlns+32qSshQeBEHXQl
         s77wI5LOY102cmescO18m+c5L5z2aqH1NbowygZVGyPYQ+VkiJvnkSYHTM/CiOjfLVLu
         sxhiOrtgmk6jrEDBIMwuv53uZx2EgDd1egcXpNvD6aIYcdHkyNxR2/DFnFkDNgppUyos
         oV7BdS45JL+cfvqqnlaoklgBQ0p+AoNufbttcJExkOj6rlfi1qPyECWffxtWd56AEBfR
         /1foQsnqXzqiOMxoqEeL9RpsLc9/8YpUhEbbsJEcSQBFaR40pasJF/ReTz7sacpBaGYI
         5tpw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1746224676; x=1746829476;
        h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post
         :list-id:mailing-list:precedence:x-original-authentication-results
         :x-original-sender:thread-index:content-language
         :content-transfer-encoding:mime-version:message-id:date:subject
         :in-reply-to:references:to:from:x-beenthere:x-gm-message-state
         :sender:from:to:cc:subject:date:message-id:reply-to;
        bh=bslp+BIUJ94txuQVFCB4bEpJaERRDuLwHLGcVc/VKms=;
        b=Z90wDemjzIqRnhklajnvQihQYbeToijP8o0XBPijUN7YMDzWYzcD0XCTiaZjDFH2Ho
         JUp5QX03apEb+Wf58va33PoxVJ5AYVQwJKMGLazel1yJzVluKYQB1zquWkfNmXaH3DDx
         cbstGJDNV0GZlt1SuZzWUoizCN5vetVyM8QkZ//ZZ6BlGQJEvJzp9lRtJXl64HDis7sa
         jikDR3wqwJEct3SxWF9WMtuzZZTI0v3Z6Y9ewCsqnkiyo+pA/lnuT9sw1npJxlMxb5iD
         dkX6JBBrWrOldkb+FEzvm3hbwjqiw4bRl1E5naO7dDU2HhJs0TmPMMXaL2/LnMIAAqwx
         iAdg==
Sender: bitcoindev@googlegroups.com
X-Forwarded-Encrypted: i=2; AJvYcCUwuJmUfyykc8801UH6hYSCBuA7PkW6GKdIvE9w2ZVIbzAAR2OXBmR/vAzzOXjzBKTVIXu/jLEuV922@gnusha.org
X-Gm-Message-State: AOJu0YzqrTQkRrubUVda3SgUQnJB7yEcoyVGX9Z7QwL04yVlpkhlUvjh
	7cf9QU3g7Ih2rQpEBB0EReQe4g65I2V1iXMAXwJspIyrBQz8QHow
X-Google-Smtp-Source: AGHT+IFz8LGAPn5jb18p/zbk8giOQDYlRcKuJu8H64OGF7H/mftiKy/qqWwvQv7SF+3akg3GNBUUBg==
X-Received: by 2002:a05:6214:224e:b0:6e8:fcc9:a291 with SMTP id 6a1803df08f44-6f51538de61mr87067876d6.23.1746224675889;
        Fri, 02 May 2025 15:24:35 -0700 (PDT)
X-BeenThere: bitcoindev@googlegroups.com; h=AVT/gBGgQErraniDC4vuByCWPLuprMDGyHRNSRE/2gqA9OvcCA==
Received: by 2002:a05:6214:189:b0:6e8:efc0:7a39 with SMTP id
 6a1803df08f44-6f508538c67ls33487336d6.2.-pod-prod-06-us; Fri, 02 May 2025
 15:24:32 -0700 (PDT)
X-Forwarded-Encrypted: i=2; AJvYcCXlR1IuAMWKAMpwburvZO2NJweMnLgQ+vTt016ZJVE3ATwus2pNxV+XyycO0DcJCCVjOo8GICE+60QF@googlegroups.com
X-Received: by 2002:a05:620a:6209:b0:7ca:df98:2f6 with SMTP id af79cd13be357-7cadf98069fmr106807185a.43.1746224672563;
        Fri, 02 May 2025 15:24:32 -0700 (PDT)
Received: by 2002:a05:620a:3602:b0:7b6:d2da:e6ae with SMTP id af79cd13be357-7cad5cae937ms85a;
        Fri, 2 May 2025 14:16:45 -0700 (PDT)
X-Forwarded-Encrypted: i=2; AJvYcCU8jEQqlVexhZB3ZXo0odaKDXA3QP5ehMrqsUzDD2Z8MW5COvlbLEKFtTmtf78z37yZLjwV8wMWCe4h@googlegroups.com
X-Received: by 2002:a05:6214:1bcb:b0:6eb:28e4:8519 with SMTP id 6a1803df08f44-6f51538d979mr68736826d6.21.1746220604434;
        Fri, 02 May 2025 14:16:44 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1746220604; cv=none;
        d=google.com; s=arc-20240605;
        b=aRF13qLOygKNq0of4/hHDya/Ou9tjk+SqR6CBvHSHqrMeAOUlSaSdXQzbdbeptEAgp
         wZ63Dor2mopkLjrG6988fHcJr6wG+cyXv6s8CEBlRXT/5+511mmFj29uceb7PPO+c8GR
         0ppbb8XzUolz7PpytWJnMRBj1D32Isgm+TIHU6HVQme9ucscBGS0N1ZizHcU0xdaJ2aY
         czIG4ShnrDc58qjHj6bk5HcZxiZj6pKxGOe4+lsIHVlyW7AvIVI49VrG8zBh6bJb05hw
         S9FmDVCFGzcJ7LvAA94A8xQDLrJkoe9nPect6YuUv546BGVrFWdLfFo2JuDJ3Ipy7+CA
         y16Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605;
        h=thread-index:content-language:content-transfer-encoding
         :mime-version:message-id:date:subject:in-reply-to:references:to:from
         :dkim-signature;
        bh=rXCfnXNe4w3E4vZjr1ejc18dzAaxAhVPhEtyuYcWIfw=;
        fh=fI++9PkZmWm5vlxQWjWLdPbe4TNFNhp7TfiBb7Dkdcs=;
        b=e7hOkXavcy5eQOG+NnQnLOC5RQCHpDpXdXcRMGV1xUXTdQqYOle3bWZ278AITwMUh0
         2fc7QfnPmgz59mXo8roGjRfwLTncNRCLV3JkDHFOzUTegUXKLWqJKi6TEbwHOLYR/FOV
         ONBBIIJpnQ9ar4/XWxZ4J6oG/It2wkgg2KES62bUaMYbLI7jHIk+DeTWE+FjZwPTPkLC
         5jBWMei/9zXerNUkYIoBCGp6aC/HE3y9G5bhB5v1OpVEs2EVWM79RyBSosQ/Sdji6jd9
         f65qBfxNBpdHQ63Co0VQcyyVpJ7p9XC+Q1TQYMjU447iOv12sY0xyDapS9MUIwGrJW1x
         EGDg==;
        dara=google.com
ARC-Authentication-Results: i=1; gmr-mx.google.com;
       dkim=pass header.i=@voskuil-org.20230601.gappssmtp.com header.s=20230601 header.b=0YHeZ3g7;
       spf=none (google.com: eric@voskuil.org does not designate permitted sender hosts) smtp.mailfrom=eric@voskuil.org;
       dara=pass header.i=@googlegroups.com
Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com. [2607:f8b0:4864:20::72a])
        by gmr-mx.google.com with ESMTPS id 6a1803df08f44-6f50f3acf36si542836d6.1.2025.05.02.14.16.44
        for <bitcoindev@googlegroups.com>
        (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
        Fri, 02 May 2025 14:16:44 -0700 (PDT)
Received-SPF: none (google.com: eric@voskuil.org does not designate permitted sender hosts) client-ip=2607:f8b0:4864:20::72a;
Received: by mail-qk1-x72a.google.com with SMTP id af79cd13be357-7cadd46ea9aso86264985a.1
        for <bitcoindev@googlegroups.com>; Fri, 02 May 2025 14:16:44 -0700 (PDT)
X-Forwarded-Encrypted: i=1; AJvYcCVOGgcP+2yIpwIYZEYMz+9dQA2mmCBRaPKWyFsxpMyzHbTWF4hVOiIS4lr5OMOjAI6+bqAQqaxYnECr@googlegroups.com
X-Gm-Gg: ASbGncs+d/f9pButmmIB2xg8w/x7OZEmHPJDctwLJNgai7L2jFOHmW3K54R6KaSVgXT
	hxT/DxxAEa3fM+Vrhz7Bu4XXXu7jqsWaQLUwQ8TYVRNdCr2700x/y5Fy9Itzv+z9g081nRax/rM
	SXJ1Dv+m7iYrvXk/CBaBo+pOBMXM00H2bBoFu6WmLmxJNCadVPj7sjsWVzveO6lGbJS8nZLPApa
	Re25tywInqDOyuYIjFud2ImflE1EDrQTV7DV8pbDSK3b3o9nrhRyXO/XB1vSqTv7XgOhdrrcxqU
	NrpPZLH1+TweUluPqcF2JcQ9gmQdyYab1ZM6kHVKffDAXvKV6LX4YbpDJg7fbeSs9ci3YuT/IrR
	qp/CTyQ==
X-Received: by 2002:a05:620a:400c:b0:7c5:4be5:b0b3 with SMTP id af79cd13be357-7cad5ba38a3mr682442085a.48.1746220603725;
        Fri, 02 May 2025 14:16:43 -0700 (PDT)
Received: from ERICDESKTOP (c-73-227-67-43.hsd1.nh.comcast.net. [73.227.67.43])
        by smtp.gmail.com with ESMTPSA id af79cd13be357-7cad23d24besm236865285a.58.2025.05.02.14.16.42
        (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
        Fri, 02 May 2025 14:16:43 -0700 (PDT)
From: <eric@voskuil.org>
To: "'Sjors Provoost'" <sjors@sprovoost.nl>,
	"'Bitcoin Development Mailing List'" <bitcoindev@googlegroups.com>
References: <F8E9B25A-5198-4A5E-B3D7-9DAD6B709825@sprovoost.nl> <8E4DFC2E-23D4-4D22-87C5-415A3CFC7E57@sprovoost.nl>
In-Reply-To: <8E4DFC2E-23D4-4D22-87C5-415A3CFC7E57@sprovoost.nl>
Subject: RE: [bitcoindev] Removing checkpoints in Bitcoin Core v30
Date: Fri, 2 May 2025 17:16:42 -0400
Message-ID: <035701dbbba7$807f23b0$817d6b10$@voskuil.org>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Mailer: Microsoft Outlook 16.0
Content-Language: en-us
Thread-Index: AQGpX3yTIPgtJr/2QclkLvTcE2GpaQIzdeJRtBLR2fA=
X-Original-Sender: eric@voskuil.org
X-Original-Authentication-Results: gmr-mx.google.com;       dkim=pass
 header.i=@voskuil-org.20230601.gappssmtp.com header.s=20230601
 header.b=0YHeZ3g7;       spf=none (google.com: eric@voskuil.org does not
 designate permitted sender hosts) smtp.mailfrom=eric@voskuil.org;
       dara=pass header.i=@googlegroups.com
Precedence: list
Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com
List-ID: <bitcoindev.googlegroups.com>
X-Google-Group-Id: 786775582512
List-Post: <https://groups.google.com/group/bitcoindev/post>, <mailto:bitcoindev@googlegroups.com>
List-Help: <https://groups.google.com/support/>, <mailto:bitcoindev+help@googlegroups.com>
List-Archive: <https://groups.google.com/group/bitcoindev
List-Subscribe: <https://groups.google.com/group/bitcoindev/subscribe>, <mailto:bitcoindev+subscribe@googlegroups.com>
List-Unsubscribe: <mailto:googlegroups-manage+786775582512+unsubscribe@googlegroups.com>,
 <https://groups.google.com/group/bitcoindev/subscribe>
X-Spam-Score: -0.8 (/)

Hi Sjors,

> In the context of BIP30 [0] Eric Voskuil brought up performance:

I didn't originate the point on performance. I said (in the BIP30 thread):

> One reference states that =E2=80=9Cassume valid=E2=80=9D speeds IBD, but =
of course it does so by not validating.

Which was in response to your OP of this thread, which provided that refere=
nce, specifically:

"Assumed Valid Blocks: a feature designed to replace the secondary use of c=
heckpoints for (optionally) speeding up Initial Block Download (IBD) by ski=
pping validation of signatures in old blocks. This was deployed in Bitcoin =
Core 0.14" - David A. Harding

This in turn references the following discussion:

"Bitcoin 0.5.0 built upon those checkpoints to speed up IBD by skipping ver=
ification of signatures in blocks that were earlier in the block chain than=
 the most recent checkpoint."

https://bitcoincore.org/en/2017/03/08/release-0.14.0/#assumed-valid-blocks

> >  The top checkpoint is consensus for over 11 years and materially reduc=
es
> the validation cost of 295,000 blocks.
>=20
> I don't think performance should be a consideration when removing
> checkpoints.

To be perfectly clear, I am not arguing against this hard fork because it r=
educes IBD cost. I=E2=80=99m pointing out that one of the arguments *in fav=
or* of removing checkpoints is that "assume valid" now serves this "seconda=
ry use". However, as I pointed out, assume valid is not consensus, it achie=
ves this outcome by trusting, not validating. The existing checkpoints are =
consensus - they provide this advantage when fully validating.

I'm not suggesting that checkpoints need to be added to improve performance=
. I'm pointing out that removing them hurts performance. So performance is =
obviously not a reason to accept such a hard fork.

> Afaik checkpoints were not introduced as a performance feature, but rathe=
r as
> a DoS vulnerability fix.

It appears from the above discussion that there was more than one reason. H=
owever this isn't relevant. What matters is the consequences of removing th=
em, not why they were introduced.

> Even if they were, imo they shouldn't be used for performance enhancement=
,
> as it creates a temptation to add more - although Eric clearly says:
>=20
> > I would never advocate for adding more
>=20
> But perhaps someone else would fork Bitcoin Core (or Libbitcoin) and star=
t
> with honest checkpoints, in order to gain adoption due its excellent
> performance - especially when combined with a UTXO snapshot. And then
> suddenly it has an "upgrade".
>=20
> But even if this wasn't a concern,

This feels more like speculation than a reasonable concern. To date bitcoin=
d has introduced two trust-based mechanisms (assume valid and assume utxo) =
both with the objective of avoiding validation to improve IBD time. As I un=
derstand it, BC ships with a very recent assume valid configuration by defa=
ult. If you have this concern, it should be directed these current practice=
s, not at ancient checkpoints. Removing the latter does not eliminate the f=
ormer - which should be the current concern. I've recently seen declaration=
s that the IBD problem is "solved" because of assume valid and/or assume ut=
xo. That's not speculation.

> the performance benefit of the latest
> checkpoint from 2014 is just very small. That's because the early blocks =
don't
> have that many signatures to check. The first 5 years of Bitcoin represen=
t only
> 3% of all historical transactions.

This assumes that signature validation is the major cost, which is not the =
case for bitcoind architectures.

> On an M4 MacBook I did an IBD up to the last checkpoint at height 295,000=
.
> This took 17.5 minutes.

I have done this many times, on powerful machines, and it's generally at le=
ast an hour - even with checkpoints (and headers excluded).

> I then ran it again with -assumevalid=3D0, i.e. checking all signatures, =
which took
> 18 minutes.

The major cost for the bitcoind architecture is unspent output accumulation=
. There is only a modest distinction between full validation and assume val=
id across the entire chain. As a result this measurement doesn't reflect th=
e true (necessary) cost differential. I just performed a 1-295,000 block li=
bbitcoin sync with and without checkpoints (and no milestone/assume valid).=
 The checkpointed time is 3.4 minutes and without is 8.3 minutes (on the sa=
me machine BC takes 69.3 minutes with checkpoints). So checkpoints validate=
d at 41% of the time vs. without (vs. your 97%). Again, this is not the bas=
is of my comments, just clarifying for the record.

> These times are from connecting the first block until connecting block
> 295,000. They exclude the 1.5 minutes spent on the header pre-sync
> mechanism, which is required to operate safely with or without (new)
> checkpoints.

Same in both cases.

> Libbitcoin would probably find a different ratio, as might a machine with
> severe CPU limitations, but I suspect it will be equally negliable. And w=
ithout
> new checkpoints the ratio keeps going down over time.

Of course, the ratio is small and will keep decreasing. On this machine it'=
s a difference of 4.9 minutes out of 4.4 hours for libbitcoin. The ratio is=
 smaller for BC, as it takes 24.5 hours to sync to the same height even wit=
h assume valid.

> That said, I've never been a fan of -assumevalid conceptually.=20

I agree, not a fan. We provide milestones (same security model) so that peo=
ple can rapidly reproduce a previously synced node from the network by spec=
ifying a milestone that they have previously validated. On my current low e=
nd ($350) benchmark on a 2.3 Gbps Internet, this takes under 40 minutes to =
block 850k, and checkpoints don't make a difference either way. It can be f=
aster to sync from the p2p network than to copy the store, and of course do=
esn't require a local copy. So it's a useful feature, but not a substitute =
for validation.

> Hopefully it can
> be replaced by something like AssumeUTXO + SwiftSync [1] for the
> background with full validation using block Undo data.

I'm not so clear on why BC wants to give people the impression that a node =
is usable when its chain hasn't been validated. I don't see this as the "ho=
peful" outcome, but that's another topic I suppose.

I also don't see the reason for all of this complexity to end up with a sec=
urity model identical to assume valid (until it's actually validated). Why =
not just sync the strong blocks and validate them in the background without=
 all of the fuss, extra bandwidth, and precomputations?

In any case, I still do not see ANY justification for hard forking out the =
checkpoints, and there are of course legitimate reasons to avoid doing so.

https://groups.google.com/g/bitcoindev/c/aqHRfa0UWFo

Best,
Eric

--=20
You received this message because you are subscribed to the Google Groups "=
Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to bitcoindev+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/bitcoindev/=
035701dbbba7%24807f23b0%24817d6b10%24%40voskuil.org.