Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from <tamas@bitsofproof.com>) id 1WXFYi-0004Jm-5Z for bitcoin-development@lists.sourceforge.net; Mon, 07 Apr 2014 19:50:20 +0000 X-ACL-Warn: Received: from wp059.webpack.hosteurope.de ([80.237.132.66]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1WXFYg-0001Ws-JP for bitcoin-development@lists.sourceforge.net; Mon, 07 Apr 2014 19:50:20 +0000 Received: from [37.143.74.116] (helo=[192.168.0.102]); authenticated by wp059.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) id 1WXFYa-0006W9-1S; Mon, 07 Apr 2014 21:50:12 +0200 Content-Type: multipart/signed; boundary="Apple-Mail=_6B6B5EE8-080F-41CA-BBAB-EBBB71364511"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Tamas Blummer <tamas@bitsofproof.com> In-Reply-To: <BLU170-W124FFC4A5D42BC516663032A5680@phx.gbl> Date: Mon, 7 Apr 2014 21:50:26 +0200 Message-Id: <AAC2AFE2-2305-48CF-850A-522CFF2A0ACE@bitsofproof.com> References: <CANEZrP2rgiQHpekEpFviJ22QsiV+s-F2pqosaZOA5WrRtJx5pg@mail.gmail.com>, <5342C833.5030906@gmail.com>, <CAAS2fgTqBfEPXh2dfcL_ke6c0wfXw4qUM1rAYMkAHcAM6mYH+g@mail.gmail.com>, <6D430188-CE00-44B1-BD8C-B623CF04D466@icloudtools.net>, <CANEZrP1-9LpPw4WuY8bfsEG0OLoDECXTfQCoZsZ4tmOn2H7Omw@mail.gmail.com>, <6D6E55CE-2F04-4C34-BEE6-98AEF1478346@bitsofproof.com>, <CAAS2fgQaJ6P4Aj2A5Zox+CiGQK6jHvF1CkLH1R6xbadYhUXO2g@mail.gmail.com>, <8A6DEBA4-EA59-4BAE-95CF-C964C2DBB84B@bitsofproof.com>, <CAAS2fgT_9tXCxOHX_sN0wa=GRMn5seu3-o1UdiLjvbivFr46_w@mail.gmail.com>, <8222EAFD-813E-4046-A751-FD3D04FF6764@bitsofproof.com> <BLU170-W124FFC4A5D42BC516663032A5680@phx.gbl> To: Paul Lyon <pmlyon@hotmail.ca> X-Mailer: Apple Mail (2.1874) X-bounce-key: webpack.hosteurope.de; tamas@bitsofproof.com; 1396900218; 21e6c4e6; X-Spam-Score: 1.0 (+) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 1.0 HTML_MESSAGE BODY: HTML included in message X-Headers-End: 1WXFYg-0001Ws-JP Cc: Bitcoin Development <bitcoin-development@lists.sourceforge.net> Subject: Re: [Bitcoin-development] Why are we bleeding nodes? X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: <bitcoin-development.lists.sourceforge.net> List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>, <mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe> List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development> List-Post: <mailto:bitcoin-development@lists.sourceforge.net> List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help> List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>, <mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe> X-List-Received-Date: Mon, 07 Apr 2014 19:50:20 -0000 --Apple-Mail=_6B6B5EE8-080F-41CA-BBAB-EBBB71364511 Content-Type: multipart/alternative; boundary="Apple-Mail=_9C53F092-AA2C-4AFA-8FE3-8D476553258C" --Apple-Mail=_9C53F092-AA2C-4AFA-8FE3-8D476553258C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 You have to load headers sequantially to be able to connect them and = determine the longest chain. Blocks can be loaded in random order once you have their order given by = the headers. Computing the UTXO however will force you to at least temporarily store = the blocks unless you have plenty of RAM.=20 Regards, Tamas Blummer http://bitsofproof.com On 07.04.2014, at 21:30, Paul Lyon <pmlyon@hotmail.ca> wrote: > I hope I'm not thread-jacking here, apologies if so, but that's the = approach I've taken with the node I'm working on. >=20 > Headers can be downloaded and stored in any order, it'll make sense of = what the winning chain is. Blocks don't need to be downloaded in any = particular order and they don't need to be saved to disk, the UTXO is = fully self-contained. That way the concern of storing blocks for seeding = (or not) is wholly separated from syncing the UTXO. This allows me to do = the initial blockchain sync in ~6 hours when I use my SSD. I only need = enough disk space to store the UTXO, and then whatever amount of block = data the user would want to store for the health of the network. >=20 > This project is a bitcoin learning exercise for me, so I can only hope = I don't have any critical design flaws in there. :) >=20 > From: tamas@bitsofproof.com > Date: Mon, 7 Apr 2014 21:20:31 +0200 > To: gmaxwell@gmail.com > CC: bitcoin-development@lists.sourceforge.net > Subject: Re: [Bitcoin-development] Why are we bleeding nodes? >=20 >=20 > Once headers are loaded first there is no reason for sequential = loading.=20 >=20 > Validation has to be sequantial, but that step can be deferred until = the blocks before a point are loaded and continous. >=20 > Tamas Blummer > http://bitsofproof.com >=20 > On 07.04.2014, at 21:03, Gregory Maxwell <gmaxwell@gmail.com> wrote: >=20 >> On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer = <tamas@bitsofproof.com> wrote: >>> therefore I guess it is more handy to return some bitmap of = pruned/full >>> blocks than ranges. >>=20 >> A bitmap also means high overhead and=97 if it's used to advertise >> non-contiguous blocks=97 poor locality, since blocks are fetched >> sequentially. >>=20 >=20 >=20 > = --------------------------------------------------------------------------= ---- Put Bad Developers to Shame Dominate Development with Jenkins = Continuous Integration Continuously Automate Build, Test & Deployment = Start a new project now. Try Jenkins in the = cloud.http://p.sf.net/sfu/13600_Cloudbees > _______________________________________________ Bitcoin-development = mailing list = Bitcoin-development@lists.sourceforge.nethttps://lists.sourceforge.net/lis= ts/listinfo/bitcoin-development --Apple-Mail=_9C53F092-AA2C-4AFA-8FE3-8D476553258C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">You = have to load headers sequantially to be able to connect them and = determine the longest chain.<div><br></div><div>Blocks can be loaded in = random order once you have their order given by the = headers.</div><div>Computing the UTXO however will force you to at least = temporarily store the blocks unless you have plenty of = RAM. <br><div apple-content-edited=3D"true"><br style=3D"color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = normal; font-variant: normal; font-weight: normal; letter-spacing: = normal; line-height: normal; text-align: -webkit-auto; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; orphans: 2; widows: 2;"><span = style=3D"color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = font-style: normal; font-variant: normal; font-weight: normal; = letter-spacing: normal; line-height: normal; text-align: -webkit-auto; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; orphans: 2; widows: = 2; float: none; display: inline !important;">Regards,</span><br = style=3D"color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = font-style: normal; font-variant: normal; font-weight: normal; = letter-spacing: normal; line-height: normal; text-align: -webkit-auto; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; orphans: 2; widows: = 2;"><br style=3D"color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant: normal; font-weight: normal; = letter-spacing: normal; line-height: normal; text-align: -webkit-auto; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; orphans: 2; widows: = 2;"><span style=3D"color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant: normal; font-weight: = normal; letter-spacing: normal; line-height: normal; text-align: = -webkit-auto; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; orphans: 2; = widows: 2; float: none; display: inline !important;">Tamas = Blummer</span><span style=3D"color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 12px; font-style: normal; font-variant: normal; = font-weight: normal; letter-spacing: normal; line-height: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px;"><br style=3D"color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = line-height: normal; text-align: -webkit-auto; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; orphans: 2; widows: 2;"><span = style=3D"color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; = font-style: normal; font-variant: normal; font-weight: normal; = letter-spacing: normal; line-height: normal; text-align: -webkit-auto; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; orphans: 2; widows: = 2; float: none; display: inline !important;"><a = href=3D"http://bitsofproof.com">http://bitsofproof.com</a></span> </span></div> <br><div><div>On 07.04.2014, at 21:30, Paul Lyon <<a = href=3D"mailto:pmlyon@hotmail.ca">pmlyon@hotmail.ca</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><blockquote = type=3D"cite"><div class=3D"hmmessage" style=3D"font-size: 12pt; = font-family: Calibri; font-style: normal; font-variant: normal; = font-weight: normal; letter-spacing: normal; line-height: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px;"><div dir=3D"ltr">I hope I'm not = thread-jacking here, apologies if so, but that's the approach I've taken = with the node I'm working on.<div><br></div><div>Headers can be = downloaded and stored in any order, it'll make sense of what the winning = chain is. Blocks don't need to be downloaded in any particular order and = they don't need to be saved to disk, the UTXO is fully self-contained. = That way the concern of storing blocks for seeding (or not) is wholly = separated from syncing the UTXO. This allows me to do the initial = blockchain sync in ~6 hours when I use my SSD. I only need enough disk = space to store the UTXO, and then whatever amount of block data the user = would want to store for the health of the = network.</div><div><br></div><div>This project is a bitcoin learning = exercise for me, so I can only hope I don't have any critical design = flaws in there. :)<br><br><div><hr id=3D"stopSpelling">From:<span = class=3D"Apple-converted-space"> </span><a = href=3D"mailto:tamas@bitsofproof.com">tamas@bitsofproof.com</a><br>Date: = Mon, 7 Apr 2014 21:20:31 +0200<br>To:<span = class=3D"Apple-converted-space"> </span><a = href=3D"mailto:gmaxwell@gmail.com">gmaxwell@gmail.com</a><br>CC:<span = class=3D"Apple-converted-space"> </span><a = href=3D"mailto:bitcoin-development@lists.sourceforge.net">bitcoin-developm= ent@lists.sourceforge.net</a><br>Subject: Re: [Bitcoin-development] Why = are we bleeding nodes?<br><br><div><br></div><div>Once headers are = loaded first there is no reason for sequential = loading. </div><div><br></div><div>Validation has to be sequantial, = but that step can be deferred until the blocks before a point are loaded = and continous.</div><br><div apple-content-edited=3D"true"><span = style=3D"font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = line-height: normal; text-align: -webkit-auto; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; orphans: 2; widows: 2; float: none; = display: inline !important;">Tamas Blummer</span><span = style=3D"font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant: normal; font-weight: normal; letter-spacing: normal; = line-height: normal; orphans: auto; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; widows: auto; word-spacing: = 0px; -webkit-text-stroke-width: 0px;"><br style=3D"font-family: = Helvetica; font-size: 12px; font-style: normal; font-variant: normal; = font-weight: normal; letter-spacing: normal; line-height: normal; = text-align: -webkit-auto; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = orphans: 2; widows: 2;"><span style=3D"font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant: normal; font-weight: = normal; letter-spacing: normal; line-height: normal; text-align: = -webkit-auto; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; orphans: 2; = widows: 2; float: none; display: inline !important;"><a = href=3D"http://bitsofproof.com/">http://bitsofproof.com</a></span></span><= /div><br><div><div>On 07.04.2014, at 21:03, Gregory Maxwell <<a = href=3D"mailto:gmaxwell@gmail.com">gmaxwell@gmail.com</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><blockquote = type=3D"cite">On Mon, Apr 7, 2014 at 12:00 PM, Tamas Blummer <<a = href=3D"mailto:tamas@bitsofproof.com">tamas@bitsofproof.com</a>> = wrote:<br><blockquote type=3D"cite">therefore I guess it is more handy = to return some bitmap of pruned/full<br>blocks than = ranges.<br></blockquote><br>A bitmap also means high overhead and=97 if = it's used to advertise<br>non-contiguous blocks=97 poor locality, since = blocks are = fetched<br>sequentially.<br><br></blockquote></div><br><br>---------------= --------------------------------------------------------------- Put Bad = Developers to Shame Dominate Development with Jenkins Continuous = Integration Continuously Automate Build, Test & Deployment Start a = new project now. Try Jenkins in the cloud.<a = href=3D"http://p.sf.net/sfu/13600_Cloudbees">http://p.sf.net/sfu/13600_Clo= udbees</a><br>_______________________________________________ = Bitcoin-development mailing list<span = class=3D"Apple-converted-space"> </span><a = href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-developm= ent@lists.sourceforge.net</a><a = href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development">= https://lists.sourceforge.net/lists/listinfo/bitcoin-development</a></div>= </div></div></div></blockquote></div><br></div></body></html>= --Apple-Mail=_9C53F092-AA2C-4AFA-8FE3-8D476553258C-- --Apple-Mail=_6B6B5EE8-080F-41CA-BBAB-EBBB71364511 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJTQwGCAAoJEPZykcUXcTkcbFkIAIOtdjpl3mMiacF8/0PijDrn SsPCaWT4JIHlMlNvfawRY8Qn7obCCr8yN6QKo6yJg/+hNxQq14TNoEC6a4GHke0E LzOSGeHwpBv6f0cWRHDhO+pbQt6Y1gOsfH7Qld7fNej3KdEv/ZmT5Q2/dMIBzGCx aDZ1UQqYHq4VDylAE/rpLOyZxLQddqDMY/LxRQHvTv9PjJ5nm+Ivr3wwvd+sqW0G xj0p6J9qSqcBsM/4naVDcCybUDWR1DG/qi1foQP8mnlFPLkwDwARbAraqeUzESgS ECzMVh91CGWiTnB3G0Pemt4badisA5/hN7LqV7l41Mqf4FOXMSuF9PsRoJHjExM= =Zjro -----END PGP SIGNATURE----- --Apple-Mail=_6B6B5EE8-080F-41CA-BBAB-EBBB71364511--