Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 938D16C for ; Sun, 20 May 2018 05:12:29 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-oln040092253028.outbound.protection.outlook.com [40.92.253.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 12417284 for ; Sun, 20 May 2018 05:12:26 +0000 (UTC) Received: from PU1APC01FT040.eop-APC01.prod.protection.outlook.com (10.152.252.58) by PU1APC01HT130.eop-APC01.prod.protection.outlook.com (10.152.253.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.696.11; Sun, 20 May 2018 05:12:23 +0000 Received: from PS2P216MB0179.KORP216.PROD.OUTLOOK.COM (10.152.252.54) by PU1APC01FT040.mail.protection.outlook.com (10.152.253.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.20.696.11 via Frontend Transport; Sun, 20 May 2018 05:12:23 +0000 Received: from PS2P216MB0179.KORP216.PROD.OUTLOOK.COM ([10.171.225.19]) by PS2P216MB0179.KORP216.PROD.OUTLOOK.COM ([10.171.225.19]) with mapi id 15.20.0776.015; Sun, 20 May 2018 05:12:23 +0000 From: Damian Williamson To: Dave Scotese , "bitcoin-dev@lists.linuxfoundation.org" Thread-Topic: [bitcoin-discuss] Checkpoints in the Blockchain. Thread-Index: AQHT7foFpbanfYqmX0m970nJ+zBo+qQ3hH2VgABb03WAADLFwg== Date: Sun, 20 May 2018 05:12:23 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-AU, en-US Content-Language: en-AU X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:1E492902D904316E8661306EE0273015CBC03D8425F631279BDA4903E6A0D1BE; UpperCasedChecksum:BAA2E233AA57B949983CC6BC86F17F294CB6306551731B2A5BF15FF0DAFBE7AD; SizeAsReceived:7381; Count:46 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [gD9PIzGBuUtHzrJ6i+WzCIfqm66rC0UH] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; PU1APC01HT130; 7:q35EDjq0HRhO/ZP2NtSwZ3kg9v6JMUbE0CL69l32CHS295YpD8QAyGnawJ0S7HgP+xKtFhuNWVJbbPIyRybpF6u+gfNC5ylieD5lzItU4Aln32E7g16bJjVAzEET8bp2WS31lQriVeZVN8fQfqfddlM2QPYRHep35b5e0SDQhhuUAI8IbV+NkIgBVVLZqgxp4zyaj3t42EZKpIWuGd0SexMNm4jQ8z58w6tX9t4Va4fyoRwMLNfm1JKIypUeqNY4 x-incomingheadercount: 46 x-eopattributedmessage: 0 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1603101448)(1601125466)(1701031045); SRVR:PU1APC01HT130; x-ms-traffictypediagnostic: PU1APC01HT130: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444000031); SRVR:PU1APC01HT130; BCL:0; PCL:0; RULEID:; SRVR:PU1APC01HT130; x-forefront-prvs: 06780E24F8 x-forefront-antispam-report: SFV:NSPM; SFS:(7070007)(189003)(199004)(52314003)(110136005)(6306002)(6346003)(236005)(86362001)(3660700001)(99286004)(8676002)(11346002)(476003)(76176011)(74482002)(2501003)(7696005)(446003)(53386004)(966005)(54896002)(93886005)(55016002)(6246003)(9686003)(53546011)(81156014)(59450400001)(25786009)(53376002)(2900100001)(6506007)(486006)(3280700002)(102836004)(14454004)(551934003)(8936002)(106356001)(229853002)(104016004)(82202002)(19627405001)(606006)(6436002)(74316002)(105586002)(68736007)(14971765001)(26005)(5660300001)(97736004)(33656002)(6606003)(77096007)(7066003); DIR:OUT; SFP:1901; SCL:1; SRVR:PU1APC01HT130; H:PS2P216MB0179.KORP216.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:; received-spf: None (protection.outlook.com: live.com.au does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=willtech@live.com.au; x-microsoft-antispam-message-info: NkhO5kpUcmCXp6ex/Bbwdq96RyEiRl5cjaerkGri+tEQAiQtMOKJ+4N262zg9NZEIxPr9oTiwOuMaYqvAwyXkOz+BOmiY5tnVcpOBPZvDTKgkCCb7lJdYd4sbC/g2Wo7UX1XRo8kjoBzKMROGy25dB6MaFotO+IXXQ1Hmi5v51jr5jo5b24hX8i2jLuZA182 Content-Type: multipart/alternative; boundary="_000_PS2P216MB01792E0070D82D4E155EDC139D960PS2P216MB0179KORP_" MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 54121694-ff20-4fb7-714b-08d5be10455a X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: c001924d-3e68-4f40-89c2-901a49278da7 X-MS-Exchange-CrossTenant-Network-Message-Id: 54121694-ff20-4fb7-714b-08d5be10455a X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: c001924d-3e68-4f40-89c2-901a49278da7 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 May 2018 05:12:23.4633 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT130 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Sun, 20 May 2018 13:29:29 +0000 Subject: Re: [bitcoin-dev] [bitcoin-discuss] Checkpoints in the Blockchain. X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 May 2018 05:12:29 -0000 --_000_PS2P216MB01792E0070D82D4E155EDC139D960PS2P216MB0179KORP_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I do understand your point, however, 'something like stuxnet' cannot be use= d to create valid data without re-doing all the PoW. Provided some valid co= pies of the blockchain continue to exist, the network can re-synchronise. Unrelated, it would seem useful to have some kind of deep blockchain corrup= tion recovery mechanism if it does not exist; where blocks are altered at a= depth exceeding the re-scan on init, efficient recovery is possible on det= ection. Presumably, it would be possible for some stuxnet like thing to mod= ify blocks by modifying the table data making blocks invalid but without ca= using a table corruption. I would also suppose that if the node is delving = deep into the blockchain for transaction data, that is would also validate = the block at least that it has a valid hash (apart from Merkle tree validat= ion for the individual transaction?) and that the hash of its immediate anc= estor is also valid. Regards, Damian Williamson ________________________________ From: bitcoin-discuss-bounces@lists.linuxfoundation.org on behalf of Dave Scotese via bitcoin-disc= uss Sent: Sunday, 20 May 2018 11:58 AM To: Scott Roberts Cc: Bitcoin Discuss Subject: Re: [bitcoin-discuss] Checkpoints in the Blockchain. I wouldn't limit my rendering to words, but that is a decent starting point= . The richer the rendering, the harder it will be to forget, but it needn'= t all be developed at once. My goal here is to inspire the creation of art = that is, at the same time, highly useful and based on randomness. Anyway, I asked what "premise that this is needed" you meant and I still do= n't know the answer. "The archive is a shared memory" - yes, a shared computer memory, and growi= ng larger (ie more cumbersome) every day. If something like stuxnet is used= to change a lot of the copies of it at some point, it seems likely that pe= ople will notice a change, but which history is correct won't be so obvious= because for the humans whose memories are not so easily overwritten, compu= ter data is remarkably non-memorable in it's normal form (0-9,a-f, whatever= ). If we ever want to abandon the historical transaction data, having a sh= ared memory of the state of a recent UTXO Set will help to obviate the need= for it. Yes, of course the blockchain is the perfect solution, as long as= there is exactly one and everyone can see that it's the same one that ever= yone else sees. Any other number of archives presents a great difficulty. In that scenario, there's no other way to prove that the starting point is = valid. Bitcoin has included a hardcoded-checkpoint in the code which serve= d the same purpose, but this ignores the possibility that two versions of t= he code could be created, one with a fake checkpoint that is useful to a po= werful attacker. If the checkpoint were rendered into something memorable = at the first opportunity, there would be little question about which one is= correct when the difference is discovered. On Sat, May 19, 2018 at 5:22 PM, Scott Roberts > wrote: I just don't see the point of needing to know it any different from the hex= value. Or maybe I should say I can't imagine it being useful because I can= 't imagine what you're after is possible. There might be a theoretical proo= f that what you're after is impossible. Hard to forget is almost the opposi= te of many options and what we're trying to do is decide between many optio= ns. I'll assume English because it's the only starting point I have that's= even in the ballpark of being possible. You might need to constrain the wo= rd selection and the structure in which you put it. I can only imagine that= you are talking about putting the words into some sort of story. The only = kind of story that would be hard to forget is one that fits into an overa= ll structure that we are familiar with but those types of structures are fe= w compared to the possibilities that we're trying to encode. "Hard to deny= " is a type of "hard to forget". Besides trying to connect it to external r= eality or history that we can all agree on there is also an internal consis= tency that could be used like a checksum such as the structure I mentioned.= The only thing that seems to fit the bill is the blockchain itself. It's o= n everyone's computer so it's impossible to forget and it has internal cons= istency. Is the only shared memory we have that can't be subject to a Sybil= attack or other hijacking of our memory of the true history. Our translati= on of the hash into words and a story could potentially be subject to hijac= king if it's not done perfectly correct. It just seems best to me to use th= e hash itself. They archived existence of the prior parts of the blockchain= are what make that particular hash hard to forget. Supposedly it can't be = forged to reference a fake history. The archive is a shared memory that fi= ts the encoding rules. On Sat, May 19, 2018, 4:30 PM Dave Scotese > wrote: Did you mean the premise that we have "the need to retain the full blockcha= in in order to validate the UTXO Set"? I hadn't thought of just making it easier to remember, as your suggestion d= oes (12-13 words), and that's a great idea. I have passphrases of that kin= d, but I noticed a kind of mandela effect just with myself. For example, I one of the wor= ds I chose was like "olfactory" but after a few months, what I remembered w= as like "ontology". The solution I came up with is to couple the data with = far more items than we normally do. Every ten minutes, we get a new set of= 256 bits that can be used to create something potential very difficult to = forget, and that's what I'm after. An algorithm could be used to do this with the Bip39 word list. If we cate= gorized the words according to parts of speech, the bits could be used to d= etermine which word goes next in a kind of ad-lib, but this creates a phras= e that is only memorable in the language for which the algorithm is develop= ed. As a thought experiment, I'll try adjective noun verb(as past tense) n= oun preposition adjective adjective noun: Stuff would come out like "Able a= bstract abused accident across actual adult affair." not memorable at all,= but sense can be made of it and sometimes the sense will be remarkable. A= s it is, I will have forgotten it in an hour or two. On Thu, May 17, 2018 at 4:29 PM, Scott Roberts > wrote: I disagree with your premise that this is needed, I like the question. Huma= ns are experts at language and I don't think we have another repository at = hand that we can categorize for memory that is better than words. Using wor= ds is a common way of doing what you're thinking about. If the checkpoint c= ould be a hash that meets a difficulty target the possibilities are 2^182 a= t the current hashrate instead of 2^256. So we need only 12 or 13 common wo= rds with their various possible endings (30,000). I would find it easier t= o insert a letter and 3 numbers after every 2 words so there would be only= 8 words used. There are probably other tricks people have figured out but = there can't be any kind of advanced encoding because it wouldn't benefit mo= re than one word. There might be a way to convert the words into something = that almost sounds like English sentences but it would probably come out a = cost of at least doubling the number of words. On Thu, May 17, 2018, 12:13 PM Dave Scotese via bitcoin-discuss > wrote: I got the idea that a SHA256 hash could be rendered into something difficul= t to forget. The rendering would involve using each of the 256 bits to spe= cify something important about the rendering - important in an instinctive = human-memory way. Let's assume such a rendering is possible, and that at any time, any person= can execute the rendering against the SHA256 hash of a consistent represen= tation of the UTXO Set. Sometimes, someone will execute the rendering and = discover that it is remarkable in some way (making it even more memorable),= and therefore will publish it. The published, memorable rendering now becomes a kind of protection against= any possible re-writing of the blockchain from any point prior to that UTX= O Set. When everyone involved in Bitcoin recognizes this protection, it re= lieves us of the need to retain the full blockchain in order to validate th= e UTXO Set at that point, because enough people will recognize it, and it c= an be validated without reference to any kind of prior computer record. This does leave open the possibility that an attacker could create a more f= avorable UTXO Set that happens to have a rendering similar enough to fool p= eople, or one that has exactly the same SHA256-hash, but that possibility i= s remote enough to ignore (just as we all ignore the possibility that whate= ver creates the master seed for our HD wallet will create a unique master s= eed). I've been working on how such a rendering could happen. It could describe = music, characters, colors, plot points, memorable elements of characters, e= tc. Dave Scotese _______________________________________________ bitcoin-discuss mailing list bitcoin-discuss@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-discuss -- I like to provide some work at no charge to prove my value. Do you need a t= echie? I own Litmocracy and Meme Racing (in alpha). I'm the webmaster for The Voluntaryist which n= ow accepts Bitcoin. I also code for The Dollar Vigilante. "He ought to find it more profitable to play by the rules" - Satoshi Nakamo= to -- I like to provide some work at no charge to prove my value. Do you need a t= echie? I own Litmocracy and Meme Racing (in alpha). I'm the webmaster for The Voluntaryist which n= ow accepts Bitcoin. I also code for The Dollar Vigilante. "He ought to find it more profitable to play by the rules" - Satoshi Nakamo= to --_000_PS2P216MB01792E0070D82D4E155EDC139D960PS2P216MB0179KORP_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

I do understand your point, howev= er, 'something like stuxnet' cannot be used to create valid data without re= -doing all the PoW. Provided some valid copies of the blockchain continue t= o exist, the network can re-synchronise.


Unrelated, it would seem useful t= o have some kind of deep blockchain corruption recovery mechanism if it doe= s not exist; where blocks are altered at a depth exceeding the re-scan on i= nit, efficient recovery is possible on detection. Presumably, it would be possible for some stuxnet like= thing to modify blocks by modifying the table data making blocks invalid b= ut without causing a table corruption. I would also suppose that if the nod= e is delving deep into the blockchain for transaction data, that is would also validate the block at least that = it has a valid hash (apart from Merkle tree validation for the individual t= ransaction?) and that the hash of its immediate ancestor is also valid.


Regards,

Damian Williamson



From: bitcoin-discuss-boun= ces@lists.linuxfoundation.org <bitcoin-discuss-bounces@lists.linuxfounda= tion.org> on behalf of Dave Scotese via bitcoin-discuss <bitcoin-discuss@lists.linuxfoundation.org>
Sent: Sunday, 20 May 2018 11:58 AM
To: Scott Roberts
Cc: Bitcoin Discuss
Subject: Re: [bitcoin-discuss] Checkpoints in the Blockchain.
 
I wouldn't limit my rendering to words, but that is a decent starting = point.  The richer the rendering, the harder it will be to forget, but= it needn't all be developed at once. My goal here is to inspire the creati= on of art that is, at the same time, highly useful and based on randomness.

Anyway, I asked what "premise that this is needed" you meant= and I still don't know the answer.

"The archive is a shared memory" - yes, a shared computer= memory, and growing larger (ie more cumbersome) every day. If somethin= g like stuxnet is used to change a lot of the copies of it at some point, i= t seems likely that people will notice a change, but which history is correct won't be so obvious because for the humans= whose memories are not so easily overwritten, computer data is remarka= bly non-memorable in it's normal form (0-9,a-f, whatever).  If we ever= want to abandon the historical transaction data, having a shared memory of the state of a recent UTXO Set will help t= o obviate the need for it.  Yes, of course the blockchain is the perfe= ct solution, as long as there is exactly one and everyone can see that it's the same one that everyon= e else sees.  Any other number of archives presents a great difficulty= .

In that scenario, there's no other way to prove that the starting poin= t is valid.  Bitcoin has included a hardcoded-checkpoint in the code w= hich served the same purpose, but this ignores the possibility that two ver= sions of the code could be created, one with a fake checkpoint that is useful to a powerful attacker.  If the= checkpoint were rendered into something memorable at the first opportunity= , there would be little question about which one is correct when the differ= ence is discovered.

On Sat, May 19, 2018 at 5:22 PM, Scott Roberts= <wordsgalore@gmail.com<= /a>> wrote:
I just don't see the point of needing to know it any diff= erent from the hex value. Or maybe I should say I can't imagine it being us= eful because I can't imagine what you're after is possible. There might be = a theoretical proof that what you're after is impossible. Hard to forget is almost the opposite of many options= and what we're trying to do is decide between many options. I'll assume En= glish because  it's the only starting point I have that's even in the = ballpark of being possible. You might need to constrain the word selection and the structure in which you put it= . I can only imagine that you are talking about putting the words into some= sort of story. The only kind of story that would be hard to forget  i= s one that  fits into an overall structure that we are familiar with but those types of structures are few  comp= ared to the possibilities that we're trying to encode. "Hard to deny&q= uot; is a type of "hard to forget". Besides trying to connect it = to external reality or history that we can all agree on there is also an internal consistency that could be used like a checksum such as= the structure I mentioned. The only thing that seems to fit the bill is th= e blockchain itself. It's on everyone's computer so it's impossible to forg= et and it has internal consistency. Is the only shared memory we have that can't be subject to a Sybil attack = or other hijacking of our memory of the true history. Our translation of th= e hash into words and a story could potentially be subject to hijacking if = it's not done perfectly correct. It just seems best to me to use the hash itself. They archived existence o= f the prior parts of the blockchain are what make that particular hash hard= to forget. Supposedly it can't be forged to reference  a fake history= . The archive is a shared memory that fits the encoding rules.

Did you mean the premise that we have "the need to retain the ful= l blockchain in order to validate the UTXO Set"?

I hadn't thought of just making it easier to remember, as your = suggestion does (12-13 words), and that's a great idea.  I have passph= rases of that kind, but I noticed a kind of mandela effect just with myself.  For example, I one of the words = I chose was like "olfactory" but after a few months, what I remem= bered was like "ontology". The solution I came up with is to coup= le the data with far more items than we normally do.  Every ten minutes, we get a new set of 256 bits that can be used to create somet= hing potential very difficult to forget, and that's what I'm after.

An algorithm could be used to do this with the Bip39 word list.  = If we categorized the words according to parts of speech, the bits could be= used to determine which word goes next in a kind of ad-lib, but this creat= es a phrase that is only memorable in the language for which the algorithm is developed.  As a thought expe= riment, I'll try adjective noun verb(as past tense) noun preposition adject= ive adjective noun: Stuff would come out like "Able abstract abused ac= cident across actual adult affair."  not memorable at all, but sense can be made of it and sometimes the sense will be remark= able.  As it is, I will have forgotten it in an hour or two.

On Thu, May 17, 2018 at 4:29 PM, Scott Roberts= <wor= dsgalore@gmail.com> wrote:
I disagree with your premise that this is needed, I like = the question. Humans are experts at language and I don't think we have anot= her repository at hand that we can categorize for memory that is better tha= n words. Using words is a common way of doing what you're thinking about. If the checkpoint could be a hash tha= t meets a difficulty target the possibilities are 2^182 at the current hash= rate instead of 2^256. So we need only 12 or 13 common words with their var= ious possible endings (30,000).  I would find it easier to insert  a letter and 3 numbers after every = 2 words so there would be only 8 words used. There are probably other trick= s people have figured out but there can't be any kind of advanced encoding = because it wouldn't benefit more than one word. There might be a way to convert the words into something that al= most sounds like English sentences but it would probably come out a cost of= at least doubling the number of words.

On Thu, May 17, 2018, 12:13 PM Dave Scotese via bitcoin-di= scuss <bitcoin-discuss@lists.linuxfoundation.org&= gt; wrote:
I got the idea that a SHA256 hash could be rendered into something dif= ficult to forget.  The rendering would involve using each of the 256 b= its to specify something important about the rendering - important in an in= stinctive human-memory way.

Let's assume such a rendering is possible, and that at any time, any p= erson can execute the rendering against the SHA256 hash of a consistent rep= resentation of the UTXO Set.  Sometimes, someone will execute the rend= ering and discover that it is remarkable in some way (making it even more memorable), and therefore will publish it= .

The published, memorable rendering now becomes a kind of protection ag= ainst any possible re-writing of the blockchain from any point prior to tha= t UTXO Set.  When everyone involved in Bitcoin recognizes this protect= ion, it relieves us of the need to retain the full blockchain in order to validate the UTXO Set at that point, becau= se enough people will recognize it, and it can be validated without referen= ce to any kind of prior computer record.

This does leave open the possibility that an attacker could create a m= ore favorable UTXO Set that happens to have a rendering similar enough to f= ool people, or one that has exactly the same SHA256-hash, but that possibil= ity is remote enough to ignore (just as we all ignore the possibility that whatever creates the master seed for= our HD wallet will create a unique master seed).

I've been working on how such a rendering could happen.  It could= describe music, characters, colors, plot points, memorable elements of cha= racters, etc.

Dave Scotese

_______________________________________________
bitcoin-discuss mailing list
bitcoin-discuss@lists.linuxfoundation.org<= br> https://lists.linuxfou= ndation.org/mailman/listinfo/bitcoin-discuss



--
I like to provide some work at no charge to prove my value= . Do you need a techie? 
I own Litmocracy and Meme Racing (in alpha).
I'm the webmaster for The Voluntaryist which now accepts Bitcoin.
I also code for The Dollar Vigilante.
"He ought to find it more profitable to play by the rules" - Sato= shi Nakamoto



--
I like to provide some work at no charge to prove my value= . Do you need a techie? 
I own Litmocracy and Meme Racing (in alpha).
I'm the webmaster for The Voluntaryist which now accepts Bitcoin.
I also code for The Dollar Vigilante.
"He ought to find it more profitable to play by the rules" - Sato= shi Nakamoto
--_000_PS2P216MB01792E0070D82D4E155EDC139D960PS2P216MB0179KORP_--