Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.213.47 as permitted sender)
	client-ip=209.85.213.47; envelope-from=etotheipi@gmail.com;
	helo=mail-yw0-f47.google.com; 
Message-ID: <4FE0C538.3090001@gmail.com>
Date: Tue, 19 Jun 2012 14:30:16 -0400
From: Alan Reiner <etotheipi@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64;
	rv:12.0) Gecko/20120428 Thunderbird/12.0.1
MIME-Version: 1.0
To: Mark Friedenbach <mark@monetize.io>
References: <CAF7tpEyEWCbcB+jSpWOMyeZUBjQ=FbVEC8kLt3j2Yzv3YJOgiA@mail.gmail.com>
	<4FE0B7EB.6000100@gmail.com>
	<CACh7GpEehHFEJGRTtijgM7UAa2jeEWRKrQo5dym8F_YgXAEhFA@mail.gmail.com>
In-Reply-To: <CACh7GpEehHFEJGRTtijgM7UAa2jeEWRKrQo5dym8F_YgXAEhFA@mail.gmail.com>
Content-Type: multipart/alternative;
	boundary="------------040508030500040107050704"
Cc: bitcoin-development@lists.sourceforge.net
Subject: Re: [Bitcoin-development] Ultimate Blockchain Compression w/
 trust-free lite node
Precedence: list

This is a multi-part message in MIME format.
--------------040508030500040107050704
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 06/19/2012 02:18 PM, Mark Friedenbach wrote:
> On Tue, Jun 19, 2012 at 10:33 AM, Alan Reiner <etotheipi@gmail.com 
> <mailto:etotheipi@gmail.com>> wrote:
>
>     If we were to use a raw trie structure, then we'd have all the above
>     issues solved:  a trie has the same configuration no matter how
>     elements
>     are inserted or deleted, and accesses to elements in the tree are
>     constant time -- O(1).  There is no such thing as an unbalanced trie.
>     But overall space-efficiency is an issue.
>
>     A PATRICIA tree/trie would be ideal, in my mind, as it also has a
>     completely deterministic structure, and is an order-of-magnitude more
>     space-efficient.  Insert, delete and query times are still O(1).
>     However, it is not a trivial implementation.  I have occasionally
>     looked
>     for implementations, but not found any that were satisfactory.
>
>
> No, a trie of any sort is dependent upon distribution of input data 
> for balancing. As Peter Todd points out, a malicious actor could 
> construct transaction or address hashes in such a way as to grow some 
> segment of the trie in an unbalanced fashion. It's not much of an 
> attack, but in principle exploitable under particular timing-sensitive 
> circumstances.
>
> Self-balancing search trees (KVL, RB, 2-3-4, whatever) don't suffer 
> from this problem.
>
> Mark

I was using "unbalanced" to refer to "query time" (and also 
insert/delete time).  If your trie nodes branch based on the next byte 
of your key hash, then the max depth of your trie is 32.  Period.  No 
one can do anything to ever make you do more than 32 hops to 
find/insert/delete your data.   And if you're using a raw trie, you'll 
always use /exactly/ 32 hops regardless of the distribution of the 
underlying data.  Hence, the trie structure is deterministic 
(history-independent) and cannot become unbalanced in terms of access time.

My first concern was that a malicious actor could linearize parts of the 
tree and cause access requests to take much longer than log(N) time.  
With the trie, that's not only impossible, you're actually accessing in 
O(1) time.

However, you are right that disk space can be affected by a malicious 
actor.  The more branching he can induce, the more branch nodes that are 
created to support branches with only one leaf.


--------------040508030500040107050704
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    On 06/19/2012 02:18 PM, Mark Friedenbach wrote:
    <blockquote
cite="mid:CACh7GpEehHFEJGRTtijgM7UAa2jeEWRKrQo5dym8F_YgXAEhFA@mail.gmail.com"
      type="cite">
      <div class="gmail_quote">On Tue, Jun 19, 2012 at 10:33 AM, Alan
        Reiner <span dir="ltr">&lt;<a moz-do-not-send="true"
            href="mailto:etotheipi@gmail.com" target="_blank">etotheipi@gmail.com</a>&gt;</span>
        wrote:<br>
        <br>
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">
          If we were to use a raw trie structure, then we'd have all the
          above<br>
          issues solved:  a trie has the same configuration no matter
          how elements<br>
          are inserted or deleted, and accesses to elements in the tree
          are<br>
          constant time -- O(1).  There is no such thing as an
          unbalanced trie.<br>
          But overall space-efficiency is an issue.<br>
          <br>
          A PATRICIA tree/trie would be ideal, in my mind, as it also
          has a<br>
          completely deterministic structure, and is an
          order-of-magnitude more<br>
          space-efficient.  Insert, delete and query times are still
          O(1).<br>
          However, it is not a trivial implementation.  I have
          occasionally looked<br>
          for implementations, but not found any that were satisfactory.<br>
        </blockquote>
        <div><br>
        </div>
        <div>No, a trie of any sort is dependent upon distribution of
          input data for balancing. As <span class="Apple-style-span"
style="border-collapse:collapse;color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px">Peter
            Todd points out, a malicious actor could construct
            transaction or address hashes in such a way as to grow some
            segment of the trie in an unbalanced fashion. It's not much
            of an attack, but in principle exploitable under particular
            timing-sensitive circumstances.</span></div>
        <div><span class="Apple-style-span"
style="border-collapse:collapse;color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px"><br>
          </span></div>
        <div><span class="Apple-style-span"
style="border-collapse:collapse;color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px">Self-balancing
            search trees (KVL, RB, 2-3-4, whatever) don't suffer from
            this problem.</span></div>
        <div><span class="Apple-style-span"
style="border-collapse:collapse;color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px"><br>
          </span></div>
        <div><span class="Apple-style-span"
style="border-collapse:collapse;color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px">Mark</span></div>
      </div>
    </blockquote>
    <br>
    I was using "unbalanced" to refer to "query time" (and also
    insert/delete time).  If your trie nodes branch based on the next
    byte of your key hash, then the max depth of your trie is 32. 
    Period.  No one can do anything to ever make you do more than 32
    hops to find/insert/delete your data.   And if you're using a raw
    trie, you'll always use <i>exactly</i> 32 hops regardless of the
    distribution of the underlying data.  Hence, the trie structure is
    deterministic (history-independent) and cannot become unbalanced in
    terms of access time.<br>
    <br>
    My first concern was that a malicious actor could linearize parts of
    the tree and cause access requests to take much longer than log(N)
    time.  With the trie, that's not only impossible, you're actually
    accessing in O(1) time.<br>
    <br>
    However, you are right that disk space can be affected by a
    malicious actor.  The more branching he can induce, the more branch
    nodes that are created to support branches with only one leaf.  <br>
    <br>
    <br>
  </body>
</html>

--------------040508030500040107050704--