Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Ysb1o-0008LG-91 for bitcoin-development@lists.sourceforge.net; Wed, 13 May 2015 18:05:08 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.215.51 as permitted sender) client-ip=209.85.215.51; envelope-from=decker.christian@gmail.com; helo=mail-la0-f51.google.com; Received: from mail-la0-f51.google.com ([209.85.215.51]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Ysb1h-0007pM-Vq for bitcoin-development@lists.sourceforge.net; Wed, 13 May 2015 18:05:08 +0000 Received: by lagv1 with SMTP id v1so35265938lag.3 for ; Wed, 13 May 2015 11:04:55 -0700 (PDT) X-Received: by 10.152.29.198 with SMTP id m6mr98670lah.11.1431540295600; Wed, 13 May 2015 11:04:55 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Christian Decker Date: Wed, 13 May 2015 18:04:54 +0000 Message-ID: To: Pieter Wuille Content-Type: multipart/alternative; boundary=089e0158c7dccd11350515fa7223 X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (decker.christian[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1Ysb1h-0007pM-Vq Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] [BIP] Normalized Transaction IDs X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 18:05:08 -0000 --089e0158c7dccd11350515fa7223 Content-Type: text/plain; charset=UTF-8 If the inputs to my transaction have been long confirmed I can be reasonably safe in assuming that the transaction hash does not change anymore. It's true that I have to be careful not to build on top of transactions that use legacy references to transactions that are unconfirmed or have few confirmations, however that does not invalidate the utility of the normalized transaction IDs. The resource doubling is not optimal, I agree, but compare that to dragging around malleability and subsequent hacks to sort-of fix it forever. Additionally if we were to decide to abandon legacy transaction IDs we could eventually drop the legacy index after a sufficient transition period. I remember reading about the SIGHASH proposal somewhere. It feels really hackish to me: It is a substantial change to the way signatures are verified, I cannot really see how this is a softfork if clients that did not update are unable to verify transactions using that SIGHASH Flag and it is adding more data (the normalized hash) to the script, which has to be stored as part of the transaction. It may be true that a node observing changes in the input transactions of a transaction using this flag could fix the problem, however it requires the node's intervention. Compare that to the simple and clean solution in the proposal, which does not add extra data to be stored, keeps the OP_*SIG* semantics as they are and where once you sign a transaction it does not have to be monitored or changed in order to be valid. There certainly are merits using the SIGHASH approach in the short term (it does not require a hard fork), however I think the normalized transaction ID is a cleaner and simpler long-term solution, even though it requires a hard-fork. Regards, Christian On Wed, May 13, 2015 at 7:14 PM Pieter Wuille wrote: > Normalized transaction ids are only effectively non-malleable when all > inputs they refer to are also non-malleable (or you can have malleability > in 2nd level dependencies), so I do not believe it makes sense to allow > mixed usage of the txids at all. They do not provide the actual benefit of > guaranteed non-malleability before it becomes disallowed to use the old > mechanism. That, together with the +- resource doubling needed for the UTXO > set (as earlier mentioned) and the fact that an alternative which is only a > softfork are available, makes this a bad idea IMHO. > > Unsure to what extent this has been presented on the mailinglist, but the > softfork idea is this: > * Transactions get 2 txids, one used to reference them (computed as > before), and one used in an (extended) sighash. > * The txins keep using the normal txid, so not structural changes to > Bitcoin. > * The ntxid is computed by replacing the scriptSigs in inputs by the empty > string, and by replacing the txids in txins by their corresponding ntxids. > * A new checksig operator is softforked in, which uses the ntxids in its > sighashes rather than the full txid. > * To support efficiently computing ntxids, every tx in the utxo set > (currently around 6M) stores the ntxid, but only supports lookup bu txid > still. > > This does result in a system where a changed dependency indeed invalidates > the spending transaction, but the fix is trivial and can be done without > access to the private key. > On May 13, 2015 5:50 AM, "Christian Decker" > wrote: > >> Hi All, >> >> I'd like to propose a BIP to normalize transaction IDs in order to >> address transaction malleability and facilitate higher level protocols. >> >> The normalized transaction ID is an alias used in parallel to the current >> (legacy) transaction IDs to address outputs in transactions. It is >> calculated by removing (zeroing) the scriptSig before computing the hash, >> which ensures that only data whose integrity is also guaranteed by the >> signatures influences the hash. Thus if anything causes the normalized ID >> to change it automatically invalidates the signature. When validating a >> client supporting this BIP would use both the normalized tx ID as well as >> the legacy tx ID when validating transactions. >> >> The detailed writeup can be found here: >> https://github.com/cdecker/bips/blob/normalized-txid/bip-00nn.mediawiki. >> >> @gmaxwell: I'd like to request a BIP number, unless there is something >> really wrong with the proposal. >> >> In addition to being a simple alternative that solves transaction >> malleability it also hugely simplifies higher level protocols. We can now >> use template transactions upon which sequences of transactions can be built >> before signing them. >> >> I hesitated quite a while to propose it since it does require a hardfork >> (old clients would not find the prevTx identified by the normalized >> transaction ID and deem the spending transaction invalid), but it seems >> that hardforks are no longer the dreaded boogeyman nobody talks about. >> I left out the details of how the hardfork is to be done, as it does not >> really matter and we may have a good mechanism to apply a bunch of >> hardforks concurrently in the future. >> >> I'm sure it'll take time to implement and upgrade, but I think it would >> be a nice addition to the functionality and would solve a long standing >> problem :-) >> >> Please let me know what you think, the proposal is definitely not set in >> stone at this point and I'm sure we can improve it further. >> >> Regards, >> Christian >> >> >> ------------------------------------------------------------------------------ >> One dashboard for servers and applications across Physical-Virtual-Cloud >> Widest out-of-the-box monitoring support with 50+ applications >> Performance metrics, stats and reports that give you Actionable Insights >> Deep dive visibility with transaction tracing using APM Insight. >> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> >> --089e0158c7dccd11350515fa7223 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
If the inputs to my transaction have been long confirmed I= can be reasonably safe in assuming that the transaction hash does not chan= ge anymore. It's true that I have to be careful not to build on top of = transactions that use legacy references to transactions that are unconfirme= d or have few confirmations, however that does not invalidate the utility o= f the normalized transaction IDs.=C2=A0

The resource dou= bling is not optimal, I agree, but compare that to dragging around malleabi= lity and subsequent hacks to sort-of fix it forever. Additionally if we wer= e to decide to abandon legacy transaction IDs we could eventually drop the = legacy index after a sufficient transition period.

I remember reading about the SIGHASH proposal somewhere. It feels really h= ackish to me: It is a substantial change to the way signatures are verified= , I cannot really see how this is a softfork if clients that did not update= are unable to verify transactions using that SIGHASH Flag and it is adding= more data (the normalized hash) to the script, which has to be stored as p= art of the transaction. It may be true that a node observing changes in the= input transactions of a transaction using this flag could fix the problem,= however it requires the node's intervention.

= Compare that to the simple and clean solution in the proposal, which does n= ot add extra data to be stored, keeps the OP_*SIG* semantics as they are an= d where once you sign a transaction it does not have to be monitored or cha= nged in order to be valid.

There certainly are mer= its using the SIGHASH approach in the short term (it does not require a har= d fork), however I think the normalized transaction ID is a cleaner and sim= pler long-term solution, even though it requires a hard-fork.
Regards,
Christian

On Wed, May 13, 2015 at 7:14 PM Pieter Wuille <<= a href=3D"mailto:pieter.wuille@gmail.com" target=3D"_blank">pieter.wuille@g= mail.com> wrote:

No= rmalized transaction ids are only effectively non-malleable when all inputs= they refer to are also non-malleable (or you can have malleability in 2nd = level dependencies), so I do not believe it makes sense to allow mixed usag= e of the txids at all. They do not provide the actual benefit of guaranteed= non-malleability before it becomes disallowed to use the old mechanism. Th= at, together with the +- resource doubling needed for the UTXO set (as earl= ier mentioned) and the fact that an alternative which is only a softfork ar= e available, makes this a bad idea IMHO.

Unsure to what extent this has been presented on the mailing= list, but the softfork idea is this:
* Transactions get 2 txids, one used to reference them (computed as before)= , and one used in an (extended) sighash.
* The txins keep using the normal txid, so not structural changes to Bitcoi= n.
* The ntxid is computed by replacing the scriptSigs in inputs by the empty = string, and by replacing the txids in txins by their corresponding ntxids.<= br> * A new checksig operator is softforked in, which uses the ntxids in its si= ghashes rather than the full txid.
* To support efficiently computing ntxids, every tx in the utxo set (curren= tly around 6M) stores the ntxid, but only supports lookup bu txid still.

This does result in a system where a changed dependency inde= ed invalidates the spending transaction, but the fix is trivial and can be = done without access to the private key.

On May 13, 2015= 5:50 AM, "Christian Decker" <decker.christian@gmail.com> wrote:
Hi All,

I'd lik= e to propose a BIP to normalize transaction IDs in order to address transac= tion malleability and facilitate higher level protocols.

The normalized transaction ID is an alias used in parallel to the current = (legacy) transaction IDs to address outputs in transactions. It is calculat= ed by removing (zeroing) the scriptSig before computing the hash, which ens= ures that only data whose integrity is also guaranteed by the signatures in= fluences the hash. Thus if anything causes the normalized ID to change it a= utomatically invalidates the signature. When validating a client supporting= this BIP would use both the normalized tx ID as well as the legacy tx ID w= hen validating transactions.


@gmaxwell: I'd like to request a BIP number, unless there is somethi= ng really wrong with the proposal.

In addition to = being a simple alternative that solves transaction malleability it also hug= ely simplifies higher level protocols. We can now use template transactions= upon which sequences of transactions can be built before signing them.

I hesitated quite a while to propose it since it does= require a hardfork (old clients would not find the prevTx identified by th= e normalized transaction ID and deem the spending transaction invalid), but= it seems that hardforks are no longer the dreaded boogeyman nobody talks a= bout.
I left out the details of how the hardfork is to be done, a= s it does not really matter and we may have a good mechanism to apply a bun= ch of hardforks concurrently in the future.

I'= m sure it'll take time to implement and upgrade, but I think it would b= e a nice addition to the functionality and would solve a long standing prob= lem :-)

Please let me know what you think, the pro= posal is definitely not set in stone at this point and I'm sure we can = improve it further.

Regards,
Christian

---------------------------------------------------------------------= ---------
One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm= /clk/290420510;117567292;y
_________________________________________= ______
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment

--089e0158c7dccd11350515fa7223--