Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.213.172 as permitted sender)
	client-ip=209.85.213.172; envelope-from=pieter.wuille@gmail.com;
	helo=mail-ig0-f172.google.com; 
MIME-Version: 1.0
Date: Mon, 10 Feb 2014 13:28:42 +0100
Message-ID: <CAPg+sBi-phaw3hDgguk-LYrPsKX4M5snTJBv_NsQML1M=XgZEw@mail.gmail.com>
From: Pieter Wuille <pieter.wuille@gmail.com>
To: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Content-Type: text/plain; charset=ISO-8859-1
Subject: [Bitcoin-development] Malleability and MtGox's announcement
Precedence: list

Hi all,

I was a bit surprised to see MtGox's announcement. The malleability of
transactions was known for years already (see for example the wiki
article on it, https://en.bitcoin.it/wiki/Transaction_Malleability it,
or mails on this list from 2012 and 2013). I don't consider it a very
big problem, but it does make it harder for infrastructure to interact
with Bitcoin. If we'd design Bitcoin today, I'm sure we would try to
avoid it altogether to make life easier for everyone.

But we can't just change all infrastructure that exists today. We're
slowly working towards making malleability harder (and hopefully
impossible someday), but this will take a long time. For example, 0.8
not supporting non-DER encoded signatures was a step in that direction
(and ironically, the trigger that caused MtGox's initial problems
here). In any case, this will take years, and nobody should wait for
this.

There seem to be two more direct problems here.
* Wallets which deal badly with modified txids.
* Services that use the transaction id to detect unconfirming transactions.

The first is something that needs to be done correctly in software -
it just needs to be aware of malleability.

The second is something I was unaware of and would have advised
against. If you plan on reissuing a transaction because on old version
doesn't confirm, make sure to make it a double spend of the first one
- so that not both can confirm.

I certainly don't like press making this sound like a problem in the
Bitcoin protocol or clients. I think this is an issue that needs to be
solved at the layer above - the infrastructure building on the Bitcoin
system. Despite that, I do think that we (as a community, not just
developers) can benefit from defining a standard way to identify
transactions unambiguously. This is something Mark Karpeles suggested
a few days ago, and my proposal is this:

We define the normalized transaction id as SHA256^2(normalized_tx +
0x01000000), where normalized_tx is the transaction with all input
scripts replaced by empty scripts. This is exactly what would be
signed inside transaction signatures using SIGHASH_ALL (except not
substituting the previous scriptPubKey to be signed, and not dealing
with the input being signed specially). An implementation is here:
https://github.com/sipa/bitcoin/commits/normtxid.

Note that this is not a solution for all problems related to
malleability, but maybe it can make people more aware of it, in
tangible way.

-- 
Pieter