Message-ID: <55B79146.70309@gmail.com>
Date: Tue, 28 Jul 2015 16:27:18 +0200
From: Pieter Wuille <pieter.wuille@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
	rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: bitcoin-dev@lists.linuxfoundation.org
References: <55B78F56.3080802@gmail.com>
In-Reply-To: <55B78F56.3080802@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Subject: [bitcoin-dev] Disclosure: consensus bug indirectly solved by BIP66
Precedence: list

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all,

I'd like to disclose a vulnerability I discovered in September 2014,
which became unexploitable when BIP66's 95% threshold was reached
earlier this month.

## Short description:

A specially-crafted transaction could have forked the blockchain
between nodes:
* using OpenSSL on a 32-bit systems and on 64-bit Windows systems
* using OpenSSL on non-Windows 64-bit systems (Linux, OSX, ...)
* using some non-OpenSSL codebases for parsing signatures

## Upgrade instructions:

None. Transactions that could trigger this problem have become invalid
on the network since BIP66's deployment of version 3 blocks reached 95%
on July 4th, 2015.

## Long description:

The problem is related to the signature encoding rules.

Bitcoin's signatures are ASN.1 BER encoded. BER is a complex standard
that allows many different encodings for the same data. Since Bitcoin
Core 0.8, a standardness rule has been in effect that only allowed
subset of encodings (DER) for relay and mining, even though any BER
remained valid in the blockchain - at least in theory.

In practice, BER has many weird edge cases, and I have not found a
single cryptographic codebase that can parse all of them correctly.
This includes OpenSSL, Crypto++, BouncyCastle, btcec, and our own
libsecp256k1 library.

This on itself would not be a problem, as full nodes on the network
currently use OpenSSL. However, while researching what was needed to
make libsecp256k1 compatible with it, I discovered that OpenSSL is even
inconsistent with itself across different platforms.

One of the features of BER is the ability for internal structures to
have a length descriptor whose size itself is up to 126 bytes (see
X.690-0207 8.1.3.5). A 1 terabyte data structure would for example use
a 5-byte length descriptor. However, there is no requirement to use the
shortest possible descriptor, so even a 70-byte ECDSA signature could
use a 5-byte length descriptor and be valid. Unfortunately, OpenSSL
supports length descriptors only as long as their size is at most that
of a C 'long int', a type whose size depends on the platform (Windows
and 32-bit Linux/OSX have a 4-byte long int, 64-bit Linux/OSX have an
8-byte long int). See
https://github.com/openssl/openssl/blob/bfa34f551c2d38e826deb44a269cb0f720f9f63b/crypto/asn1/asn1_lib.c#L178.
Some non-OpenSSL based signature validation
systems don't support such length descriptors at all, resulting in an
extra forking risk on top for them if used for blockchain validation.

This effectively means that a block chain containing a transaction with
a valid signature using such a 5-byte length descriptor would be
accepted by some systems and not by others, resulting in a fork if it
were mined into a block.

## Timeline:

* 2013-Feb-19: Bitcoin Core 0.8.0 was released, which made non-DER
signatures non-standard. No release since then would relay or mine
transactions that could trigger the vulnerability. However, such a
transaction was still valid inside blocks.

* 2014-Feb-10: I proposed BIP62 to deal with transaction malleability.
The BIP62 draft includes a rule that would make version 2 transactions
with non-DER signatures invalid.

* 2014-Jul-18: In order to make Bitcoin's signature encoding rules not
depend on OpenSSL's specific parser, I modified the BIP62 proposal to
have its strict DER signatures requirement also apply to version 1
transactions. No non-DER signatures were being mined into blocks
anymore at the time, so this was assumed to not have any impact. See
https://github.com/bitcoin/bips/pull/90 and
http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2014-July/006299.html.
Unknown at the time, but if deployed this would have solved the
vulnerability.

* 2014-Sep-01: While analysing OpenSSL's source code for BER parsing, I
discovered the architecture dependency listed above and the associated
vulnerability. The best means to fix it at the time was by getting
BIP62 adopted.

* 2014-Sep-07, 2014-Oct-17, 2014-Oct-26, 2014-Dec-06, 2015-Jan-09:
Several proposed changes to BIP62. See
https://github.com/bitcoin/bips/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aclosed+bip62.

* 2015-Jan-10: OpenSSL releases versions 1.0.0p and 1.0.1k, with a fix
for CVE-2014-8275. The fix introduced a restriction on ECDSA signatures
to be strict DER, which would have solved all problems related to
signature encodings, except Bitcoin's consensus-critical nature
requires bug-for-bug compatibility between nodes. Worse, it seemed that
there was again a small (1%) number of blocks being created with
non-DER signatures in it, resulting in actual forks. The only immediate
solution that did not introduce more risk for forks was parsing and
re-encoding signatures using OpenSSL itself before verification to
bypass the restriction, making the problem persist. See
http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-January/007097.html.

* 2015-Jan-21: The new attention to signature encoding might have
revealed the vulnerability, and the presence of miners not enforcing
strict DER might have made the vulnerability actually exploitable.
BIP62 was still a moving target, so we wanted a faster means to solve
this. Therefore, a new BIP was proposed with just the strict DER
requirement, numbered BIP66. This would both allow non-OpenSSL
verification, and solve the vulnerability, without needing to fix the
less urgent malleability problem that BIP62 wanted to address. See
http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-January/007156.html.

* 2015-Feb-17: Bitcoin Core 0.10.0 was released, with support for
BIP66. See
http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-February/007480.html.

* 2015-Jul-04: BIP66's 95% threshold was reached, enabling a consensus
rule for strict DER signatures in the blockchain. This solved the
vulnerability, and opens the door to using non-OpenSSL signature
verification in the near future.

- -- 
Pieter Wuille

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQGcBAEBAgAGBQJVt5FGAAoJEFeJbS/48LZX3ccMAJdPrpa8ggcYEyy8naqc7ewL
1Mwv24p/6Q8+T7Q6EWmgoApY1jljF+AzgSpfaf310QZf9yuC/JC++AmHfUaa9UQG
Mq1+duX64uDWIeNKTfuCwZvU0ataARZKmFUpp60UF+VtiJyLo9tpHTVajM0lv9Oq
OX40qHVC/iBogRLNREC1ggWH1JPMTbEch50YX1bgNi3gE5gtMggSQ2OXrGCCtrvR
7cVFlIyOhlLtvSAnxzmHyY8Iol+qVhIZi4mCmDgOoQKVaiYm1cODQ+nrMHx02DKC
Wqstwb/mET/vbCX4qxSNQ2B+mQk0WO/gSrWiQkBLi/AfLBh/8A/kL1RpKxVQzoaP
O165LbXye42w8Js/sE/zT6d4CIbYaW0GpF6m4agwDYgPLomhdk/elPRojKYsEab+
oFWPVagqKI9e/pjFBxqfIv3iyx1hHB6YIaX5TfFRVjsWzag5Qi2ssQYOQymyjg4J
UHNR/xW0PMPAOg5KS/uFja4OWxstHhTW9G+rslEK9g==
=7F3K
-----END PGP SIGNATURE-----