Date: Tue, 10 Dec 2019 01:50:38 +0000
To: Pieter Wuille <pieter.wuille@gmail.com>,
 Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
From: ZmnSCPxj <ZmnSCPxj@protonmail.com>
Reply-To: ZmnSCPxj <ZmnSCPxj@protonmail.com>
Message-ID: <2YyEOYXhcEvfGJLUX4zswzSpBA0vWOC_Jwl_MOcphySLZqjfBDqqDkBvZB6kX7nvVsGNPqwuh63lgBGO5BcURaig2r5tqxFoaEZTLDMTG7U=@protonmail.com>
In-Reply-To: <CAPg+sBgdspmfK1qpG=9N9fAtVNkMDd+xym7yHdjq=zYnqeh5dA@mail.gmail.com>
References: <CAPg+sBgdspmfK1qpG=9N9fAtVNkMDd+xym7yHdjq=zYnqeh5dA@mail.gmail.com>
Feedback-ID: el4j0RWPRERue64lIQeq9Y2FP-mdB86tFqjmrJyEPR9VAtMovPEo9tvgA0CrTsSHJeeyPXqnoAu6DN-R04uJUg==:Ext:ProtonMail
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: [bitcoin-dev] Analysis of Bech32 swap/insert/delete detection
	and next steps
Precedence: list

Good morning Pieter,

> Hi all,
>
> I've made a writeup on Bech32's detection abilities, analysing how it
> behaves in the presence of not just substitution errors, but also
> swapping of characters, and insertions and deletions:
> https://gist.github.com/sipa/a9845b37c1b298a7301c33a04090b2eb
>
> It shows that the "insert or delete a 'q' right before a final 'p'" is
> in fact the only deviation from the expected at-most-1-in-a-billion
> failure to detect chance, at least when restricted to the classes of
> errors analyzed with various uniformity assumptions. There is some
> future work left, such as analyzing combinations of insertions and
> substitutions, but I would be surprising if additional weaknesses
> exist there.
>
> It also shows that changing one constant in Bech32 would resolve this
> issue, while not affecting the error detection properties for other
> classes of errors.
>
> So my suggestion for the next steps are:
>
> -   Update BIP173 to include the insertion weakness as an erratum, and
>     the results of this analysis.
>

To clarify, this step does not modify anything about the implementation of =
BIP173, only adds this as an additional erratum section?

> -   Amend segwit addresses (either by amending BIP173, or by writing a
>     short updated BIP to modify it) to be restricted to only length 20 or
>     32 (as fixed-length strings are unaffected by the insertion issue, an=
d
>     I don't think inserting 20 characters is an interesting error class).

To clarify, this refers to all SegWit address versions from 1 to 15, as thi=
s restriction exists for SegWit address v0?

>
> -   Define a variant of Bech32 with the modified constant, so that
>     non-BIP173 uses of Bech32 can choose a non-impacted version if they
>     worry about this class of errors.
>

Okay, this probably needs to be raised in lightning-dev as well, for invoic=
e formats, as well as planned offers feature.

By my understanding, best practice for readers of Bech32-based formats woul=
d be something like the below:

1.  Define two variants of checksum, the current Bech32 checksum and the mo=
dified Bech32 checksum.
2.  Support both variants (software tries one first, then tries the other i=
f it fails).
3.  Flag or signal some deprecation warning if current Bech32 checksum was =
detected.
4.  At some undefined point in the future, drop support for the current Bec=
h32 checksum.

> -   Later, if and when we expect a need for non-32-byte witness programs
>     in the medium term, define an updated segwit address scheme that uses
>     the modified Bech32 variant.


Okay, so we will only use the modified Bech32 if and only if we expect to n=
eed a non-32-byte witness program for a particular non-0 SegWit version.


Regards,
ZmnSCPxj