For the second time in about a month, btcd / LND exploited a bug that led them to deviate from Bitcoin Core in consensus. Once again, Burak was the developer who triggered this vulnerability – this time it was clearly intentional – and again it was a problem with the code for analyzing Bitcoin transactions above the consensus level. As I discussed in my piece on the previous bug triggered by Burak, prior to Taproot there were limits on how large the script and witness data could be in a transaction. With the activation of Taproot, these limits have been removed leaving only the limitations on the size of the block itself to limit these parts of the individual transactions. The problem with the latest bug was that although the consent code in btcd had been correctly updated to reflect this change, the code that handled peer-to-peer transmission, including parsing the data before submission or during receiving, was not updated correctly. So blocks and code processing transactions before they were actually passed to be validated for consent failed data, never passed it to consent validation logic, and the block in question was never validated.This time a very similar thing happened. Another limitation in the peer-to-peer section of the code base was to impose a restriction on the witness data incorrectly, limiting it to a maximum of 1/8 of the block size of the whole block size. Burak made a transaction with witness data just a single unit of weight over the strict limit and once again locked the btcd and LND nodes at that block height. This transaction was a non-standard transaction, which means that even though it is perfectly valid for the consent rules, it is not valid according to the default mempool policy and therefore nodes will not broadcast it across the network. It is perfectly possible to mine it in a block, but the only way to do this is to supply it directly to a miner, as Burak did with the help of F2Pool.
This really makes it clear that any piece of code whose purpose is to analyze and validate Bitcoin data needs to go through a thorough check to ensure it is in line with what Bitcoin Core will do. It doesn’t matter if that code is the consensus engine for implementing a node or just a piece of code that passes transactions for a Lightning node. This second bug was literally above that of last month in the code base. It wasn’t even discovered by anyone at Lightning Labs. AJ Towns reported this on October 11, two days after the original bug was triggered by Burak’s 998 on 999 multisig transaction. It was publicly posted on Github for 10 hours before being deleted. A fix has therefore been made, but not released, with the intention of silently fixing the problem in the next version of LND.
Now, this is a pretty standard procedure for a severe vulnerability, especially with a project like Bitcoin Core where such a vulnerability can actually cause serious damage to the base layer network / protocol. But in this specific case, it presented a serious risk to LND users’ funds, and as it was literally next to the previous bug that presented the same risks, the chances of it being found and exploited were very high, as demonstrated by Burak. This raises the question of whether the silent patch approach is the way to go when it comes to vulnerabilities like this that can leave users open to theft of funds (because their node cannot detect old channel states and penalize them appropriately).
As I explained in my article on the latest bug, if an attacker found the bugs before a well-meaning developer, they could tactically open new channels to vulnerable nodes, redirect the entire content of those channels to themselves, and then exploit them. the bug. From there they would have those funds under their control and could also close the channel with the initial state, literally doubling their money. What Burak did by actively exploiting this problem in an ironic way actually protected LND users from such an attack.
Once exploited, users were open to such attacks by pre-existing peers with whom they already had open channels, but were no longer able to be specifically targeted with new channels. Their nodes were blocked and they would never recognize or process payments through the channels that someone tried to open after the block that blocked their node. So while it didn’t completely remove the risk of user exploitation, it limited that risk to the people they already had a channel with. Burak’s action mitigated it. Personally I think this kind of action in response to the bug made sense; it limited the damage, made users aware of the risk, and led to a quick fix.
LND wasn’t the only thing affected either. Liquids the pegging process was also stoppedrequesting updates from federation officials to resolve it. Previous versions of Rust Bitcoin were also affected, which caused the deadlock to affect some block explorers and elecrs instances (a backend server implementation for Electrum Wallet). Now, with the exception of Liquid’s peg which ultimately exposed the funds to the disaster recovery keys held by Blockstream after a time limit – and, realistically in the plot of the heist-style film where Blockstream stole these funds, all they know exactly who to hunt – these other problems never jeopardize anyone’s funds at any time. Furthermore, Rust Bitcoin had actually fixed this specific bug in more recent versions, which apparently did not lead to any communication with the maintainers of other codebases to highlight the potential of such problems. It was only the active exploitation of the live bug on the network that largely revealed that the problem existed in multiple codebases.
This raises some big problems when it comes to vulnerabilities like this in tier 2 software on Bitcoin. First, the seriousness with which these code bases are checked for security bugs and how they are prioritized over integrating new features. I think it’s very telling that security doesn’t always take priority as this second bug wasn’t even found by the maintainers of the code base it was in, even though it was literally next to the initial bug discovered last month. After a major bug that put users’ funds at risk, no internal audit of that code base was performed? Did it take someone outside the project to find out? This does not prove a priority for safeguarding user funds over creating new features to attract more users. Secondly, the fact that this problem has already been fixed in Rust Bitcoin demonstrates a lack of communication between maintainers of different code bases regarding bugs like this one. This is quite understandable, since being completely different code bases it doesn’t immediately make someone think they found a bug in one: “I should contact other teams who write similar software in totally different programming languages to warn them of the potential of such a bug” . You do not find a bug in Windows and then immediately think about going to report the bug to the Linux kernel maintainers. Bitcoin as a distributed consensus protocol across a global network is a very different beast, however; perhaps Bitcoin developers should start thinking this way when it comes to vulnerabilities in Bitcoin software. Especially when it comes to analyzing and interpreting consent data.
Finally, perhaps when it comes to protocols like Lightning, which depend on observing the blockchain at all times to be able to react to old channel states in order to maintain independent data security, analysis and verification should be reduced to a minimum, if not completely removed and delegated to Bitcoin Core or data directly derived from it. Core Lightning is designed this way, connecting to a Bitcoin Core instance and depending entirely on that for validating blocks and transactions. If LND had worked the same way, none of these btcd bugs would have affected LND users in a way that would put their funds at risk.
Whatever way things are handled, either by outsourcing validation completely or simply by minimizing internal validation and addressing it much more carefully, this incident shows that something needs to change in addressing the problem of how level 2 software handles. interaction with consent data. Again, everyone is very lucky that this wasn’t exploited by a malicious actor, but instead by a developer who proved a point. That said, Bitcoin cannot count on luck or hope that there are no malicious actors.
Developers and users should focus on improving processes to prevent incidents like this from happening again and not on the game of blaming like a hot potato.
This is a guest post from Shinobi. The views expressed are entirely their own and do not necessarily reflect those of BTC Inc or Bitcoin Magazine.