Dive into the Dedaub blog for expert insights on smart contract security and blockchain innovation. Explore the latest advancements in the Ethereum ecosystem, including program analysis and DeFi exploits, through concise, expert-driven content designed for developers and blockchain enthusiasts.
A few hours ago, the Dedaub team discovered a smart contract vulnerability in a number of uniBTC vault smart contracts in the Bedrock project. We disclosed the issue to the Bedrock account on Twitter and soon thereafter (after no response in 20 mins) to SEAL 911 for immediate investigation and action.
A SEAL 911 war room, under the guidance of @pcaversaccio, was created and we frantically tried for two hours to reach Bedrock developers. At that time, blackhats exploited the vulnerability for a $1.8m loss. However, given that this was an infinite-mint vulnerability on the uniBTC token, it is perhaps fair to assess that the damage was contained. Most of the potential losses were averted by pausing third party protocols exposed to the at-risk funds, including Pendle and Corn. Notably, Pendle had over $30M of liquidity on the Corn network for the vulnerable asset. On Ethereum, the market cap of uniBTC was $75M, which an infinite mint renders worthless, and the asset was deployed in (at least) 8 networks.
Root Cause
The root cause of this vulnerability is a mismatched calculation of the exchange rate between Ethereum and Bitcoin, in one path of the minting logic. In turn, this allows anyone who deposits Ethereum to the vulnerable smart contract vault to mint uniBTC in equal amounts. (Up until the vulnerability, uniBTC could exit to Wrapped Bitcoin at 1-1 rates.) Since the price of Ethereum is many times lower than the price of BTC, this creates an instant profit for any attacker exploiting any of these vaults. The vulnerable vault contract was a permissioned minter for uniBTC, so infinite amounts could be minted. The only adjustment made during this minting function is appropriate scaling in the number decimals of the assets.
In order to appreciate the gravity of this issue, we can illustrate this directly on the following code, straight from the vulnerable uniBTC vault smart contract (the implementation behind the proxy for the Vault):
function mint() external payable { require(!paused[NATIVE_BTC], "SYS002"); // Dedaub: adjust decimals and mint equal amount _mint(msg.sender, msg.value); }
Once the issue is exploited, the next step of a potential attacker would be to make use of this ill-gotten token on a number of other DeFi protocols, such as decentralized exchanges like Uniswap.
Reporting the issue to Bedrock and exploitation
As soon as our team had the issue, we contacted Bedrock on Twitter and entered a war room on SEAL 911.
Initial X.com exchange (time in UTC+2).
Unfortunately, even though we found the issue in the smart contract several hours before, by the time the team responded, the vulnerability had been exploited. The vulnerability could be discovered via shallow means (e.g., fuzzing bots) and the smart contract had only been deployed for under two days.
Timeline prior to exploit:
UTC 16:00 – issue discovered by Dedaub team and confirmed through simulation
The exploiter(s) subsequently minted large amounts of uniBTC and swapped them on a number of Uniswap and other AMM pools, stealing around $2M in funds directly. Note that the market cap of uniBTC on the Ethereum mainnet is $75M, which is the real potential loss for an infinite mint vulnerability.
Notably, the vulnerable contract was deployed on (at least) 8 different chains. We are aware of Ethereum, Binance, Arbitrum, Optimism, Mantle, Mode, BOB, and ZetaChain.
Averting Larger Losses
In addition to a number of pools on Uniswap (and Pancakeswap on Binance), the largest holder of uniBTC was Pendle. Luckily through war room actions, the Pendle team disabled the uniBTC token on their platform. With the main exit liquidity gone, the Bedrock team reacted some hours later (with the main devs in a 2-5am timezone) to also pause the relevant vaults.
This article will be updated with more detail and context on the discovery (which happened as part of a challenge task during our company retreat) in the next days.
On July 19th, Rho Markets — a Compound V2 fork on Scroll — was involved in an incident that led to the creation of $7.5mil in bad debt. The root cause of the vulnerability was the misconfiguration of the oracle for ETH, i.e., setting the address to a wrong price feed, at initialization time, and not a bug in the code. The price mis-alignment was quickly exploited by an MEV bot that observed the opportunity.
Fortunately, lost funds were returned due to the quick and coordinated efforts between the protocol team and SEAL 911 (of which we are also part). That being said, the willingness of the MEV Bot operator behind the incident to co-operate with the protocol and return the funds significantly sped up the recovery.
One may also read Rho Markets’ official statement [tweet | blog] on the incident, which does a great job of explaining the circumstances that led to the loss of funds.
Technical Details
The misconfiguration
Quoting Rho Markets’ incident report:
This issue occurred due to the erroneous configuration of the ETH oracle price feed to the BTC price feed. Normally, such settings are validated before any changes are implemented. However, due to a human error in overseeing the deployment process, this validation check was missed in the case of the oracle price.
As can be seen from the emitted event, the transaction has the effect of erroneously setting the oracle for the underlying asset at address(0) to be the WBTC/USD oracle.
At the time of the transaction, the configuration for the rWBTC token ( 0x1d7.. ) had not been set inside the oracle contract:
This is also evident by the fact that no calls to the PirceOracleV2‘s setRTokenConfig function had been performed, prior to the oracle update on block 7580111 and after rWBTC ’s deployment on block 7579842
The problem with setting the oracle for the asset at address(0) is that, as stated before, this address represents the underlying token of rETH (ETH) [ configuration of rETH ]
Which is a notion inherited from Compound’s semantics:
Screenshot depicting the configuration of ETH in Compound’s UniswapAnchoredView contract — the second address in the tuple is the underlying address
The consequences of the misconfiguration
At this point we can note that no contract code was vulnerable or broken, since the setter of the price oracle functioned as intended.
However, this misconfiguration alone was enough to enable the arbitrage opportunity that an MEV bot exploited. Since all ETH collateral would be priced at the value of WBTC —a 20X increase in the actual value of ETH— this allowed the bot to borrow more funds than one should normally get when using ETH as collateral (which lead to the creation of bad debt).
But it successfully manages to borrow ~942 wstETH which is then swapped into ETH
The total amount of bad debt that was created by this method ended up being ~7.5 million.
Return of funds
The war room set up with the protocol team and SEAL 911 was quick in gathering information on the attack and the operator of the MEV bot. However, the bot operator acted in good will and contacted the protocol team for the return of funds:
Hello RHO team, our MEV bot have profited from your price oracle misconfiguration. We understand that the funds belong to users and are willing to fully return. But first we would like you to admit that it was not an exploit or a hack, but a misconfiguration on your end. Also, please provide what are you going to do to prevent it from happening again.
The incident highlights the importance of rigorously reviewing deployment procedures. Even when there are no smart contract vulnerabilities, protocols must ensure that updates do not break any invariants posed by the protocol’s complex modules.
We’d like to thank Rho Markets for their quick action and transparency on the issue, as well as all the members of SEAL 911 who also participated in the war room with us.
Dedaub’s Security Audit teams comprise at least two senior security researchers, as well as any support they may need (e.g., cryptography expertise, financial modeling, testing) from the rest of our team. We carefully match the team’s expertise to your project’s specific nature and requirements. Our auditors conduct a meticulous, line-by-line review of every contract within the audit scope, ensuring that each researcher examines 100% of the code. There is no substitute for deep understanding of the code and forming a thorough mental model of its interactions and correctness assumptions.
Web3 Audit Methodology | 4 Main Strategies
Reaching this level of understanding is the goal of a Dedaub audit. To achieve this, we employ strategies such as:
Two-phase review: during phase A, the auditors understand the code in terms of functionality, i.e., in terms of legitimate use. During phase B, the auditors assume the role of attackers and attempt to subvert the system’s assumptions by abusing its flexibility.
Constant challenging between the two senior auditors: the two auditors will continuously challenge each other, trying to identify dark spots. An auditor who claims to have covered and to understand part of the code is often challenged to explain difficult elements to the other auditor.
Thinking at multiple levels: beyond thinking of adversarial scenarios in self-contained parts of the protocol, the auditors explicitly attempt to devise complex combinations of different parts that may result in unexpected behavior.
Use of advanced tools: every project is uploaded to the Dedaub Security Suite for analysis by over 70 static analysis algorithms, AI, and automated fuzzing. The auditors often also write and run manual tests on possible leads for issues. Before the conclusion of the audit, the development team gets access to the online system with our automated analyses, so they can see all the machine-generated warnings that the auditors also reviewed.
Dedaub’s auditors also identify gas inefficiencies in your smart contracts and offer cost optimization recommendations. We thoroughly audit integrations with external protocols and dependencies, such as AMMs, lending platforms, and Oracle services, to ensure they align with their specifications.
Understanding and Mitigating Solidity Security Vulnerabilities
Solidity Security Vulnerabilities are critical concerns for developers building smart contracts on the Ethereum blockchain and other EVM-compatible platforms. Solidity is the primary language for creating smart contracts on the Ethereum blockchain and other EVM-compatible platforms. It enables developers to build decentralized applications (DApps) that automate complex processes. However, blockchains’ immutability and decentralized nature make vulnerabilities in smart contracts especially critical. A single security flaw can result in the loss of millions of dollars in cryptocurrency, as demonstrated by numerous high-profile hacks.
This guide looks at the most common Solidity security vulnerabilities.
Access Control Failures
One of the most common Solidity security vulnerabilities is the failure to protect sensitive external functions with an access control modifier. Usually, a contract will have some privileged functionality that should only be called by the contract’s owner, for instance, to configure some of its parameters. Failing to protect this with an access control modifier such as onlyOwner can lead to disastrous consequences, as any attacker will be able to modify the core behavior of the smart contract.
Unchecked External Calls
Unchecked external calls can introduce significant Solidity security vulnerabilities.
Developers should control which contracts their application interacts with. Trusting arbitrary contracts and handing control to them means accepting malicious contracts could potentially interact with your application. This can lead to unintended behavior or even attacks.
In general, developers should interact only with trusted contracts. If these contracts are unknown, they should implement a system that allows the contract owner to whitelist contracts on demand.
Reentrancy Attacks
Reentrancy is a notorious Solidity security vulnerability where an external contract can hijack the control flow of the target contract. These can occur when one of the smart contract’s external functions temporarily transfers control to another contract before continuing to execute its own state-modifying code.
If the other contract is malicious, it can call the external function again, making it re-execute the code before the transfer of control happens. This takes place without the external function having completed the execution of the original call beyond the transfer of the control point. The procedure is called a re-entry and can allow an attacker to change the state of the contract in an undesirable manner.
Developers can prevent this kind of attack by using the checks-effects-interactions pattern. This pattern ensures that all state changes occur before transferring control to an external contract. Any re-entry attempt produces a fresh call, avoiding interaction with partially executed computations.
Integer Overflow/Underflow
Solidity uses integer data types for various calculations. Exceeding these data types’ maximum or minimum values can result in overflows or underflows. Solidity versions equal to or above 0.8.0 will revert automatically if this happens, and this can lead to unexpected reverts in your smart contracts. On the other hand, integer variables will wrap around if they are part of unchecked code, although this can lead to unexpected behavior unless there is a particular reason why overflow and underflow cannot occur.
Out-of-Gas Situations
Ethereum sets gas limits on transactions to prevent infinite loops and resource exhaustion. Smart contract developers must know these limits and gracefully handle out-of-gas situations.
For example, a resource-intensive loop can cause a transaction to fail due to hitting the gas limit, which may result in a frustrating user experience.
Sometimes, this situation can also lead to denial of service (DoS) attacks, where an attacker arbitrarily extends the length of the loop, effectively causing the functionality to become disabled.
An example of this scenario would be a function that loops over all registered users and sends them some funds. An attacker could increase the length of this loop by registering many bogus accounts with this system.
In general, it is preferable to avoid nested loops and adopt a pull system in which individual users request an operation rather than a push system that performs the operation for all users.
Oracle Staleness and Manipulation
Some contracts interact with oracles, which make off-chain data available on the blockchain, such as a Chainlink price feed. Developers should perform basic sanity checks on the provided data when interacting with oracles. It’s essential, therefore, to have a backup plan if the oracle fails.
For instance, contracts should always check that the data is not stale by checking the timestamp of the last data point is not further than a specified amount in the past. They should also check that the data is not an anomalous value such as zero or a negative number. In these cases, the contract should resort to a sensible default or pause the application until the feed starts reporting correct data again.
Developers should use only high-quality oracles to avoid some of the issues mentioned above. Some oracles, such as pricing data from an automated market maker (AMM), cannot be relied upon because they may suffer from value manipulation. Such manipulations will then have a ripple effect on your application as well.
Conclusion: Solidity Security Vulnerabilities
Solidity is a powerful language, but it’s easy to make mistakes that lead to severe vulnerabilities. These are only a tiny sample of the many vulnerabilities that can have a damaging effect on a smart contract. Therefore, before going live on the mainnet, it’s essential to audit your code.
You can also use tools like the Dedaub Security Suite to catch issues early. This tool helps you find and fix vulnerabilities in your smart contracts, giving you confidence in your code before deployment. Create your free account today at app.dedaub.com.
Most Dapp developers have heard of and probably use the excellent Multicall contract to bundle their eth_calls and reduce latency for bulk ETL in their applications (we do too, we even have a python library for it: Manifold).
Unfortunately, we cannot use this same trick when getting storage slots, as we discovered when developing our storage explorer, forcing developers to issue an eth_getStorageAt for each slot they want to query. Luckily, Geth has a trick up its sleeve, the “State Override Set”, which, with a little ingenuity, we can leverage to get bulk storage extraction.
Bulk Storage Extraction | Geth Trickery
The “state-override set” parameter of Geth’s eth_call implementation is a powerful but not very well-known feature. (The feature is also present in other Geth-based nodes, which form the base infrastructure for most EVM chains!) This feature enables transaction simulation over a modified blockchain state without any need for a local fork or other machinery!
Using this, we can change the balance or nonce for any address, as well set the storage or the code for any contract. The latter modification is the important one here, as it allows us to replace the code at an address we want to query the storage for with our own contract that implements arbitrary storage lookups.
Here is the detailed structure of state-override set entries:
FIELD
TYPE
BYTES
OPTIONAL
DESCRIPTION
balance
Quantity
<32
Yes
Fake balance to set for the account before executing the call.
nonce
Quantity
<8
Yes
Fake nonce to set for the account before executing the call.
code
Binary
any
Yes
Fake EVM bytecode to inject into the account before executing the call.
state
Object
any
Yes
Fake key-value mapping to override all slots in the account storage before executing the call.
stateDiff
Object
any
Yes
Fake key-value mapping to override individual slots in the account storage before executing the call.
Bulk Storage Extraction | Contract Optimizoor
To better understand what’s going on we can take a look at the high level code (this was actually generated by our decompiler)
The following handwritten smart contract has been optimized to maximize the number of storage slots we can read in a given transaction. Before diving into the results I’d like to take an aside to dive into this contract as it’s a good example of an optimized single-use contract, with some clever (or at least we think so) shortcuts.
function function_selector() public payable {
v0 = v1 = 0;
while (msg.data.length != v0) {
MEM[v0] = STORAGE[msg.data[v0]];
v0 += 32;
}
return MEM[0: msg.data.length];
}
Walking through the code we can see that we loop through the calldata, reading each word, looking up the corresponding storage location, and writing the result into memory.
The main optimizations are:
removing the need for a dispatch function
re-using the loop counter to track the memory position for writing results
removing abi-encoding by assuming that the input is a contiguous array of words (32-byte elements) and using calldatalength to calculate the number of elements
If you think you can write a shorter or more optimized bytecode please submit a PR to storage-extractor and @ us on twitter.
Bulk Storage Extraction | Results
THEORETICAL RESULTS
To calculate the maximum number of storage slots we can extract we need three equations, the execution cost (calculated as the constant cost plus the cost per iteration), the memory expansion cost $$(3x+(x^2/512))$$ and the calldata cost.
We can break down the cost of the execution as follows:
The start, the range check and the exit will always run at least once
Each storage location will result in 1 range check and 1 lookup
Calculating the calldata cost is slightly more complex as its variably-priced: empty (zero-byte) calldata is priced at 4 gas per byte and non-zero calldata is priced at 16 gas. Therefore we need to calculate a placeholder for the average price of a word (32-bytes).
zero_byte_gas = 4
non_zero_byte_gas = 16
# We calculate this as the probability each bit of a byte is a 0
prob_rand_byte_is_zero = (0.5**8) # 0.00390625
prob_rand_byte_non_zero = 1 - prob_rand_byte_is_zero # 0.99609375
avg_cost_byte = (non_zero_byte_gas * prob_rand_byte_non_zero) + \
(zero_byte_gas * prob_rand_byte_is_zero) # (16 * 0.99609375) + (04 * .00390625) = 15.953125
Therefore the average word costs: $$15.953125 * 32 * x$$
We can combine all of these equations and solve for the gas limit to get the maximum number of storage slots that can be read in one call.
Therefore given a 50 million gas limit (which is the default for Geth) we can read an average of 18514 slots.
This number will change based on the actual storage slots being accessed, with most users being able to access more. This is due to the fact that most storage variables are in the initial slots of the contract, with only mapping and dynamic arrays being pushed to random slots (or people using advanced storage layouts such as those used in Diamond proxies).
PRACTICAL RESULTS
To show the impact of this approach, we wrote a python script which queries a number of storage slots, first using normal RPC requests and batched RPC requests for the normal eth_getStorageAt, and then comparing to the optimized eth_call with state-override set. All the testing code can be found in the storage-extractor repo, along with the bytecode and results.
To isolate variable latency as a concern, we ran the tests on the same machine as our node, with latency being re-added by utilizing asyncio.sleep to have a controlled testing environment. To properly understand the results, lets look at the best-case scenario of 200 concurrent connections.
In order to properly represent the three methods we need to set the y-axis to be logarithmic since standard parallel `eth_getStorageAt`s are too slow. As you can see even with 200 connections standard RPC calls are 57 times slower than RPC batching and 103 times slower than `eth_call` with state-override.
We can take a closer look at the difference between batching and call overrides in the next graph. As you can see, call overrides are faster in all scenarios since they require fewer connections, this is most noticeable with the graph in the top left which highlights the impact of latency on the overall duration.
Conclusion
To wrap up this Dedaub blog post, I’d like to thank the Geth developers for all the hard work they’ve been doing, and the extra thought they put into their RPC to enable us to do funky stuff like this to maximize the performance of our applications.
If you have a cool use of the state-override set please tweet us, and, if you’d like to collaborate, you can submit a PR on the accompanying github repo (storage-extractor).
The Arbitrum network experienced significant downtime on December 15 due to problems with its sequencer and feed. The network had been down for almost three hours. The major outage began at 10:29 a.m. ET amid a substantial increase in a type of network traffic called Inscriptions. Arbitrum’s layer-2 network had processed over 22.29 million transactions and had a total value locked of $2.3 billion. Despite the success of the network, the current design suffers from a significant chokepoint when posting transactions to L1, causing the sequener to stall. While advancements such as Arbitrum Nova and Proto-danksharding might alleviate these design issues, this is not the first time Arbitrum has experienced such issues – a bug in the sequencer also halted the network in June 2023.
Arbitrum Sequencer Outage | Background
Arbitrum is a Layer-2 (L2) solution which settles transactions off the Ethereum mainnet. L2s provide lower gas fees and reduce congestion on the primary blockchain (In this case, Ethereum, L1). The current incarnation of Arbitrum is called Nitro. Arbitrum Nitro processes transactions in two stages: sequencing, where transactions are ordered and committed to this sequence, and deterministic execution, where each transaction undergoes a state transition function. Nitro combines Ethereum emulation software with extensions for cross-chain functionalities and uses an optimistic rollup protocol based on interactive fraud proofs. The Sequencer is a key component in the Nitro architecture. Its primary role is to order incoming transactions honestly, typically following a first-come, first-served policy. This is a centralized component operated by Offchain Labs. The Sequencer publishes its transaction order both as a real-time feed and to Ethereum, in the calldata of an “Inbox” smart contract. This publication ensures the final and authoritative transaction ordering. Additionally, a Delayed Inbox mechanism exists for L1 Ethereum contracts to submit transactions and as a backup for direct submission in case of Sequencer failure or censorship.
Arbitrum Sequencer Outage | Root cause
In the two hours prior to the outage more than 90% of Arbitrum traffic consisted of Ethscriptions. Ethscriptions are digital artifacts on EVM chains created using Ethereum calldata. Unlike traditional NFTs managed by smart contracts, Ethscriptions make the blockchain data itself a unique NFT. They are inspired by Bitcoin inscriptions (Ordinals) but function differently. Creating an Ethscription involves selecting an image, converting it to data URI format, then to hexadecimal format, and finally embedding it into a 0 ETH transaction’s Hex data field. Each Ethscription must be unique; duplicate data submissions are ignored. Owners can use Ethscriptions IDs for proof or transfer of ownership. In practice the calldata or Ethscriptions look like the code below:
Calldata example of an Ethscription. This represents a token mint.
Since Ethscriptions are very cheap, one can do a lot of them for the same unit of cost. Indeed, a staggering 90% of transactions posted on-chain were Ethscriptions. Also, for a relatively low cost, the amount of transaction entropy that needed to be committed to L1 increased to 80MB/hr vs. the 3MB/hr that was typical before the traffic spike. We calculated this by looking at average on-chain transaction postings for the sequencer.
Now, look at the architecture diagram of Arbitrum below. Note that in order to commit transaction sequences to L1, the data poster needs to post the increased amount of data over a larger number of transactions. Prior to the outage, the number of transactions posted per hour was around 10 – 20x higher than the December mean.
However, the code responsible for posting these transactions has an in-built limitation that imposes limits to the rate at which L1 batches are posted. Prior to the outage, if there are 10 batches still in the L1 mempool, no more batches are sent to L1, stalling the sequencer. This limit was subsequently raised to 20 batches after the outage. This is probably not a good long-term solution however, as it increases the chances of batches needing to be reposted due to transaction nonce issues.
// Check that posting a new transaction won't exceed maximum pending
// transactions in mempool.
if cfg.MaxMempoolTransactions > 0 {
unconfirmedNonce, err := p.client.NonceAt(ctx, p.Sender(), nil)
if err != nil {
return fmt.Errorf("getting nonce of a dataposter sender: %w", err)
}
if nextNonce >= cfg.MaxMempoolTransactions+unconfirmedNonce {
return fmt.Errorf(
"... transaction with nonce: %d will exceed max mempool size ...",
nextNonce, cfg.MaxMempoolTransactions, unconfirmedNonce
)
}
}
return nil
Batch poster is responsible for posting the sequenced transaction sequence as Ethereum calldata.
Arbitrum Sequencer Outage | Recommendations
There are several indications that point towards the sequencer, and thus the network, not being tested enough in a realistic setting or in an adversarial environment. However, luckily the upcoming Proto-Danksharding upgrade to Ethereum should also help for reducing L1-induced congestion. Irrespective of this the Arbitrum engineers can consider the following recommendations:
Whether the Arbitrum gas price of L2 calldata is set too low, compared to other kinds of operations. Gas is an anti-DoS mechanism, which is intimately tied to the L1 characteristics. If this increase in L2 calldata causes a proportionally large increase in batch size, then attackers can craft L2 transactions with large calldatas that result in batches that don’t compress well under Brotli compression, causing a DoS attack on the sequencer. Note that Arbitrum Nova should not suffer as much from this issue as the transaction data is not stored on L1, only a hash is.
Whether there is a tight feedback loop between the size of the L1 batches currently in the mempool and L2 gas price. There is an indirect feedback loop, via the gas price on L1 and backlog sizes, but this may not be too tight. In addition, since the sequencer is centralized anyway, anti-DoS measures might be encoded directly into it to reject transactions. (Note: A more decentralized sequencer is being considered for the future, so this last measure wouldn’t work)
Long-term, the engineers more research into making the rollups more efficient to decrease the sizes of batches committed to L1. This may include ZKP rollups at some point.
Additionally, security audits to the sequencer should consider DoS situations, both through simulation/fuzzing and also by having auditors think of hostile situations through adversarial thinking based off their deep knowledge of the involved chains.
Finally, the Arbitrum team made a small change to the way transactions are soft-committed. In this change the feed backlog is populated irrespective of whether the sequencer coordinator is running, which carries its own risks but enables dApps running on Arbitrum to be more responsive during certain periods.
Disclaimer: The Arbitrum sequencer is solely operated by Offchain labs. Thus, most of the information regarding its operational issues (such as logs) are not publicly available so it’s hard to get a complete picture of the issue. Dedaub has not audited Arbitrum or Offchain labs software. Dedaub has however audited other (non-Arbitrum) software and projects running on Arbitrum such as GMX, Chainlink, Rysk & Stella.
Hello everyone, this is Yannis Bollanos, Security Researcher at Dedaub. A few days ago, we published a tweet about the thestandard.io exploit that took place on November 6th, 2023, which you can find here: https://twitter.com/dedaub/status/1734598398055981471.
The positive response from the X audience indicates a strong interest in the topic. As a result, I have decided to expand it into a blog post that can be used as a reference in the future.
Thestandard.io exploit occurred on November 6th, 2023, and according to Crypto.news, approximately 280K EUROs were at risk. Fortunately, most of the funds have been recovered, so this is a hack story with a happy ending.
After the excitement and tension of the moment subside, it is important to reflect on what happened and how we can prevent similar attacks in the future. It’s a great opportunity to re-emphasize that protocols should use defensive checks/assertions at every point their code interacts with a decentralized exchange (DEX).
The @thestandard.io protocol issues coins to users who open over-collateralized positions, helping the protocol’s assets maintain a stable value by adjusting liquidity provision by actual market rates.
In the @thestandard.io attack scenario, a SmartVault contract oversees the management of each user’s position, taking responsibility for adequately verifying the position’s liquidity. Users can issue coins by calling `mint`:
The SmartVault allows the exchanging of deposited collateral tokens through Uniswap’s V3 router (0xe592427a0aece92de3edee1f18e0157c05861564 on Arbitrum). Here is where things get interesting:
With amountOutMinimum set to 0, the swap operation would succeed no matter the extent of the slippage incurred.
There were no other safeguards in place to ensure a fair exchange for the value provided in the contract.
This enabled the owner of the vault contract to initiate a swap on a pool that might have been maliciously ’tilted,’ allowing for an exchange at an arbitrarily different price from the market price.
There are two ways to profit from this:
(1) Utilize a flash loan and purposely sandwich the swap operation between a tilting and an un-tiling swap on the pool. This is a fairly typical attack pattern commonly used in exploits.
OR
(2) Have the swap operation occur on a pool, the liquidity of which (as well as the execution price) is entirely controlled by the attacker. This can be done only on freshly created pools or in pools with near-0 liquidity.
The attacker chose option (2) since a Uniswap V3 pool for PAXG-WBTC didn’t exist then. Here’s how everything is put together to form the attack:
Attack Transaction:
The attacker creates the Uniswap v3 PAXG-WBTC pool
The attacker flash borrows 10 WBTC ( and a tiny extra amount to provide as initial liquidity)
The attacker provides 10 WBTC as collateral and mints as many EUROs as possible
The attacker provides liquidity to the PAXG/WBTC pool. WBTC and PAXG are at a 1:1 ratio within the tick range in which liquidity is minted. This is over-valuing PAXG by a lot.
The attacker swaps the deposited WBTC for PAXG, and the swap operation goes through the attacker-controlled pool. The vault is now under-collateralized, in terms of real value: the PAXG it obtained has much less value than the EUROs issued.
The attacker then burns all of his liquidity on Uniswap, and he notably receives ~9.9 WBTC. At this point, the attacker still holds the originally minted EUROs.
The attacker swaps 10k of his EUROs for USDCs. Some USDCs are then employed to obtain the few remaining WBTCs needed to repay the flash loan.
In the end, the attacker walks away with 280k EUROs and ~8.5k USDC.
Fortunately, the attacker has returned ~240k EUROs back to the protocol:
Smart Contract developers should not solely rely on assumptions about on-chain liquidity/asset prices. The code should consistently enforce these assumptions (within a reasonable deviation).
Transaction simulation tools improve developer and user experience when operating decentralized Web3 applications (Smart Contracts running on programmable blockchains).
These tools can lower the risk and guesswork during development, deployment, and subsequent operation of Web3 applications. And they’re particularly useful in hostile security environments such as public blockchains.
Transaction simulation tools allow developers and users to “dry-run” the execution of transactions on the blockchain without committing the state changes of this transaction to the ledger.
For example, an end user can deposit funds in a yield farming vault and understand what proportion of the vault the deposit would be entitled to.
Another example is the simulation of a decentralized autonomous organization (DAO) proposal to evaluate its integrity and functionality, ensuring it’s not malicious before implementation.
In this article, we will explore the user experience and security issues that users and developers face when interacting with Web3 applications and how transaction simulation tools can help mitigate them.
By the end of this article, you’ll better understand what transaction simulation tools do, how they work, and how they can improve both user and developer experience.
The Need for Transaction Simulation Solutions in Blockchain
Web3 applications, such as DeFi applications, enable novel financial primitives with many more possibilities for end users. However, the complexity and irreversibility of blockchain transactions have led to unexpected fund losses for many users, often due to poorly designed interfaces in these applications.
Loss of funds is not the only issue for Web3 applications. We often face reverted or out-of-gas transactions, wasting funds, which is especially detrimental to our experience when interacting with Web3 applications.
The impact of these challenges is not limited to regular end-users. Developers and Web3 teams face the complex task of ensuring their contracts perform as intended.
Interacting with a blockchain protocol in a complex manner, for instance, through a multisig account, is a highly daunting task. Typically, it can be accomplished by forking the blockchain, but this is time-consuming.
Real-world scenarios underscore how critical transaction simulation solutions are. For instance, in platforms Yearn Finance or Uniswap Labs, where complex financial transactions are constant, the necessity to simulate transactions is invaluable.
In these cases, simulations allow users to review the outcomes of Smart Contract transactions in a controlled environment, giving teams time to identify and address potential issues before running them on-chain.
Types of Transaction Simulation Solutions Available
The market offers a variety of transaction simulation solutions, each catering to different needs and preferences.
Browser Extensions are popular for their ease of use, integrating with web browsers to offer simulation capabilities alongside wallet interactions.
In-Wallet Simulations integrate with the wallet software, providing a seamless experience for users to simulate transactions within the wallet interface.
Standalone Tools are comprehensive software solutions. These offer advanced features and greater flexibility for complex simulations. Developers and organizations needing detailed analyses and custom simulation scenarios prefer standalone tools.
Advantages of Using Transaction Simulation Tools
ERROR PREVENTION
Error prevention is a crucial advantage of transaction simulation tools, as they enable developers to simulate transactions in a controlled environment.
This process helps identify and correct errors before executing them on the blockchain, significantly reducing the likelihood of costly mistakes such as failed transactions that consume resources without achieving their intended outcomes.
Consequently, these tools greatly enhance blockchain applications’ overall reliability and efficiency.
EDUCATIONAL VALUE
For newcomers to blockchain development, transaction simulation solutions are invaluable educational resources. They provide a hands-on, risk-free platform for understanding the intricacies of blockchain transactions.
They allow developers to experiment with different scenarios, gaining practical insights into the operation of Smart Contracts. This experiential learning accelerates any developer’s expertise in blockchain technology, empowering them to build more sophisticated and secure dApps.
Choosing the Right Transaction Simulation Solution
Selecting an appropriate transaction simulation solution is crucial for blockchain developers. These tools come in various forms, each suited to different needs and environments.
Factors to Consider:
Network Support: Ensure the tool supports all relevant blockchain networks your project interacts with. For instance, if your Smart Contract runs on Ethereum and Polygon, the chosen transaction simulation solution must accommodate both.
Ease of Integration: Assess how seamlessly the tool integrates into your existing development. A smooth integration minimizes disruptions and maintains development flow.
User Experience: Assess the tool’s user interface and usability. A good simulator should offer clear insights into the transaction process, aiding decision-making and error identification.
Type of Tool: Decide between browser extensions and wallet-based simulators. Browser extensions are generally more flexible and accessible to test across various wallets, whereas wallet-based solutions offer a more integrated experience.
EVALUATION CRITERIA:
Reliability and Support: Investigate the tool’s performance history and the provider’s responsiveness to support queries and updates.
Track Record: Consider the provider’s reputation within the blockchain community. Long-standing, positively reviewed tools often indicate reliability and efficacy.
RECOMMENDATIONS:
Opt for solutions that prioritize security and accuracy in transaction simulation.
Avoid tools that are overly complex or do not offer transparent processes, as these can hinder rather than help your development efforts.
Stay informed about the latest developments in transaction simulation technologies to ensure your choice remains relevant and effective.
Selecting the right tool is crucial. It must meet technical requirements and adhere to the highest security and efficiency standards in the blockchain space.
Dedaub Watchdog Transaction Simulator
The Dedaub Watchdog Transaction Simulator allows users to simulate transactions when interacting with complex Smart Contracts before committing to the main chain.
It allows an understanding of all the various actions that would happen without the risk of losing funds. The Dedaub Watchdog transaction simulation provides three approaches, depending on specific use cases:
Through the Dedaub Simulation API, developers can integrate simulation directly into their applications.
Through the read/write/simulate feature on any Smart Contract page in Watchdog.
When used by an end-user, such as in the latter two approaches, the transaction simulation presents relevant information in convenient formats through the Watchdog UI.
One such format is the (i) trace format, which contains all intermediate Smart Contract functions called, new Smart Contracts created, and events fired.
The other format contains fund transfer, and (ii) includes the amount of funds transferred, both for the user and other participants in the transaction.
(Trace format above)
(funds transferred above)
When used by Web3 users, an important use case is checking the legitimacy and reliability of the accounts and Smart Contracts involved in the transaction. By simulating transactions, users can also gain insight into potential outcomes, allowing them to identify risks proactively.
The Dedaub Watchdog Transaction Simulator leverages the Dedaub Smart Contract database. The database offers detailed, real-time information on all deployed Smart Contracts on-chain, providing deep insights into the workings of Smart Contracts.
Conclusion
In conclusion, transaction simulation tools, particularly those exemplified by the Dedaub Watchdog Transaction Simulator, represent an advancement in Web3 application development and user interaction. They provide an extra layer of security and insight, allowing developers and end-users to identify and rectify potential issues in Smart Contract transactions promptly. These tools prevent costly errors and fund losses and serve as educational resources for those new to blockchain technology. With their ability to simulate complex financial transactions in a controlled environment, transaction simulation solutions enhance the efficiency, reliability, and overall user experience of interacting with Web3 applications.
As most programmers would admit, the most annoying bugs are often the “little” ones. Tiny logic errors caused by a few wrong characters in a single line of code, compiling fine and remaining undetected, patiently waiting to crash our program at the worst possible moment. We’ve all written such bugs, spent countless hours debugging them, and uttered the most horrific profanities when we finally discovered that we lost our sleep over a couple of wrong characters.
But losing a night’s sleep over a little bug isn’t the worst of our worries. At least not if one writes software for NASA, whose Mars Climate Orbiter famously burned up in the Martian atmosphere due to a software bug. Well, NASA software is complex; such a catastrophic bug should clearly be complicated, impossible to understand by mere mortals, right? Far from it, the bug that led to the loss of the $125 million Mars Climate Orbiter was a trivial but crucial missing multiplication by 4.45. Europeans aren’t immune to little bugs either; the loss of ESA’s $370 million Ariane V rocket in just 39 seconds was caused but a simple integer overflow error.
Thankfully, for the longest time, one needed to be employed by a space agency to worry about a little bug having such enormous financial consequences. That is until Smart Contracts arrived! Now, programs consisting of a few hundred lines of relatively “simple” code, developed by small teams over a relatively short period of time, are directly responsible for safeguarding various types of multi-million-dollar assets. All it takes is one undetected little bug and we get, not a spectacular rocket explosion, but an equally spectacular crypto hack that makes the Mars Climate Orbiter seem like pocket change.
So, let’s look at an instructive example of such a little bug. Smart Contracts typically use Solidity modifiers to guard their functions, performing crucial security checks.
modifier isOwner() {
// Make sure we're called by our trusted owner before doing anything.
require(msg.sender == owner, "Caller is not owner");
_;
}
Writing such a check is simple, no need to be a NASA engineer to do it. But better double and triple-check it because the consequences of the tiniest of bugs in that line are enormous.
error CallerNotOwner(); // gas efficient and easy to recognize
modifier isOwner() {
// I wish this were valid code, but it isn't.
require(msg.sender == owner, CallerNotOwner());
_;
}
Not a big deal, you’ll say, require is just a combination of a check and a revert; we can rewrite it and perform the two steps manually.
modifier isOwner() {
// This works fine
if(msg.sender != owner)
revert CallerNotOwner();
_;
}
Mission accomplished, but you might have noticed a small detail. In the code above msg.sender == owner was replaced by its negation: msg.sender != owner. This is because require expects a condition that should hold, while its if/revert replacement expects a condition that should not. So, in general, we should replace
This negation of the Boolean expression is exactly the beginning of our “little bugs” story. Well, how hard is it to simply add a “!”? But that’s not exactly what we did above, is it? No programmer that appreciates code simplicity and elegance writes
if(!(msg.sender == owner))
Everyone would simplify it to
if(!(msg.sender == owner))
bringing the negation inside the Boolean expression. And what if the negated expression is more complex? Logic, being the foundation of computer science, provides us with simple rules:
!(A && B) is equivalent to (!A || !B)
!(A || B) is equivalent to (!A && !B)
Just carefully follow the rules inside the complex Boolean expression, and you’ll be fine. Easier said than done; I bet every single programmer with a few years of experience has incorrectly negated a Boolean formula at some point in their career.
So, it shouldn’t be surprising that this exact bug appeared in one of our recent audits. The above commit aimed at replacing a string error with a custom one and, in doing so, changed:
Did you spot the negation error? The expression is of the form A || (B && C), so its negation becomes !A && (!B || !C), the && in B && C should change to ||. So, the correct check should be
These two wrong characters (&& instead of ||) completely change the logic of the modifier; now an unauthorized call with updater != address(0) and _msgSender() != address(this)) will not trigger the error as it should, which could easily lead to a total loss of funds for this specific contract.
Of course, the point is not that Smart Contracts are impossible to secure: this bug was caught by the audit (the chances of catching it were very high), and even if it weren’t, we are confident that it would still have been found before releasing the code, either by manual inspection or automated tests.
But its mere existence shows that Smart Contracts, as with all programs, are not immune to little bugs. Even the simplest of changes require caution and should be properly tested and audited, both internally and by external teams, to minimize the chances of a catastrophic little bug as much as possible.
Summary: The root cause of the thirdweb critical vulnerability is that independent libraries implementing ERC2771 & Multicall, such as OpenZeppelin Libraries, interact badly, when combined. This allows attackers to spoof the _msgSender() with all sorts of access control implications including loss of funds.
The issue is complex, but can be explained using a simple analogy. Imagine a bank that will let one of the bank officials carry out a transaction on your behalf, as long as the instruction is written on a piece of paper with your verified signature. This is a very common scenario, for instance with some preferred bank clients. So, you go to the bank official and hand him a signed piece of paper. Your instructions are “take this sealed box to the cashier, open it, and give him what’s inside”. The bank official happily executes your signed instructions, after checking your id against your signature. The sealed box contains another piece of paper reading …”do a withdrawal on behalf of Elon Musk”, signed with a fake signature. The cashier takes this piece of paper from the bank official, thinking that the signature was checked, when, really, the only signature that was checked was on the instructions to deliver and open the box. That’s it!
Now let’s look into the technical mechanics for how this vulnerability works, and how to protect your project from this issue.
The Critical Thirdweb Vulnerability | Background
First, we need to cover some preliminaries. In particular we need to first understand the implementation of the ERC2771 standard and the OpenZeppelin Multicall library. ERC2771 gives the ability to have a “virtual” msg.sender, i.e., caller of a public function of a smart contract.
ERC2771 defines a contract-level protocol for Recipient contracts to accept meta-transactions through trusted Forwarder contracts. No protocol changes are made. Recipient contracts are sent the effective msg.sender (referred to as _msgSender()) and msg.data (referred to as _msgData()) by appending additional calldata.
ETHEREUM ERC-2771
Therefore, this virtual msg.sender, called _msgSender() is set by a trusted external party, the forwarder. And how does the forwarder tell the contract what is the virtual msg.sender? It appends an extra parameter to all calls. This means that all functions of a contract that supports such virtual msg.senders need to take in an extra parameter which they interpret as the msg.sender. The other side of the vulnerability is Multicall. It is a way to have a single call that becomes many calls (to the same contract) in sequence. How does this happen? By making all the info of the “many calls” be parameters of the “outer” single call.
The Critical Thirdweb Vulnerability | Root cause?
The problem with these two libraries is that the forwarders (in ERC2771) were not designed to work with multicall. They add a single _msgSender() parameter to the outer call of a multicall. But remember: all functions now expect this parameter! Where can they get it from? The parameters of the *outer* multicall.
So, if an attacker uses multicall to call, say, 3 functions in sequence, the attacker can define all the parameters to these function calls, including the _msgSender()! This means that the attacker can make a call appear to be coming from anyone!
The Critical Thirdweb Vulnerability | Evaluating the impact
We have tried to reach out to most large projects that might have been affected (in collaboration with thirdweb and OpenZeppelin) over the last few days. However, if you are worried about this issue affecting your contract, we have flagged any contract affected on Watchdog and made this information available to the public. However, the extent to which your contract is affected depends on actual implementation of the contract. First, evaluate functions with access to _msgSender() (transitively). Do these functions contract check access control mechanisms using _msgSender()? For example, can someone withdraw or burn coins for the _msgSender()? In that case, the issue affects your contract, critically. In many of these contracts there may be onlyOwner or onlyRole modifiers that make use of _msgSender(). In addition, look for common transfer functions such as safeTransferFrom() or transfer(). The effect is also modulated by the value of the assets held by the contract, or if this contract represents an asset. Make sure to find out if your contract is a token in a Uniswap-like liquidity pool. It is possible that all the liquidity in this pool could be stolen due to this issue.
The Critical Thirdweb Vulnerability | Mitigation
The rest of the article outlines mitigation. Thirdweb has developed and deployed a mitigation tool that can possibly assist you. A large number of affected contracts were deployed by their product. However, oftentimes you’d need to take additional actions. Should you require assistance the team at Dedaub can at least point you to the right information. You may contact us here. In the rest of the article we list some some mitigation options we’ve observed over the last few days to be successful. Legal disclaimer: This should not be construed to be professional advice by our team.
PREFERRED MITIGATION: DISABLE TRUSTED FORWARDER
Some ERC2771 library implementations allow resetting a trusted forwarder. Doing so will prevent any gasless transaction from being executed through the forwarder, solving the issue (albeit at the cost of missing functionality). Unfortunately there are many instances where resetting the trusted forwarder is not possible, so the rest of the mitigation steps apply. Otherwise, your smart contract is probably safe from this issue.
ADVANCED MITIGATION METHODS
These mitigation methods may take time and expertise to successfully execute. If time is critical, you can consider decreasing the blast radius in the next section in case an attacker hacks the contract while you are in the process of planning a mitigation.
If your contract is Upgradeable, prepare an upgrade. Removing either multicall (and all functionality that delegatecalls to the same smart contract) prevents the attack. In addition, removing ERC-2771 functionality also prevents the attack. Other ways to prevent this attack involve adding a module that allows doctoring of the contract’s storage and removing the trusted forwarders in this way. This latter is difficult to execute correctly.
DECREASING THE BLAST RADIUS
Some steps can be taken in cases you do not manage to mitigate the issue in a timely fashion to limit the amount of stolen assets from your contract. This can be done in several ways:
Ask your users to remove approvals from your contract. You can additionally check which users have approved your smart contract to transfer funds by checking on app.dedaub.com, navigating to your smart contract, navigating to balances and then allowers. Note that publicly announcing removal of approvals can work both in favor and against you since malicious hackers could be tipped off.
Pausing your contract may help users from continuing to use it, but, depending on the implementation it might not prevent the attack.
Remove liquidity from Uniswap-like pools in case the token is held by a pool, otherwise the liquidity in this pool may be drained in some cases.
Conclusion
The thirdweb vulnerability is an unfortunate issue that came about due to the composability of libraries in a single smart contract, through inheritance mechanisms. Unfortunately, although libraries are supposed to be abstractions, when it comes to security abstractions can easily be broken and implementations can affect each other in unforseen ways. This was even the case despite the overwhelming majority of affected libraries were developed by the same organization. In their defence however, it is very hard to make libraries interoperable, and, furthermore even harder to make them upgradable. Our audit team at Dedaub regularly finds issues in smart contracts that employ “safe” 3rd party libraries. Our decompiler and contract analysis tools really help in such cases as they work on the actual deployed code of a smart contract. We regularly find issues related to upgradability, but other issues may be lurking.
We would like to commend the work of countless other security engineers who have helped reach out to affected projects!
