Project overview


✅ **Prerequisites**

- Basic knowledge about Ethereum.
- How to run Dune Queries.

If you are not familiar with the above, We've got you covered. 

Please take a quick detour to our introductory project: [Build Ethereum Dashboard](https://www.twigblock.com/projects/eth-intro-dune/t/eit-overview)

⏲️ **30 min**

📖 In this **Guided project**, you will

- Learn about <span style="color:#ed5a00;font-weight:bold;">transaction fees</span> and why they exist.
- Understand various <span style="color:#ed5a00;font-weight:bold;">fee structures</span> (first price auction and EIP-1559).
- Write <span style="color:#ed5a00;font-weight:bold;">SQL</span> to retrieve gas information from the Ethereum blockchain.
- Build <span style="color:#ed5a00;font-weight:bold;">charts</span> that answers questions about gas prices.
- Get ideas for your analysis.

Let's get started 🎬

&nbsp;

*This project wouldn't have been possible without the great work done by others. This is not an exhaustive list.*

- *[Gas and Fees](https://ethereum.org/en/developers/docs/gas/)*
- *[the daily gwei](https://thedailygwei.substack.com/)*
- *[Empirical analysis of EIP-1559](https://arxiv.org/pdf/2201.05574.pdf)*
- *[EIP-1559 compilations](https://hackmd.io/@timbeiko/1559-resources)*
- *Dune dashboards by [@hildobby](https://dune.com/hildobby/Gas) and [@msilb7](https://dune.com/msilb7/EIP1559-Base-Fee-x-Tip-by-Block)*


What you'll be building

Learn and Analyze Ethereum Gas Prices

Gas Basics


#### What is a Transaction?

Ethereum is a global computer that executes smart contract and stores its data (generally referred as contract state).

In addition to smart contract's state, it also stores account balances i.e ETH owned by accounts.

![Ethereum global state](/images/tasks/contract_state.png)Ethereum global state

Ethereum's global state is updated on the following three scenarios.

- Transfer of ETH between two accounts.
- Creation of a smart contract.
- Execution of a method in an existing smart contract.

Any changes to the Ethereum's global state are made through transactions. Transactions are the means by which external users interact with the system.

Every transaction gets executed in every node, and the resulting state gets saved in all the nodes. It involves CPU, memory, and network costs. Miners/Validators are in charge of setting up and maintaining nodes.

![Ethrereum Validator Expenses](/images/tasks/ethereum_validator_expense.png)Miner/Validator Expenses

#### Why would miners(validators) spend time and effort to execute every transaction and store the state?

The primary objective for miners(validators in Ethereum 2.0) is to earn money. They get incentives in the form of

- Block reward for every block produced.
- <span style="color:#ed5a00;font-weight:bold;">Transaction fees</span> paid by the users.
- Miner Extracted Value (MEV).

#### You may be wondering why, in addition to the other incentives, they require transaction fees.

Without a fee to include transactions in the block, lazy miners(validators) may choose to just propose empty blocks, thus saving their CPU and network costs.

Furthermore, if there are no transaction fees, malicious users may flood the network and disrupt other transactions. Since the resources are constrained, users must pay a fee based on the level of transaction complexity.

#### How are transaction fees computed?

The transaction cost is expressed in <span style="color:#ed5a00;font-weight:bold;">gas units</span>.

Each unit of gas represents a certain amount of computational work done by EVM (Ethereum Virtual Machine).

- An ETH transfer would cost 21,000 gas units in transaction fee.
- An NFT transfer or a more complex transaction would cost more.

#### How much is 21,000 gas units?

It depends on how much gas price the user is willing to pay per gas unit.

<span style="color:#ed5a00;font-weight:bold;">Gas prices</span> are expressed in gwei (equal to 0.000000001 ETH).

![Transaction Gas Fees](/images/tasks/eth_gas_price_compute.png)Transaction Gas Fees

In the above case, if the user is willing to pay 10 gwei per gas unit, they must spend 0.00021 ETH in transaction (gas) fees to have their transaction processed by the network.

Now that we understand how gas fees are calculated, let's dive into Dune Analytics to query the blockchain and calculate the total gas fees paid by users thus far.

When the user initiates a transaction, they input the gas price they are willing to pay. 
The EVM records the transaction as it is in the blockchain.
Once the transaction is executed, the EVM updates the outcome that includes actual gas units consumed in ```transaction receipt trie```.

![Dune Data Load](/images/tasks/eg_dune_analytics_load.png)Dune Data Load

Dune analytics extracts the transaction input and transaction outcome data and loads them into the ```ethereum.transactions``` table.


We can query the ```gas_used``` and ```gas_price``` columns in ```ethereum.transactions``` table to figure out the gas price paid by the users for every transaction. We can then add up the gas fees to get the total amount paid so far.


```sql
SELECT
  --Divided by 1e9 to convert gas_price to gwei and again by 1e9 to convert gwei to eth
  --Divided by 1e6 to report the results in millions
  SUM(gas_used * gas_price/1e18)/1e6 AS total_gas_fees_in_eth 
FROM
  ethereum.transactions tx
```

👨🏽‍💻 [Fork](https://dune.com/queries/1245188/2133607) the above query and give it a try.

<iframe src="https://dune.com/embeds/1245188/2133607/5184ba2a-3f5e-4c5c-b738-3ff5ea6da710" width="100%"></iframe>

&nbsp;

Users have paid ~7 million in ETH thus far (as of Sep'22) to transact on the Ethereum network 😮. 


Transaction Fees

Can you identify the most profitable month for miners in terms of transaction fees?

🏋️ Let's query the chain

What are transactions, and why do they have fees?


Before we go any further into gas fees, let's understand block gas limits and why they exist.

![Block space is limited](/images/tasks/block_full.png)Block space is limited

When a user initiates a transaction, it is not immediately processed by the Ethereum network. Instead, it is 
added to the node's <span style="color:#ed5a00;font-weight:bold;">mempool(waiting area)</span> and awaits processing until it is picked up.

Miners(validators) cannot process all the pending transactions into the next block. 

There is a limit on the transactions that can be included based on their complexity.

The <span style="color:#ed5a00;font-weight:bold;">block gas limit</span> is expressed in gas units. For example, if the block gas limit is 10M, it can include transactions that cumulatively consume 10M gas units.

#### Why is there a block gas limit?

For a moment, consider ticket fare gates in train stations. The job of the fare gate (during checkout) is to confirm that every passenger has a valid ticket with balance before letting them out. 

What if there was only one gate that can let out ~30 customers per second and trains kept coming every five seconds? 

The fare gate would never catch up because there would always be an excessive number of people waiting to be checked out.

It is not a hard problem to solve. Just install additional gates.

In case of the Ethereum, each node has to validate every transactions in the latest block, that are produced every ~12 seconds.

To allow for more transactions to be added to the block, the nodes must be powerful enough to complete the validation and begin producing new blocks within that time frame. This leads to centralization since only nodes with powerful hardware can now participate in the network. 

However, in order to <span style="color:#ed5a00;font-weight:bold;">achieve decentralization</span>, regular users with consumer grade hardware must be able to participate in the network. The greater the number of nodes, the greater the decentralization. Hence, the **block gas limit** exists.

![Block size impact](/images/tasks/eg_block_gas_limit.png)Block size impact

#### Let us query the blockchain to understand the block gas limit and the size of the latest blocks.

EVM records the block ```gas_limit``` and ```gas_used``` by all the included transactions in the block header. 

Dune Analytics extract and loads the block header information into ```ethereum.blocks``` table.

![Dune Data Load](/images/tasks/eg_dune_analytics_load_blocks.png)Dune Data Load


```sql
SELECT
  number as block_number,
  gas_used AS gas_used, -- gas used by all transactions in that block
  gas_limit - gas_used AS gas_remaining
FROM
  ethereum.blocks
ORDER by
  block_number DESC
LIMIT 10; --select the latest 10 blocks
```

👨🏽‍💻 [Fork](https://dune.com/queries/1228599/2109208) the above query and give it a go.

<iframe src="https://dune.com/embeds/1228599/2109208/e2c2561f-0151-494b-9858-447f33de1e68"  width="100%"  height="400px" title="Ethereum Block Capacity"></iframe>

&nbsp;

This is a live chart. If you see any full blocks, it is likely that some transactions did not make it and will have to wait for their turn in subsequent blocks.

So, if all pending transactions are unable to enter the next block, how are they selected?

What are your thoughts? 

Let's discuss it in the next section.

Block space is limited

Can you identify when the block gas limit was increased to 30 Million gas units?

Why Ethereum's blockspace is limited?

Fee Models


We know that the blockspace is limited.

Let us look at what happens when there are more transactions than the available block space.

![Block Allocation](/images/tasks/ethereum_gas_auction.png)Block Allocation

Since ETH transfer costs 21,000 gas units and there are only 25,000 gas units left in the block, only one can make it to the next block.

it is NOT decided on first come first serve basis.

Instead, it is up to the miners(validators) to pick up and order the transactions in the block. Since miners are motivated to maximize profits, they would prefer to pickup transactions that pays high gas price per unit.

In this case, it is more profitable for the miners(validators) to include the transaction #0xa1d3.

![Winner](/images/tasks/ethereum_gas_auction_winner.png)Winner

Essentially, this created a fee auction market wherein users would bid for space in the blocks and the miners(validators) would pick up the highest bidders. This created few challenges.

- Users must guess the network congestion.
- Users have no easy way to see what other users are bidding.

If they underestimate, then the transaction gets stuck in the mempool. If they overestimate, then they end up paying more.

As a result, most users pay needlessly more to process their transaction.

Consider the below example where two transactions in the same block paid different transaction fees.

![Gas Volatility](/images/tasks/ethereum_gas_volatility.png)Gas Volatility in the same block. [tx1 link](https://etherscan.io/tx/0x0a14eb58994ee1fd3ec410f45e84cb4c0cd98ee2672b57ff26f1ebedd31a7812) and [tx2 link](https://etherscan.io/tx/0x1b2c02c4d0c16928c898a1fcd6d41c47ca449816363c9025cdf472030060a5b4)

In the above example, the gas fees paid for the second transaction were ~60% higher than the first transaction, even though both of them were ETH transfers and processed in the same block.

The inefficiency of the first price auction system was a topic of [discussion](https://ethresear.ch/t/first-and-second-price-auctions-and-improved-transaction-fee-markets/2410/2).

What's the fix? EIP-1559

First Price Auction

Can you retrieve min and max gas price (in gwei) paid in the block# 12465199?

Ethereum Gas Fee Market


<span style="color:#ed5a00;font-weight:bold;">EIP-1559</span> is part of the London upgrade that happened on August 2021 at block# 12965000.

![Ethereum Gas Fee Structures](/images/tasks/eip-1559.png)Ethereum Gas Fee Structures

It changed the fee mechanism to reduce gas price volatility and transaction delays.

1. Prior to the London upgrade, the block size was fixed at 15 million gas units. This resulted in unnecessary transaction delays, particularly during the congestion period. One of the changes is to make the <span style="color:#ed5a00;font-weight:bold;">block size variable</span>. The target is 15M, but it can grow upto 30M gas units.

2. Instead of letting the user guess the network congestion, the Ethereum protocol computes the <span style="color:#ed5a00;font-weight:bold;">base fee</span> based on the network demand. The base fee is recalculated after every block depending on the size of the previous blocks. The base fee will continue to rise if the previous blocks are larger than the target, and it will decrease if they are smaller. The objective is to keep the average block size close to the target, which is 15M gas units. There is no guessing game with the base fee because most wallet software (e.g., Metamask) can look at previous blocks and then quote a base fee to users. Users can then decide whether to do a transaction or not.

3. The <span style="color:#ed5a00;font-weight:bold;">base fee is burnt</span> instead of sending it to miners(validators). If not, miners(validators) may add their own transactions into the block to keep the base fee going up. Now that they don't get the base fee, they are indifferent to the block capacity. Additionally, by burning the base fee, the ETH issuance rate is reduced. Simply put, when you pay for a transaction, you are paying it to all ETH holders.

4. Now that the miners(validators) don't get the base fee, they need an incentive to pickup your transactions. EIP-1559 introduced <span style="color:#ed5a00;font-weight:bold;">priority fee</span> that goes to the miners(validators) to disincentivize them from producing empty blocks. Users only need to guess priority fee to speeden up their transactions. This is usually a small amount.

![EIP-1559 seems complex](/images/tasks/eip_1559_confusing.png)

Fear not. We will break it down.

Assume you are doing an ETH transfer via a wallet. 

The wallet computes the base fee based on the size of previous blocks.

You enter priority fee(referred to as tip) that goes to the miners(validators) to priorize your transaction.

![Gas Input](/images/tasks/eth_wallet_gas.png)

You submit your transaction and nothing happens.

What if there is a sudden congestion (possibly an NFT launch) and the base fee has increased?

![EIP 1559 Congestion](/images/tasks/eip_1559_congestion.png)

Since the BASE FEE has increased, it has eaten up a portion of the priority fee.

Miners (validators) will now receive a 1 GWEI tip.

Your transaction remains valid, but it is less likely to be picked up due to the existing congestion, and miners(validators) prioritizing other transactions since your tip has been reduced to 1 GWEI.

What if the congestion persists and the base fee keeps on rising?

![Base Fee Increases](/images/tasks/eip_1559_basefee_increase.png)

Your transaction will not be picked up if the max fee is less than the base fee.

Assume after a while, the increase the in base fee caused more transactions to be excluded. As a result, the subsequent blocks are not full, lowering the base fee.

![Max fee refund](/images/tasks/eip_1559_refund.png)

Once the transaction is processed, any extra fee paid will be refunded to the users.

Generally, it is preferable to pay a high max fee so that the transactions don't end up waiting since any extra fees are refunded, unlike in a first price auction, where everything is paid to the miners(validators).

Overall, EIP 1559 promised an improved user experience. Let's dig into the data to validate the impact ourselves.



EIP-1559

Can you find the peak congestion time since London upgrade (and before Sep'22)?

EIP-1559 - A proposal to change fee market mechanism


Let us investigate the data to determine the impact of EIP 1559.

Before we get started, let's go over the key tables in Dune Analytics where the necessary information is loaded.

![First Price Auction Data](/images/tasks/dune_first_price_data.png)First Price Auction Data

![EIP 1559 Data](/images/tasks/dune_eip1559_data.png)EIP 1559 Data

#### How soon users adopted to using new fee model?

Though EIP-1559 promised to improve users' experience, are the users quick enough to leverage the benefit?

We can answer this question by identifying month wise transaction volume by fee type.

```sql
SELECT
  DATE_TRUNC('month',block_time) AS month,
  -- For every month, Compute Number of eip 1559 transactions / Total transactions
  COUNT(*) FILTER ( WHERE `type` = 'DynamicFee') / COUNT(*) AS eip1559_tx,
  COUNT(*) FILTER ( WHERE `type` = 'Legacy' ) / COUNT(*) AS legacy_tx,
  COUNT(*) FILTER ( WHERE `type` NOT IN ('DynamicFee', 'Legacy') ) / COUNT(*) AS other_tx
FROM
  ethereum.transactions
WHERE
  block_number >= 12965000 --London upgrade block number
GROUP BY
  month
```
👨🏽‍💻 [Fork](https://dune.com/queries/1241812/2146406) the above query and give it a try.

<iframe src="https://dune.com/embeds/1241812/2146406/e5b9cae0-4ef8-4270-9267-2386eef3a10e"  width="100%"  height="300px" margin-bottom="100px"></iframe>

&nbsp;

It is quite surprising to see that ~18% of transactions (as of Sep'22) still use the first price auction (legacy model) and end up paying more gas fees.

What could the reasons be?

- Some users may not have updated their wallets.
- Perhaps there are automated scripts that have not been updated.
- Exchanges (who are also miners) may not adopt fee standards in order to continue extracting fees.


EIP-1559 Adoption


#### How the transaction fees split changed after EIP-1559?

With the introduction of EIP 1559, the base fee is burnt. Prior to that, all transaction fees were paid to the miner(validator). 

We can figure out the split by querying the fees burnt (```base_fee * gas_used```) and miner fees (```priority_fee * gas_used```) for each transaction.

```sql
SELECT
  DATE_TRUNC('month', tx.block_time) AS month,
   --before london upgrade, all gas fees went to the miner
   --after london upgrade, only the priority fees goes to the miner
  SUM(
    CASE
      WHEN tx.block_number < 12965000 THEN tx.gas_used * tx.gas_price/1e18
      ELSE tx.gas_used * tx.priority_fee_per_gas/1e18
    END
  ) as fees_miner_eth,
  SUM(tx.gas_used * blk.base_fee_per_gas/1e18) AS fees_burnt_eth
FROM
  ethereum.transactions tx
  INNER JOIN ethereum.blocks blk ON tx.block_number = blk.number
WHERE
  tx.block_time >= '2021-01-01'
GROUP BY
  month;
```
👨🏽‍💻 [Fork](https://dune.com/queries/1243169/2130648) the above query and give it a try.

<iframe src="https://dune.com/embeds/1243169/2130648/9b585dbe-01a5-495b-be13-d51084be55e8"  width="100%"  height="300px"></iframe>

&nbsp;

We can see that a significant amount of ETH is burned following the London upgrade. This is commonly referred to as the first step in ETH becoming ultrasound money. 

ETH is continuously issued to miners/validators as block rewards to secure the network. However, by burning the base fee, the issuance rate is reduced. 

When the amount of ETH burned exceeds the amount of ETH issued, ETH becomes deflationary. 
Hence the meme term ultrasound money.


EIP-1559 Fee Split

EIP-1559 - Transaction Fee Split


#### Has EIP 1559 changed gas price?

We can find out the median gas price paid each month by querying the ```gas_price``` for every transaction and grouping it by month.

```sql
SELECT
  DATE_TRUNC('month', block_time) as month,
  --Divided by 1e9 to convert it to GWEI
  PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY gas_price)/1e9 as median_gas_price 
FROM
  ethereum.transactions
WHERE
  DATE(block_time)>= '2021-01-01'
GROUP BY
  month
```
👨🏽‍💻 [Fork](https://dune.com/queries/1252684/2146312) the above query and give it a try.

<iframe src="https://dune.com/embeds/1252684/2146312/e9351a22-5795-456a-be34-54eaa8914454"  width="100%"  height="300px"></iframe>

&nbsp;

There is no evidence to show that EIP 1559 has reduced the gas price. The gas price is influenced by the network demand.
The primary objective of EIP-1559 is not to make gas cheaper but to reduce gas volatility. Let's measure that in the next section.

EIP-1559 Gas Price Impact

EIP-1559 - Gas Price Impact


#### How do we measure gas volatility?

The difference between the gas price paid (per gas unit) by two different transactions in the same block should not be very high.

But, which transactions to pick?

We can use a measure called inter quartile range. By ordering the transactions in the block by gas price, we can pick the 25th percentile and 75th percentile and take their difference.

![Intra Block Variance](/images/tasks/iqr.png)

In the above example, we can see that IQR for block#2 is less than block #1. The less the variation in gas prices in the same block, the smaller the IQR will be.

```sql
--Compute the IQR for every block since 2021
WITH blocks_iqr_gas AS (
    SELECT 
    block_number,
    ((PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY gas_price)) - (PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY gas_price))) / PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY gas_price) AS iqr_gas 
    FROM ethereum.transactions 
    WHERE DATE(block_time)>= '2021-01-01'
    GROUP BY block_number
)
--Compute the median of IQR for every month
SELECT DATE_TRUNC('month', time) AS month,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY iqr_gas) AS median_gas_volatility
FROM blocks_iqr_gas blk_iqr
INNER JOIN ethereum.blocks blk
ON blk_iqr.block_number = blk.number
GROUP BY month;
```
👨🏽‍💻 [Fork](https://dune.com/queries/1260344/2160219) the above query and give it a try.

<iframe src="https://dune.com/embeds/1260344/2160219/efb22197-9ab2-4388-8cf8-9def9eb8e325"  width="100%"  height="300px"></iframe>

&nbsp;

As we can see, since the implementation of EIP 1559, the IQR has significantly decreased. This suggests that EIP 1559 did, in fact, simplify gas estimation.

EIP-1559 Gas Volatility

EIP-1559 - Gas Volatility


So far, we've assumed that the user is aware of the number of gas units required by the transaction.

This is true for ETH transfers, which always consume 21,000 gas units.

This is not the case for other transactions involving smart contract execution.
Since smart contracts are lines of code, they can take different code paths during execution. It will not be known until the transaction is completed.

As a result, one can't determine upfront how many gas units they are going to consume.

#### So, how much should a user set?

Users can set a gas limit, i.e., the maximum number of gas units they are willing to spend.

If any gas units remain after execution, they will be refunded to the user.

![Gas refund](/images/tasks/gas_limit_refund.png)

#### What happens if the gas units provided is insufficient?

The transaction will revert, but it will be recorded on the blockchain with a failed status. However, the miner(validator) has already done the required work to execute the transaction until no more gas is available. As a result, the consumed gas will not be refunded.

![Out of Gas](/images/tasks/out_of_gas.png)

#### Let us find out the last 10 transactions that failed due to out of gas.

EVM records only the status in ```Transaction Receipt Trie```. To figure out the failure reason, We have to look at the run time data which is captured in ```traces```.

![Dune Data Load Traces](/images/tasks/dune_data_load_traces.png)

```sql
SELECT
  block_number,
  tx_hash,
  error,
  'https://etherscan.io/tx/' || tx_hash AS etherscan_tx_link,
  'https://etherscan.io/vmtrace?txhash=' || tx_hash || '&type=gethtrace2' AS etherscan_trace_link
FROM
  ethereum.traces
WHERE
  error = "Out of gas"
AND block_time > NOW() - INTERVAL '1 hour'
ORDER BY block_time DESC
LIMIT 10;
```

👨🏽‍💻 [Fork](https://dune.com/queries/1243169/2130648) the above query and give it a try.

<iframe src="https://dune.com/embeds/1268282/2173065/b62dd0f9-dea0-4079-b26c-a66e1a9d2119"  width="100%"  height="300px" margin-bottom="100px"></iframe>

&nbsp;

Navigate to the above etherscan trace link and try to understand the trace structures. We will dissect them in our upcoming project.

How cool would it be to create an AI system that alerts the user in advance that the transaction will fail?

You may be wondering if people really pay that much as gas fees?

Let's find it out.

Transaction Gas Limits

What was the maximum gas price paid in the month of Aug'22 for a (out-of-gas) failed transaction?

Publish your story


Yeah! That's a lot to take in. Hopefully, you are now familiar with these terms.

![Ethereum Gas Terms](/images/tasks/gas_topics_review.png)Ethereum Gas Terms

Here is a list of the charts that were used in this project.

- [Total Gas Fees Paid](https://dune.com/queries/1245188/2133607)
- [Block occupancy for the recent blocks](https://dune.com/queries/1228599/2109208)
- [EIP 1559 Adoption](https://dune.com/queries/1241812/2146406)
- [EIP 1559 Fee Split](https://dune.com/queries/1243169/2130648)
- [EIP 1559 Median Gas Price](https://dune.com/queries/1252684/2146312)
- [EIP 1559 Gas Volatility](https://dune.com/queries/1260344/2160219)
- [Ethereum Gas Dashboard](https://dune.com/kikura/ethereum-gas)

Please feel free to fork and experiment.

Here are some inspirations for you to explore further.

- When are the gas prices low? 
- What is the total amount of ETH burned so far?
- How much ETH has been saved on gas fees since the introduction of EIP-1559?
- How much have the users lost so far on out-of-gas failed transactions?
- Mistake or something else? Find suspicious transactions with unusually high gas fees as compared to the actual transaction value.

Build your own narrative based on the above charts/questions.

#### What comes next?

- We have looked at Ethereum Gas topics.
- Next, let us explore the foundational topics on <span style="color:#ed5a00">Smart Contracts, Events, Token Standards, Layer 2 solutions</span>.
- Following that, we will delve deeply into sectors (such as <span style="color:#ed5a00">DeFi, NFT etc</span>).
- Finally, we will pick a specific protocol (<span style="color:#ed5a00">Uniswap, OpenSea, etc </span>) and conduct extensive research on it.
- And then, You will be ready to start working on <span style="color:#ed5a00">web3 data analytics</span>. 🎉

&nbsp;

We are currently in beta. Please DM us on [twitter](https://twitter.com/Kirubakumaresh) for any issues or questions.