Calculating DEX Liquidity Pool Fees and Volumes

Decentralized exchanges are playing an increasingly important role in supporting cryptoasset trading, particularly when it comes to tokens using the ERC-20 standard. Coin Metrics has been actively working on collecting data from major DeFi protocols. For our first release, we have added support for all major liquidity pools on Uniswap v2, Uniswap v3, and Sushiswap v1.

Resources

This notebook demonstrates basic functionality offered by the Coin Metrics Python API Client and DEX Market Data.

Coin Metrics offers a vast assortment of data for hundreds of cryptoassets. The Python API Client allows for easy access to this data using Python without needing to create your own wrappers using requests and other such libraries.

To understand the data that Coin Metrics offers, feel free to peruse the resources below.

The Coin Metrics API v4 website contains the full set of endpoints and data offered by Coin Metrics.
The Coin Metrics Product Documentation gives detailed, conceptual explanations of the data that Coin Metrics offers.
The API Spec contains a full list of functions.

File Download

Download the entire notebook as either a jupyter notebook to run yourself or as a pdf from the two links below

Setup

from os import environ
import sys
import pandas as pd
import numpy as np
import seaborn as sns
import logging
from datetime import date, datetime, timedelta
from coinmetrics.api_client import CoinMetricsClient
import json
import logging
from pytz import timezone as timezone_conv
from datetime import timezone as timezone_info
import matplotlib.dates as mdates
from IPython.display import Markdown as md
import matplotlib.pyplot as plt
import plotly.express as px 
%matplotlib inline

sns.set_theme()
sns.set_style('whitegrid')
sns.set(rc={'figure.figsize':(16,8)})
sns.set_context("notebook", font_scale=1.2, rc={"font.family": "Lato"});

logging.basicConfig(
    format='%(asctime)s %(levelname)-8s %(message)s',
    level=logging.INFO,
    datefmt='%Y-%m-%d %H:%M:%S'
)

# We recommend privately storing your API key in your local environment.
try:
    api_key = environ["CM_API_KEY"]
    logging.info("Using API key found in environment")
except KeyError:
    api_key = ""
    logging.info("API key not found. Using community client")
    
client = CoinMetricsClient(api_key)

2024-10-09 11:26:54 INFO     Using API key found in environment

DEX Market Catalog

The catalog/markets endpoint returns a list of available markets along with time ranges of available data. Users can pass in a list of markets, exchanges, or market types (spot, futures, options).

We can retrieve our DEX markets by fetching a list of all 'spot' markets, then filtering for the markets where the 'experimental' parameter equals true.

spot_markets = pd.concat([client.reference_data_markets(
    type = 'spot',
    page_size = 10000,
    exchange = "uniswap_v3_eth",
).to_dataframe(), client.reference_data_markets(
    type = 'spot',
    page_size = 10000,
    exchange = "uniswap_v2_eth",
).to_dataframe(), client.reference_data_markets(
    type = 'spot',
    page_size = 10000,
    exchange = "sushiswap_v1_eth",
).to_dataframe()])

exp_markets = spot_markets.loc[spot_markets['experimental']==True]

exp_markets

2544 rows × 39 columns

# Calculate number of markets (pools) per DEX
mkt_counts = pd.DataFrame(exp_markets['exchange'].value_counts()).reset_index()

print (mkt_counts)

           exchange  count
0    uniswap_v3_eth   1643
1    uniswap_v2_eth    676
2  sushiswap_v1_eth    225

# Plot number of markets (pools) per DEX
ax1 = plt.subplot()
# mkt_counts['exchange'].plot(kind='bar', width=0.8, color=sns.color_palette('tab10'))
ax1.bar(x = mkt_counts['exchange'], height = mkt_counts['count'], color=sns.color_palette('tab10'))
plt.setp(ax1.get_xticklabels(), rotation=0)
ax1.set_facecolor("white")
plt.grid(color='black', linestyle='--', linewidth=0.2)
plt.title('\nCount of Markets \nby Decentralized Exchange\n', fontdict={'fontsize': 24, 'font': 'arial'})
# for i, v in enumerate(mkt_counts['exchange']):
ax1.set_xticks(range(len(mkt_counts['exchange'])))
ax1.set_xticklabels([label.get_text().upper() for label in ax1.get_xticklabels()], rotation=0, ha='center', va='top', fontsize=17)
plt.xticks(fontweight='bold')
plt.show()

# Show fields
pd.DataFrame(exp_markets.loc[exp_markets['exchange'] == 'uniswap_v3_eth'].iloc[0]).dropna()

DEX markets include metadata for 3 different smart contracts:

contract_address: The address of the liquidity pool contract. Each liquidity pool is a unique instance of a smart contract, deployed at a dedicated address. The pool contract holds both the base and the quote asset.
base_address: The address of the ERC-20 token contract associated with the base asset.
quote_address: The address of the ERC-20 token contract associated with the quote asset.

# DEX fields
exp_markets[['pool_config_id','contract_address','base_address','quote_address','fee']]

2544 rows × 5 columns

Each liquidity pool is also associated with a corresponding fee percentage. With each trade, fees are distributed pro-rata to the pool's liquidity providers.

# Range of pool fees
pool_fees = pd.DataFrame(exp_markets.fee.unique().dropna())
pool_fees

Due to the permissionless nature of provisioning a liquidity pool, users can easily add arbitrary ERC-20 tokens to the decentralized exchange. There are over 50,000 trading pairs available on Uniswap today. To prioritize only the most relevant and liquid markets, we cover the subset of markets where both tokens are part of Coin Metrics reference rate coverage.

# Assets covered
asset_coverage = np.unique(exp_markets[['base', 'quote']].values)
print(asset_coverage)
print('\nTotal number of assets: \033[1m' + str(len(asset_coverage)) + '\033[0m\n')

['1inch' 'aave' 'ageur_eth' 'alcx' 'alpha' 'alusd' 'ampl_eth' 'ant' 'ape'
 'api3' 'audio' 'ausd_eth' 'axs_1_eth' 'axs_2_eth' 'badger' 'bal'
 'band_eth' 'bat' 'bit' 'bnb_eth' 'bnt' 'boost_eth' 'btm_eth' 'busd'
 'cbat' 'cbeth' 'ccomp' 'cdai' 'cel' 'cennz' 'ceth' 'chz_eth' 'comp' 'cro'
 'crv' 'crvusd_eth' 'cuni' 'cusdc' 'cusdcv3' 'cusdt' 'cvc' 'cvx' 'cwbtc'
 'czrx' 'dai' 'dar' 'degen_eth' 'deusd_eth' 'dgx' 'dorkl_eth' 'dpi' 'drgn'
 'eeth_eth' 'elf' 'eng' 'enj' 'ens' 'esd' 'ethos' 'ethx_eth' 'eul'
 'eurc_eth' 'eurcv_eth' 'eure_eth' 'eurs_eth' 'eurt_eth' 'fdusd_eth'
 'fei_eth' 'flx' 'fox_eth' 'frax_eth' 'frxeth_eth' 'ftm_eth' 'ftt' 'fun'
 'fwb' 'fxc_eth' 'fxs' 'gala' 'gbpt_eth' 'gho_eth' 'glm' 'gno' 'grt'
 'gt_eth' 'gusd' 'gyen_eth' 'hbot' 'hbtc' 'hedg' 'hex_eth' 'hpos10i_eth'
 'hpos1m_eth' 'ht' 'husd' 'idrt_eth' 'imx' 'inst' 'kin1' 'knc' 'ldo'
 'lend' 'leo_eth' 'link' 'linq_eth' 'looks' 'loom' 'lpt' 'ltx_eth'
 'lusd_eth' 'mana' 'matic_eth' 'mco' 'meth_eth' 'mim_eth' 'mkr_2_eth'
 'mnt' 'mpl' 'mtl_metal' 'myc' 'myth' 'nexo' 'nftx' 'nmr' 'ogn'
 'ohm_2_eth' 'okb' 'omg' 'ousd_eth' 'paid' 'parrot_eth' 'pax' 'paxg' 'pay'
 'pdt' 'pepe_eth' 'perp' 'pndc_eth' 'pog_eth' 'pol_eth' 'poly' 'pow_eth'
 'powr' 'ppt' 'pufeth_eth' 'pvpbot_eth' 'pxeth_eth' 'pyusd_eth' 'qash'
 'qnt' 'rad' 'radar' 'rai_eth' 'ren' 'renbtc' 'renfil_eth' 'rep_3_eth'
 'reth_eth' 'rev_eth' 'rez_eth' 'rlb_eth' 'rook' 'rseth_eth' 'rsr'
 'rsweth_eth' 'safe_eth' 'sai' 'salt' 'sand' 'sbtc_eth' 'sdai_eth'
 'seth_synthseth_eth' 'shepe_eth' 'shia_eth' 'shib' 'skl' 'slp_1_eth'
 'slp_2_eth' 'snt' 'snx_3_eth' 'socks' 'spell' 'srm' 'srn' 'steth_lido'
 'stg_eth' 'stkaave' 'stmx' 'stone_eth' 'storj' 'susd_eth' 'susde_eth'
 'sushi' 'sweth_eth' 'swise' 'swrv' 't_eth' 'taud_eth' 'tbtc_eth'
 'tcad_eth' 'tgbp_eth' 'thkd_eth' 'toke' 'trx_eth' 'tusd_2_eth' 'ubt'
 'uma' 'uni' 'univ2aaveweth_eth' 'univ2crvweth_eth' 'univ2daiweth_eth'
 'univ2snxweth_eth' 'univ2uniweth_eth' 'univ2wbtcweth_eth'
 'univ2yfiweth_eth' 'uqc_1_eth' 'uqc_2_eth' 'usd0_eth' 'usda_eth' 'usdc'
 'usdd_eth' 'usde_eth' 'usdk' 'usdn_eth' 'usdt_eth' 'usdv_eth' 'usdy_eth'
 'ustc.t_eth_wh' 'veri' 'wbtc' 'wcelo' 'wcusd' 'weeth_eth' 'weth' 'wluna'
 'wnxm' 'wsteth' 'wust' 'wzec' 'xai_silofinance_eth' 'xaut_1_eth'
 'xaut_2_eth' 'xchf' 'xidr_eth' 'xsgd_eth' 'xsushi' 'yama_eth' 'yfi' 'zrx']

Total number of assets: [1m249[0m

The majority of DEX assets trade against Wrapped Ether (WETH). Note that Ethereum's native asset ETH is not supported in Uniswap V2/V3, as the asset was created prior to the ERC-20 standard. From the Uniswap documentation:

"Unlike Uniswap V1 pools, V2 pairs do not support ETH directly, so ETH⇄ERC-20 pairs must be emulated with WETH. The motivation behind this choice was to remove ETH-specific code in the core, resulting in a leaner codebase. End users can be kept fully ignorant of this implementation detail, however, by simply wrapping/unwrapping ETH in the periphery." docs.uniswap.org

# Enter an asset ticker to see available 'base' and 'quote' markets
asset = 'weth'

selected_markets = exp_markets.loc[(exp_markets['base']==asset) | (exp_markets['quote']==asset)]
print('\nTotal number of ' + '\033[1m' + asset + '\033[0m' + ' markets: \n\033[1m' 
      + str(len(selected_markets)) + '\n')

Total number of [1mweth[0m markets: 
[1m847

Using the min_time parameter, we can filter for the newest DEX liquidity pool deployed in our coverage

defi_market_catalogs = pd.concat([client.catalog_market_trades_v2(
    market_type = 'spot',
    page_size = 10000,
    exchange = "uniswap_v3_eth",
).to_dataframe(), client.catalog_market_trades_v2(
    market_type = 'spot',
    page_size = 10000,
    exchange = "uniswap_v2_eth",
).to_dataframe(), client.catalog_market_trades_v2(
    market_type = 'spot',
    page_size = 10000,
    exchange = "sushiswap_v1_eth",
).to_dataframe()]).sort_values('min_time').reset_index()

print (defi_market_catalogs)

      index                                      market  \
0       560               uniswap_v2_eth-usdc-weth-spot   
1       154                uniswap_v2_eth-dai-weth-spot   
2       152                uniswap_v2_eth-dai-usdc-spot   
3       387                uniswap_v2_eth-ren-usdc-spot   
4        85               uniswap_v2_eth-cdai-weth-spot   
...     ...                                         ...   
2012    207         uniswap_v3_eth-2-paxg-usdt_eth-spot   
2013    208       uniswap_v3_eth-2-paxg-xaut_2_eth-spot   
2014    302  uniswap_v3_eth-2-weeth_eth-pufeth_eth-spot   
2015     24    uniswap_v3_eth-1-deusd_eth-usdt_eth-spot   
2016    656   uniswap_v3_eth-3-usdt_eth-crvusd_eth-spot   

                      min_time                  max_time  
0    2020-05-05 21:09:32+00:00 2024-10-03 19:44:11+00:00  
1    2020-05-14 00:01:38+00:00 2024-10-03 19:22:47+00:00  
2    2020-05-15 02:18:16+00:00 2024-10-03 12:06:35+00:00  
3    2020-05-17 21:32:57+00:00 2024-09-03 22:31:59+00:00  
4    2020-05-18 17:18:03+00:00 2024-10-01 22:43:11+00:00  
...                        ...                       ...  
2012 2024-09-16 01:14:11+00:00 2024-09-17 01:00:23+00:00  
2013 2024-09-17 01:56:23+00:00 2024-09-28 12:37:23+00:00  
2014 2024-09-26 04:21:35+00:00 2024-10-03 09:49:59+00:00  
2015 2024-09-27 23:06:35+00:00 2024-10-03 10:26:47+00:00  
2016 2024-09-28 07:12:59+00:00 2024-10-03 18:50:35+00:00  

[2017 rows x 4 columns]

# Check for the newest market
newest = defi_market_catalogs.loc[defi_market_catalogs['min_time'].idxmax()].dropna()
pd.DataFrame(newest)

DEX Swaps Data

Swaps data is served through our existing /timeseries/market-trades endpoint because swaps are conceptually identical to a trade. Users can see all the standard trade fields for a swap such as time, price, and volume but can also see defi-specific fields such as the block height, block hash, transaction id, addresses involved in the swap, and more.

defi_market = 'uniswap_v3_eth-2-weth-usdt_eth-spot'
start = datetime.now() - timedelta(days=3)
end = datetime.now() - timedelta(hours=1)

defi_trades = client.get_market_trades(
    markets=defi_market,
    start_time=start,
    end_time = end
).to_dataframe()

defi_trades['amount'] = defi_trades['amount'].astype(float)
defi_trades.head()

color_map = {'buy': 'green', 'sell': 'red'}
defi_price = plt.scatter(x=defi_trades['time'], y=defi_trades['price'], s=defi_trades['amount'], c=defi_trades['side'].map(color_map))

plt.xlabel("", fontsize=15)
plt.ylabel("Price (USDT)\n", font='arial',fontsize=15)

market_string = defi_market.split("uniswap_v3_eth-", 1)[-1].split("-spot")[0].upper()
plt.title('\nUniswap V3\n Pool ' + market_string + '\nTrades\n', font='arial', size=20)

legend_labels = ['BUY', 'SELL']
legend_handles = [plt.Line2D([0], [0], marker='o', color='w', markerfacecolor=color_map['buy'], markersize=12),
                  plt.Line2D([0], [0], marker='o', color='w', markerfacecolor=color_map['sell'], markersize=12)]
legend = plt.legend(legend_handles, legend_labels, loc='lower right', fontsize=16, framealpha=0, bbox_to_anchor=(0.99, 1.02))

plt.gca().set_facecolor('white')
plt.grid(color='black', linestyle='dotted')
plt.show()

pd.DataFrame(defi_trades.iloc[0])

Unlike centralized exchanges, where there is an unknown buyer and a seller, each swap is associated with 3 different Ethereum addresses:

Initiator is the ethereum address which submitted the transaction as a result of which the swap/liquidity action occurred
Sender is the ethereum address that invoked the uniswap pool smart contract's function for swapping or adding/removing liquidity
Beneficiary is the ethereum address that got credited with the output tokens (in case of a swap or liquidity removal) or with liquidity (in case of liquidity addition)

print('\nTotal number of ' + '\033[1m' + defi_market + '\033[0m' + ' trades (' + str(start.date()) + ' to ' + str(end.date()) + '): \n\033[1m' 
      + str(len(defi_trades)) + '\033[0m\n')

print('\nTotal number of ' + '\033[1m' + defi_market + '\033[0m' + ' buys (' + str(start.date()) + ' to ' + str(end.date())+ '): \n\033[1m' 
      + str(len(defi_trades.loc[(defi_trades['side']=='buy')])) + '\033[0m\n')

print('\nTotal number of ' + '\033[1m' + defi_market + '\033[0m' + ' unique buyers (' + str(start.date()) + ' to ' + str(end.date()) + '): \n\033[1m' 
      + str(len((defi_trades.loc[(defi_trades['side']=='buy')]).beneficiary.unique())) + '\n')

Total number of [1muniswap_v3_eth-2-weth-usdt_eth-spot[0m trades (2024-10-06 to 2024-10-09): 
[1m5874[0m


Total number of [1muniswap_v3_eth-2-weth-usdt_eth-spot[0m buys (2024-10-06 to 2024-10-09): 
[1m3003[0m


Total number of [1muniswap_v3_eth-2-weth-usdt_eth-spot[0m unique buyers (2024-10-06 to 2024-10-09): 
[1m351

defi_buyers = pd.DataFrame(defi_trades)
defi_buyers['time'] = pd.to_datetime(defi_buyers['time'])
defi_buyers.set_index('time', inplace=True)

hourly_unique_buyers = defi_buyers['beneficiary'].resample('h').nunique()
sns.set_style('whitegrid')
fig, ax = plt.subplots()
sns.lineplot(data=hourly_unique_buyers, ax=ax)
ax.set_xlabel('')
ax.set_ylabel('Unique Buyers per Hour')
ax.set_title('\nPool ' + str(market_string) + '\nUnique Buyers per Hour\n', font='arial', size=20)
plt.grid(color='black', linestyle='dotted')
plt.show()

# Unique buyer addresses
((defi_trades.loc[(defi_trades['side']=='buy')]).beneficiary.unique())

<StringArray>
['1111111254eeb25477b68fb85ed929f73a960582',
 '5ced44f03ff443bbe14d8ea23bc24425fb89e3ed',
 '2bbf5243e49e8f0025988754dac6620ab0d3e162',
 '308c6fbd6a14881af333649f17f2fde9cd75e2a6',
 'def1c0ded9bec7f1a1670819833240f027b25eff',
 '313d26bf14c4362883cc798d4aa50352fc55c954',
 '3fc91a3afd70395cd496c647d5a6cc9d4b2b7fad',
 '111111125421ca6dc452d289314280a0f8842a65',
 '2580916979353d3a40c71be00b0e197a98694cdf',
 'e37e799d5077682fa0a244d46e5649f71457bd09',
 ...
 '898fcb7b4e3bee37ebb0ca3a3fbd08cefdc8c995',
 '8ef79d6c328c25da633559c20c75f638a4863462',
 '309eac50cedc3909ce0f03c9725bb7a684b75829',
 '98c640aac6b88f5e1ab14073dd2d5baf71461e9c',
 '0f388ecdbc128083bfe317b00de2c25fbe9f24d5',
 '8583c58e3b0e02d2cdf45b06e437674f2fb47eb6',
 '6905fef2f9a7825f94018f4f989dc82f88be9181',
 '0b2b99d9fa762060fdbf1657984f7ece56ca5e5c',
 'f2b6525a3d7f28113cc57cb9b73dde9230073059',
 '051ef36e55875c08e4efaea6072c6d0f66fb0b9f']
Length: 351, dtype: string

# Calculate approximate USD volume (NOTE: quote asset must be a stablecoin)
defi_trades['DEX Volume (USD)'] = (defi_trades['amount'])*(defi_trades['price'])

# Use only the amount field if base asset is a stablecoin
#defi_trades['DeFi Volume (USD)'] = (defi_trades['amount'])

# Largest trade
largest = defi_trades.loc[defi_trades['amount'].idxmax()]
trade_size_usd = '${:,.2f}'.format(largest['DEX Volume (USD)'])
pd.DataFrame(largest)

print('\nLARGEST TRADE: \n\n' + trade_size_usd + ' by address 0x' + largest['beneficiary'] + '\n')

LARGEST TRADE: 

$2,712,158.45 by address 0xddc25c965443834179438c2eb30e70f34b854a04

# ATLAS links for blockchain metadata
md('<br>**ATLAS** by Coin Metrics <br> **Blockchain Search Engine** <br><br>**BUYER ADDRESS:** https://atlas.coinmetrics.io/address-details?asset=weth&address=' + str(largest['beneficiary']) + '<br><br>**TRANSACTION INFO:** https://atlas.coinmetrics.io/transaction-details?asset=weth&tx_hash=' + str(largest['txid']) + '<br><br>**BlOCK INFO:** https://atlas.coinmetrics.io/block-details?asset=weth&block_hash=' + str(largest['block_hash']) + '<br><br>For more info on **ATLAS** visit: https://coinmetrics.io/atlas/')

ATLAS by Coin Metrics Blockchain Search Engine BUYER ADDRESS: https://atlas.coinmetrics.io/address-details?asset=weth&address=ddc25c965443834179438c2eb30e70f34b854a04 TRANSACTION INFO: https://atlas.coinmetrics.io/transaction-details?asset=weth&tx_hash=cb93842f093812b85469b576914d02907345d2624e1eee463999bade940a8851 BlOCK INFO: https://atlas.coinmetrics.io/block-details?asset=weth&block_hash=73155bd5ea974746d59ed3b05d93f24d2011f391b527b9d7d5c42e59752c6d41 For more info on ATLAS visit: https://coinmetrics.io/atlas/

defi_trades['Datetime'] = pd.to_datetime(defi_trades.time.dt.strftime('%m/%d/%y %H:00'))
defi_vol = defi_trades.groupby('Datetime')['DEX Volume (USD)'].sum()

pd.DataFrame(defi_vol)

72 rows × 1 columns

Retrieving volume for a centralized exchange via the market-candles endpoint

cex_market = 'coinbase-eth-usd-spot'

cex_vol = client.get_market_candles(
    markets=cex_market,
    frequency='1h',
    start_time=start,
    end_time=end- timedelta(hours=1)
).to_dataframe()

cex_vol = cex_vol.rename(columns={"candle_usd_volume": "CEX Volume (USD)"})
cex_vol['Datetime'] = pd.to_datetime(cex_vol.time.dt.strftime('%m/%d/%y %H:00'))
cex_vol = cex_vol.groupby('Datetime')['CEX Volume (USD)'].sum()

vol_comp = pd.merge(defi_vol, cex_vol,on="Datetime", how="left").dropna()
vol_comp["DEX Volume (USD)"] = vol_comp['DEX Volume (USD)'].astype(int)
vol_comp["CEX Volume (USD)"] = vol_comp['CEX Volume (USD)'].astype(int)

vol_comp

70 rows × 2 columns

sns.set_style('whitegrid')
vc = sns.lineplot(data=vol_comp)
vc.yaxis.set_major_formatter('${x:,.0f}')
vc.set_title(str('\n' + defi_market.upper() + '\n vs. \n' + cex_market.upper() + '\n Hourly Volume \n'),font='arial',fontsize=17.5)
vc.set_ylabel("Volume (USD) \n", font='arial',fontsize = 14)
plt.grid(color='black', linestyle='dotted')
legend = vc.legend(fontsize=15, framealpha=0, bbox_to_anchor=(1.0, 1.2))
vc.set_xlabel("");

PreviousCalculating Total Value Locked in Liquidity Pools using DEX Data NextAnalyzing DeFi Protocol Balance Sheets

Last updated 26 days ago