<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Kevin Gakio's Engineering Notes]]></title><description><![CDATA[Notes from a backend engineer learning how databases and backend systems work underneath]]></description><link>https://kevingakio.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Tue, 27 Oct 2026 14:08:51 GMT</lastBuildDate><atom:link href="https://kevingakio.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[One index, 64× faster: what a composite index actually does]]></title><description><![CDATA[A query summing one customer's payments took 1,086 ms. After adding one index, the same query took 17 ms. The query didn't change. What changed was how…]]></description><link>https://kevingakio.hashnode.dev/one-index-64-faster-what-a-composite-index-actually-does</link><guid isPermaLink="true">https://kevingakio.hashnode.dev/one-index-64-faster-what-a-composite-index-actually-does</guid><category><![CDATA[database, performance, sql, backend]]></category><category><![CDATA[Indexing strategy]]></category><category><![CDATA[Performance Optimization]]></category><dc:creator><![CDATA[Kevin gakio]]></dc:creator><pubDate>Mon, 12 Oct 2026 18:07:14 GMT</pubDate><content:encoded><![CDATA[<p>A query summing one customer's payments took <strong>1,086 ms</strong>. After adding one index, the same query took <strong>17 ms</strong>.</p>
<p>The query didn't change. What changed was how MySQL could find the rows. Here's what happened inside the database, with the real execution plans.</p>
<h2>The experiment</h2>
<p>I built a test table in MySQL 8 with random data, shaped like a real payments system:</p>
<ul>
<li><p><strong>2,020,000</strong> payments across <strong>100,000</strong> customers</p>
</li>
<li><p>one fund holding <strong>95%</strong> of all rows (most systems have one very popular product)</p>
</li>
<li><p>one big customer, <code>777</code>, with <strong>20,028</strong> payments</p>
</li>
</ul>
<pre><code class="language-sql">CREATE TABLE payments (
    id          BIGINT AUTO_INCREMENT PRIMARY KEY,
    customer_id INT NOT NULL,
    fund_id     INT NOT NULL,
    paid_on     DATE NOT NULL,
    amount      DECIMAL(12,2) NOT NULL,
    INDEX idx_customer (customer_id),
    INDEX idx_fund (fund_id)
);
</code></pre>
<p>The full data-generation script is here: <a href="https://gist.github.com/gakiokevin/39e11599a0ce989ad7e33cc66656766c">composite-index-demo.sql</a></p>
<p>The query is a classic balance lookup: one customer, one fund, everything up to today.</p>
<pre><code class="language-sql">SELECT SUM(amount) FROM payments
WHERE customer_id = 777 AND fund_id = 1 AND paid_on &lt;= '2026-10-12';
</code></pre>
<p>Both <code>customer_id</code> and <code>fund_id</code> already had indexes. So why did it take over a second?</p>
<h2>Before: two indexes and an intersection</h2>
<pre><code class="language-plaintext">-&gt; Aggregate: sum(payments.amount)  (actual time=1086..1086 rows=1)
    -&gt; Filter: (fund_id = 1 and customer_id = 777 and paid_on &lt;= '2026-10-12')
        -&gt; Intersect rows sorted by row ID  (actual time=114..1075 rows=20028)
            -&gt; Index range scan using idx_customer  (actual time=5.71..16 rows=20028)
            -&gt; Index range scan using idx_fund      (actual time=9.36..897 rows=1.92e+6)
</code></pre>
<p>Read from the bottom up, MySQL:</p>
<ol>
<li><p>used <code>idx_customer</code> to collect customer 777's row ids: <strong>20,028 ids in 16 ms</strong></p>
</li>
<li><p>used <code>idx_fund</code> to collect <strong>every</strong> fund-1 row id: <strong>1.92 million ids in 897 ms</strong></p>
</li>
<li><p>kept only the ids present in <strong>both</strong> lists (an "index intersection")</p>
</li>
<li><p>fetched those rows, checked the date, and summed the amounts</p>
</li>
</ol>
<p>It read almost <strong>two million</strong> entries to answer a question about <strong>twenty thousand</strong>.</p>
<p>Why choose that plan? The optimizer's statistics estimated about <strong>1 million</strong> fund-1 rows. There were actually <strong>1.92 million</strong>. With the smaller estimate, intersecting looked cheaper than it really was.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6acce90b2a32ac45bd221221/6281e8f2-bec2-4a9e-9d33-fa5347b6ccac.png" alt="" style="display:block;margin:0 auto" />

<h2>A quick reminder: what an index is</h2>
<p>An index is a sorted copy of one or more columns, stored as a B-tree. Because it's sorted, MySQL can walk down a few levels and land on the value it wants instead of reading every row.</p>
<p>In InnoDB, each entry in a secondary index is <code>(indexed value, primary key)</code>, one entry per row. So <code>idx_fund</code> holds 1.92 million entries that start with <code>1</code>. An index on a value shared by 95% of the table can't narrow anything down.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6acce90b2a32ac45bd221221/db863654-3a2c-4ee5-8d89-59c2c734f459.png" alt="" style="display:block;margin:0 auto" />

<h2>The composite index</h2>
<pre><code class="language-sql">CREATE INDEX idx_cust_fund_date_amount
ON payments (customer_id, fund_id, paid_on, amount);
</code></pre>
<p>This index is sorted by customer, then by fund within each customer, then by date within each fund, and it stores the amount alongside. All of customer 777's fund-1 payments now sit in one continuous block, already in date order.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6acce90b2a32ac45bd221221/1fb4ca6c-e352-465a-9b94-f9593b7c0e4c.png" alt="" style="display:block;margin:0 auto" />

<h2>After: 17 ms</h2>
<pre><code class="language-plaintext">-&gt; Aggregate: sum(payments.amount)  (actual time=17..17 rows=1)
    -&gt; Filter: (fund_id = 1 and customer_id = 777 and paid_on &lt;= '2026-10-12')
        -&gt; Covering index range scan using idx_cust_fund_date_amount
           over (customer_id = 777 AND fund_id = 1 AND paid_on &lt;= '2026-10-12')
           (actual time=1.37..9.43 rows=20028)
</code></pre>
<p>One jump to <code>(777, 1)</code>, then one walk forward through 20,028 entries, stopping at the date limit.</p>
<p><strong>Covering</strong> means everything the query needs (customer, fund, date and amount) is inside the index, so MySQL never opened the table. I didn't force anything: the optimizer picked this index on its own.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6acce90b2a32ac45bd221221/9d3ba5ae-ac88-4b2a-b3c7-21a14621287d.png" alt="" style="display:block;margin:0 auto" />

<table>
<thead>
<tr>
<th></th>
<th>Before (two indexes)</th>
<th>After (composite)</th>
</tr>
</thead>
<tbody><tr>
<td>Entries read</td>
<td>20,028 + <strong>1,920,000</strong></td>
<td><strong>20,028</strong></td>
</tr>
<tr>
<td>Table visited</td>
<td>yes</td>
<td><strong>no</strong></td>
</tr>
<tr>
<td>Time</td>
<td><strong>1,086 ms</strong></td>
<td><strong>17 ms</strong></td>
</tr>
</tbody></table>
<h2>The rules that make it work</h2>
<p><strong>1. Leftmost prefix.</strong> A composite index works like a phone book sorted by surname, then first name. You can find all the Mwangis, or "Mwangi, Kevin", but not all the Kevins, because they're spread under every surname.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6acce90b2a32ac45bd221221/8a342b9f-380b-445a-a46e-bec808a9baa7.png" alt="" style="display:block;margin:0 auto" />

<p>My index helps queries on <code>customer_id</code>, or <code>customer_id + fund_id</code>, or all three. A query on <code>fund_id</code> alone can't use it:</p>
<pre><code class="language-sql">EXPLAIN SELECT SUM(amount) FROM payments WHERE fund_id = 1;
-- key: idx_fund   (the composite index starts with customer_id)
</code></pre>
<p><strong>2. Equality first, range last.</strong> <code>customer_id =</code> and <code>fund_id =</code> come before <code>paid_on &lt;=</code>. Once the index reaches a range, the columns after it can't narrow the search any further.</p>
<p><strong>3. Covering is a bonus.</strong> Adding <code>amount</code> let MySQL skip the table entirely. It's worth it for frequent queries, but every extra column makes the index bigger.</p>
<h2>The trade-offs</h2>
<ul>
<li><p>every <code>INSERT</code>, <code>UPDATE</code> and <code>DELETE</code> has to update every index</p>
</li>
<li><p>each index takes disk space and memory</p>
</li>
<li><p>a composite index only helps queries that match its column order</p>
</li>
</ul>
<p>So I wouldn't add one just in case. Measure the slow query, read the plan with <code>EXPLAIN ANALYZE</code>, add the index, then measure again.</p>
<h2>Takeaway</h2>
<p><strong>Single-column indexes each know one thing. A composite index knows the combination.</strong></p>
<hr />
<p><em>I'm Kevin Gakio, a backend engineer learning how databases work underneath.</em></p>
]]></content:encoded></item></channel></rss>