When rugo makes your Lambda cheaper, and when it doesn't
Correct at time of publishing. For up-to-date performance and feature information, see rugo.dev.
On AWS Lambda, rugo costs 15% to 35% less to run than PyArrow: 15% when every call is warm, 35% when every call is a cold start. Against DuckDB and Polars, the answer depends on one number: the share of your calls that are cold starts. Above about 10%, rugo is the cheapest of the four.
Reading Parquet in AWS Lambda without PyArrow
covered package sizes, deployment and read speed. This post
turns the same comparison into money: what each library
costs per million calls, where the break-even points sit,
and how to find out which side of them your function is on.
Every number here comes from functions deployed to AWS and
read off Lambda's own REPORT lines.
How Lambda bills you
- Requests: $0.20 per million calls, whatever the function does.
- Duration: $0.0000166667 per GB-second on x86, $0.0000133334 on arm64, billed by the millisecond. This is almost always most of the bill.
Duration includes the INIT phase: the time
a new execution environment spends loading your code and
running your imports before the first call. You can see it
in the logs. On a cold call, Billed Duration is
Duration plus Init Duration:
Duration: 780.30 ms Billed Duration: 1342 ms Init Duration: 560.73 ms
So every call is one of two kinds. A cold call pays for INIT plus the handler. A warm call reuses an environment and pays for the handler alone. A slow import is a cost on every cold call, not just a delay.
The test
One Parquet file in S3: 38.6 MB, 2 million rows, 8 columns,
20 row groups, ZSTD compressed, web-traffic shaped. Four
functions, one per library, each deployed on x86 and arm64
and run at 1024, 512 and 256 MB. Each function downloads the
file to /tmp on its first call, then runs one
of four queries:
-
broad:
status == 200, 2 of 8 columns. 92% of rows pass, so this measures raw decoding. -
selective:
status == 500. 2% of rows pass, but no row group can be skipped. -
pruned: a 50,000-row
event_idrange. 19 of 20 row groups are skipped using the footer statistics. - footer: row count and schema only, the shape of a function that checks files as they land.
All four libraries returned identical answers on every query, in every configuration. 1,440 calls in all, none failed.
What each library costs to start
At 1024 MB on x86:
| Library | Package | Init duration | Peak memory |
|---|---|---|---|
| rugo 0.5.4 | 13.5 MB | 551 ms | 174 MB |
| duckdb 1.5.6 | 61.7 MB | 845 ms | 205 MB |
| pyarrow 26.0.0 | 172.5 MB | 1,011 ms | 319 MB |
| polars 2.0.0 | 196.3 MB | 1,201 ms | 280 MB |
Init duration is the median over 20 cold starts. It doesn't depend on the query, because it happens before the handler runs.
What one call costs
Billed duration for the broad query, at 1024 MB on x86:
| Library | Cold call | Warm call |
|---|---|---|
| rugo | 1,350 ms | 128 ms |
| duckdb | 1,625 ms | 102 ms |
| pyarrow | 2,076 ms | 154 ms |
| polars | 2,486 ms | 82 ms |
A cold call includes downloading the file from S3, about
600 ms here for every library. A warm call reuses the file
already on /tmp.
rugo has the cheapest cold call by a wide margin, and its warm call is faster than PyArrow's. Polars and DuckDB have the faster warm calls: their scan engines decode a whole-file query quicker.
What a million calls costs
Mix cold and warm calls in different proportions, and the broad query at 1024 MB on x86 costs this per million calls, requests included:
| Cold starts | rugo | duckdb | pyarrow | polars | rugo vs pyarrow |
|---|---|---|---|---|---|
| 0% | $2.33 | $1.90 | $2.76 | $1.57 | −15% |
| 10% | $4.37 | $4.44 | $5.96 | $5.57 | −27% |
| 25% | $7.42 | $8.25 | $10.77 | $11.58 | −31% |
| 50% | $12.52 | $14.59 | $18.78 | $21.60 | −33% |
| 100% | $22.70 | $27.29 | $34.80 | $41.64 | −35% |
If every call is warm and scans most of the file, Polars is cheapest and DuckDB second. From about one cold start in ten upwards, rugo is cheapest, and the gap widens the colder the function runs. PyArrow is never the cheapest: it starts slower than rugo and DuckDB and reads slower than all three.
The break-even point
You don't need the whole table to compare two libraries. If rugo saves time on cold calls but loses some on warm ones, it comes out cheaper when:
cold-start rate > warm penalty ÷ (warm penalty + cold saving)
Against DuckDB on the broad query, the warm penalty is 26 ms (128 − 102) and the cold saving is 275 ms (1,625 − 1,350). That gives 26 ÷ 301, about 9%.
The break-even moves with the query, because the warm penalty is the part that changes. At 1024 MB on x86, rugo is cheaper when the cold-start rate is:
| Query | vs pyarrow | vs duckdb | vs polars |
|---|---|---|---|
| broad (92% pass) | any | above 9% | above 4% |
| selective (2% pass, no skipping) | any | above 6% | above 1% |
| pruned (19 of 20 row groups skipped) | any* | any | any* |
| footer only | any | any | any |
* On the pruned query every library's warm call bills 8 to 10 ms, so rugo is 1 ms dearer than PyArrow and Polars, and cheaper as soon as more than about one call in 500 is cold.
The less of the file a query has to decode, the less the read speed matters and the more the start-up time does. When the query reads a footer or skips most row groups, every library reads in a few milliseconds, and rugo's 300 to 650 ms head start on each cold call is the whole difference.
Other architectures and memory sizes
The broad query again, in every configuration we ran:
| Configuration | vs pyarrow | vs duckdb | vs polars | rugo, all warm | rugo, all cold |
|---|---|---|---|---|---|
| x86, 1024 MB | any | above 9% | above 4% | $2.33 | $22.70 |
| x86, 512 MB | any | above 16% | above 6% | $2.55 | $18.27 |
| x86, 256 MB | any | above 24% | above 5% | $2.42 | $16.13 |
| arm64, 1024 MB | any | any | above 2% | $1.70 | $18.25 |
| arm64, 512 MB | any | above 7% | above 6% | $1.98 | $14.21 |
| arm64, 256 MB | any | above 20% | above 7% | $2.15 | $12.50 |
Three things in this table are worth knowing whichever library you use.
- rugo is cheaper than PyArrow everywhere. Every architecture and every memory size. The only exception is the pruned query, where every library's warm call takes a few milliseconds, and rugo is cheaper once more than 1 to 2% of calls are cold.
- arm64 is cheaper for everyone. It's 20% cheaper per GB-second, and most calls ran no slower. On arm64 at 1024 MB, rugo beats DuckDB at every cold-start rate.
- Less memory isn't a straight saving. Lambda scales CPU and network bandwidth with memory, so at 256 MB every call takes longer. The S3 download alone went from about 600 ms at 1024 MB to about 2,400 ms at 256 MB. For cold-heavy functions the lower price still wins. For warm-heavy ones, cutting memory saves almost nothing.
One thing we expected and didn't find: a memory size only rugo fits in. rugo peaked at 174 MB against PyArrow's 319 MB at 1024 MB, but at 256 MB every library ran without failing. PyArrow peaked at 248 MB, close to the limit. On a bigger file, or with more dependencies loaded, that margin would matter. On this file it didn't.
Find your own cold-start rate
Lambda writes a REPORT line for every call, and
only cold calls include an Init Duration. In
CloudWatch Logs Insights, against your function's log group:
filter @type = "REPORT"
| stats count(*) as calls, count(@initDuration) as cold by bin(1d)
Divide cold by calls. As a rough
guide:
- A steady, busy API keeps its environments warm, and cold starts can be a small fraction of calls. Read speed is most of what you pay for.
- Functions triggered by S3 uploads, schedules or queues that empty and refill sit idle long enough for Lambda to retire their environments. A large share of their calls can be cold.
- Spiky traffic makes Lambda start many environments at once, and each one pays INIT.
What this leaves out
-
Downloading on every call. Our warm
calls reused a file already on
/tmp. A function triggered by each new upload downloads a new object every time. That adds the same time to every library, so the break-even points don't move, but the percentage saved shrinks. Adding a 600 ms download to every warm call cuts rugo's all-warm saving over PyArrow from 15% to about 3%. - Provisioned concurrency. Some teams pay to keep environments warm purely to hide slow imports. If rugo's start-up time makes that unnecessary, the saving is the whole provisioned-concurrency charge, which can dwarf everything in these tables.
- Your other imports. If the function also imports pandas, PyArrow comes along anyway and most of rugo's start-up saving disappears.
- The rest of the bill. CloudWatch Logs, S3 requests and data transfer cost the same whichever reader you use, and for small functions they can be the bigger number.
- One file. A file ten times bigger shifts the balance towards whichever library decodes fastest: DuckDB and Polars on broad scans.
So should you switch?
If your function uses PyArrow only to read Parquet: yes. On Lambda, rugo was cheaper in every configuration we measured, and the colder the function runs, the bigger the saving.
Against DuckDB and Polars, count your cold starts first. If more than about one call in ten is cold, or your queries read a footer or a few row groups rather than the whole file, rugo is cheapest. If your function stays warm and scans whole files, DuckDB or Polars will cost less, and it has SQL or dataframes as well.
The rule of thumb: rugo pays for itself on Lambda when functions start often and read little. That's the usual shape of event-driven work: validating files as they land, reading a footer, pulling a few rows out of a well-partitioned lake.
Reproducing this
Measured 17 October 2026 in us-east-1, on the
python3.12 runtime, against rugo 0.5.4,
pyarrow 26.0.0, duckdb 1.5.6 and polars 2.0.0. Each cold
sample changes the function's configuration first, which
retires its execution environment. There were five cold
samples per query and configuration, then ten warm calls on
the same environment. Costs are calculated, not read off an
invoice: billed duration × memory × us-east-1 price, plus
the request charge.
To reduce noise in the cold figures, init time is the median
of all 20 cold starts per library and configuration, and the
S3 download, which varied by about ±10% between calls and
is the same work for every library, uses one shared median.
The harness, the data generator and the raw results for all
1,440 calls are in
benchmarks/lambda-cost.
The formula works with your own numbers too. Deploy your
function both ways, force a few cold starts, read
Init Duration and Billed Duration
from the REPORT lines, and put in your own
cold-start rate.