Read and write Parquet, CSV, and JSONL.
No PyArrow. No Pandas. No NumPy.

Traditional libraries optimise decoding speed. Rugo optimises reading less data in the first place — pushing column projection and predicate pushdown ahead of decode, so data you don't want is never touched. Extracted from years of work on the Opteryx SQL engine.

Performance doesn't only come from decoding faster.

Often it comes from avoiding decoding data you don't need in the first place. Rugo was built as the file layer for the Opteryx SQL engine, where the goals were simple: read as little data as possible, hold as little in memory as possible. Column projection and row-group pruning happen before any decoding — so the columns and rows a query doesn't need are never paid for. The result is a library that gets faster the more selective your query is, stays tiny on disk, and carries no PyArrow, Pandas, or NumPy into your environment.

Measured with Rugo 0.4.10 and PyArrow 25.0.0 on Python 3.14.5t, Apple M-series (macOS arm64 wheels). Import times on a cold process (first load off disk), best of 10. Footprint is the on-disk size of the installed package; neither library pulls a runtime dependency.

Where PyArrow is faster: full-table scans with no filtering. PyArrow's Parquet engine has had over a decade of optimisation poured into raw decode throughput — Rugo doesn't compete there. It competes on how little it has to decode in the first place.

A different niche, not a smaller PyArrow.

Rugo is the file layer for applications that read data selectively rather than scan it wholesale — places where footprint, cold-start time, and memory matter as much as throughput.

One install, three formats.

Rugo reads and writes Parquet, CSV, and JSONL. The API is the same shape across all three: pass a path or bytes, get columnar data back.

from rugo import parquet

# Read — context manager, yields Morsels
with parquet.read_parquet("planets.parquet", columns=["id", "name"], predicates=[("id", ">", 4)]) as reader:
    for morsel in reader:
        print(morsel.column("name").to_pylist())

# Metadata — no column decode
meta = parquet.read_metadata("planets.parquet")
print(meta.num_rows, [c.name for c in meta.schema_columns])

# Write
data = parquet.write_parquet(morsel, compression="zstd")

from rugo import csv

# Same shape — context manager, yields Morsels
with csv.read_csv("data.csv", columns=["name"], predicates=[("age", ">", 30)]) as reader:
    for morsel in reader:
        print(morsel.column("name").to_pylist())

meta = csv.read_metadata("data.csv")
data = csv.write_csv(morsel)

from rugo import jsonl

# Same shape again
with jsonl.read_jsonl("events.jsonl", columns=["id", "status"], predicates=[("status", "==", "active")]) as reader:
    for morsel in reader:
        print(len(morsel), "rows matched")

meta = jsonl.read_metadata("events.jsonl")
data = jsonl.write_jsonl(morsel)

# Look at a file
rugo info events.parquet              # rows, columns, size, format
rugo schema events.parquet            # column names, types, nullability
rugo preview -n 5 events.parquet       # first rows as a table
rugo describe events.parquet          # per-column null counts, min/max

# Reshape and move data between formats
rugo convert events.parquet events.jsonl   # format inferred from extensions
rugo merge part-*.parquet all.parquet      # concatenate schema-identical files
rugo split --rows 100000 big.parquet       # chunk into smaller files
rugo diff before.parquet after.parquet     # schema differences

# Every verb takes --json, so it composes with jq and shell pipelines
rugo count --json events.parquet | jq .num_rows

A complete worked example on a real dataset is available in Google Colab or as a notebook on GitHub.

Parquet, CSV, JSONL — read and write.

All three formats support column projection and predicate filtering. Writers produce standard output readable by any compliant tool.

from rugo import parquet

# Metadata only — no column decode
meta = parquet.read_metadata("planets.parquet")
print(meta.num_rows)
print([c.name for c in meta.schema_columns])

# Selective streaming read
with parquet.read_parquet(
    "planets.parquet",
    columns=["id", "name"],
    predicates=[("id", ">", 4)],
) as reader:
    for morsel in reader:
        names = morsel.column("name").to_pylist()

# Write (ZSTD default)
data = parquet.write_parquet(morsel)
with open("out.parquet", "wb") as f:
    f.write(data)

from rugo import csv

# Schema without reading all data
meta = csv.read_metadata("data.csv")
print(meta.num_rows, [c["name"] for c in meta.schema_columns])

# Projected + filtered read
with csv.read_csv(
    "data.csv",
    columns=["name", "score"],
    predicates=[("age", ">", 30)],
) as reader:
    for morsel in reader:
        print(morsel.column("name").to_pylist())

# TSV variant
with csv.read_csv("data.tsv", delimiter="\t") as reader:
    for morsel in reader: ...

# Write
csv_bytes = csv.write_csv(morsel)

from rugo import jsonl

# Schema without reading all data
meta = jsonl.read_metadata("events.jsonl")
print(meta.num_rows, [c["name"] for c in meta.schema_columns])

# Projected + filtered read
with jsonl.read_jsonl(
    "events.jsonl",
    columns=["id", "status"],
    predicates=[("status", "==", "active")],
) as reader:
    for morsel in reader:
        print(len(morsel), "rows matched")

# Write a Morsel to JSONL bytes
jsonl_bytes = jsonl.write_jsonl(morsel)

Vectors and Morsels.

Rugo speaks Draken — a bundled columnar substrate that ships inside the wheel. There are two objects you will encounter.

vec = morsel.column("name")
values = vec.to_pylist()  # ["Mercury", "Venus", …]

for morsel in reader:
    print(len(morsel), "rows")
    vec = morsel.column("id")

A read → write round-trip works naturally: read_parquet yields Morsels, write_parquet / write_csv / write_jsonl all consume a Morsel.

from rugo import parquet
from rugo.csv import write_csv
from rugo.jsonl import write_jsonl

with parquet.read_parquet("planets.parquet") as reader:
    for morsel in reader:
        csv_bytes   = write_csv(morsel)
        jsonl_bytes = write_jsonl(morsel)
        pq_bytes    = parquet.write_parquet(morsel)

Rugo isn't designed to decode data faster.
It's designed to avoid decoding data you don't need.

Rugo was born out of Opteryx's need for a fast, low-memory Parquet reader — thin by design, and opinionated about what not to load.

Where to go next.