mq-conv

mq-conv converts PDFs, Office docs, markup, spreadsheets, data formats, media and archives into Markdown. It runs locally: no API keys, no network calls, no Python runtime. It is a single binary, built to feed mq, the jq-like query tool for Markdown.

Most "convert to Markdown" tools either dump raw text or call a cloud parser. mq-conv reconstructs structure (headings, tables, lists, links) from the file format's own layout, so the output is Markdown you can query, diff and give to an LLM.

mq-conv report.pdf                 # whole document
mq-conv report.pdf -q '.h2'        # only what a query selects
mq-conv book.pdf --max-tokens 4000 # one prompt-sized chunk at a time

Features

• Layout-aware PDF parsing: columns, headings, lists, ruled and borderless tables, CJK fonts
• Documents, markup, spreadsheets, media and archives in one CLI, detected by extension and magic bytes
• Parallel conversion of PDF pages and input files
• Token budgeting and page selection
• Built-in mq queries
• Image extraction and OCR
• Any input to docx, HTML, LaTeX, EPUB and more
• Feature flags per format and a WebAssembly build

Head to Getting Started to install it and convert your first file.

Getting Started

1. Install the mq-conv binary.
2. Follow the Quick Start to convert a file.

Install

Install script

curl -fsSL https://raw.githubusercontent.com/harehare/mq-conv/main/bin/install.sh | bash

The installer downloads the latest release for your platform, verifies its SHA-256 checksum, installs it to ~/.mq/bin/ and adds that directory to your PATH (bash, zsh or fish). Restart your terminal afterwards.

Cargo

cargo install mq-conv
cargo binstall mq-conv   # prebuilt release instead of compiling

From source

git clone https://github.com/harehare/mq-conv.git
cd mq-conv
cargo build --release    # target/release/mq-conv

Prebuilt binaries for macOS, Linux and Windows are on the releases page.

Verify

mq-conv --version

Quick Start

mq-conv input.pdf                  # convert a file to stdout
mq-conv input.bin --format rst     # force a format
cat notes.org | mq-conv --format org     # read stdin

Convert many files into a directory of .md files:

mq-conv reports/*.pdf notes/*.docx --output-dir ./out

Keep embedded images (written to ./img and linked as ![image](img/...)):

mq-conv report.pdf --extract-images ./img

Next: query the result, page through large documents or pipe into mq.

Guide

• Supported Formats
• Queries and Combine with mq
• Token Budgeting and Page Selection
• Images and OCR
• Markdown to Other Formats
• How Conversion Works

Supported Formats

The format is detected by extension and magic bytes. Override it with --format.

--format values: excel, pdf, powerpoint, word, image, zip, epub, audio, sqlite, tar, video, ocr, markdown-docx, rst, org, mediawiki, asciidoc.

CSV, HTML, JSON, YAML, TOML and XML

mq reads these directly, so mq-conv does not handle them:

mq -I csv 'include "csv" | csv_to_markdown_table(.)' data.csv
mq -I html '.' page.html
mq -I json '.' data.json   # also yaml, toml and xml

Queries

-q/--query runs an mq query over the converted Markdown and prints the result instead of the whole document. With several inputs the query runs on each file. With --max-tokens the query result is what gets paged. It needs Markdown output, so it cannot be combined with --to.

mq-conv adds three functions to the standard mq ones:

mq-conv report.pdf -q '.h2'
mq-conv report.pdf -q '.h2 | select(contains("Results"))'
mq-conv manual.pdf -q '.h2 | select(tokens(.) < 500)'

A query runs once per top-level node, as in mq. The query feature is on by default and adds about 2 MB to the release binary. Build with --no-default-features and pick formats to leave it out. It is not part of the WebAssembly builds.

Combine with mq

mq-conv is the first stage of an mq pipeline: convert to structured Markdown, then slice, filter and transform it with mq.

mq conv document.pdf | mq '.h'
mq conv data.xlsx | mq '.table'
mq conv document.docx | mq -A 'section::section("Summary")'
mq conv slides.pptx | mq view

Token Budgeting

mq-conv book.pdf --max-tokens 4000
# ...ends with: <!-- mq-conv: next-cursor=18342 chunk-tokens=3921 total-tokens=187450 -->
mq-conv book.pdf --max-tokens 4000 --cursor 18342

--max-tokens cuts at block boundaries: paragraphs, tables and code blocks stay whole. It uses a built-in estimate of about 4 ASCII characters, or one CJK character, per token.

A cursor is a byte offset into the converted Markdown. Conversion is deterministic, so the same command with --cursor resumes exactly where the previous chunk stopped. Both options take a single input.

Page Selection

--pages takes numbers and ranges (3, 2-5, 7-, -4, 1,4-6) and works on PDF pages, PowerPoint slides, Excel sheets and EPUB chapters. Other inputs reject it.

mq-conv report.pdf --pages 3-5
mq-conv report.pdf --pages 1,4-6,9-
mq-conv deck.pptx --pages 2-4
mq-conv budget.xlsx --pages 1
mq-conv novel.epub --pages 3

Slides and sheets are numbered by position. EPUB chapters are numbered in reading order, counting only chapters that have text (the sections separated by --- in the output). Numbers stay those of the source: PDF page markers, # Slide N headings and positions do not shift when others are skipped. A selection that matches nothing is an error.

For PDFs the whole document is still analysed, so heading levels and running-header removal match a full conversion. Only the selected pages are rendered, which saves time on OCR and image extraction. Unselected Excel sheets, PowerPoint slides and EPUB chapters are never read or extracted.

Images and OCR

Embedded images

--extract-images DIR writes images from PDF, Word, PowerPoint and EPUB into DIR and links them from the Markdown. With several inputs each file gets its own sub-directory.

OCR

The ocr feature needs Tesseract:

brew install tesseract              # macOS
sudo apt install tesseract-ocr      # Ubuntu/Debian
sudo pacman -S tesseract            # Arch Linux

mq-conv photo.png --format ocr
mq-conv photo.png --format ocr --ocr-lang jpn
mq-conv mixed.png --format ocr --ocr-lang eng+jpn   # combine with "+"

OCR defaults to English. Install the matching language pack for others (brew install tesseract-lang, or sudo apt install tesseract-ocr-jpn). With the feature enabled, image-only PDF pages also fall back to Tesseract.

Other Output Formats

Any input can be written in another format with --to. The input is converted to Markdown first, and that Markdown is written as the target:

mq-conv report.pdf --to html
mq-conv notes.org --to docx --output-dir ./out
mq-conv deck.pptx --to epub --output-dir ./out

A .md input is converted to Word by default:

mq-conv document.md                       # document.docx
mq-conv document.md --output-dir ./out
mq-conv document.md --to html

Targets: html, text, latex, rst, asciidoc, org, epub, json (AST) and docx. --to cannot be combined with -q.

How Conversion Works

Each converter reconstructs Markdown structure from the source format's own layout signals.

PDF

lopdf only reads PDF objects and decompresses streams. The rest is done here from glyph coordinates:

• Fonts are decoded through /ToUnicode, /Encoding + /Differences, the embedded Type 1 / CFF font's own encoding, predefined CJK CMaps and Adobe-Japan1 CIDs, so Japanese PDFs without /ToUnicode still extract.
• Glyphs become words (by gaps measured against font size) and lines. Reading order comes from an XY-cut, so multi-column pages read column by column.
• Tables are detected from drawn rules (merged cells included) or, for borderless tables, from aligned whitespace columns.
• Headings come from relative font size, lists from bullet and number markers, code from monospaced fonts, links from annotations and contents pages from dot leaders.
• Repeated running headers, footers and page numbers are removed.

Others

• Word / PowerPoint: parsed directly from OOXML (document.xml, slideN.xml): headings, bold/italic, hyperlinks, list nesting, tables. No Office shell-out.
• Excel: each sheet is split into blocks by blank rows, then each block is classified as table or free text.

Reference

• CLI
• Feature Flags
• WebAssembly

CLI

Usage: mq-conv [OPTIONS] [FILES]...

Arguments:
  [FILES]...  Input file paths (reads from stdin if omitted)

Options:
  -f, --format <FORMAT>          Force a specific format instead of auto-detecting
  -o, --output-dir <OUTPUT_DIR>  Output directory for individual output files (one per input file)
      --to <TO>                  Write this format instead of Markdown (any input is converted through Markdown)
      --ocr-lang <OCR_LANG>      Tesseract language for OCR, e.g. "jpn" or "eng+jpn" [default: eng]
  -q, --query <QUERY>            Run an mq query over the converted Markdown and print its result
      --pages <PAGES>            Convert only these PDF pages, slides, sheets or EPUB chapters, e.g. "3", "2-5", "7-" or "1,4-6"
      --extract-images <DIR>     Extract embedded images (PDF, DOCX, PPTX, EPUB) into DIR and link them
      --max-tokens <N>           Emit at most ~N tokens, cut at block boundaries (single input)
      --cursor <OFFSET>          Resume from the cursor printed by a previous --max-tokens run [default: 0]
  -h, --help                     Print help
  -V, --version                  Print version

Feature Flags

Every format is a Cargo feature, all on by default. Build with --no-default-features and list only what you need to cut binary size:

cargo install mq-conv --no-default-features --features pdf,word

Features come in groups. The default build enables documents, markup, writers, media, sqlite and query.

Approximate cost in the release binary: query 2.0 MB, sqlite 0.9 MB, image 0.5 MB, writers 0.5 MB, audio and video 0.2 MB together. The other formats add little.

WebAssembly

The wasm feature builds mq-conv as a cdylib for wasm32-unknown-unknown, for the browser via wasm-bindgen. It bundles every format except sqlite and ocr, which need native C libraries.

wasm-pack build --target web --no-default-features --features wasm --release

import init, { convert, detectFormat } from "./pkg/mq_conv.js";

await init();
const bytes = new Uint8Array(await file.arrayBuffer());
const markdown = convert(bytes, file.name, undefined); // filename, or a format name directly