The LLM Input Preprocessing Checklist
A practical checklist for preparing any input, from PDFs to transcripts, before sending it to a large language model.
The five steps
Use this as a pre-flight check:
- Extract: get text out with PDF to Clean Text or OCR.
- Clean: remove timestamps with the Transcript Cleaner.
- Structure: convert HTML to Markdown with the Markdown Converter.
- Measure: check size in the Token Counter.
- Compress: trim with the Context Compressor.
Why it works
Each step removes tokens that carry no meaning. Less noise means lower cost, fewer truncation surprises, and more model attention on the content you care about.
Make it a habit
Bookmark the tools you use most. After a week the routine takes seconds, and the quality difference in answers is noticeable.
Try it free in your browser
- PDF to Clean Text — Strips headers, footers, page numbers, and hyphenation. Outputs TXT or Markdown.
- Transcript Cleaner — Removes timestamps, filler words, and duplicate speaker labels.
- Context Compressor — Reduce token usage with compression strategies tuned for LLMs.
- Markdown Converter — Paste HTML, DOCX, or webpage text and get tidy Markdown.
- Token Counter — Live token, character, and context-window meter for any prompt. Uses official tiktoken encoders.
Tools mentioned
Frequently asked
Do I need all five steps every time?
No. Short, clean inputs only need measuring. Long or messy inputs benefit from all five.
What is the biggest win?
Removing repeated noise like headers, footers, timestamps and HTML.
Keep reading
How to Prepare PDFs for ChatGPT, Claude, and Gemini
A practical guide to extracting clean, AI-ready text from PDFs — born-digital and scanned — so ChatGPT, Claude, and Gemini answer accurately and don't waste tokens on headers, footers, and page numbers.
The Best Way to Feed Long Documents to Claude (and Other Long-Context Models)
Claude's 200K-token context is generous, but you'll still want to clean, compress, and structure long documents before sending them. Here's a step-by-step playbook.
Cleaning Whisper Transcripts for AI Summaries
OpenAI Whisper, Otter, and YouTube transcripts are full of timestamps, filler words, and speaker noise. Here's how to strip them before sending to ChatGPT or Claude — and why it matters.