Skip to content

THE FIELD JOURNAL

Chat With PDF Locally: Free Offline Apps That Keep Your Files Private

By Local AI Scout · Beginner guide

Hand-drawn laptop turning a PDF window into text and a chat window, illustrating how to chat with a PDF locally without uploading it

How do you chat with a PDF locally?

Install a free desktop app that runs an AI model on your own computer, then drop the PDF into it. AnythingLLM Desktop, LM Studio, Jan and GPT4All all read PDFs on your machine. Once the model is downloaded, LM Studio says chatting with documents needs no internet. That also makes it a backup for outages: you can keep your generator manual searchable offline with other emergency guides. Scanned pages need text recognition first.

This guide follows one reader: someone with a 16GB laptop, either Windows with no graphics card or an M-series MacBook Air, who has a scanned tenancy contract, a letter from a hospital and a 60-page work report, and does not want any of them on ChatGPT's servers. Our guide to what happens when you upload a document to ChatGPT explains the cloud side; this page is the practical how-to.

Choose AnythingLLM Desktop if:

Choose LM Studio if:

Choose Open WebUI with Ollama if:

What does RAG mean, in plain English?

RAG stands for retrieval-augmented generation. The app does not teach the model your PDF. It cuts the text into short passages called chunks, turns each one into an embedding (a list of numbers that captures meaning) and stores them in a search index. When you ask a question, it finds the few chunks that best match and pastes them into the prompt, as GPT4All's LocalDocs page and AnythingLLM's RAG explainer describe.

Diagram of how you chat with a PDF locally: the PDF's text is extracted, with OCR for scans, split into chunks and indexed; your question finds the best matches, which a local model reads to answer with sources
The model never reads your whole library: it answers from the few passages the search step hands it, so check those sources.

The catch follows from the design. The model only sees the matched passages, so a question like "list every deadline in this contract" can miss clauses that were not retrieved. AnythingLLM's docs say plainly that there is no guarantee that relevant text stays together when a document is chunked.

Full document or retrieval: which does the app use?

Most apps now put a short document in whole and switch to retrieval only for long ones. LM Studio adds the file in full if it fits in the model's context, the text the model can hold at once, measured in tokens (pieces of words; LM Studio says one token is often about 3/4 of a word). AnythingLLM inserts the full text by default and asks whether to embed when a file overflows. Open WebUI has a Using Entire Document toggle that skips retrieval for a file.

For a two-page medical letter, full text is what you want. The 60-page report will usually be too long for the context a 16GB laptop can hold, so retrieval takes over and precise questions matter.

Which free apps can chat with a PDF offline in October 2026?

Every app below runs the model and the document processing on your computer. The table compares what matters for private PDFs, using each maker's own pages.

AppCostRuns onReads scanned PDFsShows sourcesLicenseVersion, October 2026
AnythingLLM DesktopFree; optional Pro removes daily limits on Magic featuresWindows (x64, ARM), macOS (Apple Silicon, Intel), LinuxYes, built-in OCRYes, source citationsMIT1.19.0
LM StudioFree, including at workWindows (AVX2) or ARM, macOS 14+ on Apple Silicon, LinuxNot documentedNot documentedClosed source0.4.27, October 29
Open WebUI with OllamaFreeDocker or Python on Windows, macOS, LinuxYes, OCR switch or DoclingYes, citationsOpen WebUI License (BSD-3 plus branding clause)0.12.5, October 29
JanFreeWindows 10+ (AVX2), macOS 13.6+, LinuxNot documentedNot documentedApache 2.00.8.8, October 28
GPT4AllFreeWindows 10, macOS 12.6+, Ubuntu 22.04No, text layer onlyYes, SourcesMIT3.10.0, February 2025
Msty StudioFree tier with limited Knowledge Stacks; Aurum $149 a yearWindows 10/11, macOS 11+, LinuxNot documentedNot documentedCommercial app2.9.11, September 15 (3.0 in beta)

Two notes on the table. "Not documented" means the official docs make no claim, not that the feature is definitely absent. And GPT4All still works, but our GPT4All guide explains why a project with no release since February 2025 is a weak place to start.

How to chat with a PDF in AnythingLLM Desktop

AnythingLLM Desktop bundles its own engine: on the desktop app the built-in LLM is the default provider, running GGUF models (the common file format for local models) through llama.cpp. You do not need Ollama or LM Studio as well. If you already run Ollama, our guide shows how to set up AnythingLLM with Ollama.

  1. Download the installer from anythingllm.com/desktop. There are native builds for macOS (Apple Silicon and Intel), Windows (x64 and ARM) and Linux.
  2. Pick a model on first launch. When you first open the app, the built-in provider lets you select the model to download, in GGUF format from Hugging Face. Pick a small one on the Windows laptop.
  3. Switch off telemetry. AnythingLLM collects anonymous telemetry unless you disable it; the project's README says to open Privacy in the sidebar and disable telemetry.
  4. Create a workspace for one topic, such as "Tenancy", and drag the PDF into the chat or click the + icon, as the documents guide shows.
  5. Choose Embed for long files. If the file overflows the context window, the app offers Cancel, Continue Anyway or Embed. The docs say Continue Anyway loses data because the text is pruned, so pick Embed.
  6. Pin what must be read whole. Clicking the pushpin on an embedded document puts its full text into every answer, which the docs reserve for documents that fit in the context window.

If answers say the document has nothing on a topic you can see in it, lower the document similarity threshold. AnythingLLM's own docs call it likely the cause of the issue, with a default minimum score of 20%, and suggest No Restriction when answers go wrong. The built-in engine's context window defaults to 16,384 tokens, which you can raise in the provider's advanced settings.

How do you delete a private document from AnythingLLM?

Removing a file from a workspace deletes its vectors there, but the document stays in My Documents. To erase it, delete it from My Documents, which removes the parsed file and its cached embeddings from every workspace.

How to chat with a PDF in LM Studio

LM Studio is the quickest route if the PDF already has selectable text. For downloads, models that fit your RAM and the rest of the app, see our full LM Studio guide.

  1. Install LM Studio from lmstudio.ai/download, choosing LM Studio rather than Bionic, the maker's separate agent app offered on the same page. Version 0.4.27 was released on October 29, 2026, and on x64 Windows it needs a processor with AVX2 support.
  2. Download a model in the Discover tab, for example Gemma 4 E4B on Windows or Gemma 4 12B on the Mac, picking the MLX build, made for Apple's own machine learning framework, on Apple Silicon.
  3. Attach the PDF. LM Studio accepts .docx, .pdf and .txt files; drag one into the chat.
  4. Ask with the document's own words. LM Studio's tip is to mention terms, ideas, and words you expect to be in the relevant source material, such as "notice period" rather than "how do I leave".

Its limits are worth knowing. The 0.3.0 release notes put the maximum PDF or .docx size for retrieval at 30MB, and the docs admit retrieval sometimes requires some tuning and experimentation. The document docs say nothing about OCR, so convert scans first (see below). On privacy, LM Studio says a dropped document stays on your machine and its policy says the app does not include telemetry.

GPT4All LocalDocs and Jan Projects

GPT4All LocalDocs

LocalDocs indexes a whole folder: click + Add Collection, name it, link a folder, click Create Collection and wait for the green Ready indicator, then switch the collection on from the LocalDocs button in a chat and click Sources under an answer to see which files were referenced. Its settings page lists the defaults that limit accuracy: snippets of 512 characters, and at most 3 per prompt. That is roughly 1,500 characters of your PDF per question, about a third of a page. That settings page lists .txt, .pdf, .md and .rst as the default file types, but the app's own settings code also includes .docx in the default list, which the changelog dates to version 3.4.0. Its code reads the text layer of each PDF, with no OCR step, so scans give it nothing.

Jan Projects

Jan, the open-source app reviewed in our Jan guide, works two ways. Files added to a Project are chunked and indexed for retrieval across every conversation in that project; open the Project, click Add in the Files panel and wait for the chunk count. For a one-off question, click + in the chat box and choose Add documents or files. Jan says that with a local model, your files never leave your device, while a cloud provider receives them with the request. Version 0.8.8 came out on October 28, 2026 as a security release, so update first.

How to chat with PDFs in Open WebUI with Ollama

Open WebUI is a browser chat page that sits on top of Ollama; our Open WebUI guide covers installing it. With both running:

  1. Raise Ollama's context length. Ollama gives a model a 4k context below 24 GiB of VRAM, a graphics card's own memory, and Open WebUI warns this severely limits RAG performance. Under Settings, Admin, Models, edit the model and raise the context length in Advanced Params.
  2. Build a knowledge base. Upload your PDFs under Workspace, Knowledge.
  3. Ask with #. In a chat, type # and pick the document or knowledge base, then ask your question.
  4. Turn on OCR for scans. In the Admin Panel under Settings, Documents, the PDF Extract Images (OCR) switch is off by default and runs RapidOCR on the Open WebUI server, which is your own computer.
  5. Or switch the reader to Docling. In the same Documents page, Open WebUI lists Docling as a content extraction engine; our Docling guide walks through running docling-serve and keeping it on 127.0.0.1.

Two gotchas. Changing the extraction engine does not reprocess files already in a knowledge base, because Open WebUI re-indexes from the text extracted at first upload, so re-upload them. And the default embedding model is sentence-transformers/all-MiniLM-L6-v2, which runs locally. Open WebUI 0.12.5 was released on October 29, 2026.

Msty Studio and the Ollama app

Msty Studio is a commercial app whose free tier includes Knowledge Stacks with limited RAG; the Aurum plan costs $149 a year. Stacks embed with a local model, gte-tiny by default. Ollama's own app also supports drag and drop of text or PDFs, which suits a quick question about a short file.

Why scanned PDFs need OCR first

A scanned contract is a photograph of text. OCR, optical character recognition, reads the letters in that image and turns them into text an AI model can search. Apps that only read a PDF's text layer, such as GPT4All, get nothing from a scan, and a model will then answer from guesswork. Test your file: if you cannot select a sentence in a PDF viewer, it is a scan.

Info card titled Is your PDF a scan, with the commands pip install docling and docling letter.pdf with full-page OCR and text output, plus three checks: select a sentence, fix gibberish text, compare the numbers
If you cannot select a sentence, the PDF is a picture: run OCR before any app tries to answer from it.

You have three options. AnythingLLM and Open WebUI have OCR built in. For anything else, convert the PDF to text with Docling, a free converter whose FAQ says it is not using any remote service, then attach the text file. These commands come from the Docling CLI reference, where --ocr-mode full_page sends every page to OCR, --to text writes plain text and --replace-broken-text-layer fixes pages whose text copies out as gibberish:

pip install docling
docling letter.pdf --ocr-mode full_page --ocr-lang iso:en --to text

--ocr-lang iso:en sets English, since a BCP-47 tag must be prefixed with 'iso:'. Our Docling guide covers installing it in a virtual environment, tables and running it fully offline. A single photographed page can also go straight to a vision model: Gemma 4 supports vision input, and Jan shows Add Images only when the active model has the vision capability.

Which model should you use for PDF chat on 8, 16 or 32GB?

The model must fit in memory alongside the document's context, so leave headroom. Gemma 4 is a sensible family because every size handles text and images, and LM Studio lists the downloads below.

Bar chart of Gemma 4 download sizes in LM Studio for local PDF chat: E2B 4.20 GB, E4B 5.90 GB, 12B 7.40 GB and 26B A4B 15.60 GB
On 16GB, the E4B and 12B sizes leave room for the document's context; the 26B model needs 32GB.

Sizes come from LM Studio's Gemma 4 page, which says the smallest needs at least 4 GB of RAM, and the tags from Ollama's gemma4 library, where downloads run from 4.6GB to 7.5GB for E2B and 7.7GB to 8.0GB for 12B.

Your machineModelSize in LM StudioOllama tag
8GB laptopGemma 4 E2B4.20 GBgemma4:e2b
16GB Windows laptop, no graphics cardGemma 4 E4B5.90 GBgemma4:e4b
16GB MacBook Air (M-series)Gemma 4 12B (MLX build)7.40 GBgemma4:12b
32GB or more, or a 16GB+ graphics cardGemma 4 26B A4B15.60 GBgemma4:26b

Context costs memory too. LM Studio's Gemma 4 page says the small models have a 128K context window, but that is a maximum, not what a 16GB laptop can hold comfortably. On the Mac, Jan's MLX engine has no embeddings, so no document search, so use a GGUF model for Projects.

How accurate is local PDF chat, and how do you check it?

Treat every answer as a pointer to the right page, not as the final word. Retrieval may pass the model only a few passages, smaller models reason less well than the largest cloud models, and AnythingLLM's docs warn that pushing too much text in with Continue Anyway leads to inaccurate LLM behavior. These habits catch most mistakes:

  • Ask for the exact wording, then find that wording in the PDF yourself. If you cannot find it, the quote is wrong.
  • Open the sources or citations under the answer and read the passage around it.
  • Ask one question at a time, using the document's own terms.
  • Check every number, date and name against the original, especially after OCR.
  • For anything legal or medical, use the answer to find the right clause, then ask a qualified person.

Our document upload checklist adds a useful first test: start with a non-sensitive file whose answers you already know.

Is chatting with a PDF locally really private?

The document and the chat stay on your computer when the model is local, but a few things still go online, so check each app's switches.

Diagram showing that your PDF, the local app's text, chunks and index, and the local model stay on the laptop, while model downloads and app updates come in, usage statistics go out until switched off, and a cloud provider should be avoided
The document never needs to leave the laptop; downloads, updates and optional telemetry are the only traffic to watch.

Running a shop, salon or practice? Read our guide to using this for business files and customer notes.

Common questions about chatting with a PDF locally

Can I chat with a PDF offline for free?

Yes. AnythingLLM Desktop, LM Studio, Jan, GPT4All and Open WebUI with Ollama are all free and run the model on your own computer. You need the internet once to download the app and a model. After that, reading the PDF and answering questions work with no connection.

Can a local AI app read a scanned PDF?

Only if it runs OCR, which turns a picture of text into real text. AnythingLLM Desktop has built-in OCR, and Open WebUI can use its OCR switch or Docling. For other apps, convert the scan to text with Docling first, then attach the text file and check names and numbers.

How much RAM do I need to chat with a PDF locally?

8GB runs a small model such as Gemma 4 E2B, a 4.20 GB download in LM Studio. 16GB suits Gemma 4 E4B on a Windows laptop or Gemma 4 12B on an M-series Mac. Long documents need extra memory for context, so close other apps.

Why does the AI say my PDF does not mention something it does?

Most apps search the PDF and pass only a few matching passages to the model, so the right passage may not be picked. Use the document's own words, ask one question at a time, open the cited sources, and in AnythingLLM lower the document similarity threshold.

Is chatting with a PDF in a local app private?

Yes, as long as you pick a local model. The file, its text and the search index stay on your computer. Apps still go online for model downloads and updates, and AnythingLLM sends anonymous usage statistics until you switch telemetry off in its settings.

Can I trust the quotes a local AI gives from my PDF?

Not without checking. Small local models can misquote or blend passages, especially after OCR on a scan. Search the original PDF for each quoted sentence, check every date, sum and name, and for legal or medical documents confirm the meaning with a qualified person.

My verdict for a 16GB laptop: AnythingLLM Desktop, with Docling for hard scans

For this reader, AnythingLLM Desktop is the best single app. It is free and MIT licensed, runs its own engine, reads scanned PDFs with built-in OCR, shows citations, and keeps separate workspaces for the contract, the medical letter and the report. Switch off telemetry on day one.

On the Windows laptop with no graphics card: install AnythingLLM Desktop, pick a small model when it first asks, and keep each workspace to a few documents. If a scan comes out garbled, convert it with Docling and attach the text file instead.

On the 16GB MacBook Air: the same app works on Apple Silicon. If you already use LM Studio, keep it for PDFs with real text, with Gemma 4 12B in MLX, and use AnythingLLM or Docling for scans.

Add Open WebUI with Ollama only when several people need the same document library, and skip GPT4All for new setups.

Your next step: install AnythingLLM Desktop, turn telemetry off, then add a PDF you know well and ask three questions whose answers you can check, including one quote. If all three match the original, move your private documents in.

Sources

Keep exploring: Apps · Models · All guides

THE SCOUT LETTER

Good finds.
Less noise.

Useful local AI apps, practical guides and the updates worth your time. A thoughtful note in your inbox.

Free to join. Unsubscribe with one click.