9 Open Source RAG Tools for Australian Businesses (2026)

On this page · 9 sections
- 01How we pick
- 02Part 1: Open source RAG tools your team can use this week
- 03Part 2: Knowledge bases that sync from Google Drive, SharePoint and Outlook
- 04Part 3: For whoever builds it for you
- 05What we rejected, and why
- 06Before you connect SharePoint or Google Drive: five checks
- 07Which open source RAG tool should you start with?
- 08FAQ
- 09Where AI Lab Australia fits
Every business has the same problem. The answer exists. It's in a PDF on SharePoint, a Google Doc from 2022, or the head of whoever has been there longest. New starters ask that person, and that person gets less done.
RAG (retrieval-augmented generation) is the boring fix. Your documents go into a search index. When someone asks a question, the system pulls the most relevant passages and hands them to a language model to write the answer. Done properly, every answer points back to its source, so staff can check it instead of trusting it.
One honest point first. If you already pay for Microsoft 365 Copilot or Gemini in Google Workspace, test those on your own files before installing anything. Open source earns its place when you want control over which model reads your documents, where the index lives, and who sees what.
This is the third tool shortlist in our Monday series. If your documents are scanned paper, start with our open source OCR tools list, because a RAG tool can't search text it can't read.
How we pick
Every tool had to pass all six:
- Useful in under a day to a real Australian SMB, even if a technical person does the install.
- Open source or free to start, with a clear licence. Where the licence adds conditions, we say so.
- Maintained in the last six months, checked on GitHub.
- A concrete Aussie scenario. Google Workspace, Outlook and SharePoint, Xero or MYOB procedures, or a common CRM.
- Security and privacy check. Telemetry defaults, default passwords, how it installs, and where your documents go.
- No hype, no abandonware, nothing that needs access you don't legitimately have.
Stars and push dates come from the GitHub API as of 16 November 2026, rounded down. Stars measure popularity, not safety.
Part 1: Open source RAG tools your team can use this week
A chat window over your documents, with very little setup.
1. AnythingLLM
What it does: A desktop app (Mac, Windows, Linux) and Docker server for chatting with your PDFs, Word files and text, with source citations. It ships with a built-in local model engine, a local vector database and a native embedder, so one laptop can run the lot. The Docker version adds multi-user permissions and an embeddable website widget.
Who it's for: A small team that wants to try this before spending a cent.
Aussie scenario: A bookkeeping practice in Manly loads its month-end checklists, Xero and MYOB how-to notes and onboarding steps into one workspace. A junior asks "what do we do when a client's bank feed drops out?" and gets the practice's own answer, with the document cited.
Our take: The easiest start on this list. Two caveats. Anonymous telemetry (PostHog) is on unless you set DISABLE_TELEMETRY to true or switch it off under Privacy. And "local" only stays local if you pick a local model. Choose a cloud provider and your passages go to that provider with every question.
2. Open WebUI
What it does: A self-hosted chat platform for local models (via Ollama) and any OpenAI-compatible API. You build knowledge bases from uploaded files, with hybrid keyword and vector search plus reranking. It has roles, groups, per-model access control, and SSO through OIDC or LDAP.
Who it's for: A business that wants one internal "company ChatGPT" with a few curated knowledge bases.
Aussie scenario: A design studio in Surry Hills signs staff in with Google Workspace accounts via OIDC, attaches an "HR and policies" knowledge base to one model and "client brand guides" to another, restricted to the design group.
Our take: Polished and very actively developed. The first account becomes admin and sign-ups then switch off, which is the right default. The docs list every outbound call a stock install makes and how to turn each off. The catch is the licence: BSD-3 plus a branding clause. Above 50 users in a 30-day window, you can't remove the Open WebUI branding without permission or an enterprise licence. Fine for most SMBs, but it's no longer a plain open source licence.
3. LibreChat
What it does: A ChatGPT-style interface for Anthropic, OpenAI, Azure OpenAI, AWS Bedrock, Google and local models. Agents can use file search, backed by a separate RAG API service, plus MCP servers and code execution. Login supports OAuth2, LDAP and email.
Who it's for: A business with a preferred model provider, say through an Azure or AWS agreement, that wants a controlled front door for staff.
Aussie scenario: A mortgage broking team in Parramatta connects LibreChat to Azure OpenAI in an Australian region and builds an agent with file search over lender policy PDFs. Brokers ask "which lenders accept casual income under 12 months?" and see the passages behind the answer.
Our take: Strong for teams that care which cloud processes their data. Two day-one jobs. The example config ships with open registration (ALLOW_REGISTRATION=true), so turn it off once your accounts exist. And keep the RAG API updated and off the public internet: its README describes a recent fix for authorisation gaps that let one signed-in user read other users' document chunks. Fixed now, but it shows how much the plumbing matters.
Part 2: Knowledge bases that sync from Google Drive, SharePoint and Outlook
Uploading by hand works for ten documents. These connect to where your files already live.
4. Onyx
What it does: Connects to 50+ business apps, indexes content with its metadata and permissions, and gives staff chat and search over all of it. Connectors include Google Drive, Gmail, SharePoint, OneDrive, Teams, Confluence, HubSpot and Salesforce. It can also answer in Slack and act as an MCP server.
Who it's for: A 20 to 200 person business whose knowledge is spread across several systems.
Aussie scenario: An engineering consultancy in Chatswood connects its SharePoint project library, Outlook and HubSpot. A project manager asks "what did we quote the last three councils for a drainage study?" and gets an answer drawn from the actual files and CRM records.
Our take: The most complete "search everything" option here. Three checks. The quick install pipes a script from the internet into bash, so read it first or use the Docker Compose files. The environment template leaves DISABLE_TELEMETRY unset, so set it. And it's open-core: the Community Edition is MIT, while some features sit in the paid Enterprise Edition, so confirm what the free version does with document permissions before you connect anything sensitive.
5. RAGFlow
What it does: A RAG engine built on "deep document understanding": layout analysis, OCR and table recognition, so messy PDFs, spreadsheets and scans are chunked sensibly. You can see and fix how each document was chunked, and answers come with traceable citations. Release notes list connectors for SharePoint, OneDrive, Outlook, Teams, Gmail and Google Drive.
Who it's for: Businesses whose documents are ugly: long PDFs with tables, policy wordings, manuals.
Aussie scenario: An insurance broker in North Sydney indexes policy wordings from SharePoint. Account managers ask "does this policy cover flood for a strata building in Lismore?" and the answer cites the exact clause.
Our take: The best document handling here, and the heaviest. The README recommends 4 CPU cores, 16 GB RAM and 50 GB of disk, and the backend doesn't currently support macOS, so plan a Linux server. The Docker config ships with default database passwords and an admin password of "admin", with a warning not to deploy them. Change every one before anyone else can reach it.
6. DocsGPT
What it does: Builds agents grounded in your documents, with cited answers. Knowledge comes from PDFs, Office files and web pages, synced from Google Drive, SharePoint, Confluence and S3. Agents go on your website as a chat widget, behind an OpenAI-compatible API, or to MCP clients. Guardrails can flag, redact or block PII, secrets and prompt injection.
Who it's for: A business that wants a customer-facing bot that only says what's in its own published documents.
Aussie scenario: A property management agency in Penrith points DocsGPT at its public tenant handbook and FAQ pages and puts the widget on its website. A tenant asking "how do I report an urgent repair after hours?" gets the agency's actual process, with a link.
Our take: A good public-facing option, as long as a public bot only ever sees public documents. The one-command installer pipes a script into bash, so read it first, and read the security checklist the README links before exposing it beyond your machine.
Part 3: For whoever builds it for you
These need a developer. If you're paying one, they're worth naming in the brief.
7. Dify
What it does: A visual platform for AI workflows and agents, with a RAG pipeline for PDFs, slides and other common formats. It runs on Docker Compose with a minimum of 2 CPU cores and 4 GB RAM.
Who it's for: A business that wants RAG as one step in a process, not just a chat box.
Aussie scenario: A recruitment firm in the Sydney CBD passes each new client enquiry to a Dify workflow that retrieves its terms of business and fee schedule, drafts a reply, and holds it for a consultant to review in Outlook before anything is sent.
Our take: Capable and well supported, but read the licence. It's Apache 2.0 with conditions: no multi-tenant service without written permission, and no removing the Dify logo from the frontend. For one business using it internally, neither usually bites. For an agency planning to resell it, both do.
8. PrivateGPT
What it does: An API layer for private AI applications: file ingestion, retrieval with citations, agentic RAG, tools and MCP connectors. It doesn't run models itself; it connects to an OpenAI-compatible server such as Ollama or vLLM, and ships a workbench UI for testing.
Who it's for: Developers building private document AI into other systems.
Aussie scenario: An IT provider runs Ollama and PrivateGPT on a client's server in a Sydney data centre and calls it from the client's n8n workflows, so policy questions are answered from local documents without the text leaving that server.
Our take: Honest about what it is: the README says the UI is a demonstrator and the API is the product. The SharePoint connectors, LDAP and audit logs it mentions belong to Zylon, the company's paid platform, not the open source project.
9. Haystack
What it does: A Python framework for building RAG, search and agents from explicit, modular pipelines, with control over retrieval, routing, memory and generation.
Who it's for: A developer building something custom who wants to see every step.
Aussie scenario: A compliance assistant for a financial services firm in Barangaroo that retrieves only from approved policy versions and logs which passages fed each answer, so the risk team can audit it.
Our take: Our pick when no off-the-shelf app fits, because the pipelines are easy to reason about. Anonymous telemetry is on by default; set HAYSTACK_TELEMETRY_ENABLED to False to opt out.
What we rejected, and why
- Flowise: the repository shows as archived on GitHub (last push 13 August 2026).
- Verba: archived by Weaviate.
- R2R: last push 7 November 2025.
- Quivr: the repo now redirects to a project that has pivoted to monitoring content streams.
- Khoj: last commit 2 August 2026, latest release a March 2026 beta, and the team's focus has moved to a new product.
- Kotaemon: a tidy document Q&A app, but its last main-branch commit was 30 May 2026, right at our cutoff. We'll check again next round.
- LlamaIndex, LightRAG and Langflow: solid developer projects, but they overlap with Haystack and PrivateGPT for this audience.
Before you connect SharePoint or Google Drive: five checks
A RAG tool is a very efficient way to show staff documents they were never meant to see.
- Permissions. Does the tool respect who could open each file? If unsure, start with one folder everyone can already read.
- The model. Each question sends retrieved passages to your chosen model. An overseas API means personal information may cross the border, which brings in the Privacy Act. Our Privacy Act guide for AI covers what that means.
- Defaults. Change default passwords, close open sign-up, and set the telemetry switches above.
- Exposure. Keep it behind your VPN or SSO. A public widget should only see public documents.
- Freshness. A bot quoting last year's leave policy is worse than no bot. Use sync connectors or a monthly re-upload.
Which open source RAG tool should you start with?
- Trying it on one machine: AnythingLLM.
- One internal AI chat for the team: Open WebUI or LibreChat.
- Search across SharePoint, Google Drive, email and CRM: Onyx.
- Ugly PDFs with tables and scans: RAGFlow.
- A public answer bot on your website: DocsGPT.
- RAG inside a process, built by a developer: Dify, PrivateGPT or Haystack.
FAQ
What is a RAG tool, in plain English? Search plus a language model. It finds the passages in your documents most relevant to a question, and the model writes an answer from those passages, ideally linking each source.
Are open source RAG tools free? The software is. You still pay for a server, setup and upkeep, and either a GPU for local models or API usage for cloud ones. Watch licences with extra conditions, like Open WebUI's branding clause and Dify's multi-tenant rule.
Can I keep my documents in Australia? Yes, if you self-host on an Australian server and use a local model or a provider region in Australia. See our private AI in Australia page for how we approach it.
Will a RAG chatbot make things up? Less often than a plain chatbot, but sometimes. Pick a tool that shows sources, and train staff to click them before acting on anything that matters.
Where AI Lab Australia fits
We put AI agents to work inside Australian businesses, and we show you the controls that make them safe to keep. For a knowledge-base chatbot, that usually means one well-scoped folder, a model chosen on purpose, permissions that match your drive, and a clear answer to "where does this text go?" before launch. See how we approach business automation.
Whether you're in the Sydney CBD, Parramatta, the Northern Beaches or elsewhere in Australia, email info@ailabaustralia.com with where your documents live and what your team keeps asking, and we'll tell you honestly which of these tools fits, or whether you need any of them.

WRITTEN BY
Roman Silantev
CEO and Lead AI Engineer. Scopes and builds AI systems for Australian businesses.
← PREVIOUSAI News Australia 2026: A New AI Safety Regulator, APRA Stress Tests and an Aussie Model
NEXT ARTICLE →Claude Code Skills: Five Public Packs, What They Actually Do, and What to Check First



