Add and manage sources
Contents
Business Knowledge is in private beta. If Business Knowledge isn't in your PostHog sidebar, your project isn't in the beta yet. Send feedback in the app.
Source types
Text
Paste a policy, an FAQ, or any other text, and give the source a name. Edit the source when the text changes. Changes apply to new conversations.
URL
Paste a public http or https URL and choose how PostHog should read it. PostHog rejects a URL it can't reach, including a private or internal address, with URL is not reachable from this environment.
- Single page fetches that page.
- Sitemap.xml reads the sitemap at that URL, or
/sitemap.xmlon the same site, and indexes each listed page. - Same-origin crawl indexes that page and the pages under its path on the same site. A URL at the site root covers the whole site.
| Crawl option | What it does |
|---|---|
| Skip paths | Leaves out matching paths, one pattern per line. /docs/private/* skips that section. |
| Max pages | Caps how many pages the source indexes. |
| Max depth | Caps how many links deep a same-origin crawl goes. |
| Include globs | Under Advanced: override include scope. Replaces the path PostHog takes from the URL. |
The crawler follows robots.txt and identifies itself as PostHog-BusinessKnowledge. If a refresh fails, the row shows refresh failed and the previous text keeps answering questions.
File
Upload a PDF, DOCX, Markdown, CSV, or plain text file. See Limits for size caps.
PostHog stores the text it reads from the file. It doesn't keep the original file. To change a file, delete the source and upload it again. You can still rename the source and change Always include in replies.
Settings
Auto-refresh
A URL source can refresh on a schedule: every hour, every 6 hours, every day, or every week. Manual only means PostHog fetches the source when you add it, and again when you refresh that row yourself.
Changing the auto-refresh interval doesn't fetch the source immediately. Changing the URL or the crawl settings does.
Text and file sources don't refresh on a schedule.
Always include in replies
Turn this on for guidance that should be in every reply: tone, policies, and company direction. PostHog still runs the content check. PostHog includes the source even when the question doesn't match it.
The text injected from these sources is capped at 20,000 characters combined. PostHog leaves out content past that cap.
Status and checks
Source status
| Status | Meaning |
|---|---|
| Pending | Waiting to start |
| Processing | Fetching or indexing |
| Indexing | The content check has passed and keyword search works. Semantic search is still building. This can take up to an hour. |
| Ready | Keyword and semantic search are both available |
| Error | The source failed. Open it to read the reason. |
| Disabled | A learned source that a newer learned answer replaced. PostHog keeps it and doesn't use it in replies. |
| Unsafe content | At least one page was flagged and search leaves it out |
Search leaves out documents that haven't passed the content check, for both keyword and semantic search.
Content checks
PostHog checks each document before using it in an answer. Search leaves out a document marked unsafe. Search also leaves a document out until that check finishes. The check doesn't run until the organization has approved AI data processing.
On a URL source, Indexed pages lists each page as safe, unsafe, or pending. Pending means search skips that page until the check finishes. Unsafe means the page stays listed and search excludes it.
The source shows unsafe content when it has at least one flagged page.
Manage sources
Edit or delete a source
Open a row to change it.
- A text source: edit the name and the content.
- A URL source: edit the name, URL, crawl settings, auto-refresh, and Always include in replies. Use Refresh to fetch it now.
- A file source: edit the name and Always include in replies. Replace the file by deleting the source and uploading again.
- A learned source: edit or delete it. See Learn from support.
Delete removes the source and its chunks. PostHog records the change in the activity log.
Limits
| Limit | Value |
|---|---|
| Sources you add | 500 per project |
| Learned sources | 10,000 per project |
| Chunks | 250,000 per project |
| Pasted text | 1 MB |
| Uploaded file | 50 MB |
| One fetched page | 10 MB |
| Pages in one URL source | 5,000 (default 200) |
| Same-origin depth | 5 (default 2) |
| PDF pages | 2,000 |
| CSV rows kept | 10,000 |
| Always-included text | 20,000 characters combined |