Website Sources
Add a website to your organization's Knowledge Base, and Arketic keeps its pages up to date automatically — recrawling the site on whatever schedule you choose. This page explains how to add a website, what the access levels and refresh schedules mean, how to read a site's status, and how to manage or remove it.
The Websites Tab
Open Knowledge Base from the sidebar. At the top of the page, next to Documents, there is a Websites tab.
Switch to it and the page's subtitle changes to "Websites your assistants can read from." Underneath, a line tells you how many websites are already feeding your knowledge base, next to a Refresh button and an Add Website button.
No websites yet? You'll see "No websites yet", a short explanation, and an Add Website button to get started.
Adding a Website
Adding a website works like adding a document: anyone who can open the Knowledge Base can add a Personal or Organization site. Adding a Department site requires managing that department, or being an Admin or Executive.
- Open Knowledge Base from the sidebar and switch to the Websites tab.
- Click Add Website. The Add Website dialog opens.
- Enter the Website address to start from — for example
https://example.com. Arketic follows links within the same site from there, and re-reads the pages on the schedule you set below. - Optionally give it a Name. Leave it blank and Arketic uses the address instead.
- Under Who can use it, choose an access level. If you choose Department, also choose Which department.
- Under Re-read the site, choose a refresh schedule.
- Click Add and start crawling. The first crawl starts right away — you don't need to wait for the schedule.
Who can use it
| Level | Who can use the site |
|---|---|
| Personal | Only you |
| Department | Everyone in the department you choose |
| Organization | Everyone in your company |
Websites support three access levels. Team and Confidential, which exist for documents, aren't available for websites.
When you choose Department, a Which department list appears with every department in your organization. Only picking one you actually manage will save — Admins and Executives can choose any department. Picking one you don't manage shows an error asking you to choose a department you manage.
Re-read the site
| Option | What it means |
|---|---|
| Manual | Arketic only recrawls the site when you press Crawl now |
| Hourly | Recrawled about once an hour |
| Daily | Recrawled about once a day |
| Weekly | Recrawled about once a week |
Organization and Weekly are the defaults. Change either one before adding the site if you need something different.
Each website can only be added once. If you try to add a site that's already in your knowledge base, you'll see "This website is already a knowledge source."
Reading a Website's Status
The Websites table shows one row per site:
| Column | Shows |
|---|---|
| Website | The site's name (or its address, if you didn't give it one) and, underneath, the full address. Click it to see the site's indexed pages. |
| Status | Whether the site is healthy right now. |
| Access | Who can use the site, with a pencil icon to change it. |
| Pages | How many pages are indexed, and a coverage badge. |
| Last crawl | When the site was last crawled. |
| Refresh | The recrawl schedule — change it directly from this dropdown. |
| Actions | Crawl now, Pause or Resume, view pages, and remove. |
Status badges
| Badge | Meaning |
|---|---|
| Active | Healthy. The last crawl completed normally and nothing is running right now. |
| Crawling… | A crawl is running right now. The page count and last-crawl date on this row are still from the previous crawl until this one finishes. |
| Paused | Scheduled crawls are off. The site stays in your knowledge base, but it won't refresh on its own. |
| Last crawl failed | The most recent crawl hit a problem. Click the badge to open the site's pages and see why. |
| Crawl didn't finish | A crawl started but never reported back. Click the badge to try again. |
Coverage badges
| Badge | Meaning |
|---|---|
| Not crawled yet | The first crawl is queued. It can take several minutes on a large site. |
| Partial | Pages were indexed, but the crawl stopped at a depth or page limit. |
| Complete | Every reachable page on the site was crawled. |
Partial doesn't mean something is wrong. Most real websites are large enough that a crawl stops at a limit before reaching every corner of the site — Partial is the normal, healthy result for most sites, not a warning. Arketic doesn't show a percentage for it either: without a reliable way to know how big the whole site really is, a percentage would just be a guess.
If some pages couldn't be read, a line appears under the page count, such as "2 pages returned nothing usable." The rest of the site is still indexed normally.
Changing Who Can Use a Website
- In the Access column, click the pencil icon next to the site's badge.
- Choose a new level. If you choose Department, also choose which department.
- Click Save.
A confirmation dialog explains what's about to change before it takes effect:
- Widening access (for example, Personal to Organization) confirms with "Widen access to this website?" and tells you assistants will start answering from it for the new audience right away.
- Narrowing access confirms with "Change who can use this website?" and tells you that anyone outside the new audience stops getting answers from it right away.
Changes take effect immediately — there's no separate publish step. If a crawl is running when you change access, pages it writes over the next few minutes keep the old access until the next crawl. Press Crawl now once it finishes to apply the new access to everything.
Refreshing, Pausing, and Removing a Website
Use the icons in the Actions column:
- Crawl now — queues a crawl immediately, regardless of the schedule. It's disabled while a crawl is already running.
- Pause / Resume — Pause turns off scheduled crawls; the site and its pages stay in your knowledge base, but it won't refresh on its own. A paused site also won't respond to Crawl now until you resume it first.
- Remove — permanently deletes the site and every page it indexed. Assistants stop answering from it immediately, and adding the same site again means crawling it from scratch. A confirmation dialog spells this out before anything is deleted.
Viewing a Website's Indexed Pages
Click a site's name, or the list icon in Actions, to open its pages.
Crawled pages don't appear in the regular Knowledge Base document list or its search — a single website can add hundreds of pages, which would bury your uploaded documents. To see what a site actually indexed, open it from the Websites tab.
The pages panel shows:
- The site's name and a link to the live site.
- The same coverage line shown in the table, and — if the last crawl failed — the reason why.
- A search box to find a page in the list.
- Each page's name and address. A page that's been crawled but isn't searchable yet shows an Indexing badge.
If a site has many pages, use Previous and Next at the bottom to page through them.
Getting Website Answers in Chat
Adding and crawling a website does not by itself let any assistant answer questions from it. That's a separate setting on the assistant itself.
If your organization has websites in the knowledge base but no assistant is set up to use them yet, the Websites tab shows a reminder:
Not used in answers yet. Crawling a site does not by itself let an assistant answer from it. Open AI Assistants → Edit → Knowledge Base and turn on "Answer from Websites" for the assistants that should use these sites.
See AI Assistant Configuration for how to set up an assistant's knowledge sources.
Messages You May See
| Message | What it means | What to do |
|---|---|---|
| "This website is already a knowledge source" | This exact site is already in your knowledge base | Find it in the list instead of adding it again |
| "You do not have permission to add a website to this department" | You tried to publish to a department you don't manage | Choose a department you manage, or ask an Admin or Executive |
| "This website is not available to you. You can only change sites you own, or ask an admin." | You tried to change or remove a site someone else added | Ask the site's owner or an Admin |
| "Resume this website source before refreshing it" | You clicked Crawl now on a paused site | Click Resume, then Crawl now |
| "The crawl queue is not accepting jobs right now. Try again shortly." | The system is too busy to start a crawl immediately | Wait a moment and try again |