REST API

All endpoints are under the /api prefix. Document processing runs asynchronously: digitization, rewriting, and embedding are background jobs. Upload or update a document, then poll document and knowledge-base status.

Authentication

Supply your API key in the X-API-Key header.

curl -X GET https://knowledge.hidoba.com/api/knowledgebases/my-partner \
  -H "X-API-Key: YOUR_API_KEY"

Knowledge Character Limits

Quota-aware callers can send an optional positive character_limit. It is a total digitized-character limit for the whole knowledge base, not a per-document limit and not a file-size limit.

When digitized content is about to be saved, the processor atomically checks:

existing KB digitized characters
- current characters for the document being replaced
+ proposed digitized characters

The change is accepted only when that total is within character_limit. An edit that reduces the knowledge base's usage remains allowed even when the knowledge base is already above its current limit. Omit character_limit for unrestricted processing.

The limit applies to the resulting digitized text, including OCR/parsing output; it does not impose a separate per-document character cap. The normal upload file-size limit still applies independently.

For a newly uploaded document that is rejected during synchronous or asynchronous digitization, the document metadata remains visible with:

processing_error = QUOTA_EXCEEDED:knowledge_chars

Its content files, character counts, and processing queues are cleared. Poll the document or knowledge-base status to observe an asynchronous rejection. A replacement of existing digitized content instead returns 403 with:

{
  "detail": "QUOTA_EXCEEDED:knowledge_chars"
}

The previous accepted content is preserved in that case.

Content Types

List Content Types

GET /api/content_types

Returns the available content types for rewriting. No authentication required.

Response

{
  "content_types": [
    { "id": "auto", "name": "Auto (detect from source)" },
    { "id": "article", "name": "Article" },
    { "id": "youtube", "name": "YouTube" },
    { "id": "book", "name": "Book" }
  ]
}

Knowledge Bases

List Knowledge Bases

GET /api/knowledgebases/{partner}

Returns all knowledge bases for the partner, including aggregate embedding status, compact latest-rewrite usage grouped by model, and stored Apify digitization spend.

Response

[
  {
    "name": "my-kb",
    "embedding_status": "success",
    "expert_name": "Ada Lovelace",
    "user_email": "[email protected]",
    "rewrite_usage": [
      {
        "model": "google/gemini-3-flash-preview",
        "rewritten_count": 35,
        "rejected_count": 2,
        "complete_usage_count": 36,
        "complete_cost_count": 20,
        "cost_usd_total": 0.78,
        "input_tokens_total": 1200000,
        "output_tokens_total": 450000,
        "fallback_complete_usage_count": 16,
        "fallback_input_tokens_total": 500000,
        "fallback_output_tokens_total": 180000
      }
    ],
    "digitization_usage": {
      "apify_document_count": 12,
      "complete_cost_count": 12,
      "cost_usd_total": 0.1875
    },
    "deletion_requested_at": null
  }
]

embedding_status is one of success, in_progress, fail, none, or delete_pending.

Each rewrite_usage row aggregates the latest stored rewrite attempt, grouped by rewrite_model. This includes final rewrites and LLM-generated attempts that were subsequently rejected:

Field	Type	Description
`model`	String or null	Stored model for the latest rewrite attempt.
`rewritten_count`	Integer	Documents with a final rewrite in this model group.
`rejected_count`	Integer	Documents whose latest LLM-generated rewrite attempt was rejected after consuming tokens.
`complete_usage_count`	Integer	Final or rejected rewrite attempts with both input and output token counts.
`complete_cost_count`	Integer	Attempts with a stored multi-model list-price estimate.
`cost_usd_total`	Number	Sum of stored rewrite estimates in this group.
`input_tokens_total`	Integer	Sum of `rewrite_input_tokens_total` for documents with complete usage.
`output_tokens_total`	Integer	Sum of `rewrite_output_tokens` for documents with complete usage.
`fallback_complete_usage_count`	Integer	Legacy attempts without `rewrite_cost_usd` that have complete token usage.
`fallback_input_tokens_total`	Integer	Legacy input tokens used by the UI's model-price fallback.
`fallback_output_tokens_total`	Integer	Legacy output tokens used by the UI's model-price fallback.

digitization_usage summarizes documents whose current successful digitization records an Apify provider:

Field	Type	Description
`apify_document_count`	Integer	Documents with a current successful Apify digitization.
`complete_cost_count`	Integer	Those documents with a stored allocated Apify cost.
`cost_usd_total`	Number	Sum of the stored Apify Actor-run shares in USD.

This is a snapshot of the latest fields stored on each document, not a history of processing attempts. Previous rewrites, retries, and failed digitization fallbacks are not included. Rejected or truncated generated output is included because the LLM tokens were consumed; a rejection made before any LLM call contributes no rewrite component. The Knowledge web UI uses stored multi-model rewrite estimates first, falls back to legacy model/token totals, adds Apify cost, and marks incomplete totals as lower bounds. See Spend in the Web UI.

Create Knowledge Base

POST /api/knowledgebases/{partner}
Content-Type: multipart/form-data

curl -X POST https://knowledge.hidoba.com/api/knowledgebases/my-partner \
  -H "X-API-Key: YOUR_API_KEY" \
  -F "name=my-kb"

Parameter	Type	Default	Description
`name`	String	—	Knowledge base name. Required.
`expert_name`	String	—	Knowledge base-level expert/author metadata.
`user_email`	String	—	Knowledge base-level user email metadata.

Delete Knowledge Base

DELETE /api/knowledgebases/{partner}/{kb}

Deletes the knowledge base immediately when possible. If documents or vector cleanup are still active, the response status is delete_pending and includes pending document/vector cleanup details.

Folders

List Folders

GET /api/folders/{partner}/{kb}

Response

{
  "folders": ["chapter-1", "chapter-2"]
}

Create Folder

POST /api/folders/{partner}/{kb}
Content-Type: multipart/form-data

curl -X POST https://knowledge.hidoba.com/api/folders/my-partner/my-kb \
  -H "X-API-Key: YOUR_API_KEY" \
  -F "name=chapter-1"

Send name as a form parameter.

Delete Folder

DELETE /api/folders/{partner}/{kb}/{folder}

Deletes the folder immediately when possible. If documents or vector cleanup are still active, the response status is delete_pending.

Documents

List Documents

GET /api/documents/{partner}/{kb}

Returns the knowledge base with all documents and aggregate embedding info.

Get Combined Data

GET /api/kb-data/{partner}/{kb}

Returns documents, folders, pending folder deletions, and aggregate embedding info in a single call.

Get KB Statistics

GET /api/kb-stats/{partner}/{kb}

Response

{
  "document_count": 42,
  "digitized_count": 40,
  "draft_rewrite_count": 38,
  "final_rewrite_count": 35,
  "total_digitized_chars": 250000,
  "total_draft_rewrite_chars": 200000,
  "total_final_rewrite_chars": 180000,
  "error_count": 2,
  "embedded_count": 35
}

Get Single Document

GET /api/document/{partner}/{kb}/{doc_id}

Create Document (File Upload)

POST /api/documents/{partner}/{kb}
Content-Type: multipart/form-data

curl -X POST https://knowledge.hidoba.com/api/documents/my-partner/my-kb \
  -H "X-API-Key: YOUR_API_KEY" \
  -F "type=file" \
  -F "folder=chapter-1" \
  -F "[email protected]" \
  -F "auto_rewrite=true" \
  -F "character_limit=250000" \
  -F "model_tier=standard"

Form Parameters

Parameter	Type	Default	Description
`type`	String	—	`file` or `url`. Required.
`folder`	String	—	Target folder name. Required.
`file`	File	—	File to upload. Required when `type=file`.
`url`	String	—	URL to fetch. Required when `type=url`.
`name`	String	—	Document name. Auto-detected from file/URL if omitted.
`expert_name`	String	—	Content author or speaker name.
`user_email`	String	—	Email of the uploading user.
`auto_rewrite`	String	`false`	Automatically queue for rewriting after digitization. Pass `"true"` to enable.
`model_tier`	String	`standard`	LLM model tier: `standard` or `economy`.
`use_original_as_rewritten`	String	`false`	Skip LLM rewriting and use the original/digitized content as final rewritten content. Pass `"true"` to enable.
`character_limit`	Integer	—	Optional positive total digitized-character limit for this knowledge base. Enforcement occurs when digitized content is committed. Omit for unrestricted processing.

When final rewritten content is created, embedding is queued automatically. This happens only after digitized content has passed any supplied character limit, then after a successful rewrite, after use_original_as_rewritten=true, or after uploading/updating rewritten content.

Uploads are validated by file extension and file signature. Supported extensions are PDF, DOC/DOCX, PPT/PPTX, XLS/XLSX, EPUB, MD/MARKDOWN, HTML/HTM, TXT/PY/JS/CSS, CSV/JSON/XML/YAML/YML, MP4/MOV/M4V/AVI, MP3/WAV/M4A, and JPG/JPEG/PNG/TIF/TIFF/GIF/WebP. Archives such as ZIP, TAR, GZ/TGZ, RAR, 7Z, BZ2, and XZ are blocked.

Create Document (URL)

curl -X POST https://knowledge.hidoba.com/api/documents/my-partner/my-kb \
  -H "X-API-Key: YOUR_API_KEY" \
  -F "type=url" \
  -F "folder=articles" \
  -F "url=https://example.com/article"

Batch URL Import

POST /api/documents/{partner}/{kb}/batch
Content-Type: application/json

curl -X POST https://knowledge.hidoba.com/api/documents/my-partner/my-kb/batch \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "folder": "articles",
    "urls": [
      "https://example.com/article-1",
      "https://example.com/article-2"
    ],
    "use_original_as_rewritten": false
  }'

note

Maximum 1,000 URLs per batch request.

If use_original_as_rewritten is true, each URL is still digitized first, then the digitized content is saved as final rewritten content and embedded automatically.

Response

{
  "summary": {
    "total": 2,
    "succeeded": 2,
    "failed": 0
  },
  "results": [
    {
      "url": "https://example.com/article-1",
      "status": "success",
      "document_id": "a1b2c3d4-e5f6-..."
    },
    {
      "url": "https://example.com/article-2",
      "status": "success",
      "document_id": "f7g8h9i0-j1k2-..."
    }
  ]
}

Create Empty Document

POST /api/documents/{partner}/{kb}/empty
Content-Type: multipart/form-data

Creates a document with no content, for manual content editing.

Parameter	Type	Default	Description
`name`	String	—	Document name. Required.
`folder`	String	—	Target folder name. Required.
`type`	String	—	`file` or `url`. Required.
`url`	String	—	URL (if type is `url`).

Rename Document

PUT /api/documents/{partner}/{kb}/{doc_id}/name

Send name as a form parameter.

Update Document Type

PUT /api/documents/{partner}/{kb}/{doc_id}/type

Send type as a form parameter. Valid values are file and url.

Update Document URL

PUT /api/documents/{partner}/{kb}/{doc_id}/url

Send url as a form parameter.

Delete Document

DELETE /api/documents/{partner}/{kb}/{doc_id}

Deletion is allowed by default even if a document is being processed. If active work or vector cleanup still blocks final deletion, the response status is delete_pending and includes pending queue/vector cleanup details. Pass force=false to return 409 instead when active processing would block deletion.

curl -X DELETE "https://knowledge.hidoba.com/api/documents/my-partner/my-kb/DOC_ID?force=false" \
  -H "X-API-Key: YOUR_API_KEY"

Document Content

Get Content

GET /api/documents/{partner}/{kb}/{doc_id}/content/{type}

Where {type} is one of: original, digitized, rewritten, draft.

Query Parameters

Parameter	Type	Default	Description
`format`	string	`raw`	`markdown` returns `rewritten`/`draft` content rendered as clean markdown — the same rendered version shown in the dashboard's rewritten document view. Ignored for `original`/`digitized`.

Response

{
  "content": "# Document Title\n\nProcessed document content...",
  "format": "raw"
}

format in the response reflects what was actually returned: "markdown" when rendered markdown was returned, otherwise "raw".

note

Without format=markdown, rewritten/draft content is returned as the document's raw stored content, which is not guaranteed to be clean, readable markdown. Use format=markdown to reliably fetch the fully rendered markdown version.

Returns 400 if format is not raw or markdown, or if format=markdown is requested for a rewritten/draft document that hasn't been successfully rewritten yet.

Update Content

PUT /api/documents/{partner}/{kb}/{doc_id}/content/{type}

Send content as a form parameter. For updates, {type} is one of: original, digitized, rewritten.

For type=digitized, you can also send an optional positive character_limit form parameter. A rejected replacement returns 403 with detail set to QUOTA_EXCEEDED:knowledge_chars; the previous digitized content remains in place. The parameter has no quota effect for original or rewritten updates.

When {type} is rewritten, existing vectors for that document are cleaned up and a fresh embedding job is queued automatically.

Upload Content as File

POST /api/documents/{partner}/{kb}/{doc_id}/content/{type}
Content-Type: multipart/form-data

For uploads, {type} is one of: original, digitized, rewritten.

Upload the content in the file multipart field.

For type=digitized, you can also send an optional positive character_limit form parameter. The uploaded digitized file must be UTF-8. If it exceeds the total knowledge-base limit, the endpoint returns 403 with detail set to QUOTA_EXCEEDED:knowledge_chars and preserves the previous accepted content.

When {type} is rewritten, existing vectors for that document are cleaned up and a fresh embedding job is queued automatically.

Download Document File

GET /api/documents/{partner}/{kb}/{doc_id}/download/{type}

Where {type} is one of: original, digitized, rewritten, draft.

Download Rewritten Bundle

GET /api/documents/{partner}/{kb}/rewritten-bundle

Returns a ZIP archive of all rewritten documents in the knowledge base. The archive contains documents_info.json and one {doc_id}.md file per rewritten document.

Processing

Queue Document for Processing

POST /api/queue/process

Parameter	Type	Default	Description
`document`	Object	—	Document object (JSON body).
`action`	String	`digitize`	Processing action: `digitize` or `rewrite`.
`partner`	String	—	Partner name.
`kb`	String	—	Knowledge base name.
`content_type`	String	`auto`	`auto`, `article`, `youtube`, or `book`.
`person_name`	String	—	Author/speaker name for the rewrite prompt.
`character_limit`	Integer	—	Optional positive total digitized-character limit for the knowledge base. Used when `action=digitize`; omit for unrestricted processing.

Send document in the JSON body. Send action, partner, kb, content_type, person_name, and optional character_limit as query parameters.

Rewriting produces final rewritten content. When that content is saved, embedding is queued automatically. For action=digitize, rewrite/embedding work begins only after the digitized content passes any supplied character limit.

Content Type Auto-Detection

When content_type is set to auto:

YouTube URLs and audio files → youtube template
PDF, EPUB, and Word documents → book template
Everything else → article template

Bulk Rewrite (Folder)

POST /api/bulk/rewrite/folder/{partner}/{kb}/{folder}

Parameter	Type	Default	Description
`skip_rewritten`	Boolean	`true`	Skip documents that already have rewritten content.
`content_type`	String	`auto`	Content type template to use.
`person_name`	String	—	Author/speaker name.

These options are query parameters.

Bulk Rewrite (Knowledge Base)

POST /api/bulk/rewrite/kb/{partner}/{kb}

Parameter	Type	Default	Description
`skip_rewritten`	Boolean	`true`	Skip documents that already have rewritten content.
`content_type`	String	`auto`	Content type template to use.
`person_name`	String	—	Author/speaker name.

These options are query parameters.

Embedding

Embedding is automatic. You do not need to call an endpoint to start embedding for a knowledge base.

The embedding worker indexes final rewritten content only. A document becomes eligible for embedding when file_rewritten is set. Use the document fields and aggregate KB status below to monitor progress.

Get Embedding Status

GET /api/embedding/{partner}/{kb}

Returns aggregate embedding status for the knowledge base. This is a read/status endpoint; it does not start embedding.

Response

{
  "last_successful_update_time": "2025-01-15T10:30:00Z",
  "last_update_time": "2025-01-15T10:30:00Z",
  "last_update_status": "success",
  "last_update_details": {
    "chunk_count": 35,
    "point_count": 35
  },
  "last_successful_details": {
    "chunk_count": 35,
    "point_count": 35
  },
  "knowledge_base_id": "42",
  "embedding_profile_id": 1,
  "embedding_profile_name": "default",
  "eligible_document_count": 35,
  "pending_document_count": 0,
  "failed_document_count": 0,
  "embedded_document_count": 35,
  "last_error": null
}

last_update_status is in_progress, success, fail, or null. Per-document embedding state is available on document objects.

Document Model

A document returned from the API includes the following fields:

Field	Type	Description
`id`	String	UUID identifier.
`name`	String	Document name.
`folder`	String	Folder name.
`type`	String	`file` or `url`.
`url`	String	Source URL (if type is `url`).
`file_original`	String	Original uploaded filename.
`file_digitized`	String	Digitized content filename.
`file_rewritten_draft`	String	Draft rewrite filename.
`file_rewritten`	String	Final rewrite filename.
`user_email`	String	Email of the uploading user.
`in_queues`	Array	Active processing queues (e.g., `["digitize"]`, `["rewrite"]`, `["embedding"]`).
`expert_name`	String	Content author/speaker.
`auto_rewrite`	Boolean	Whether auto-rewrite is enabled.
`model_tier`	String	`standard` or `economy`.
`digitization_provider`	String	Provider for the latest successful digitization when billing metadata was captured. Currently populated for Apify routes; null for other and legacy digitizations.
`digitization_cost_usd`	Number	This document's allocated share of the successful Apify Actor run's actual `usageTotalUsd`; null when unavailable or not applicable.
`rewrite_model`	String	LLM model used for rewriting.
`input_file_tokens`	Integer	Estimated source tokens for the rewritten input. For LLM rewrites, this is derived from first-pass provider prompt usage minus prompt overhead; split inputs follow the actual chunk prompts, including overlap. For content accepted as rewritten without an LLM rewrite, such as `use_original_as_rewritten`, this is calculated locally from the original or digitized text stored as rewritten. Null for pre-3.0.2 rewrite stats until the document is rewritten again or backfilled.
`rewrite_input_tokens_total`	Integer	Provider-reported prompt tokens across all calls in the latest rewrite attempt, including generated output later rejected by validation. Null for pre-3.0.2 rewrite stats until the document is rewritten again.
`rewrite_input_tokens_passes`	Array	Per-call token breakdown. New rows include `model_name` and `output_tokens`; first-pass rows also include `prompt_overhead_tokens` and `source_input_tokens` when available.
`rewrite_output_tokens`	Integer	Provider-reported candidate plus thinking tokens across all calls in the latest rewrite attempt, including generated output later rejected by validation. Null for pre-3.0.2 rewrite stats until the document is rewritten again.
`rewrite_cost_usd`	Number	Stored list-price estimate across every model used by the latest rewrite attempt. Null for legacy attempts or when a configured model has no price.
`rewrite_completed_at`	String	ISO timestamp when the latest rewrite attempt completed or was rejected after generation.
`processing_error`	String	Error message if digitization or rewriting failed.
`embedding_status`	String	Per-document embedding state: `pending`, `in_progress`, `success`, `failed`, or `null`.
`is_embedded`	Boolean	Backward-compatible flag; `true` when `embedding_status` is `success`.
`embedding_error`	String	Error message if embedding failed.
`embedded_at`	String	ISO timestamp of the latest successful embedding.
`embedding_chunk_count`	Integer	Number of chunks embedded for this document.
`deletion_requested_at`	String	ISO timestamp when deletion has been requested but is still waiting for processing/vector cleanup.
`digitized_char_count`	Integer	Character count of digitized content.
`draft_rewrite_char_count`	Integer	Character count of draft rewrite.
`final_rewrite_char_count`	Integer	Character count of final rewrite.

When use_original_as_rewritten is enabled, the document still receives file_rewritten content and can be embedded/synced normally, but no LLM rewrite pass occurs. In that case, input_file_tokens describes the accepted source text, while rewrite_model, rewrite_input_tokens_total, rewrite_input_tokens_passes, rewrite_output_tokens, rewrite_cost_usd, and rewrite_completed_at remain null or unchanged until a real rewrite runs.

Error Responses

Most errors use FastAPI's standard JSON shape with a detail field. The exact message varies by endpoint.

Unauthorized (401)

{
  "detail": "Invalid API key"
}

Forbidden (403)

{
  "detail": "Access denied to partner 'my-partner'"
}

Not Found (404)

{
  "detail": "Knowledge base not found"
}

Rate Limited (429)

Rate-limit responses are produced by the shared SlowAPI rate-limit handler.

Authentication​

Knowledge Character Limits​

Content Types​

List Content Types​

Response​

Knowledge Bases​

List Knowledge Bases​

Response​

Create Knowledge Base​

Delete Knowledge Base​

Folders​

List Folders​

Response​

Create Folder​

Delete Folder​

Documents​

List Documents​

Get Combined Data​

Get KB Statistics​

Response​

Get Single Document​

Create Document (File Upload)​

Form Parameters​

Create Document (URL)​

Batch URL Import​

Response​

Create Empty Document​

Rename Document​

Update Document Type​

Update Document URL​

Delete Document​

Document Content​

Get Content​

Query Parameters​

Response​

Update Content​

Upload Content as File​

Download Document File​

Download Rewritten Bundle​

Processing​

Queue Document for Processing​

Content Type Auto-Detection​

Bulk Rewrite (Folder)​

Bulk Rewrite (Knowledge Base)​

Embedding​

Get Embedding Status​

Response​

Document Model​

Error Responses​

Unauthorized (401)​

Forbidden (403)​

Not Found (404)​

Rate Limited (429)​

Authentication

Knowledge Character Limits

Content Types

List Content Types

Response

Knowledge Bases

List Knowledge Bases

Response

Create Knowledge Base

Delete Knowledge Base

Folders

List Folders

Response

Create Folder

Delete Folder

Documents

List Documents

Get Combined Data

Get KB Statistics

Response

Get Single Document

Create Document (File Upload)

Form Parameters

Create Document (URL)

Batch URL Import

Response

Create Empty Document

Rename Document

Update Document Type

Update Document URL

Delete Document

Document Content

Get Content

Query Parameters

Response

Update Content

Upload Content as File

Download Document File

Download Rewritten Bundle

Processing

Queue Document for Processing

Content Type Auto-Detection

Bulk Rewrite (Folder)

Bulk Rewrite (Knowledge Base)

Embedding

Get Embedding Status

Response

Document Model

Error Responses

Unauthorized (401)

Forbidden (403)

Not Found (404)

Rate Limited (429)