# Hidoba Research Docs This file contains all documentation content in a single document following the llmstxt.org standard. ## Data Format ## Input Format The input object contains conversation data and schema definitions. | Field | Type | Required | Description | |-------|------|----------|-------------| | `messages` | array | Yes | Conversation history | | `language` | string | Yes | Language of the conversation | | `current_profile` | object | Yes | Current user profile values | | `profile_schema` | object | Yes | Schema defining profile fields | | `feedback_schema` | object | Yes | Schema defining feedback fields | | `prompt_extra_text` | string | No | Additional instructions for the analysis | ### Messages Messages represent the conversation history, ordered chronologically. ```json { "messages": [ {"text": "Hello, how are you?", "originator": "bot"}, {"text": "I'm doing well, thanks!", "originator": "user"} ] } ``` | Field | Type | Required | Values | |-------|------|----------|--------| | `text` | string | Yes | The message content | | `originator` | string | Yes | `bot` or `user` | ### Profile Schema Defines the structure of user profile fields. **All properties must have type `string`.** ```json { "profile_schema": { "location": { "type": "string", "description": "User's current city" }, "age": { "type": "string", "description": "User's age" } } } ``` | Field | Required | Description | |-------|----------|-------------| | `type` | Yes | Must be `"string"` | | `description` | No | Describes what this field represents | :::note Schema Constraints - Only flat objects are allowed (no nested structures) - All properties must have `type: "string"` - All defined properties are required in the output ::: ### Feedback Schema Defines the structure of conversation feedback. Supports multiple types. ```json { "feedback_schema": { "summary": { "type": "string", "description": "Summary of the conversation" }, "sentiment_score": { "type": "number", "description": "Sentiment score from -1 to 1" }, "is_satisfied": { "type": "boolean", "description": "Whether user seems satisfied" } } } ``` | Type | Description | |------|-------------| | `string` | Text values | | `number` | Decimal numbers | | `integer` | Whole numbers | | `boolean` | True/false values | ### Current Profile The current values of user profile fields. Values must match the profile schema. Use `null` for unknown values. ```json { "current_profile": { "location": "Paris", "age": null } } ``` ### Prompt Extra Text Optional additional instructions for the analysis. Use this for detailed guidance on how to analyze specific fields. ```json { "prompt_extra_text": "Estimate sentiment based on word choice and tone. Consider cultural context for location inference." } ``` ## Output Format The output contains the updated profile and conversation feedback. ```json { "output": { "profile": { "location": "London", "age": "25" }, "feedback": { "summary": "User mentioned moving to a new city.", "sentiment_score": 0.7, "is_satisfied": true } } } ``` ### Profile Output Updated user profile values based on the conversation analysis. - Values conform to the `profile_schema` structure - All values are strings - `null` indicates the value remains unknown ### Feedback Output Analysis results about the conversation. - Values conform to the `feedback_schema` structure - Types match what was specified in the schema ## Unknown Value Handling Profile fields may have unknown values, represented as `null` in the API. **Input**: When a profile field has `null`, it means the value is unknown. ```json { "current_profile": { "location": "Paris", "age": null } } ``` **Output**: If the analysis cannot determine a value, it remains `null`. ```json { "profile": { "location": "London", "age": null } } ``` If the conversation reveals new information, the value is updated: ```json { "profile": { "location": "London", "age": "25" } } ``` --- ## Examples ## Complete Request/Response Example ### Request ```bash curl -X POST https://api.example.com/v2/dialog_analyses \ -H "Content-Type: application/json" \ -H "X-Api-Key: YOUR_API_KEY" \ -d '{ "input": { "messages": [ {"text": "Hello, can you hear me?", "originator": "bot"}, {"text": "Yes, I can!", "originator": "user"}, {"text": "Where are you calling from?", "originator": "bot"}, {"text": "I just moved to London last week", "originator": "user"} ], "language": "English", "current_profile": { "location": "Paris", "age": null }, "profile_schema": { "location": {"type": "string", "description": "User current city"}, "age": {"type": "string", "description": "User age"} }, "feedback_schema": { "summary": {"type": "string", "description": "Brief conversation summary"}, "sentiment": {"type": "string", "description": "Overall user sentiment"} } } }' ``` ### Initial Response ```json { "id": "/v2/dialog_analyses/123", "state": "in-progress", "created_at": "2024-12-06T15:30:00.000Z" } ``` ### Status Check ```bash curl -X GET https://api.example.com/v2/dialog_analyses/123 \ -H "X-Api-Key: YOUR_API_KEY" ``` ### Completed Response ```json { "id": "/v2/dialog_analyses/123", "state": "completed", "created_at": "2024-12-06T15:30:00.000Z", "server_runtime": 8, "output": { "profile": { "location": "London", "age": null }, "feedback": { "summary": "User confirmed they recently relocated to London.", "sentiment": "Positive" } } } ``` ## Example with Numeric Feedback ### Request ```json { "input": { "messages": [ {"text": "How was your experience today?", "originator": "bot"}, {"text": "It was okay, nothing special", "originator": "user"} ], "language": "English", "current_profile": { "satisfaction_level": null }, "profile_schema": { "satisfaction_level": {"type": "string", "description": "User satisfaction: low, medium, or high"} }, "feedback_schema": { "nps_score": {"type": "integer", "description": "Net Promoter Score estimate 0-10"}, "needs_followup": {"type": "boolean", "description": "Whether user needs additional support"} } } } ``` ### Response ```json { "id": "/v2/dialog_analyses/456", "state": "completed", "created_at": "2024-12-06T16:00:00.000Z", "server_runtime": 5, "output": { "profile": { "satisfaction_level": "medium" }, "feedback": { "nps_score": 6, "needs_followup": false } } } ``` ## Example with Extra Prompt Text ### Request ```json { "input": { "messages": [ {"text": "Tell me about yourself", "originator": "bot"}, {"text": "I'm a software developer working remotely", "originator": "user"} ], "language": "English", "current_profile": { "occupation": null, "work_style": null }, "profile_schema": { "occupation": {"type": "string", "description": "User job title or profession"}, "work_style": {"type": "string", "description": "remote, hybrid, or office"} }, "feedback_schema": { "topics_mentioned": {"type": "string", "description": "Comma-separated list of topics"} }, "prompt_extra_text": "For work_style, only use values: remote, hybrid, or office. If unclear, use the most likely option based on context." } } ``` ### Response ```json { "id": "/v2/dialog_analyses/789", "state": "completed", "created_at": "2024-12-06T16:30:00.000Z", "server_runtime": 6, "output": { "profile": { "occupation": "software developer", "work_style": "remote" }, "feedback": { "topics_mentioned": "profession, work arrangement" } } } ``` --- ## Overview The Dialog Analysis API enables you to analyze user conversations and extract structured insights. Pass in recent dialog history along with a user profile, and receive an updated profile and feedback about the conversation. ## Use Cases - **Profile enrichment**: Extract and update user attributes from conversations - **Conversation feedback**: Generate summaries, sentiment analysis, or custom metrics - **User understanding**: Build richer user profiles over time through ongoing dialog ## Typical Flow 1. Submit an analysis request with conversation history, current user profile, and schemas 2. Receive a request ID to track progress 3. Poll for results until the analysis completes 4. Retrieve the updated profile and feedback ## Key Features - **Asynchronous processing**: Long-running analysis handled in the background - **Flexible schemas**: Define your own profile and feedback fields - **Unknown value handling**: Profile fields with unknown values are tracked and updated when discovered - **Structured output**: Results conform exactly to your specified schemas :::important Important Considerations - **Retention**: Output is available for 24 hours after completion - **Rate limits**: Requests are rate-limited per API key and globally - **Input size**: Total request payload must not exceed 100,000 characters ::: --- ## REST API Dialog analysis is a long-running operation. Submit a request and poll for the result. Output is retained for 24 hours after completion. ## Authentication Supply your API key in the `X-Api-Key` header. ```bash curl -X POST https://api.example.com/v2/dialog_analyses \ -H "Content-Type: application/json" \ -H "X-Api-Key: YOUR_API_KEY" \ -d '{"input": {...}}' ``` ## Requesting Analysis ### Request ```http POST /v2/dialog_analyses ``` ```json { "input": { "messages": [...], "language": "English", "current_profile": {...}, "profile_schema": {...}, "feedback_schema": {...}, "prompt_extra_text": "..." } } ``` ### Response ```json { "id": "/v2/dialog_analyses/123", "state": "in-progress", "created_at": "2024-12-06T15:30:00.000Z" } ``` ## Checking Status ### Request ```http GET /v2/dialog_analyses/{id} ``` ### Response (Completed) ```json { "id": "/v2/dialog_analyses/123", "state": "completed", "created_at": "2024-12-06T15:30:00.000Z", "server_runtime": 12, "output": { "profile": {...}, "feedback": {...} } } ``` ## Possible States | State | Description | |-------|-------------| | `in-progress` | Analysis is being processed. Poll again later. | | `completed` | Analysis finished successfully. The `output` field contains results. | | `failed` | Analysis failed after all retry attempts. | ## Error Responses ### Validation Error (422) Returned when the request payload is invalid. ```json { "message": "The given data was invalid.", "errors": { "input.profile_schema.age.type": ["The type must be string."] } } ``` ### Input Size Exceeded (422) Returned when the total payload exceeds the maximum allowed size. ```json { "message": "The given data was invalid.", "errors": { "input": ["Total payload size (150000 characters) exceeds maximum allowed (100000 characters)."] } } ``` ### Rate Limited (429) Returned when rate limits are exceeded. ```json { "message": "Too Many Attempts." } ``` --- ## Introduction Welcome to the official documentation for Hidoba Research APIs. This documentation provides comprehensive guides and references for integrating our AI-powered services into your applications. ## API Offerings Hidoba Research provides the following API offerings: ### Voice Calls SDK Enable real-time voice conversations with AI characters in your web applications. The SDK handles all the complex audio processing, streaming, and communication with our backend services. [Get started with Voice Calls SDK](./voice-calls/overview.md) ### Messages API v2 (legacy) Legacy text/audio conversations through the old `/v2/completions` REST API. Do not use this API for new projects; use Messages API v3 instead. [View legacy Messages API v2 docs](./texting-api/overview.md) ### Messages API v3 Send OpenAI-compatible text generation requests to `https://msg.hidoba.com`. Use it for chat completions, Responses API calls, character prompts, configured knowledge context, streaming, fallback models, and quota-tracked usage. [Get started with Messages API v3](./msg-api/overview.md) ### Transcript API Convert audio to text. Provide an audio URL or upload a file, and receive a full speech-to-text transcription. [Get started with Transcript API](./transcript-api/overview.md) ### TTS API Convert text into speech audio with the OpenAI-compatible `/v2/audio/speech` endpoint. [Get started with TTS API](./tts-api/overview.md) ### Dialog Analysis API Analyze conversations to extract user insights and generate structured feedback. Pass in dialog history and user profile, and receive updated profile data along with custom feedback metrics. [Get started with Dialog Analysis API](./dialog-analysis/overview.md) ## For LLMs and AI assistants This documentation is also available in LLM-friendly plain text: `/llms.txt` links to every page as Markdown, and `/llms-full.txt` contains the full content of all pages in a single file. Every individual page is also available as raw Markdown by appending `.md` to its URL. --- ## Overview(Knowledge-api) The Knowledge API lets you manage knowledge bases of documents that are processed asynchronously. Upload files or URLs, organize them into folders, digitize and rewrite content, and read embedding status for search/retrieval readiness. ## Typical Flow 1. Create a knowledge base for your partner account 2. Upload documents (files or URLs) into the knowledge base 3. Documents are automatically digitized (OCR/parsing) 4. Optionally rewrite documents using LLM processing 5. Final rewritten content is embedded automatically 6. Poll documents or knowledge-base embedding status until ready 7. Retrieve processed content or download rewritten bundles ## Features - **File and URL upload**: Upload documents directly or provide URLs for automatic fetching - **Batch URL import**: Submit up to 1,000 URLs in a single request - **Automatic digitization**: Supported uploads are extracted, OCRed, or transcribed into text - **LLM rewriting**: Documents are rewritten using configurable content-type templates (article, youtube, book) - **Automatic embedding**: Rewritten documents are queued for embedding automatically; clients do not need to start KB embedding manually - **Folder organization**: Organize documents into folders within knowledge bases - **Embedding status**: Track per-document and knowledge-base embedding progress for search and retrieval - **Bulk operations**: Rewrite all documents in a folder or knowledge base at once :::important Important Considerations - **Supported uploads**: PDF, DOC/DOCX, PPT/PPTX, XLS/XLSX, EPUB, MD/MARKDOWN, HTML/HTM, TXT/PY/JS/CSS, CSV/JSON/XML/YAML/YML, MP4/MOV/M4V/AVI, MP3/WAV/M4A, and JPG/JPEG/PNG/TIF/TIFF/GIF/WebP. Archives such as ZIP, TAR, GZ/TGZ, RAR, 7Z, BZ2, and XZ are blocked. - **File size limit**: 100MB per upload by default - **Async processing**: Digitization, rewriting, and embedding run in the background — poll document and KB status - **Embedding source**: Only final rewritten content is embedded. Updating rewritten content automatically refreshes embeddings. - **Audio/video access**: Audio and video transcription can be disabled per partner. If disabled, audio/video documents fail digitization with `processing_error`. - **Rate limits**: Requests are rate-limited ::: --- ## REST API(Knowledge-api) All endpoints are under the `/api` prefix. Document processing runs asynchronously: digitization, rewriting, and embedding are background jobs. Upload or update a document, then poll document and knowledge-base status. ## Authentication Supply your API key in the `X-API-Key` header. ```bash curl -X GET https://knowledge.hidoba.com/api/knowledgebases/my-partner \ -H "X-API-Key: YOUR_API_KEY" ``` ## Content Types ### List Content Types ```http GET /api/content_types ``` Returns the available content types for rewriting. No authentication required. #### Response ```json { "content_types": [ { "id": "auto", "name": "Auto (detect from source)" }, { "id": "article", "name": "Article" }, { "id": "youtube", "name": "YouTube" }, { "id": "book", "name": "Book" } ] } ``` ## Knowledge Bases ### List Knowledge Bases ```http GET /api/knowledgebases/{partner} ``` Returns all knowledge bases for the partner, including aggregate embedding status. ### Response ```json [ { "name": "my-kb", "embedding_status": "success", "expert_name": "Ada Lovelace", "user_email": "ada@example.com", "deletion_requested_at": null } ] ``` `embedding_status` is one of `success`, `in_progress`, `fail`, `none`, or `delete_pending`. ### Create Knowledge Base ```http POST /api/knowledgebases/{partner} Content-Type: multipart/form-data ``` ```bash curl -X POST https://knowledge.hidoba.com/api/knowledgebases/my-partner \ -H "X-API-Key: YOUR_API_KEY" \ -F "name=my-kb" ``` | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `name` | String | — | Knowledge base name. Required. | | `expert_name` | String | — | Knowledge base-level expert/author metadata. | | `user_email` | String | — | Knowledge base-level user email metadata. | ### Delete Knowledge Base ```http DELETE /api/knowledgebases/{partner}/{kb} ``` Deletes the knowledge base immediately when possible. If documents or vector cleanup are still active, the response status is `delete_pending` and includes pending document/vector cleanup details. ## Folders ### List Folders ```http GET /api/folders/{partner}/{kb} ``` #### Response ```json { "folders": ["chapter-1", "chapter-2"] } ``` ### Create Folder ```http POST /api/folders/{partner}/{kb} Content-Type: multipart/form-data ``` ```bash curl -X POST https://knowledge.hidoba.com/api/folders/my-partner/my-kb \ -H "X-API-Key: YOUR_API_KEY" \ -F "name=chapter-1" ``` Send `name` as a form parameter. ### Delete Folder ```http DELETE /api/folders/{partner}/{kb}/{folder} ``` Deletes the folder immediately when possible. If documents or vector cleanup are still active, the response status is `delete_pending`. ## Documents ### List Documents ```http GET /api/documents/{partner}/{kb} ``` Returns the knowledge base with all documents and aggregate embedding info. ### Get Combined Data ```http GET /api/kb-data/{partner}/{kb} ``` Returns documents, folders, pending folder deletions, and aggregate embedding info in a single call. ### Get KB Statistics ```http GET /api/kb-stats/{partner}/{kb} ``` #### Response ```json { "document_count": 42, "digitized_count": 40, "draft_rewrite_count": 38, "final_rewrite_count": 35, "total_digitized_chars": 250000, "total_draft_rewrite_chars": 200000, "total_final_rewrite_chars": 180000, "error_count": 2, "embedded_count": 35 } ``` ### Get Single Document ```http GET /api/document/{partner}/{kb}/{doc_id} ``` ### Create Document (File Upload) ```http POST /api/documents/{partner}/{kb} Content-Type: multipart/form-data ``` ```bash curl -X POST https://knowledge.hidoba.com/api/documents/my-partner/my-kb \ -H "X-API-Key: YOUR_API_KEY" \ -F "type=file" \ -F "folder=chapter-1" \ -F "file=@document.pdf" \ -F "auto_rewrite=true" \ -F "model_tier=standard" ``` #### Form Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `type` | String | — | `file` or `url`. Required. | | `folder` | String | — | Target folder name. Required. | | `file` | File | — | File to upload. Required when `type=file`. | | `url` | String | — | URL to fetch. Required when `type=url`. | | `name` | String | — | Document name. Auto-detected from file/URL if omitted. | | `expert_name` | String | — | Content author or speaker name. | | `user_email` | String | — | Email of the uploading user. | | `auto_rewrite` | String | `false` | Automatically queue for rewriting after digitization. Pass `"true"` to enable. | | `model_tier` | String | `standard` | LLM model tier: `standard` or `economy`. | | `use_original_as_rewritten` | String | `false` | Skip LLM rewriting and use the original/digitized content as final rewritten content. Pass `"true"` to enable. | When final rewritten content is created, embedding is queued automatically. This happens after a successful rewrite, after `use_original_as_rewritten=true`, or after uploading/updating rewritten content. Uploads are validated by file extension and file signature. Supported extensions are PDF, DOC/DOCX, PPT/PPTX, XLS/XLSX, EPUB, MD/MARKDOWN, HTML/HTM, TXT/PY/JS/CSS, CSV/JSON/XML/YAML/YML, MP4/MOV/M4V/AVI, MP3/WAV/M4A, and JPG/JPEG/PNG/TIF/TIFF/GIF/WebP. Archives such as ZIP, TAR, GZ/TGZ, RAR, 7Z, BZ2, and XZ are blocked. ### Create Document (URL) ```bash curl -X POST https://knowledge.hidoba.com/api/documents/my-partner/my-kb \ -H "X-API-Key: YOUR_API_KEY" \ -F "type=url" \ -F "folder=articles" \ -F "url=https://example.com/article" ``` ### Batch URL Import ```http POST /api/documents/{partner}/{kb}/batch Content-Type: application/json ``` ```bash curl -X POST https://knowledge.hidoba.com/api/documents/my-partner/my-kb/batch \ -H "X-API-Key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "folder": "articles", "urls": [ "https://example.com/article-1", "https://example.com/article-2" ], "use_original_as_rewritten": false }' ``` :::note Maximum 1,000 URLs per batch request. ::: If `use_original_as_rewritten` is `true`, each URL is still digitized first, then the digitized content is saved as final rewritten content and embedded automatically. #### Response ```json { "summary": { "total": 2, "succeeded": 2, "failed": 0 }, "results": [ { "url": "https://example.com/article-1", "status": "success", "document_id": "a1b2c3d4-e5f6-..." }, { "url": "https://example.com/article-2", "status": "success", "document_id": "f7g8h9i0-j1k2-..." } ] } ``` ### Create Empty Document ```http POST /api/documents/{partner}/{kb}/empty Content-Type: multipart/form-data ``` Creates a document with no content, for manual content editing. | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `name` | String | — | Document name. Required. | | `folder` | String | — | Target folder name. Required. | | `type` | String | — | `file` or `url`. Required. | | `url` | String | — | URL (if type is `url`). | ### Rename Document ```http PUT /api/documents/{partner}/{kb}/{doc_id}/name ``` Send `name` as a form parameter. ### Update Document Type ```http PUT /api/documents/{partner}/{kb}/{doc_id}/type ``` Send `type` as a form parameter. Valid values are `file` and `url`. ### Update Document URL ```http PUT /api/documents/{partner}/{kb}/{doc_id}/url ``` Send `url` as a form parameter. ### Delete Document ```http DELETE /api/documents/{partner}/{kb}/{doc_id} ``` Deletion is allowed by default even if a document is being processed. If active work or vector cleanup still blocks final deletion, the response status is `delete_pending` and includes pending queue/vector cleanup details. Pass `force=false` to return `409` instead when active processing would block deletion. ```bash curl -X DELETE "https://knowledge.hidoba.com/api/documents/my-partner/my-kb/DOC_ID?force=false" \ -H "X-API-Key: YOUR_API_KEY" ``` ## Document Content ### Get Content ```http GET /api/documents/{partner}/{kb}/{doc_id}/content/{type} ``` Where `{type}` is one of: `original`, `digitized`, `rewritten`, `draft`. #### Query Parameters | Parameter | Type | Default | Description | |---|---|---|---| | `format` | string | `raw` | `markdown` returns `rewritten`/`draft` content rendered as clean markdown — the same rendered version shown in the dashboard's rewritten document view. Ignored for `original`/`digitized`. | #### Response ```json { "content": "# Document Title\n\nProcessed document content...", "format": "raw" } ``` `format` in the response reflects what was actually returned: `"markdown"` when rendered markdown was returned, otherwise `"raw"`. :::note Without `format=markdown`, `rewritten`/`draft` content is returned as the document's raw stored content, which is not guaranteed to be clean, readable markdown. Use `format=markdown` to reliably fetch the fully rendered markdown version. ::: Returns `400` if `format` is not `raw` or `markdown`, or if `format=markdown` is requested for a `rewritten`/`draft` document that hasn't been successfully rewritten yet. ### Update Content ```http PUT /api/documents/{partner}/{kb}/{doc_id}/content/{type} ``` Send `content` as a form parameter. For updates, `{type}` is one of: `original`, `digitized`, `rewritten`. When `{type}` is `rewritten`, existing vectors for that document are cleaned up and a fresh embedding job is queued automatically. ### Upload Content as File ```http POST /api/documents/{partner}/{kb}/{doc_id}/content/{type} Content-Type: multipart/form-data ``` For uploads, `{type}` is one of: `original`, `digitized`, `rewritten`. Upload the content in the `file` multipart field. When `{type}` is `rewritten`, existing vectors for that document are cleaned up and a fresh embedding job is queued automatically. ### Download Document File ```http GET /api/documents/{partner}/{kb}/{doc_id}/download/{type} ``` Where `{type}` is one of: `original`, `digitized`, `rewritten`, `draft`. ### Download Rewritten Bundle ```http GET /api/documents/{partner}/{kb}/rewritten-bundle ``` Returns a ZIP archive of all rewritten documents in the knowledge base. The archive contains `documents_info.json` and one `{doc_id}.md` file per rewritten document. ## Processing ### Queue Document for Processing ```http POST /api/queue/process ``` | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `document` | Object | — | Document object (JSON body). | | `action` | String | `digitize` | Processing action: `digitize` or `rewrite`. | | `partner` | String | — | Partner name. | | `kb` | String | — | Knowledge base name. | | `content_type` | String | `auto` | `auto`, `article`, `youtube`, or `book`. | | `person_name` | String | — | Author/speaker name for the rewrite prompt. | Send `document` in the JSON body. Send `action`, `partner`, `kb`, `content_type`, and `person_name` as query parameters. Rewriting produces final rewritten content. When that content is saved, embedding is queued automatically. ### Content Type Auto-Detection When `content_type` is set to `auto`: - YouTube URLs and audio files → `youtube` template - PDF, EPUB, and Word documents → `book` template - Everything else → `article` template ### Bulk Rewrite (Folder) ```http POST /api/bulk/rewrite/folder/{partner}/{kb}/{folder} ``` | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `skip_rewritten` | Boolean | `true` | Skip documents that already have rewritten content. | | `content_type` | String | `auto` | Content type template to use. | | `person_name` | String | — | Author/speaker name. | These options are query parameters. ### Bulk Rewrite (Knowledge Base) ```http POST /api/bulk/rewrite/kb/{partner}/{kb} ``` | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `skip_rewritten` | Boolean | `true` | Skip documents that already have rewritten content. | | `content_type` | String | `auto` | Content type template to use. | | `person_name` | String | — | Author/speaker name. | These options are query parameters. ## Embedding Embedding is automatic. You do not need to call an endpoint to start embedding for a knowledge base. The embedding worker indexes final rewritten content only. A document becomes eligible for embedding when `file_rewritten` is set. Use the document fields and aggregate KB status below to monitor progress. ### Get Embedding Status ```http GET /api/embedding/{partner}/{kb} ``` Returns aggregate embedding status for the knowledge base. This is a read/status endpoint; it does not start embedding. #### Response ```json { "last_successful_update_time": "2025-01-15T10:30:00Z", "last_update_time": "2025-01-15T10:30:00Z", "last_update_status": "success", "last_update_details": { "chunk_count": 35, "point_count": 35 }, "last_successful_details": { "chunk_count": 35, "point_count": 35 }, "knowledge_base_id": "42", "embedding_profile_id": 1, "embedding_profile_name": "default", "eligible_document_count": 35, "pending_document_count": 0, "failed_document_count": 0, "embedded_document_count": 35, "last_error": null } ``` `last_update_status` is `in_progress`, `success`, `fail`, or `null`. Per-document embedding state is available on document objects. ## Document Model A document returned from the API includes the following fields: | Field | Type | Description | |-------|------|-------------| | `id` | String | UUID identifier. | | `name` | String | Document name. | | `folder` | String | Folder name. | | `type` | String | `file` or `url`. | | `url` | String | Source URL (if type is `url`). | | `file_original` | String | Original uploaded filename. | | `file_digitized` | String | Digitized content filename. | | `file_rewritten_draft` | String | Draft rewrite filename. | | `file_rewritten` | String | Final rewrite filename. | | `user_email` | String | Email of the uploading user. | | `in_queues` | Array | Active processing queues (e.g., `["digitize"]`, `["rewrite"]`, `["embedding"]`). | | `expert_name` | String | Content author/speaker. | | `auto_rewrite` | Boolean | Whether auto-rewrite is enabled. | | `model_tier` | String | `standard` or `economy`. | | `rewrite_model` | String | LLM model used for rewriting. | | `rewrite_input_tokens` | Integer | Input tokens consumed. | | `rewrite_output_tokens` | Integer | Output tokens generated. | | `rewrite_completed_at` | String | ISO timestamp of rewrite completion. | | `processing_error` | String | Error message if digitization or rewriting failed. | | `embedding_status` | String | Per-document embedding state: `pending`, `in_progress`, `success`, `failed`, or `null`. | | `is_embedded` | Boolean | Backward-compatible flag; `true` when `embedding_status` is `success`. | | `embedding_error` | String | Error message if embedding failed. | | `embedded_at` | String | ISO timestamp of the latest successful embedding. | | `embedding_chunk_count` | Integer | Number of chunks embedded for this document. | | `deletion_requested_at` | String | ISO timestamp when deletion has been requested but is still waiting for processing/vector cleanup. | | `digitized_char_count` | Integer | Character count of digitized content. | | `draft_rewrite_char_count` | Integer | Character count of draft rewrite. | | `final_rewrite_char_count` | Integer | Character count of final rewrite. | ## Error Responses Most errors use FastAPI's standard JSON shape with a `detail` field. The exact message varies by endpoint. ### Unauthorized (401) ```json { "detail": "Invalid API key" } ``` ### Forbidden (403) ```json { "detail": "Access denied to partner 'my-partner'" } ``` ### Not Found (404) ```json { "detail": "Knowledge base not found" } ``` ### Rate Limited (429) Rate-limit responses are produced by the shared SlowAPI rate-limit handler. --- ## Examples(Msg-api) ## Chat Completion ```bash curl https://msg.hidoba.com/v3/chat/completions \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "Hi, can you help me test the API?" }, { "role": "assistant", "content": "Yes. Send me the exact reply you want." }, { "role": "user", "content": "Reply with exactly: hello" } ], "max_completion_tokens": 32 }' ``` ## Streaming Chat Completion ```bash curl -N https://msg.hidoba.com/v3/chat/completions \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "Can you write short poems?" }, { "role": "assistant", "content": "Yes, I can keep them concise." }, { "role": "user", "content": "Write a two-line poem about rain." } ], "stream": true, "max_completion_tokens": 80 }' ``` ## Responses API ```bash curl https://msg.hidoba.com/v3/responses \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "input": "Explain semantic search in one sentence.", "max_output_tokens": 80 }' ``` ## Streaming Responses With Knowledge Sources When a character has knowledge context enabled, streaming Responses calls can include status and source events before the model's answer. ```bash curl -N https://msg.hidoba.com/v3/responses \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "stream": true, "input": [ { "role": "user", "content": "What should I know from the knowledge base?" } ], "max_output_tokens": 240, "metadata": { "hidoba": { "character": "github:partner/KnowledgeCharacter" } } }' ``` Example stream fragments: ```text event: response.output_item.added data: {"type":"response.output_item.added","output_index":0,"item":{"id":"rs_hidoba_rag_loading_123","type":"reasoning","summary":[{"type":"summary_text","text":"Looking through the knowledge base..."}]}} event: response.file_search_call.completed data: {"type":"response.file_search_call.completed","output_index":1,"item":{"id":"fs_hidoba_123","type":"file_search_call","status":"completed","queries":["What should I know from the knowledge base?"],"results":[{"file_id":"file_hidoba_07d6ebeddf6a421c","filename":"Product Notes","score":0.91,"text":"Product Notes # Overview The product supports...","attributes":{"chunk_index":0,"retrievers":"dense,splade"}}]}} event: response.output_item.added data: {"type":"response.output_item.added","output_index":2,"item":{"id":"rs_hidoba_rag_found_123","type":"reasoning","summary":[{"type":"summary_text","text":"Found 1 relevant documents, thinking..."}]}} ``` Render the `reasoning.summary[].text` values as status messages and `file_search_call.results[]` as sources. Each source `text` value is a short preview, not the full document chunk. If model thinking is enabled, you may also receive `response.reasoning_text.delta` and `response.reasoning_text.done` events. Display those as thinking text, not as the final answer. ## GitHub Character ```bash curl https://msg.hidoba.com/v3/chat/completions \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "I want a practical coaching answer." }, { "role": "assistant", "content": "I will keep it practical and encouraging." }, { "role": "user", "content": "How should I start practicing today?" } ], "max_completion_tokens": 120, "metadata": { "hidoba": { "character": "github:partner/Coach", "character_params": { "person_name": "Alex" } } } }' ``` ## Inline Character Use an inline character when you want to provide the character prompt in the request. ```json { "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "Give me a warm one-sentence greeting." } ], "max_completion_tokens": 80, "metadata": { "hidoba": { "character": { "parameters": { "name": "Mira", "language": "en", "person_name": "Alex", "robot_name": "Mira", "name_colon": "Mira:", "conversation_prefix": "", "interruption_message": "", "easy_questions": [], "difficult_questions": [], "voice": null, "greetings": [], "max_new_tokens": 128 }, "personality": "You are {{ robot_name }}. Be concise, friendly, and helpful." } } } } ``` ## Reasoning Effort ```json { "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "I need a direct answer." }, { "role": "assistant", "content": "Understood. I will keep it direct." }, { "role": "user", "content": "Think briefly, then answer with one paragraph." } ], "reasoning": { "effort": "low" }, "max_completion_tokens": 200 } ``` Use `none` to turn reasoning off: ```json { "reasoning": { "effort": "none" } } ``` Use a token budget when the model supports it: ```json { "reasoning": { "max_tokens": 1024 } } ``` ## Fallback Model ```json { "model": "google/gemini-2.5-flash", "fallback_model": "qwen/qwen3.6-27b", "messages": [ { "role": "user", "content": "I am brainstorming features." }, { "role": "assistant", "content": "Great. I can suggest concise ideas." }, { "role": "user", "content": "Give me one concise product idea." } ], "max_completion_tokens": 100 } ``` `fallback_model` is a Hidoba routing option. It is not part of the model conversation. ## Character With Knowledge Context Knowledge context is configured in the character settings. With a GitHub character, the settings live in the saved character. With an inline character, put them in the character `parameters`. ```json { "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "What should I remember from the knowledge base?" } ], "max_completion_tokens": 160, "metadata": { "hidoba": { "character": { "parameters": { "robot_name": "Mira", "rag_index_required": "partner/knowledge-base-name", "rag_enabled": true }, "personality": "You are {{ robot_name }}. Use the available knowledge context when relevant." } } } } ``` Here, `rag_enabled` turns knowledge context on, and `rag_index_required` selects the saved knowledge base for the character. --- ## Hidoba Metadata Messages API v3 accepts optional Hidoba-specific request metadata under `metadata.hidoba`. Only these fields are allowed: | Field | Type | Description | | --- | --- | --- | | `character` | String or object | GitHub character reference or inline character object. | | `character_params` | Object | Flat template parameters for the character personality. Values may be string, number, boolean, or null. | All other `metadata.hidoba` fields are rejected. ## GitHub Character Use a GitHub character reference when the character is saved in Hidoba: ```json { "metadata": { "hidoba": { "character": "github:partner/CharacterName", "character_params": { "person_name": "Alex" } } } } ``` `github-force:` is also accepted for force-refresh flows: ```json { "metadata": { "hidoba": { "character": "github-force:partner/CharacterName" } } } ``` Hidoba validates that the quota is allowed to use the requested character. ## Inline Character Use an inline character object for ad hoc testing or cases where the character is supplied with the request: ```json { "metadata": { "hidoba": { "character": { "parameters": { "name": "Mira", "language": "en", "person_name": "Alex", "robot_name": "Mira", "name_colon": "Mira:", "conversation_prefix": "", "interruption_message": "", "easy_questions": [], "difficult_questions": [], "voice": null, "greetings": [], "max_new_tokens": 128 }, "personality": "You are {{ robot_name }}. Be concise and helpful." } } } } ``` Inline characters must pass Hidoba character validation. Some existing character schemas include `max_new_tokens` as legacy character metadata. Messages API v3 does not use character `max_new_tokens` to control generation length; use request-level token limits instead. ## Character Params `character_params` are rendered into Jinja character personality templates. ```json { "metadata": { "hidoba": { "character": "github:partner/CharacterName", "character_params": { "person_name": "Alex", "plan": "premium" } } } } ``` Rules: - `character_params` must be a JSON object. - Values must be string, number, boolean, or null. - `character_params.context` is reserved and rejected. ## Knowledge Context Knowledge context is configured in the character settings. This works the same way whether the character is referenced from GitHub or supplied inline. For a GitHub character, configure the knowledge settings in the saved character file and reference the character: ```json { "metadata": { "hidoba": { "character": "github:partner/CharacterName" } } } ``` For an inline character, include the knowledge settings inside the character `parameters` object: ```json { "metadata": { "hidoba": { "character": { "parameters": { "robot_name": "Mira", "rag_index_required": "partner/knowledge-base-name", "rag_enabled": true }, "personality": "You are {{ robot_name }}. Use the available knowledge context when relevant." } } } } ``` When knowledge context is configured, Messages API v3 may add relevant context automatically before the model answers. In character settings: - `rag_enabled` turns knowledge context on for that character. - `rag_index_required` selects which saved knowledge base the character should use. ## Model Fallback Do not send routing config in `metadata.hidoba`. Invalid: ```json { "metadata": { "hidoba": { "routing": { "primary_model": "google/gemini-2.5-flash" } } } } ``` To request a fallback model, use top-level `fallback_model`: ```json { "model": "google/gemini-2.5-flash", "fallback_model": "qwen/qwen3.6-27b" } ``` Messages API v3 uses this as a routing option when supported. ## Model Conversation Visibility `metadata.hidoba` is Hidoba-only request metadata and is not included in the model conversation. Non-Hidoba metadata is preserved: ```json { "metadata": { "client": "kept", "hidoba": { "character": "github:partner/CharacterName" } } } ``` The model request keeps: ```json { "metadata": { "client": "kept" } } ``` --- ## Messages API v3 Messages API v3 is Hidoba's OpenAI-compatible text generation endpoint. Base URL: ```text https://msg.hidoba.com ``` Use this API when you want synchronous chat or Responses API generations with Hidoba quota tracking, character prompts, configured knowledge context, streaming, fallback models, and usage attribution. The older Messages API v2 docs cover the legacy `/v2/completions` text/audio flow. Use Messages API v3 for new OpenAI-compatible text generation integrations. ## Typical Flow 1. Send an OpenAI-compatible request with a quota API key. 2. Hidoba validates the API key, quota, and character access. 3. Messages API v3 applies character prompts and configured knowledge context when available. 4. The model response is returned directly or streamed. 5. Usage is recorded automatically. ## Features - **Chat Completions**: `POST /v3/chat/completions` - **Responses API**: `POST /v3/responses` - **Authentication**: `Authorization: Bearer ` or `X-API-Key: ` - **Characters**: Optional GitHub or inline characters under `metadata.hidoba.character` - **Knowledge context**: Configured through GitHub or inline character settings - **Knowledge visibility**: Responses API calls can include status messages and source items when knowledge context is used - **Fallback models**: Optional top-level `fallback_model` for supported requests - **Reasoning controls**: Optional `reasoning` settings for supported models - **Streaming**: Standard streaming responses for supported models ## Important Considerations :::important - `metadata.hidoba` may contain only `character` and `character_params`. - `metadata.hidoba.routing`, `metadata.hidoba.request_id`, and unknown `metadata.hidoba` fields are rejected. - `metadata.hidoba` is Hidoba-only request metadata and is not part of the model conversation. - Knowledge context is configured in character settings, either in GitHub or inline. - Character `max_new_tokens`, when present in old character schemas, is not used as the output-token limit. Use request-level token fields such as `max_completion_tokens`, `max_tokens`, or `max_output_tokens`. ::: --- ## REST API(Msg-api) All public Messages API v3 endpoints are under: ```text https://msg.hidoba.com ``` ## Authentication Send a quota API key with each generation request. Bearer auth is preferred. ```bash curl https://msg.hidoba.com/v3/chat/completions \ -H "Authorization: Bearer YOUR_QUOTA_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"google/gemini-2.5-flash","messages":[{"role":"user","content":"Say hello"}]}' ``` `X-API-Key` is also accepted: ```http X-API-Key: YOUR_QUOTA_API_KEY ``` ## Health ```http GET /health GET /health/deep ``` Health endpoints are intended for uptime checks and deployment verification. ## Chat Completions ```http POST /v3/chat/completions ``` The request body follows the OpenAI Chat Completions shape. Messages API v3 also accepts a small number of Hidoba-specific additions. | Field | Type | Required | Description | | --- | --- | --- | --- | | `model` | String | Yes | Requested model name, for example `google/gemini-2.5-flash`. | | `messages` | Array | Yes | OpenAI-compatible chat messages. Use `assistant` for prior chatbot messages and `user` for user messages. | | `stream` | Boolean | No | Set to `true` for streaming responses. | | `max_completion_tokens` | Number | No | Preferred output-token limit for compatible models. | | `max_tokens` | Number | No | Compatibility output-token limit. | | `reasoning` | Object | No | Optional thinking/reasoning control for supported models. See [Reasoning](#reasoning). | | `fallback_model` | String | No | Optional fallback model to try when supported. This is a Hidoba routing option, not part of the model conversation. | | `metadata.hidoba` | Object | No | Hidoba character metadata. See [Hidoba Metadata](./metadata.md). | ### Example ```bash curl https://msg.hidoba.com/v3/chat/completions \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "Hi, can you help me write concise replies?" }, { "role": "assistant", "content": "Yes. I will keep replies brief and clear." }, { "role": "user", "content": "Reply with exactly: hello" } ], "max_completion_tokens": 32 }' ``` The response is an OpenAI-compatible chat completion response. ## Responses API ```http POST /v3/responses ``` The request body follows the OpenAI Responses API shape. | Field | Type | Required | Description | | --- | --- | --- | --- | | `model` | String | Yes | Requested model name. | | `input` | String or Array | Yes | Responses API input. | | `instructions` | String | No | Additional system instructions. Character prompts and knowledge context may be added when configured. | | `stream` | Boolean | No | Set to `true` for streaming responses when supported. | | `max_output_tokens` | Number | No | Output-token limit for Responses API requests. | | `reasoning` | Object | No | Optional thinking/reasoning control for supported models. See [Reasoning](#reasoning). | | `fallback_model` | String | No | Optional fallback model to try when supported. | | `metadata.hidoba` | Object | No | Hidoba character metadata. See [Hidoba Metadata](./metadata.md). | ### Example ```bash curl https://msg.hidoba.com/v3/responses \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "input": "Write one concise sentence about Mars.", "max_output_tokens": 80 }' ``` The response is an OpenAI-compatible Responses API response. ## Responses API Visibility When a `/v3/responses` request uses a character with knowledge context enabled, Hidoba adds OpenAI-shaped visibility items to the Responses output. These items are intended for user interfaces that want to show what is happening before the final answer arrives. This visibility is added only for `/v3/responses`. Chat Completions responses keep the normal Chat Completions shape. ### Output Items When knowledge context runs, the Responses `output` array includes Hidoba-added items before the model answer: 1. A `reasoning` item saying that Hidoba is looking through the knowledge base. 2. A `file_search_call` item with the searches and retrieved source previews. 3. A `reasoning` item saying how many relevant documents were found. 4. The normal model output items. Example: ```json { "output": [ { "id": "rs_hidoba_rag_loading_123", "type": "reasoning", "summary": [ { "type": "summary_text", "text": "Looking through the knowledge base..." } ] }, { "id": "fs_hidoba_123", "type": "file_search_call", "status": "completed", "queries": ["have you ever had a horse?"], "results": [ { "file_id": "file_hidoba_07d6ebeddf6a421c", "filename": "Stephen Wolfram's Personal History with Animals and the Concept of Video Games for Pets", "score": 0.111, "text": "Q&A about Business, Innovation, and Managing Life with Stephen Wolfram...", "attributes": { "chunk_index": 0, "retrievers": "dense,splade" } } ] }, { "id": "rs_hidoba_rag_found_123", "type": "reasoning", "summary": [ { "type": "summary_text", "text": "Found 1 relevant documents, thinking..." } ] } ] } ``` If knowledge context runs and finds no sources, Hidoba still returns a completed `file_search_call` with `results: []` and a status message such as `Found 0 relevant documents, thinking...`. ### Source Result Fields Each `file_search_call.results[]` item may include: | Field | Type | Description | | --- | --- | --- | | `file_id` | String | Stable public source identifier for the returned result. | | `filename` | String | Human-readable source or document title. | | `score` | Number | Relevance score when available. | | `text` | String | Short source preview. Full knowledge chunks are not returned in public Responses output. | | `attributes` | Object | Safe source metadata, such as `chunk_index` and `retrievers`, when available. | The `text` field is a compact preview only. Hidoba trims surrounding whitespace, collapses repeated whitespace, and returns a short snippet of the retrieved chunk, currently up to 200 characters. ### Streaming Events For streaming `/v3/responses` calls, the same information is emitted as Responses stream events before the model answer. Typical event order: ```text response.created response.in_progress response.output_item.added # reasoning: looking through knowledge base response.output_item.done response.file_search_call.in_progress response.file_search_call.searching response.file_search_call.completed response.output_item.added # reasoning: found N documents response.output_item.done ...model output events... response.completed ``` Render `response.file_search_call.completed.item.results` as the source list. Render Hidoba `reasoning.summary[].text` as status text. Hidoba-added item IDs use identifiable prefixes such as `rs_hidoba_...` and `fs_hidoba_...`. ### Model Thinking Events Some models also emit their own reasoning or thinking output when you use the `reasoning` request field. This is separate from Hidoba's knowledge-status messages. In Responses streams, model thinking may appear as: - `response.reasoning_text.delta` - `response.reasoning_text.done` - `response.content_part.done` with `part.type: "reasoning_text"` - `response.output_item.done` for an item with `type: "reasoning"` and `content[].type: "reasoning_text"` User interfaces should treat these as reasoning/thinking text, not as final answer text. Preserve whitespace when displaying reasoning text so headings and paragraphs remain readable. ## Reasoning Use `reasoning` when the requested model supports explicit thinking controls. Messages API v3 accepts the object and applies it to the model request; it is not used for Hidoba quota logic, character rendering, or knowledge retrieval. Reasoning is separate from the answer output limit: - Chat Completions output limit: `max_tokens` or `max_completion_tokens` - Responses API output limit: `max_output_tokens` - Reasoning budget: `reasoning.max_tokens` If `reasoning` is omitted, no explicit reasoning instruction is sent and the model default applies. Turn reasoning off: ```json { "reasoning": { "effort": "none" } } ``` Use this shape to turn reasoning off. Use an effort level: ```json { "reasoning": { "effort": "low" } } ``` The commonly supported effort values are `none`, `low`, `medium`, and `high`. To let the model choose its default reasoning behavior, omit the `reasoning` field. Use a custom thinking-token budget: ```json { "reasoning": { "max_tokens": 1024 } } ``` Use this shape when you want to set an explicit reasoning-token budget. Chat Completions example with low reasoning effort: ```bash curl https://msg.hidoba.com/v3/chat/completions \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "I am comparing retrieval methods." }, { "role": "assistant", "content": "Got it. I can compare them clearly and briefly." }, { "role": "user", "content": "Explain semantic search in one concise paragraph." } ], "max_tokens": 120, "reasoning": { "effort": "low" } }' ``` Responses API example with a custom reasoning budget: ```bash curl https://msg.hidoba.com/v3/responses \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemini-2.5-flash", "input": "Compare semantic search and keyword search in three bullets.", "max_output_tokens": 180, "reasoning": { "max_tokens": 1024 } }' ``` Streaming uses the same request fields: ```json { "model": "google/gemini-2.5-flash", "messages": [ { "role": "user", "content": "I want a short answer." }, { "role": "assistant", "content": "Understood. I will be concise." }, { "role": "user", "content": "What does a sunrise usually symbolize?" } ], "stream": true, "max_tokens": 120, "reasoning": { "effort": "medium" } } ``` Reasoning support depends on the selected model. Unsupported combinations may ignore the field or return a model error. ## Errors Generation requests may fail before reaching the model if the API key, quota, character, or metadata is invalid. | Status | Meaning | | --- | --- | | `400` | Invalid request body, invalid `metadata.hidoba`, or character validation failure. | | `401` | Missing or invalid quota API key. | | `403` | Quota type or access is not allowed for this endpoint. | | `429` | Quota is exhausted. | | `5xx` | Model or service failure. | Example error: ```json { "detail": { "code": "invalid_api_key", "message": "Invalid API key", "context": { "request_id": "7ed3d6f1-1a9f-4f3b-90d3-7c681a9d9fb8", "endpoint": "/v3/chat/completions" } } } ``` Model errors are returned in an OpenAI-compatible response shape when possible. --- ## Data Format(Texting-api) :::warning Legacy API Messages API v2 is legacy and should not be used for new projects. Use [Messages API v3](../msg-api/overview.md) for new OpenAI-compatible messaging integrations. ::: ## Input Format Input has the following parts: - **Message history** - includes past (context) and new (to be processed) user messages - **Audio files** - contents of audio files referenced in message history - **Character** - AI character to use for responses (string identifier OR custom character object) - **Character params** - optional field, only applicable if character personality is a template (jinja). Note: `context` name is reserved for the RAG - **Generate audio** - optional flag indicating whether audio should be generated for the response (default: `true`) - **Num options** - optional number of response variants to generate (default: `1`, max: `12`) ```json { "input": { "messages": [...], "audio_files": { ... }, "character": "...", "character_params": { ... }, "generate_audio": true|false, "num_options": 1 } } ``` :::note Multiple Response Options When `num_options` is greater than 1, you must set `generate_audio: false`. The response will include an `options` array on the generated message containing all response variants. ::: ### Character Parameter You can specify a character in two ways: **Option 1: Character Identifier (String)** Use a character identifier string to reference a predefined character: ```json { "input": { "character": "github:partner/character", "messages": [...] } } ``` **Option 2: Custom Character (Object)** Define a custom character configuration inline: ```json { "input": { "character": { "description": "{\"name\": \"Assistant\", \"voice\": \"...\"}", "personality": "You are a helpful assistant..." }, "messages": [...] } } ``` When using a custom character: - `description` must be a JSON string containing the character configuration - `personality` must be a string containing the system prompt See the character editor app for valid configuration examples. **Legacy Support:** The `character_name` field is still supported for backwards compatibility but is deprecated in favor of `character`. ### Message History Messages should be ordered chronologically, past (context) messages followed by 1 or more new (to be processed) user messages - distinguished by a flag. All past messages should only have text, new messages might be either text or audio and should have unique `id`. #### Past Message Format ```json { "id": 1, "originator": "bot|user", "text": "", "is_processed": true } ``` #### New Message Format ```json { "id": 5, "type": "text|audio", "text": "", "audio_file_id": "", "is_processed": false } ``` ## Audio Files Audio files included in both input and output as base64-encoded. The only supported format is OGG, response is also OGG (Telegram compatible). Please note that max size for the whole payload is 5MB. ```json { "msg1": "", "msg2": "" } ``` ## Output Format Output includes all relevant messages: 1. New user messages with transcripts 2. Generated audio message with the transcript ```json { "messages": [...], "audio_files": { "response": "" } } ``` ### New User Message Format ```json { "id": "", "type": "", "text": "" } ``` ### Generated Message Format ```json { "type": "audio", "text": "", "audio_file_id": "response" } ``` ### Generated Message with Options The generated message always includes an `options` array containing response variants: ```json { "type": "text", "text": "First response option", "options": [ {"type": "text", "text": "First response option"}, {"type": "text", "text": "Second response option"}, {"type": "text", "text": "Third response option"} ] } ``` The `options` array is always present on the generated message. When `num_options` is 1 (or not specified), the array contains a single element. --- ## Examples(Texting-api) :::warning Legacy API Messages API v2 is legacy and should not be used for new projects. Use [Messages API v3](../msg-api/overview.md) for new OpenAI-compatible messaging integrations. ::: ## Complete Request/Response Example ### Input Request ```json { "input": { "messages": [ { "originator": "bot", "text": "Hello, can you hear me?", "is_processed": true }, { "originator": "user", "text": "How are you doing?", "is_processed": true }, { "originator": "bot", "text": "Great! You?", "is_processed": true }, { "id": 4, "type": "text", "originator": "user", "text": "Me too!!", "is_processed": false }, { "id": 5, "type": "text", "text": "What are you doing tonight?", "is_processed": false }, { "id": 6, "type": "audio", "text": null, "audio_file_id": "msg1", "is_processed": false } ], "audio_files": { "msg1": "" }, "character": "github:partner/character", "character_params": { "anything_you_want": "Those parameters will be used to render the character prompt template.", "user_tier": "Premium", "user_hair_color": "Blonde" }, "generate_audio": true } } ``` ### Output Response ```json { "messages": [ { "id": 4, "type": "text", "text": "Me too!!" }, { "id": 5, "type": "text", "text": "What are you doing tonight?" }, { "id": 6, "type": "audio", "text": "", "audio_file_id": "msg1" }, { "type": "audio", "text": "Getting dinner with friends, you?", "audio_file_id": "response", "options": [ {"type": "audio", "text": "Getting dinner with friends, you?"} ] } ], "audio_files": { "response": "" } } ``` ## Multiple Response Options Example Request multiple response variants using `num_options`. Audio generation must be disabled when requesting multiple options. ### Input Request ```json { "input": { "messages": [ { "originator": "user", "text": "What should we do tonight?", "is_processed": true }, { "id": 1, "type": "text", "text": "I'm feeling adventurous!", "is_processed": false } ], "character": "github:partner/character", "num_options": 3, "generate_audio": false } } ``` ### Output Response ```json { "messages": [ { "id": 1, "type": "text", "text": "I'm feeling adventurous!" }, { "type": "text", "text": "How about we go stargazing at the hilltop?", "options": [ {"type": "text", "text": "How about we go stargazing at the hilltop?"}, {"type": "text", "text": "Let's try that new escape room downtown!"}, {"type": "text", "text": "We could go for a midnight hike in the park."} ] } ], "audio_files": {} } ``` The `options` array contains all response variants. The main `text` field contains the first option for backwards compatibility. --- ## Messages API v2 (legacy) :::warning Legacy API Messages API v2 is legacy and should not be used for new projects. Use [Messages API v3](../msg-api/overview.md) for new OpenAI-compatible messaging integrations. ::: The Hidoba Research Messages API v2 enables legacy AI-powered conversational experiences through a REST API. The service receives chat history and returns both transcript and generated audio responses. ## Typical Flow 1. User responds in the chat 2. Service generates response from user message(s) and past chat history 3. Response is sent to the user :::important Important Considerations - **Concurrency:** User might send another message before response is generated - **Retries:** Service might occasionally fail to generate response ::: --- ## REST API(Texting-api) :::warning Legacy API Messages API v2 is legacy and should not be used for new projects. Use [Messages API v3](../msg-api/overview.md) for new OpenAI-compatible messaging integrations. ::: Completion is a long-running operation. Client needs to request it and then poll for the result. Please note that result is retained for only around 20 min. ## Authentication Supply API key in `X-Api-Key` header. ```bash curl -X POST https://api.example.com/v2/completions \ -H "Content-Type: application/json" \ -H "X-Api-Key: YOUR_API_KEY" \ -d '{"input": {...}}' ``` ## Requesting Completion ### Request ```http POST /v2/completions { "input": { "messages": [...], "audio_files": { ... }, "character": "...", "character_params": { ... } } } ``` ### Response ```json { "id": "/v2/completions/", "state": "in-progress" } ``` ## Checking Status ### Request ```http GET /v2/completions/ ``` ### Response ```json { "id": "/v2/completions/", "state": "", "output": ... } ``` ### Possible States - `in-progress` - Operation is in progress, check again later - `completed` - Response is generated successfully, `output` property contains the result - `failed` - Operation failed --- ## Overview(Transcript-api) The Transcript API converts audio into text. Provide an audio URL or upload a file, and receive a full transcription. ## Typical Flow 1. Submit a transcription request with an audio URL or file upload 2. Receive a request ID to track progress 3. Poll for results until the transcription completes 4. Retrieve the transcribed text ## Features - **URL or file upload**: Provide a link to hosted audio, or upload a file directly (up to 100MB) - **Asynchronous processing**: Transcription runs in the background; poll for results - **Language support**: Specify a language code or enable automatic language detection - **Filler words**: Optionally include filler words (um, uh) in the output - **Keyword boosting**: Improve accuracy for domain-specific terms :::important Important Considerations - **Supported formats**: mp3, wav, ogg, flac, m4a, mp4, webm - **File size limit**: 100MB for direct uploads - **Rate limits**: Requests are rate-limited per API key ::: --- ## REST API(Transcript-api) Transcription is a long-running operation. Submit a request and poll for the result. ## Authentication Supply your API key in the `X-Api-Key` header. ```bash curl -X POST https://api.example.com/v2/transcript \ -H "Content-Type: application/json" \ -H "X-Api-Key: YOUR_API_KEY" \ -d '{"audio_url": "https://example.com/audio.mp3"}' ``` ## Submitting a Transcript Audio can be provided as a URL or as a file upload. The two options are mutually exclusive. ### Option 1: Audio URL ```http POST /v2/transcript Content-Type: application/json ``` ```bash curl -X POST https://api.example.com/v2/transcript \ -H "Content-Type: application/json" \ -H "X-Api-Key: YOUR_API_KEY" \ -d '{ "audio_url": "https://example.com/audio.mp3", "language_code": "en", "filler_words": true, "format_text": true, "boosted_words": ["artificial intelligence", "machine learning"] }' ``` ### Option 2: File Upload ```http POST /v2/transcript Content-Type: multipart/form-data ``` ```bash curl -X POST https://api.example.com/v2/transcript \ -H "X-Api-Key: YOUR_API_KEY" \ -F "audio=@recording.mp3" ``` Optional parameters can be included as additional form fields: ```bash curl -X POST https://api.example.com/v2/transcript \ -H "X-Api-Key: YOUR_API_KEY" \ -F "audio=@recording.mp3" \ -F "language_code=en" \ -F "filler_words=true" \ -F "boosted_words[]=artificial intelligence" \ -F "boosted_words[]=machine learning" \ -F "boosted_words[]=neural network" ``` :::note Array fields in multipart requests For array parameters like `boosted_words`, use `[]` suffix notation to send multiple values: `-F "boosted_words[]=term1" -F "boosted_words[]=term2"`. ::: ### Request Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `audio_url` | String | — | URL of the audio file to transcribe. Required unless `audio` is provided. | | `audio` | File | — | Audio file upload (max 100MB). Supported: mp3, wav, ogg, flac, m4a, mp4, webm. Required unless `audio_url` is provided. | | `language_code` | String | `en_us` | Language code. See [supported languages](#supported-languages) below. | | `language_auto` | Boolean | `false` | Enable automatic language detection. | | `filler_words` | Boolean | `false` | Include filler words (um, uh) in the transcript. | | `format_text` | Boolean | `true` | Apply text formatting (capitalization, punctuation). | | `boosted_words` | Array | — | List of terms to boost recognition accuracy for. Max 100 items. | \* Provide either `audio_url` or `audio`, not both. ### Supported Languages | Code | Language | |------|----------| | `en` | English | | `en_us` | English (US) — default | | `en_uk` | English (UK) | | `en_au` | English (AU) | | `es` | Spanish | | `fr` | French | | `de` | German | | `id` | Indonesian | | `it` | Italian | | `ja` | Japanese | | `nl` | Dutch | | `pl` | Polish | | `pt` | Portuguese | | `ru` | Russian | | `tr` | Turkish | | `uk` | Ukrainian | | `ca` | Catalan | ### Response ```json { "id": "/v2/transcript/abc123-def456", "status": "queued" } ``` ## Checking Status ### Request ```http GET /v2/transcript/{id} ``` ### Response (Processing) ```json { "id": "/v2/transcript/abc123-def456", "status": "processing" } ``` ### Response (Completed) ```json { "id": "/v2/transcript/abc123-def456", "status": "completed", "text": "Hello, this is a sample transcription of the audio file.", "audio_duration": 45 } ``` ### Response (Error) ```json { "id": "/v2/transcript/abc123-def456", "status": "error", "error": "Download failed. The file is not accessible." } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `id` | String | Transcript identifier (path format). | | `status` | String | Current status: `queued`, `processing`, `completed`, or `error`. | | `text` | String | The transcribed text. Present when `status` is `completed`. | | `audio_duration` | Integer | Audio duration in seconds. Present when `status` is `completed`. | | `error` | String | Error description. Present when `status` is `error`. | ## Error Responses ### Validation Error (422) ```json { "message": "The given data was invalid.", "errors": { "audio_url": ["Either audio_url or an audio file is required."] } } ``` ### Rate Limited (429) ```json { "error": "Rate limit exceeded. Try again in 45 seconds." } ``` --- ## Overview(Tts-api) The TTS API converts text into speech audio. It exposes a V2 OpenAI-compatible speech endpoint backed by Hidoba TTS. Base endpoint: ```text POST /v2/audio/speech ``` ## Voice processor Use [hidoba voice processor](https://voices.hidoba.com/) to manage your voices and see their ID. Currently API only supports `ishikawa` model (other models currently work only in the calls). ## Typical Flow 1. Send text, model, voice, and optional audio settings. 2. The API validates quota access and calls the model. 3. The response returns raw audio bytes in the requested format. 4. Usage is recorded automatically for quota billing. ## Features - **OpenAI-compatible shape**: Use `model`, `voice`, `input`, `response_format`, and `speed`. - **Formats**: Generate `mp3`, `opus`, `wav`, or raw `pcm`. - **Language control**: Pass a supported short language code, or use `auto` to detect the language from the input text. - **Billing included**: Processed characters are billed automatically from provider usage. :::important Important Considerations - **Maximum input**: 2,000 characters per request. - **Streaming**: Not supported by this endpoint. - **Quota type**: Requires a server-time quota. ::: --- ## REST API(Tts-api) ## Authentication Send your quota API key with each request. Bearer auth is preferred. ```bash curl https://api.example.com/v2/audio/speech \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"ishikawa","voice":"Dennis","input":"Hello world."}' \ --output speech.mp3 ``` `X-Api-Key` and `X-API-Key` are also accepted. ## Create Speech ```http POST /v2/audio/speech Content-Type: application/json ``` ### Request Body | Field | Type | Required | Default | Description | | --- | --- | --- | --- | --- | | `model` | String | Yes | — | Public TTS model. Currently only `ishikawa`. | | `voice` | String | Yes | — | Voice ID, for example `Dennis`. | | `input` | String | Yes | — | Text to synthesize, 1-2,000 characters. | | `language` | String | No | `auto` | Optional language code. Use `auto` or one of the supported short codes below. | | `response_format` | String | No | `mp3` | One of `mp3`, `opus`, `wav`, `pcm`. | | `speed` | Number | No | Provider default | Speaking rate from `0.5` to `1.5`. | ### Language The API accepts short language codes, not provider-specific BCP-47 tags. For example, send `en`, not `en-US`. If `language` is omitted or set to `auto`, the API tries to detect the language from `input`. When detection is confident and the detected language is supported, the detected language is passed to the TTS provider. When detection is not confident, no language is passed and the provider default is used. Supported values: | Code | Language | | --- | --- | | `auto` | Auto-detect | | `en` | English | | `es` | Spanish | | `fr` | French | | `de` | German | | `it` | Italian | | `pt` | Portuguese | | `pl` | Polish | | `ru` | Russian | | `nl` | Dutch | | `ar` | Arabic | | `zh-cn` | Chinese (Simplified) | | `ja` | Japanese | | `ko` | Korean | | `hi` | Hindi | | `he` | Hebrew | ### Example: MP3 ```bash curl https://api.example.com/v2/audio/speech \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "ishikawa", "voice": "Dennis", "input": "Hello, this is generated speech.", "response_format": "mp3" }' \ --output speech.mp3 ``` ### Example: WAV With Language ```bash curl https://api.example.com/v2/audio/speech \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "ishikawa", "voice": "Dennis", "input": "Buongiorno, piacere di conoscerti.", "language": "it", "response_format": "wav", "speed": 1.05 }' \ --output speech.wav ``` ### Example: Auto Language Detection ```bash curl https://api.example.com/v2/audio/speech \ -H "Authorization: Bearer $HIDOBA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "ishikawa", "voice": "Dennis", "input": "Привет! Как дела у тебя сегодня?", "language": "auto", "response_format": "mp3" }' \ --output speech.mp3 ``` ## Response The response body is raw audio bytes. | Format | Content-Type | | --- | --- | | `mp3` | `audio/mpeg` | | `opus` | `audio/ogg` | | `wav` | `audio/wav` | | `pcm` | `application/octet-stream` | The response includes: ```http X-Hidoba-TTS-Request-Id: 123 ``` Use this ID when asking support to trace a TTS request. ## Errors ### Validation Error ```json { "message": "The given data was invalid.", "errors": { "model": ["The selected model is invalid."] } } ``` ### Quota Exhausted ```json { "error": "Quota exhausted", "code": "QUOTA_EXHAUSTED" } ``` ### Provider Rejection Voice validation or provider-side request problems are returned as client-facing errors: ```json { "error": "Invalid voiceId" } ``` --- ## Overview(Voice-api) The Voice API manages voice samples and generates voice embeddings for text-to-speech. Upload a voice recording, and the service automatically splits it into samples and creates embeddings via a processing pipeline. ## Typical Flow 1. Upload a voice recording to create a new voice 2. The recording is automatically split into opus-encoded samples 3. Embedding is generated from the samples via RunPod 4. Retrieve test samples and quality scores to verify the result 5. Download the embedding zip or individual samples as needed ## Features - **Automatic processing**: Upload an audio file and splitting + embedding run automatically - **Manual sample upload**: Create an empty voice and upload individual samples - **Re-splitting**: Re-process the original recording with custom parameters (sample count, duration) - **Quality scoring**: Each sample receives a similarity score after embedding - **Test samples**: TTS-generated test audio to preview voice quality - **Sample bundles**: Download all samples as a zip archive with metadata :::important Important Considerations - **Audio conversion**: All uploads are automatically converted to opus 96kbps mono - **Processing locks**: Voices cannot be modified while splitting or embedding is in progress - **Rate limits**: Requests are rate-limited per API key ::: --- ## REST API(Voice-api) All endpoints are under the `/api` prefix. Voice processing (splitting and embedding) runs asynchronously — upload a recording and poll the voice status. ## Authentication Supply your API key in the `X-Api-Key` header or as an `api_key` query parameter. ```bash curl -X GET https://voice.hidoba.com/api/voices/my-partner \ -H "X-Api-Key: YOUR_API_KEY" ``` ## Voices ### List Voices ```http GET /api/voices/{partner} ``` #### Response ```json [ { "name": "speaker-1", "has_original": true, "sample_count": 12, "splitting_status": "completed", "embedding_status": "completed", "creator_name": "John", "creator_email": "john@example.com" } ] ``` ### Create Voice (with Audio) ```http POST /api/voices/{partner} Content-Type: multipart/form-data ``` Uploading an audio file automatically triggers splitting and embedding. ```bash curl -X POST https://voice.hidoba.com/api/voices/my-partner \ -H "X-Api-Key: YOUR_API_KEY" \ -F "name=speaker-1" \ -F "file=@recording.wav" \ -F "creator_name=John" \ -F "creator_email=john@example.com" ``` #### Form Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `name` | String | — | Voice name (1–255 chars). Required. | | `file` | File | — | Audio file to upload. Optional. | | `creator_name` | String | — | Name of the person who created the voice. | | `creator_email` | String | — | Email of the creator. | ### Create Empty Voice ```http POST /api/voices/{partner}/empty ``` Creates a voice without an audio file. Use this when uploading samples manually. ```bash curl -X POST https://voice.hidoba.com/api/voices/my-partner/empty \ -H "X-Api-Key: YOUR_API_KEY" \ -F "name=speaker-2" ``` ### Get Voice Details ```http GET /api/voices/{partner}/{voice} ``` #### Response ```json { "name": "speaker-1", "original_filename": "recording.opus", "original_uploaded_name": "recording.wav", "original_duration": 185.4, "status": { "splitting": "completed", "embedding": "completed" }, "samples": [ { "index": 1, "source_timestamp": "00:00", "duration": 15.0 }, { "index": 2, "source_timestamp": "00:15", "duration": 15.0 } ], "embedding_info": { "tts_type": "kyutai", "last_update_status": "success", "last_update_time": "2025-01-15T10:30:00Z", "test_samples": [ { "sample_index": 1, "sentence_index": 1, "filename": "test_1_1.opus" } ], "sample_scores": [ { "sample_index": 3, "score": 0.92 }, { "sample_index": 1, "score": 0.88 } ] }, "embedding_type": "kyutai", "embedding_zip_url": "https://...", "preferred_sample_index": 3, "creator_name": "John", "creator_email": "john@example.com" } ``` ### Delete Voice ```http DELETE /api/voices/{partner}/{voice} ``` :::note Voices cannot be deleted while splitting or embedding is in progress. You will receive a `409 Conflict` response. ::: ### Rename Voice ```http PUT /api/voices/{partner}/{voice}/name Content-Type: multipart/form-data ``` | Parameter | Type | Description | |-----------|------|-------------| | `name` | String | New voice name. Required. | ## Samples ### Upload Sample ```http POST /api/voices/{partner}/{voice}/samples Content-Type: multipart/form-data ``` Upload a single audio sample to a voice that was created empty. The file is automatically converted to opus 96kbps mono. ```bash curl -X POST https://voice.hidoba.com/api/voices/my-partner/speaker-2/samples \ -H "X-Api-Key: YOUR_API_KEY" \ -F "file=@sample.wav" ``` ### Download Sample ```http GET /api/voices/{partner}/{voice}/samples/{index} ``` Returns the opus audio file for the sample at the given index. ### Delete Sample ```http DELETE /api/voices/{partner}/{voice}/samples/{index} ``` ### Download Sample Bundle ```http GET /api/voices/{partner}/{voice}/samples-bundle ``` Returns a ZIP archive containing all samples as `{index}.opus` files and a `voice_info.json` metadata file. ## Splitting ### Queue Splitting Job ```http POST /api/voices/{partner}/{voice}/split Content-Type: application/x-www-form-urlencoded ``` Re-splits the original recording with custom parameters. Clears existing samples. | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `max_samples` | Integer | `25` | Maximum number of samples to extract (1–100). | | `sample_duration` | Integer | `15` | Duration of each sample in seconds (5–60). | ```bash curl -X POST https://voice.hidoba.com/api/voices/my-partner/speaker-1/split \ -H "X-Api-Key: YOUR_API_KEY" \ -d "max_samples=30&sample_duration=10" ``` ## Embedding ### Queue Embedding Job ```http POST /api/voices/{partner}/{voice}/embed ``` Submits the voice samples for embedding generation via RunPod. | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `tts_type` | String | — | TTS type override. If omitted, uses the voice's `embedding_type` (default `kyutai`). | ### Get Test Samples ```http GET /api/voices/{partner}/{voice}/test-samples ``` Returns a list of TTS-generated test audio files created during the embedding process. | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `tts_type` | String (query) | `kyutai` | Filter test samples by TTS type. | ### Download Test Sample ```http GET /api/voices/{partner}/{voice}/test-samples/{filename} ``` Returns the test sample audio file. | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `tts_type` | String (query) | `kyutai` | TTS type folder to look in. | ### Delete Test Files ```http DELETE /api/voices/{partner}/{voice}/test-files/{tts_type} ``` ### Set Preferred Sample ```http PUT /api/voices/{partner}/{voice}/preferred-sample ``` Sets the preferred sample index based on quality scores or manual selection. | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `sample_index` | Integer (form) | — | Sample index to set as preferred. Pass empty to clear. | ## Voice Status A voice tracks two independent processing states: | Status | Values | Description | |--------|--------|-------------| | `splitting` | `idle`, `in_progress`, `completed`, `failed` | Audio splitting into samples. | | `embedding` | `idle`, `in_progress`, `completed`, `failed` | Voice embedding generation. | When either status is `in_progress`, the voice is locked — rename and delete operations will return `409 Conflict`. ## Error Responses ### Unauthorized (401) ```json { "detail": "Invalid API key" } ``` ### Forbidden (403) ```json { "detail": "Access denied" } ``` ### Not Found (404) ```json { "detail": "Voice not found" } ``` ### Conflict (409) ```json { "detail": "Voice is currently being processed" } ``` ### Rate Limited (429) ```json { "detail": "Rate limit exceeded" } ``` --- ## Backend API Reference This page documents the server-side REST API for creating voice calls. Use this API from your backend server to implement the [Server-Side Initiation](./index.md#server-side-initiation-recommended-for-production) flow. ## Overview The Backend API allows your server to create calls on behalf of your users and check their status, returning a signed URL that the client can use to connect securely without exposing your API key. ## Authentication All API requests require authentication using your API key in the `X-Api-Key` header. ```http X-Api-Key: your-api-key-here ``` ## Create Call Creates a new voice call and returns connection credentials. **Important:** You only need to make **one POST request** to create a call. The `signedUrl` returned in the response is immediately usable by the client-side SDK. ### Endpoint ```http POST /v2/calls ``` ### Headers | Header | Value | Required | |--------|-------|----------| | `Content-Type` | `application/x-www-form-urlencoded``multipart/form-data``application/json` | Yes | | `X-Api-Key` | Your API key | Yes | ### Request Body The API accepts URL-encoded form data, multipart form data, or JSON. When using form data (URL-encoded or multipart) and a parameter is an object, encode it as a JSON string. Parameters: | Parameter | Type | Description | Required | Example | |-----------|------|-------------|----------|---------| | `character` | String or Object | Character identifier (e.g., `"github:partner/character"`) **OR** custom character object | Yes | See below | | `call_time_limit_seconds` | String | Maximum call duration in seconds | Yes | `"3600"` | | `on_completion_webhook` | String | URL to receive webhook when call completes | No | `"https://your-domain.com/webhook"` | | `character_params` | Object | Dictionary of parameters that will be used in template rendering in case if the character is created in Character Editor ("github:..."). Same as in [Messages API v2 (legacy)](/texting-api/rest-api/). | No | {} | #### Character Parameter You can specify a character in two ways: **Option 1: Character Identifier (String)** ``` github:partner/character ``` **Option 2: Custom Character (Object)** ```json { "description": "{\"name\": \"Assistant\", ...}", "personality": "You are a helpful assistant..." } ``` When using a custom character: - `description` must be a JSON string containing the character configuration - `personality` must be a string containing the system prompt See the character editor app for valid configuration examples ### Response Format Returns the same format as the [Get Call Status](#get-call-status) response. The call will initially be in `"starting"` state. ## Get Call Status Retrieve the current status of an existing call. ### Endpoint ```http GET /v2/calls/{call_id} ``` ### Headers | Header | Value | Required | |--------|-------|----------| | `X-Api-Key` | Your API key | Yes | ### Response Format The API returns different response formats depending on the call state: #### Example: Starting Call ```json { "data": { "id": "/v2/calls/3267", "character": "github:partner/character", "state": "starting", "available_endpoints": [], "duration": { "call_seconds": null, "server_runtime_seconds": null }, "signedUrl": "https://backend.funtimewithaisolutions.com/v2/calls/3267?signature=xyz", "created_at": "2025-09-24T22:36:43.000000Z", "updated_at": "2025-09-24T22:36:43.000000Z" } } ``` #### Example: Ongoing Call ```json { "data": { "id": "/v2/calls/3267", "character": "github:partner/character", "state": "ongoing", "available_endpoints": [...], "duration": { "call_seconds": null, "server_runtime_seconds": null }, "signedUrl": "https://backend.funtimewithaisolutions.com/v2/calls/3267?signature=xyz", "created_at": "2025-09-24T22:36:43.000000Z", "updated_at": "2025-09-24T22:36:45.000000Z" } } ``` #### Example: Completed Call ```json { "data": { "id": "/v2/calls/3267", "character": "github:partner/character", "state": "completed", "available_endpoints": [], "duration": { "call_seconds": 4, "server_runtime_seconds": 6 }, "signedUrl": "https://backend.funtimewithaisolutions.com/v2/calls/3267?signature=xyz", "created_at": "2025-09-24T22:36:43.000000Z", "updated_at": "2025-09-24T22:36:55.000000Z" } } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `id` | String | Unique call identifier | | `character` | String | Character used for the call (`null` if character is custom) | | `state` | String | Call state: `"starting"`, `"ongoing"`, `"completed"`, or `"error"` | | `available_endpoints` | Array | **SDK-only field:** WebSocket endpoints for connection (populated when state is `"ongoing"`) | | `duration.call_seconds` | Number/null | Actual conversation duration in seconds (available when completed) | | `duration.server_runtime_seconds` | Number/null | Total server runtime in seconds (available when completed) | | `signedUrl` | String | **Most Important:** Signed URL for client-side SDK connection | | `created_at` | String | Call creation timestamp | | `updated_at` | String | Last update timestamp | ## Get Call Transcript Retrieve the conversation transcript for a completed call. ### Endpoint ```http GET /v2/calls/{call_id}/transcript ``` ### Headers | Header | Value | Required | |--------|-------|----------| | `X-Api-Key` | Your API key | Yes | ### Response Format #### Example: Transcript Available ```json { "data": { "available": true, "transcript": [ { "text": "Hello, how can I help you today?", "originator": "bot" }, { "text": "I have a question about your service.", "originator": "user" }, { "text": "Of course! I'd be happy to help. What would you like to know?", "originator": "bot" } ] } } ``` #### Example: Transcript Not Available ```json { "data": { "available": false } } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `available` | Boolean | Whether transcript data is available | | `transcript` | Array | Array of messages (only present when `available` is `true`) | | `transcript[].text` | String | The message content | | `transcript[].originator` | String | Who sent the message: `"bot"` or `"user"` | ### Notes - Transcripts are only available after a call has completed. - If a call ended unexpectedly or no conversation occurred, `available` will be `false`. ## Webhooks Webhooks allow you to receive notifications when calls complete. Include the `on_completion_webhook` parameter when creating a call to receive a webhook notification upon completion. ### Behavior When a call completes, a POST request is sent to your webhook URL: - **Payload:** The payload is the same as the [Get Call Status](#get-call-status) response for a completed call. - **HTTPS Required:** Webhook URLs must use HTTPS with valid SSL certificates. - **Response Code:** Your endpoint must return HTTP 200 for successful processing. - **Timeout:** Webhook requests timeout after 30 seconds. - **Retries:** Failed webhooks are retried up to 5 times with exponential backoff. Please note that you're responsible for authentication of the webhook. ## Call Time Limits Call time limits automatically end calls after a specified duration. Include the `call_time_limit_seconds` parameter when creating a call to set a maximum duration in seconds. ### Behavior Calls are automatically terminated when the time limit is reached. ## Error Handling ### HTTP Status Codes | Status | Description | |--------|-------------| | `200` | Success | | `400` | Bad Request - Invalid parameters | | `401` | Unauthorized - Invalid or missing API key | | `429` | Too Many Requests - Rate limit exceeded | | `500` | Internal Server Error | | `503` | Service Unavailable | ### Error Response Format ```json { "error": { "code": "INVALID_CHARACTER", "message": "Character not found or not accessible" } } ``` ## Next Steps - **Client Integration:** See [SDK Reference](../sdk/reference.md) for using the returned `signedUrl`. - **Complete Example:** Check [Quick Start](../quick-start.mdx) for end-to-end implementation. --- ## Call Initiation Hidoba Research Voice Calls SDK supports two distinct methods for initiating calls. Both methods use the same API key authentication, but differ in where and how the call is created. ## Overview | Method | Call Creation | API Key Location | Control Level | |--------|---------------|------------------|---------------| | **Client-Side Initiation** | Browser creates call directly | Client-side code | Direct SDK control | | **Server-Side Initiation** | Your server creates call, returns signed URL | Server-side (secure) | Backend-mediated control | ## Client-Side Initiation The browser directly communicates with Hidoba Research's backend to create and establish the call. ```mermaid sequenceDiagram participant User participant Browser as Browser/SDK participant HidobaAPI as Hidoba Research API User->>Browser: Click "Call Now" Browser->>HidobaAPI: POST /v2/calls (with API key) HidobaAPI-->>Browser: Call created, endpoints available Browser->>HidobaAPI: WebSocket connection HidobaAPI-->>Browser: WebSocket established Note over User, HidobaAPI: Live voice conversation User->>Browser: End call Browser->>HidobaAPI: Disconnect ``` **Characteristics:** - Direct connection: Browser → Hidoba Research API → WebSocket - Single-step process from user action to call - SDK handles call creation, authentication, and connection **Best for:** - Rapid development and prototyping - Simple applications without backend complexity - When you want the SDK to handle the entire flow **Security considerations:** - API key must be present in client-side code - Users can inspect and potentially extract the API key - No server-side control over call initiation ## Server-Side Initiation (Recommended for Production) Your backend server creates the call and provides the client with a signed URL for connection. ```mermaid sequenceDiagram participant User participant Frontend as Frontend/SDK participant Backend as Your Backend participant HidobaAPI as Hidoba Research API User->>Frontend: Click "Call Now" Frontend->>Backend: Request call creation Backend->>HidobaAPI: POST /v2/calls (with API key) HidobaAPI-->>Backend: signedUrl + call details Backend-->>Frontend: Return signedUrl Frontend->>HidobaAPI: Connect with signedUrl HidobaAPI-->>Frontend: WebSocket established Note over User, HidobaAPI: Live voice conversation User->>Frontend: End call Frontend->>HidobaAPI: Disconnect ``` **Characteristics:** - Mediated connection: Browser → Your Server → Hidoba Research API → Signed URL → Browser → WebSocket - Two-step process: server creates call, client connects using provided credentials - Your server manages call creation and can implement business logic **Best for:** - Production applications requiring security and control - User authentication and management requirements - Integration with existing backend systems - Rate limiting, usage tracking, and analytics - Compliance or security policies requiring API key protection **Security benefits:** - API key remains secure on your server - You control who can initiate calls - Call metadata and parameters managed server-side - Integration with your existing authentication system ## Key Differences ### Call Creation Process **Client-Side:** User action → SDK creates call → WebSocket connection → Conversation **Server-Side:** User action → Frontend requests call → Server creates call → Server returns signed URL → SDK connects → WebSocket connection → Conversation ### API Key Usage - **Client-Side:** API key used directly by the SDK in the browser - **Server-Side:** API key used by your server; client never sees it ### Control and Flexibility - **Client-Side:** Direct SDK control, immediate initiation, limited business logic - **Server-Side:** Full backend control, authentication, rate limiting, logging, analytics ## Implementation Details For specific implementation instructions: - **Client-Side Setup:** See [Initialization](../sdk/setup.md) - **Server-Side Setup:** See [Backend API Reference](./backend-api.md) - **SDK Usage:** See [SDK Reference](../sdk/reference.md) --- ## Overview(Voice-calls) Hidoba Research SDK allows you to integrate AI-powered voice conversations into your web applications. The SDK handles all the complex audio processing, streaming, and communication with our backend services. ## Key Features - **Real-time Voice Conversations** - Direct voice chat with AI characters - **Automatic Audio Processing** - Built-in echo cancellation, noise reduction, and audio optimization - **Device Management** - Switch microphones and speakers during calls - **RAG Integration** - AI can reference external documents and show sources - **Conversation History** - Optional real-time transcript display - **Cross-Browser Support** - Works in all modern browsers ## Browser Requirements - Modern browser with WebRTC support (Chrome 66+, Firefox 60+, Safari 11+) - HTTPS connection (required for microphone access) - AudioWorklet support for optimal audio processing --- ## Quick start import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # Quick start ## Installation Include the required SDK scripts in your HTML file: ```html ``` If you need conversation history support, include the history.js script as well: ```html ``` ## Choose Your Authentication Method Before implementing your first call, you need to choose between two authentication methods. See [Call Initiation Methods](./call-initiation/index.md) for detailed comparison. - **Client-Side Initiation:** Quick setup, API key in browser (suitable for development/prototyping) - **Server-Side Initiation:** Secure setup, API key on server (recommended for production) Here's a complete example using client-side authentication: ```html Voice Call - Client-Side Status: Ready ``` Here's a complete example using server-side authentication: ```html Voice Call - Server-Side Status: Ready ``` ## Next Steps ### For Client-Side Implementation: - Replace `YOUR_API_KEY`, `PARTNER`, and `CHARACTER` with your actual credentials - See [SDK Setup](./sdk/setup.md) for credential setup and configuration ### For Server-Side Implementation: - Implement server-side endpoint using [Backend API Reference](./call-initiation/backend-api.md) - See [Call Initiation Methods](./call-initiation/index.md) for security considerations ### Additional Features: - **Device Management:** See [SDK Reference](./sdk/reference.md#device-management) for microphone/speaker selection - **Conversation History:** See [Features](./sdk/features.md#conversation-history) for real-time transcript display - **Advanced Configuration:** See [SDK Setup](./sdk/setup.md) for detailed setup options --- ## Core Concepts ## Call Initiation Responsibility The SDK supports two approaches for call initiation, differing in who is responsible for creating the call: ### Client-Side Initiation - **Responsibility:** The SDK directly creates calls with Hidoba Research - **Process:** SDK uses your API key to create and manage the entire call lifecycle - **Use Case:** Development, prototyping, simple applications ### Server-Side Initiation - **Responsibility:** Your backend creates calls, SDK connects using provided credentials - **Process:** Your server creates the call and returns a signed URL for the SDK to use - **Use Case:** Production applications, enhanced security, user management ## Call Flow Understanding the typical flow of an AI voice call helps you implement proper event handling: 1. **Call Initiation:** User clicks call button → `onCallStart` triggered 2. **Call Creation:** Either SDK creates call directly (client-side) or your backend creates call (server-side) 3. **Permission Request:** Browser requests microphone access automatically 4. **Permission Granted:** `onCallStarting` callback triggered 5. **WebSocket Connection:** SDK establishes audio streaming connection 6. **Connected:** `onConnected` callback triggered → conversation can begin 7. **Active Conversation:** Real-time audio streaming and optional transcript display 8. **Call End:** User hangs up → `onHangUp` callback triggered ## Callback Architecture The SDK uses an event-driven architecture with callbacks to handle different states: - **Status Callbacks:** Keep your UI updated with current call state - **Error Handling:** Gracefully handle connection issues and permissions - **RAG Integration:** Display external document sources when AI references them - **User Feedback:** Show connection progress and call quality information ## Permission Management Microphone access is handled automatically by the SDK: - **Automatic Request:** Permission requested immediately after backend call creation - **Early Optimization:** Stream acquired during backend polling to reduce perceived latency - **No Manual Setup:** No need to request permissions separately - SDK handles it - **HTTPS Required:** Browser security requires secure context for microphone access ## Audio Processing The SDK includes advanced audio processing capabilities: - **Real-time Processing:** Uses AudioWorklet for low-latency audio handling - **Automatic Optimization:** Built-in echo cancellation and noise reduction - **Device Flexibility:** Switch microphones and speakers during active calls - **Quality Control:** Automatic audio format optimization for best performance --- ## Deployment Guide ## Development Setup For local development, ensure you're serving files over HTTPS or using localhost: ### Local HTTPS Server ```bash # Using Python (recommended for testing) python -m http.server 8000 --bind localhost # Using Node.js http-server npx http-server -a localhost -p 8000 # Using VS Code Live Server extension # Right-click HTML file → "Open with Live Server" ``` ## Production Deployment ### HTTPS Requirements - **Microphone Access:** Requires HTTPS in production - **AudioWorklet:** Requires secure context for optimal audio processing - **WebRTC:** Enhanced features available over HTTPS ### CDN Integration The SDK scripts are served from our CDN for optimal performance: ```html ``` ## Security Considerations :::warning API Key Security - Never expose API keys in client-side code for production - Consider using your backend service for API calls - Implement rate limiting and usage monitoring - Rotate keys regularly ::: ## Performance Optimization - **Preload Scripts:** Use `rel="preload"` for faster loading - **Connection Pooling:** SDK handles WebSocket connection optimization - **Error Handling:** Implement comprehensive error handling for production - **Monitoring:** Set up logging for call success rates and error patterns --- ## Implementation Examples We provide reference implementations demonstrating different integration patterns: ## Basic Implementation Minimal setup with essential call functionality: [View Basic Example](https://sdk.hidoba.com/sdk_test.html) - Call and hang up buttons - Status display - Device selection - Mute functionality ## Advanced Implementation Full-featured implementation with conversation history: [View Advanced Example](https://sdk.hidoba.com/sdk_test_history.html) - Real-time conversation transcript - RAG document display - Audio waveform visualization - Complete device management ## Common Implementation Patterns ### Mute Button Implementation ```html ``` ### Device Selector Implementation ```html ``` ## Testing with URL Parameters Our examples support URL parameters for easy testing: `sdk_test.html?partner=github:partner&character=character&apiKey=YOUR_API_KEY` --- ## Features ## Audio Device Management The SDK provides comprehensive device management capabilities for handling microphones and speakers. **Note:** Microphone permission is automatically requested when you start a call - no manual permission request is needed. For complete API documentation, see [Device Management](./reference#device-management) in the API Reference section. :::tip Quick Reference All SDK methods are documented in the [API Reference](./reference.md) section. This Features section focuses on practical implementation patterns and examples. ::: ### Device Selection Workflow This example shows the proper flow: query devices first, then optionally set preferences: ```javascript // Step 1: Query available devices const devices = await CallManager.getAvailableDevices(); console.log('Available microphones:', devices.microphones); console.log('Available speakers:', devices.speakers); // Step 2: Let user choose or find preferred devices const preferredMic = devices.microphones.find(m => m.label.includes('USB')); const preferredSpeaker = devices.speakers.find(s => s.label.includes('Headphones')); // Step 3: Create CallManager with selected devices (or null for defaults) const config = { microphoneId: preferredMic?.deviceId || null, // null = system default speakerId: preferredSpeaker?.deviceId || null }; const callManager = new CallManager(callbacks, config); // Step 4: Make the call await callManager.handleCall2(PARTNER, CHARACTER, API_KEY); // Step 5: Change devices during call if needed // await callManager.changeDevices(newMicId, newSpeakerId); ``` ### Device Persistence Example Save user device preferences and restore them on next session: ```javascript // Function to save device preferences function saveDevicePreferences(microphoneId, speakerId) { const preferences = { microphoneId: microphoneId, speakerId: speakerId, timestamp: Date.now() }; localStorage.setItem('audioDevicePrefs', JSON.stringify(preferences)); } // Function to load and validate device preferences async function loadDevicePreferences() { try { const saved = localStorage.getItem('audioDevicePrefs'); if (!saved) return { microphoneId: null, speakerId: null }; const preferences = JSON.parse(saved); const devices = await CallManager.getAvailableDevices(); // Validate that saved devices still exist const validMic = preferences.microphoneId && devices.microphones.find(m => m.deviceId === preferences.microphoneId); const validSpeaker = preferences.speakerId && devices.speakers.find(s => s.deviceId === preferences.speakerId); return { microphoneId: validMic ? preferences.microphoneId : null, speakerId: validSpeaker ? preferences.speakerId : null }; } catch (error) { console.warn('Error loading device preferences:', error); return { microphoneId: null, speakerId: null }; } } // Usage: Initialize CallManager with saved preferences async function initializeWithSavedDevices() { const preferences = await loadDevicePreferences(); const config = { microphoneId: preferences.microphoneId, speakerId: preferences.speakerId }; const callManager = new CallManager(callbacks, config); // When user changes devices, save the new preference // saveDevicePreferences(newMicId, newSpeakerId); return callManager; } ``` ## Conversation History The SDK includes an optional conversation history feature that displays the ongoing dialogue between the user and AI character. ### Implementation Requirements #### 1. Include the History Script In addition to the core SDK scripts, you must include the history.js file: ```html ``` #### 2. Create Required HTML Structure The conversation history requires a specific HTML structure to function correctly: ```html ``` These element IDs and class names are required for the history feature to work properly. #### 3. Apply Basic Styling Add appropriate CSS styling for the conversation container: ```css .container { display: flex; flex-direction: column; height: 80%; margin: 0 2em; } .scroll-box { flex-grow: 1; display: flex; overflow: hidden; } .scroll-box-middle { width: 45em; overflow-y: auto; border: 2px solid #ccc; padding: 15px; } #conversation-inner>pre { white-space: pre-wrap; word-wrap: break-word; } .role { color: #D55; white-space: nowrap; } ``` Once implemented, the conversation history will automatically display and scroll as the conversation with the AI character progresses. :::note RAG Integration If your character uses RAG (Retrieval Augmented Generation), the conversation history will automatically show when the AI references external documents. Use the `onRagInfo` callback to display these sources to users. ::: For a complete implementation example, refer to the [Advanced Example](./examples.md). --- ## Reference This page documents the client-side JavaScript SDK for voice calls. The SDK supports both [Client-Side and Server-Side Initiation](../call-initiation/index.md) methods. ## Call Management ### handleCall2(partner, character, credential, customMetadata) Initiates a call to the AI character. The method supports both authentication flows: #### Client-Side Initiation For direct client-side calls using an API key: | Parameter | Type | Description | Example | |-----------|------|-------------|---------| | partner | String | Partner identifier (same as your login id in [Character Editor](https://editor.funtimewithaisolutions.com)) | `"github:partner"` | | character | String | Name of the AI character created in the [Character Editor](https://editor.funtimewithaisolutions.com) | `"character"` | | credential | String | Your API key | `"your-api-key"` | | customMetadata | Object | Custom metadata for statistics collection | `{ userType: "premium", userName: "James"}` | Example: ```javascript await callManager.handleCall2("github:partner", "character", "your-api-key", { userType: "premium", userName: "James" }); ``` #### Server-Side Initiation (Recommended) For secure calls using a signed URL from your backend: | Parameter | Type | Description | Example | |-----------|------|-------------|---------| | partner | null | Not used with signed URLs | `null` | | character | null | Not used with signed URLs | `null` | | credential | String | Signed URL from your backend | `"https://backend.funtimewithaisolutions.com/v2/calls/3267?signature=..."` | | customMetadata | Object | Custom metadata for statistics collection | `{ userType: "premium", userName: "James"}` | Your application should initiate calls through your backend and provide the SDK with the returned signed URL: ```javascript // signedUrl obtained from your backend await callManager.handleCall2(null, null, signedUrl, { userType: "premium", userName: "James" }); ``` ### hangUp() Ends the current call. ```javascript callManager.hangUp(); ``` ### mute(isMuted) Mutes or unmutes the microphone during an active call. | Parameter | Type | Description | |-----------|------|-------------| | isMuted | Boolean | true to mute the microphone (sends silence), false to unmute | Example: ```javascript // During a call, mute the microphone callManager.mute(true); // Unmute to resume talking callManager.mute(false); ``` ## Device Management ### CallManager.getAvailableDevices() _(Static Method)_ Returns a list of available audio input and output devices. | Returns | Type | Description | |---------|------|-------------| | Promise<Object> | Object | Object with `microphones` and `speakers` arrays | Each device object contains: - `deviceId` - Unique identifier for the device - `label` - Human-readable device name - `isDefault` - Whether it's the system default device Example: ```javascript // Get all available audio devices const devices = await CallManager.getAvailableDevices(); console.log("Microphones:", devices.microphones); console.log("Speakers:", devices.speakers); ``` ### changeDevices(microphoneId, speakerId) Changes audio input/output devices during an active call. | Parameter | Type | Description | |-----------|------|-------------| | microphoneId | String or null | Device ID of the microphone to switch to. Pass null to keep current. | | speakerId | String or null | Device ID of the speaker to switch to. Pass null to keep current. | Example: ```javascript // Change both devices await callManager.changeDevices(microphoneId, speakerId); // Change only microphone await callManager.changeDevices(microphoneId, null); // Change only speaker await callManager.changeDevices(null, speakerId); ``` ## Callback Functions The SDK uses a callback-based approach to handle different events. You may use them in order to have a visual feedback / change UI based on the call events. Define these in the callbacks object when initializing CallManager: **Required vs Optional Callbacks:** - **Required callbacks** are called directly by the SDK and must be implemented to prevent errors - **Optional callbacks** are checked before calling - the SDK will safely skip them if not provided - You can provide empty functions for required callbacks if you don't need the functionality: `onCallStart: () => {}` | Callback | Parameters | Required | Description | |----------|------------|----------|-------------| | onCallStart | None | **Required** | Triggered when a call initiation begins (at the start of handleCall2) | | onCallError | None | **Required** | Triggered when there's an error starting or during a call | | onHangUp | None | **Required** | Triggered when a call ends | | onStatusUpdate | status (String) | **Required** | Triggered when the call status changes | | onCallStarting | None | Optional | Triggered after microphone permission is granted but before WebSocket connection begins | | onConnected | None | Optional | Triggered when WebSocket connection is successfully established | | onUserInfo | info (String) | Optional | Triggered when there's new user information | | onRagInfo | ragData (Object) | Optional | Triggered when the AI uses RAG (Retrieval Augmented Generation) to fetch relevant documents. Contains metadata and array of documents used for context. | ### RAG Data Structure The `ragData` object contains the following fields: | Field | Type | Description | |-------|------|-------------| | documents | Array | Array of document objects retrieved by RAG search | | count | Number | Total number of documents (optional, defaults to documents.length) | | total_tokens | Number | Total tokens across all retrieved documents | ### Document Object Structure Each document in the `documents` array contains: | Field | Type | Description | |-------|------|-------------| | title | String | Document title or identifier | | fusion_score | Number | Relevance score from RAG fusion algorithm | | token_count | Number | Number of tokens in this document | Example handling RAG information: ```javascript const callbacks = { // ... other required callbacks (onCallStart, onCallError, onHangUp, onStatusUpdate) ... onRagInfo: (ragData) => { // Optional callback console.log(`Found ${ragData.count || ragData.documents.length} documents (${ragData.total_tokens || 0} tokens)`); // Display each document with its metadata ragData.documents.forEach((doc, index) => { console.log(`${index + 1}. ${doc.title}`); console.log(` Score: ${doc.fusion_score || 'N/A'}, Tokens: ${doc.token_count}`); }); // Example: Show RAG sources in your UI const ragPanel = document.getElementById('ragPanel'); if (ragData.documents.length > 0) { ragPanel.style.display = 'block'; ragPanel.innerHTML = ` Sources Used (${ragData.count || ragData.documents.length}) ${ragData.documents.map((doc, i) => ` ${i + 1}. ${doc.title} Score: ${doc.fusion_score || 'N/A'} | ${doc.token_count} tokens `).join('')} `; } } }; ``` ## Initialization ### CallManager Constructor ```javascript const callManager = new CallManager(callbacks, config); ``` | Parameter | Type | Description | |-----------|------|-------------| | callbacks | Object | **Required** - Object containing callback functions | | config | Object | Optional configuration object | #### Configuration Options | Option | Type | Description | Default | |--------|------|-------------|---------| | characterRole | String | Display name for the AI character in conversation history | Character name | | personRole | String | Display name for the user in conversation history | "You" | | microphoneId | String | Preferred microphone device ID to use | System default | | speakerId | String | Preferred speaker device ID to use | System default | Example: ```javascript const callbacks = { onCallStart: () => console.log("Call starting..."), onStatusUpdate: (status) => console.log(status), onCallError: () => console.log("Call error"), onHangUp: () => console.log("Call ended"), onConnected: () => console.log("Connected!") }; const config = { characterRole: "Assistant", personRole: "User", microphoneId: "device123", speakerId: "speaker456" }; const callManager = new CallManager(callbacks, config); ``` ## Next Steps - **Authentication Methods:** See [Call Initiation Methods](../call-initiation/index.md) to choose between client-side and server-side flows - **Backend Integration:** See [Backend API Reference](../call-initiation/backend-api.md) for server-side implementation - **Getting Started:** Check [Quick Start](../quick-start.mdx) for complete examples --- ## Setup This guide covers everything you need to configure and initialize the Hidoba Research Voice Calls SDK. ## Required Credentials Before using the SDK, you need to obtain these credentials from Hidoba Research: ### Step-by-Step Setup **1. Obtain Your Credentials** Contact Hidoba Research to receive: - **Partner ID:** Your unique identifier (e.g., "github:partner") - **API Key:** Your API key for authentication **2. Create AI Characters** Visit the [Character Editor](https://editor.funtimewithaisolutions.com) to create and customize your AI characters. You can define personality traits, conversation styles, and more. Your Partner ID is your login for the Character Editor. **3. Configure Your Application** Set up your constants and initialize the CallManager as shown below. :::warning Security Note Your API Key is tied to your account and usage limits. Keep it secure and don't share it publicly. ::: ## SDK Configuration Parameters Configure your SDK with these required parameters: | Parameter | Type | Description | Example | |-----------|------|-------------|---------| | BACKEND_URL | String | Backend URL connected to your account (provided by Hidoba Research) | "https://backend.funtimewithaisolutions.com" | ## SDK Initialization To initialize the SDK, you need to set up your configuration constants and create a CallManager instance: ```javascript // Set up your configuration constants const BACKEND_URL = 'https://backend.funtimewithaisolutions.com'; // Define your callback functions const callbacks = { onCallStart: () => { /* Handle call start */ }, // Required onCallError: () => { /* Handle call errors */ }, // Required onHangUp: () => { /* Handle hang up event */ }, // Required onStatusUpdate: (status) => { /* Handle status updates */ }, // Required onCallStarting: () => { /* Handle call starting (after permissions) */ }, // Optional onConnected: () => { /* Handle successful connection */ }, // Optional onUserInfo: (info) => { /* Handle user info updates */ }, // Optional onRagInfo: (ragData) => { /* Handle RAG document info */ } // Optional }; // Create the CallManager instance const callManager = new CallManager(callbacks); ``` ## Advanced Configuration Options The CallManager constructor accepts an optional `config` object with these settings: | Option | Type | Description | Default | |--------|------|-------------|---------| | characterRole | String | Display name for the AI character in conversation history | Character name | | personRole | String | Display name for the user in conversation history | "You" | | microphoneId | String or null | Device ID from getAvailableDevices() or saved user preference. Use null for system default | null (system default) | | speakerId | String or null | Device ID from getAvailableDevices() or saved user preference. Use null for system default | null (system default) | ### Basic Configuration Example ```javascript // Basic usage with display names only const config = { characterRole: "Assistant", personRole: "User" }; const callManager = new CallManager(callbacks, config); ``` ### Advanced Configuration Example ```javascript // Advanced usage with device selection const config = { characterRole: "Assistant", personRole: "User", microphoneId: "device123", // Specific microphone device ID speakerId: "speaker456" // Specific speaker device ID }; const callManager = new CallManager(callbacks, config); ``` ## UI Implementation Requirements After initializing the SDK, you need to implement these essential UI components: ### Call Control Buttons Your application needs UI elements to trigger call start and end actions (such as buttons or other interactive elements). These can be implemented as separate buttons or as a single button that changes its function based on call state. Example implementation: ```html ``` ### Status Display Your application should include an element to display call status and user information: ```html Status: Not connected User Info: ``` These UI elements are essential for proper functionality and user experience. ## Next Steps - **Call Initiation Methods:** See [Call Initiation Methods](../call-initiation/index.md) to choose between client-side and server-side flows - **SDK Reference:** See [SDK Reference](./reference.md) for complete API documentation - **Getting Started:** Check [Quick Start](../quick-start.mdx) for complete examples - **Device Management:** See [SDK Reference](./reference#device-management) for microphone/speaker selection --- ## Troubleshooting ## Common Issues ### Connection Issues - **Call fails to connect** - Verify API credentials, internet connection, and backend URL - **WebSocket connection timeout** - Check firewall settings and network stability - **Unexpected disconnections** - Usually caused by poor network conditions or browser tab backgrounding ### Audio Issues - **Microphone access denied** - Ensure HTTPS context and user has granted permissions - **No sound output** - Check device connections, browser audio settings, and system volume - **Echo or feedback** - SDK includes echo cancellation; use headphones for best results - **Audio cutting out** - May indicate network issues or browser resource constraints ### Device Management - **Device list empty** - Ensure microphone permission granted before calling getAvailableDevices() - **Device switching fails** - Verify device is still connected and permission is granted - **Device labels show "Unknown"** - Grant microphone permission first for device labels to appear ## Error Messages | Error | Cause | Solution | |-------|-------|----------| | "AudioWorklet requires secure context" | Page loaded over HTTP in production | Use HTTPS or localhost for development | | "getUserMedia not supported" | Outdated browser or insecure context | Update browser and ensure HTTPS | | "WebSocket connection failed" | Network issues or invalid endpoint | Check network and verify backend URL | ## Browser Compatibility - **Chrome 66+** - Full support, recommended - **Firefox 60+** - Full support - **Safari 11+** - Supported, some limitations on older versions - **Edge 79+** - Full support (Chromium-based) ## Debug Tips - **Console Logs:** Check browser developer console for detailed error messages - **Network Tab:** Monitor WebSocket connections and HTTP requests - **Callback Logging:** Add console.log to all callbacks to track event flow - **Permission Status:** Check browser's site settings for microphone permissions