> ## Documentation Index > Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Update Storage Connection > Update connection metadata or credentials. Allows partial updates to connection metadata without changing credentials. Credentials can be updated via provider_config. All changes are logged in audit trail. **What You Can Update:** - Connection name and description - Metadata tags - Status (active/suspended) - Provider credentials (via provider_config) **Example:** ```bash curl -X PATCH "http://localhost:8000/v1/organizations/connections/conn_abc123" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "Updated Drive Name", "status": "suspended" }' ``` ## OpenAPI ````yaml patch /v1/organizations/connections/{connection_identifier} openapi: 3.1.0 info: title: Mixpeek API description: >- This is the Mixpeek API, providing access to various endpoints for data processing and retrieval. termsOfService: https://mixpeek.com/terms contact: name: Mixpeek Support url: https://mixpeek.com/contact email: info@mixpeek.com version: '0.82' servers: - url: https://api.mixpeek.com description: Production security: [] paths: /v1/organizations/connections/{connection_identifier}: patch: tags: - Organization Connections summary: Update Storage Connection description: >- Update connection metadata or credentials. Allows partial updates to connection metadata without changing credentials. Credentials can be updated via provider_config. All changes are logged in audit trail. **What You Can Update:** - Connection name and description - Metadata tags - Status (active/suspended) - Provider credentials (via provider_config) **Example:** ```bash curl -X PATCH "http://localhost:8000/v1/organizations/connections/conn_abc123" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "Updated Drive Name", "status": "suspended" }' ``` operationId: >- update_storage_connection_v1_organizations_connections__connection_identifier__patch parameters: - name: connection_identifier in: path required: true schema: type: string description: >- Connection identifier - either connection ID (conn_...) or name. The system will automatically resolve names to IDs. examples: - conn_abc123def456ghi - Marketing Google Drive title: Connection Identifier description: >- Connection identifier - either connection ID (conn_...) or name. The system will automatically resolve names to IDs. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/StorageConnectionUpdateRequest' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/StorageConnectionModel' '400': description: Bad Request content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '401': description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '403': description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '404': description: Not Found content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' '500': description: Internal Server Error content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' components: schemas: StorageConnectionUpdateRequest: properties: name: anyOf: - type: string maxLength: 100 minLength: 1 - type: 'null' title: Name description: >- OPTIONAL. New name for the connection. Must be unique within the organization if provided. Format: 1-100 characters. examples: - Updated Drive Name - New S3 Connection description: anyOf: - type: string maxLength: 500 - type: 'null' title: Description description: >- OPTIONAL. New description for the connection. Set to empty string to clear existing description. Format: Up to 500 characters. metadata: anyOf: - additionalProperties: true type: object - type: 'null' title: Metadata description: >- OPTIONAL. New metadata dictionary. Replaces existing metadata entirely (partial updates not supported). Set to empty dict {} to clear all metadata. status: anyOf: - $ref: '#/components/schemas/TaskStatusEnum' - type: 'null' description: >- OPTIONAL. New operational status. ACTIVE: Connection is healthy and ready for use. SUSPENDED: Temporarily disabled, credentials preserved. FAILED: Health checks failing. ARCHIVED: Permanently retired (cannot be reactivated). is_active: anyOf: - type: boolean - type: 'null' title: Is Active description: >- OPTIONAL. Quick boolean flag for filtering. True when status is ACTIVE, False otherwise. Automatically maintained when status changes. provider_config: anyOf: - additionalProperties: true type: object - type: 'null' title: Provider Config description: >- OPTIONAL. Updated provider configuration including credentials. Replaces entire provider_config (partial updates not supported). SECURITY: Sensitive fields are encrypted at rest. type: object title: StorageConnectionUpdateRequest description: >- Request payload for updating storage connection metadata. Allows partial updates to connection metadata without changing credentials. Credentials can be updated via provider_config. **What You Can Update:** - Connection name and description - Metadata tags - Status (active/suspended) - Provider credentials (via provider_config) **Examples:** ```python # Update name and description { "name": "Updated Drive Name", "description": "New description" } # Suspend connection { "status": "suspended", "is_active": False } # Refresh credentials { "provider_config": { "credentials": {...} } } ``` examples: - description: New description name: Updated Drive Name - is_active: false status: suspended - provider_config: credentials: access_key_id: AKIA... StorageConnectionModel: properties: connection_id: type: string title: Connection Id description: >- Unique identifier for the storage connection. Auto-generated with 'conn_' prefix followed by secure random token. Format: conn_{15-character alphanumeric}. Used for API operations and audit trails. examples: - conn_abc123def456ghi - conn_xyz789uvw012qrs internal_id: type: string title: Internal Id description: >- REQUIRED. Organization internal identifier for multi-tenancy scoping. All connection operations are scoped to this organization. Format: int_{24-character secure token}. examples: - int_org123abc456def789xyz012 provider_type: $ref: '#/components/schemas/StorageProvider' description: >- REQUIRED. Storage provider implementation to use. Determines which client adapter is loaded for sync operations. Supported: google_drive, s3, snowflake, sharepoint, tigris. provider_config: anyOf: - $ref: '#/components/schemas/GoogleDriveConfig' - $ref: '#/components/schemas/S3Config' - $ref: '#/components/schemas/SnowflakeConfig' - $ref: '#/components/schemas/SharePointConfig' - $ref: '#/components/schemas/TigrisConfig' - $ref: '#/components/schemas/PostgreSQLConfig' - $ref: '#/components/schemas/InstagramConfig' - $ref: '#/components/schemas/TikTokConfig' - $ref: '#/components/schemas/RSSConfig' - $ref: '#/components/schemas/HTTPAPIConfig' - $ref: '#/components/schemas/BoxConfig' - $ref: '#/components/schemas/BrightDataConfig' - $ref: '#/components/schemas/BackblazeConfig' - $ref: '#/components/schemas/MuxConfig' - $ref: '#/components/schemas/IconikConfig' - $ref: >- #/components/schemas/shared__organizations__connections__provider_configs__EmailConfig - additionalProperties: true type: object title: Provider Config description: >- REQUIRED. Provider-specific configuration payload including credentials. Type depends on provider_type (GoogleDriveConfig, S3Config, etc.). SECURITY: Sensitive credential fields are encrypted at rest via MongoDB client-side field level encryption (CSFLE). Credentials never appear in API responses or logs. See provider_configs.py for detailed schemas. name: type: string maxLength: 100 minLength: 1 title: Name description: >- REQUIRED. Human-readable connection name for identification. Displayed in dashboards, sync logs, and API responses. Must be unique within the organization for clarity. Format: 1-100 characters, descriptive of the connection's purpose. examples: - Marketing Google Drive - Production S3 Bucket - Customer Assets Archive description: anyOf: - type: string maxLength: 500 - type: 'null' title: Description description: >- NOT REQUIRED. Optional description explaining the connection's purpose and scope. Helpful for team collaboration and documentation. Format: Up to 500 characters. examples: - Shared drive for marketing team assets and campaign materials - Production S3 bucket containing customer-uploaded videos status: $ref: '#/components/schemas/TaskStatusEnum' description: >- Operational status of the connection. ACTIVE: Connection is healthy and ready for use in sync operations. SUSPENDED: Temporarily disabled by user, credentials preserved but sync paused. FAILED: Health checks failing, credentials may be invalid or expired. ARCHIVED: Permanently retired, cannot be reactivated. Status transitions automatically based on health checks and user actions. default: ACTIVE is_active: type: boolean title: Is Active description: >- Quick boolean flag for filtering active connections in queries. True when status is ACTIVE, False for SUSPENDED/FAILED/ARCHIVED. Maintained automatically when status changes. Use for efficient filtering: db.connections.find({'is_active': True}) default: true last_used_at: anyOf: - type: string format: date-time - type: 'null' title: Last Used At description: >- NOT REQUIRED. UTC timestamp of the most recent successful sync operation. Updated automatically after each successful file sync/list operation. None if connection has never been used. Useful for identifying stale connections and usage analytics. last_error: anyOf: - type: string maxLength: 1000 - type: 'null' title: Last Error description: >- NOT REQUIRED. Most recent error message from failed health check or sync. Populated when authentication fails, network errors occur, or permissions denied. None when connection is healthy. Format: Error message truncated to 1000 characters. Used for diagnostics and troubleshooting. examples: - 'Authentication failed: Invalid credentials' - 'Permission denied: Unable to access shared drive' - 'Network timeout: Failed to connect to storage.googleapis.com' consecutive_failures: type: integer minimum: 0 title: Consecutive Failures description: >- Counter tracking consecutive failed health checks or sync attempts. Incremented on each failure, reset to 0 on success. Used to implement automatic connection suspension. Auto-suspend after 5 consecutive failures to prevent account lockout. Range: 0 to infinity (typically 0-10). default: 0 created_at: type: string format: date-time title: Created At description: >- UTC timestamp when the connection was created. Auto-generated using shared.utilities.helpers.current_time(). Immutable after creation. Format: ISO 8601 datetime. updated_at: type: string format: date-time title: Updated At description: >- UTC timestamp of the most recent update to the connection. Updated automatically on any field modification. Tracks configuration changes, status updates, and credential refreshes. Format: ISO 8601 datetime. created_by_user_id: type: string title: Created By User Id description: >- REQUIRED. User identifier of the user who created this connection. Used for audit trails and permission checks. Format: usr_{15-character alphanumeric}. Immutable after creation. examples: - usr_34bbf6ded1b749 metadata: additionalProperties: true type: object title: Metadata description: >- Arbitrary key-value metadata provided by the user. Useful for tagging, categorization, and custom annotations. NOT REQUIRED - defaults to empty dictionary. Common uses: team tags, cost center codes, project identifiers. examples: - cost_center: CC-1234 team: marketing - environment: production project: q4-campaign - tags: - customer-facing - high-priority type: object required: - internal_id - provider_type - provider_config - name - created_by_user_id title: StorageConnectionModel description: >- Canonical representation of an external storage provider connection. Storage connections enable Mixpeek to access external cloud storage providers (Google Drive, S3, etc.) for automated file ingestion and synchronization. Each connection represents a configured integration with credentials, health monitoring, and usage tracking. Lifecycle States: - ACTIVE: Connection is healthy and ready for sync operations - SUSPENDED: Temporarily disabled by user (credentials preserved) - FAILED: Health checks failing (may need credential refresh) - ARCHIVED: Permanently retired (cannot be reactivated) Security: - Sensitive credential fields are encrypted at rest using MongoDB client-side field level encryption (CSFLE) - Credentials never appear in API responses or logs - Failed authentication attempts are logged in last_error - Consecutive failures trigger automatic suspension Use Cases: - Connect to team Google Drive for document ingestion - Sync files from customer S3 buckets - Monitor and process uploaded media files - Schedule periodic sync operations Health Monitoring: - Automatic health checks validate connectivity and credentials - consecutive_failures tracks authentication/network issues - Auto-disable after 5 consecutive failures to prevent lockout - last_error stores diagnostic information for debugging examples: - connection_id: conn_abc123def456ghi created_by_user_id: usr_admin123abc456 description: Team drive for marketing assets internal_id: int_org123 is_active: true metadata: team: marketing name: Marketing Google Drive provider_config: credentials: client_email: sync@project.iam.gserviceaccount.com type: service_account provider_type: google_drive shared_drive_id: 0AH-Xabc123 provider_type: google_drive status: active - connection_id: conn_xyz789uvw012qrs created_by_user_id: usr_admin123abc456 internal_id: int_org123 is_active: true name: Production S3 Bucket provider_config: credentials: access_key_id: AKIA... provider_type: s3 region: us-east-1 provider_type: s3 status: active ErrorResponse: properties: success: type: boolean title: Success description: Always false for error responses default: false status: type: integer title: Status description: HTTP status code for this error error: $ref: '#/components/schemas/ErrorDetail' description: Error details payload type: object required: - status - error title: ErrorResponse description: Error response model. examples: - error: details: id: ns_123 resource: namespace message: Namespace not found type: NotFoundError status: 404 success: false HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError TaskStatusEnum: type: string enum: - PENDING - QUEUED - IN_PROGRESS - PROCESSING - COMPLETED - COMPLETED_WITH_ERRORS - FAILED - CANCELED - INTERRUPTED - UNKNOWN - SKIPPED - DRAFT - ACTIVE - ARCHIVED - SUSPENDED title: TaskStatusEnum description: |- Enumeration of task statuses for tracking asynchronous operations. Task statuses indicate the current state of asynchronous operations like batch processing, object ingestion, clustering, and taxonomy execution. Status Categories: Operation Statuses: Track progress of async operations Lifecycle Statuses: Track entity state (buckets, collections, namespaces) Values: PENDING: Task is queued but has not started processing yet IN_PROGRESS: Task is currently being executed PROCESSING: Task is actively processing data (similar to IN_PROGRESS) COMPLETED: Task finished successfully with no errors COMPLETED_WITH_ERRORS: Task finished but some items failed (partial success) FAILED: Task encountered an error and could not complete CANCELED: Task was manually canceled by a user or system UNKNOWN: Task status could not be determined SKIPPED: Task was intentionally skipped DRAFT: Task is in draft state and not yet submitted ACTIVE: Entity is active and operational (for buckets, collections, etc.) ARCHIVED: Entity has been archived SUSPENDED: Entity has been temporarily suspended Terminal Statuses: COMPLETED, COMPLETED_WITH_ERRORS, FAILED, CANCELED are terminal statuses. Once a task reaches these states, it will not transition to another state. Partial Success Handling: COMPLETED_WITH_ERRORS indicates that the operation completed but some documents/items failed. The task result includes: - List of successful items - List of failed items with error details - Success rate percentage This allows clients to handle partial success scenarios appropriately. Polling Guidance: - Poll tasks in PENDING, QUEUED, IN_PROGRESS, or PROCESSING states - Stop polling when task reaches COMPLETED, COMPLETED_WITH_ERRORS, FAILED, or CANCELED - Use exponential backoff (1s → 30s) when polling StorageProvider: type: string enum: - google_drive - s3 - snowflake - sharepoint - tigris - postgresql - instagram - tiktok - rss - http_api - box - brightdata - backblaze - mux - email - supabase - iconik title: StorageProvider description: |- Supported external storage providers for ingestion and sync. Mixpeek can connect to external storage providers to automatically ingest objects and keep them synchronized with your namespaces. Providers: GOOGLE_DRIVE: Google Drive and Google Workspace shared drives. - Authentication: Service account or OAuth2 - Features: Shared drive support, real-time sync, metadata preservation - Use cases: Marketing assets, team documents, knowledge bases - Limitations: Rate limits apply (10,000 requests/100 seconds per user) S3: Amazon S3 and S3-compatible storage (MinIO, DigitalOcean Spaces, etc). - Authentication: Access keys or IAM role assumption - Features: Bucket notifications, prefix filtering, versioning support - Use cases: Data lakes, video archives, ML datasets, backups - Limitations: IAM role assumption preferred over access keys SNOWFLAKE: Snowflake data warehouse tables. - Authentication: Key pair or username/password - Features: Incremental sync via watermarks, row-level mapping, schema introspection - Use cases: Customer data tables, product catalogs, transaction logs, metadata tables - Limitations: Each row becomes one object; large tables require incremental column SHAREPOINT: Microsoft SharePoint and OneDrive for Business. - Authentication: Azure AD OAuth2 (client credentials or delegated) - Features: Site/drive selection, folder sync, delta queries for incremental sync - Use cases: Enterprise documents, team collaboration files, compliance archives - Limitations: Requires Azure AD app registration; throttling limits apply Connection Requirements: - Valid credentials with read access to target files/buckets - Network connectivity from Mixpeek infrastructure - Appropriate IAM policies or share permissions configured Examples: - Use GOOGLE_DRIVE for syncing team marketing materials - Use S3 for ingesting video archives from data lakes - Use S3 with IAM role for secure production deployments - Use SHAREPOINT for syncing enterprise SharePoint document libraries TIGRIS: Tigris Data globally distributed object storage (S3-compatible). - Authentication: Access keys (same format as S3) - Features: S3-compatible API, global distribution, zero egress fees - Use cases: Globally distributed media, low-latency content delivery - Endpoint: https://fly.storage.tigris.dev POSTGRESQL: PostgreSQL relational database. - Authentication: Username/password - Features: SQL queries, incremental sync via watermarks, row-level mapping - Use cases: Customer data tables, product catalogs, transaction logs - Limitations: Each row becomes one object; large tables require incremental column INSTAGRAM: Instagram Business/Creator accounts via Meta Graph API. - Authentication: OAuth 2.0 via Meta Developer Console - Features: Media sync (posts, reels, carousels), engagement metrics, captions - Use cases: Social media content analysis, talent scouting, competitor monitoring - Limitations: Requires Instagram Business/Creator account; long-lived tokens (60 days) need refresh TIKTOK: TikTok accounts via TikTok Content API. - Authentication: OAuth 2.0 via TikTok Login Kit - Features: Video sync, engagement metrics (likes, views, shares), descriptions - Use cases: Social media content analysis, creator discovery, trend monitoring - Limitations: Access tokens expire in 24h (refresh_token flow); rate limits apply RSS: RSS/Atom feed entries. - Authentication: Optional HTTP headers (most feeds are public) - Features: Entry-level sync with title, author, categories, content - Use cases: News monitoring, blog ingestion, content aggregation - Limitations: No pagination; feeds are fetched in full each poll HTTP_API: Arbitrary REST/HTTP JSON APIs. - Authentication: Optional HTTP headers (API keys, Bearer tokens, etc.) - Features: Configurable JSONPath to items array, dedup via item ID field, incremental sync via timestamp field, GET or POST methods, JSON or JSONL responses - Use cases: Public APIs (Hacker News, GitHub), private APIs (Stripe, internal services), any JSON endpoint that returns a list of items - Limitations: No built-in pagination; API is fetched in full each poll BOX: Box cloud content management and file sharing. - Authentication: OAuth 2.0 (JWT or Client Credentials Grant with CCG) - Features: Folder sync, enterprise content management, metadata, versioning - Use cases: Enterprise document management, compliance archives, collaboration files - Limitations: Rate limits apply (10 API calls per second per user) BRIGHTDATA: BrightData web data platform for dataset collection and web scraping. - Authentication: API token from BrightData dashboard - Features: Pre-built datasets (LinkedIn, Amazon, etc.), custom scrapers, scheduled collection, geo-targeting, output format selection - Use cases: Competitor monitoring, market research, lead generation, e-commerce pricing, social media data ingestion - Limitations: Dataset availability depends on subscription; rate limits vary BACKBLAZE: Backblaze B2 Cloud Storage (S3-compatible object storage). - Authentication: Application key ID + application key from Backblaze console - Features: S3-compatible API, auto-discovers regional endpoint via B2 auth, full bucket/prefix sync, include/exclude patterns, modified_since filtering - Use cases: Cold storage sync, media archives, backup ingestion - Limitations: Buckets are region-specific; key must have listFiles/readFiles MUX: Mux video infrastructure platform. - Authentication: Access token ID + access token secret (HTTP Basic Auth) - Features: Video asset listing, static rendition download (MP4), metadata sync - Use cases: Video library ingestion, media asset processing, content analysis - Limitations: Assets must be in 'ready' status; static renditions must be enabled EMAIL: Inbound email connector (push-based). - Authentication: Per-connection webhook signing secret (HMAC-SHA256) - Features: MIME parsing, attachment extraction, sender allowlist, raw .eml archival - Use cases: Document intake via email forwarding, compliance mailbox, support inbox - Limitations: Requires external inbound email service (SES/Postmark/CloudMailin) SUPABASE: Supabase Storage (S3-compatible object storage built on open source). - Authentication: Dedicated S3 access keys (recommended) OR service_role JWT as session token (access_key_id=project_ref, secret=anon_key, session_token=service_role) - Features: S3-compatible API, per-project endpoint auto-derived from project_ref, bucket-scoped sync, include/exclude patterns, modified_since incremental sync - Use cases: Syncing user-uploaded assets from Supabase-backed apps, unified search across Supabase Storage + other sources - Limitations: Endpoint is project-specific (https://.storage.supabase.co); S3 keys must be generated from the Supabase dashboard (Storage → S3 Access Keys) GoogleDriveConfig: properties: provider_type: type: string const: google_drive title: Provider Type default: google_drive credentials: oneOf: - $ref: '#/components/schemas/GoogleDriveServiceAccountCredentials' - $ref: '#/components/schemas/GoogleDriveOAuthCredentials' title: Credentials description: >- REQUIRED. Authentication credentials for Google Drive API access. Choose service_account for production (recommended) or oauth for personal access. The 'type' field determines which credential type is used. discriminator: propertyName: type mapping: oauth: $ref: '#/components/schemas/GoogleDriveOAuthCredentials' service_account: $ref: '#/components/schemas/GoogleDriveServiceAccountCredentials' shared_drive_id: anyOf: - type: string - type: 'null' title: Shared Drive Id description: >- NOT REQUIRED. Google Workspace shared drive (Team Drive) identifier. When provided, sync operations are scoped to this shared drive only. When omitted, syncs from 'My Drive' of the authenticated account. Find ID: Open shared drive in browser, copy ID from URL. Format: 0A{alphanumeric-string} examples: - 0AH-Xabc123def456 - 0ABcDeFgHiJkLmNoPqR impersonate_user: anyOf: - type: string - type: 'null' title: Impersonate User description: >- NOT REQUIRED. Email address to impersonate when using service account credentials. Requires domain-wide delegation to be enabled for the service account. Used in G Suite environments to access files as a specific user. When omitted, uses the service account's own access. Format: Valid email address in the G Suite domain. examples: - user@example.com - admin@company.com type: object required: - credentials title: GoogleDriveConfig description: >- Google Drive and Google Workspace shared drive configuration. This configuration enables Mixpeek to connect to Google Drive for automated file ingestion and synchronization. Supports both personal Drive and Google Workspace shared drives (formerly Team Drives). Authentication Options: - Service Account: Recommended for production. No user interaction required. - OAuth2: Suitable for personal Drive access or development. Requirements: - Google Drive API enabled in Google Cloud Console - Appropriate authentication credentials configured - Files/folders shared with the service account or OAuth user - Network connectivity to drive.googleapis.com Use Cases: - Sync marketing materials from shared drives - Ingest documents from team collaboration folders - Monitor and process uploaded media files - Archive and search historical documents examples: - credentials: client_email: sync@mixpeek-prod-456.iam.gserviceaccount.com client_id: '123456789012345678901' private_key: | -----BEGIN PRIVATE KEY----- ... -----END PRIVATE KEY----- private_key_id: a1b2c3d4e5f6... project_id: mixpeek-prod-456 type: service_account description: Service account accessing shared drive provider_type: google_drive shared_drive_id: 0AH-Xabc123def456 - credentials: client_id: 123456789012-abc.apps.googleusercontent.com client_secret: GOCSPX-abc123def456 refresh_token: 1//0abc123def456... type: oauth description: OAuth accessing personal Drive provider_type: google_drive - credentials: client_email: sync@enterprise-sync.iam.gserviceaccount.com client_id: '987654321098765432109' private_key: | -----BEGIN PRIVATE KEY----- ... -----END PRIVATE KEY----- private_key_id: xyz789... project_id: enterprise-sync type: service_account description: Service account with domain-wide delegation impersonate_user: admin@company.com provider_type: google_drive S3Config: properties: provider_type: type: string const: s3 title: Provider Type default: s3 credentials: oneOf: - $ref: '#/components/schemas/S3AccessKeyCredentials' - $ref: '#/components/schemas/S3RoleCredentials' title: Credentials description: >- REQUIRED. AWS authentication credentials configuration. Choose 'iam_role' for production deployments (recommended) or 'access_key' for development, testing, or S3-compatible services. The 'type' field determines which credential mechanism is used. discriminator: propertyName: type mapping: access_key: $ref: '#/components/schemas/S3AccessKeyCredentials' iam_role: $ref: '#/components/schemas/S3RoleCredentials' region: type: string title: Region description: >- REQUIRED. AWS region where the S3 bucket is located. Must match the bucket's actual region to avoid routing errors. For S3-compatible services, use their documented region value or 'us-east-1' as a default if regions are not applicable. Format: AWS region code (e.g., us-east-1, eu-west-1) examples: - us-east-1 - us-west-2 - eu-west-1 - ap-southeast-1 - ca-central-1 endpoint_url: anyOf: - type: string - type: 'null' title: Endpoint Url description: >- NOT REQUIRED for AWS S3 (uses default AWS endpoints). REQUIRED for S3-compatible services to specify custom endpoint URL. Must be a valid HTTPS or HTTP URL without trailing slash. Examples: - MinIO: https://minio.example.com - DigitalOcean Spaces: https://nyc3.digitaloceanspaces.com - Wasabi: https://s3.wasabisys.com examples: - https://s3.wasabisys.com - https://nyc3.digitaloceanspaces.com - https://minio.company.com - http://localhost:9000 use_ssl: type: boolean title: Use Ssl description: >- Whether to use TLS/SSL encryption for connections to S3. RECOMMENDED: Always True for production environments. Set to False only for local development with unencrypted endpoints. Default: True default: true verify_ssl: type: boolean title: Verify Ssl description: >- Whether to verify TLS/SSL certificates when connecting. RECOMMENDED: Always True for production to prevent MITM attacks. Set to False only for development with self-signed certificates. Requires use_ssl=True to have any effect. Default: True default: true type: object required: - credentials - region title: S3Config description: >- Amazon S3 and S3-compatible storage provider configuration. This configuration enables Mixpeek to connect to Amazon S3 or S3-compatible storage services (MinIO, DigitalOcean Spaces, Wasabi, Backblaze B2, etc.) for automated object ingestion and synchronization. Authentication Methods: 1. IAM Role Assumption (RECOMMENDED for production): - Most secure option with automatic credential rotation - No long-lived credentials shared - Ideal for customer-owned S3 buckets 2. Access Keys: - Simpler setup for development and testing - Works with S3-compatible services - Requires manual credential rotation Requirements: - Valid AWS credentials or IAM role configuration - S3 bucket with appropriate permissions (s3:GetObject, s3:ListBucket) - Network connectivity to S3 endpoint - Correct region configuration Supported Services: - Amazon S3 (all regions) - MinIO (self-hosted or cloud) - DigitalOcean Spaces - Wasabi Cloud Storage - Backblaze B2 - Any S3-compatible storage with compatible API Use Cases: - Ingest videos from data lakes - Sync images from marketing asset buckets - Process documents from archive storage - Monitor and index uploaded files - Backup and disaster recovery workflows examples: - credentials: external_id: mixpeek-org_abc123 role_arn: arn:aws:iam::123456789012:role/mixpeek-storage-sync-role type: iam_role description: Production AWS S3 with IAM role (recommended) provider_type: s3 region: us-east-1 use_ssl: true verify_ssl: true - credentials: access_key_id: AKIAIOSFODNN7EXAMPLE secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY type: access_key description: Development AWS S3 with access keys provider_type: s3 region: us-west-2 use_ssl: true verify_ssl: true - credentials: access_key_id: minioadmin secret_access_key: minioadmin type: access_key description: MinIO self-hosted S3-compatible storage endpoint_url: https://minio.company.com provider_type: s3 region: us-east-1 use_ssl: true verify_ssl: true - credentials: access_key_id: DO00EXAMPLE secret_access_key: secretkeyexample123 type: access_key description: DigitalOcean Spaces endpoint_url: https://nyc3.digitaloceanspaces.com provider_type: s3 region: nyc3 use_ssl: true verify_ssl: true - credentials: access_key_id: minioadmin secret_access_key: minioadmin type: access_key description: Local development MinIO without SSL endpoint_url: http://localhost:9000 provider_type: s3 region: us-east-1 use_ssl: false verify_ssl: false SnowflakeConfig: properties: provider_type: type: string const: snowflake title: Provider Type default: snowflake credentials: oneOf: - $ref: '#/components/schemas/SnowflakeKeyPairCredentials' - $ref: '#/components/schemas/SnowflakeUsernamePasswordCredentials' title: Credentials description: >- REQUIRED. Authentication credentials for Snowflake. Choose key_pair for production (recommended) or username_password for simpler setup. The 'type' field determines which credential mechanism is used. discriminator: propertyName: type mapping: key_pair: $ref: '#/components/schemas/SnowflakeKeyPairCredentials' username_password: $ref: '#/components/schemas/SnowflakeUsernamePasswordCredentials' account: type: string title: Account description: >- REQUIRED. Snowflake account identifier. Format: {account_locator}.{cloud_region} or {org_name}-{account_name} Find in: Snowflake UI > Account dropdown > Account URL examples: - xy12345.us-east-1 - myorg-prod - acme-analytics warehouse: type: string title: Warehouse description: >- REQUIRED. Warehouse name for compute resources. Must have USAGE privilege on this warehouse. Warehouse will be used for all sync queries. Consider: Use dedicated warehouse for sync operations to isolate costs. examples: - COMPUTE_WH - MIXPEEK_SYNC_WH - ETL_SMALL database: anyOf: - type: string - type: 'null' title: Database description: >- NOT REQUIRED if fully qualified table name used in source_path. Database name for default context. Can be omitted if source_path uses {DATABASE}.{SCHEMA}.{TABLE} format. Must have USAGE privilege if specified. examples: - PRODUCTION - ANALYTICS_DB - CUSTOMER_DATA schema: anyOf: - type: string - type: 'null' title: Schema description: >- NOT REQUIRED if fully qualified table name used in source_path. Schema name for default context. Can be omitted if source_path uses {SCHEMA}.{TABLE} or {DATABASE}.{SCHEMA}.{TABLE}. Must have USAGE privilege if specified. examples: - PUBLIC - RAW_DATA - TRANSFORMED role: anyOf: - type: string - type: 'null' title: Role description: >- NOT REQUIRED. Snowflake role to use for operations. If omitted, uses user's default role. Role must have SELECT on target tables and USAGE on database/schema/warehouse. Best practice: Create dedicated read-only role for sync operations. examples: - MIXPEEK_SYNC_ROLE - READ_ONLY - DATA_ENGINEER incremental_column: anyOf: - type: string - type: 'null' title: Incremental Column description: >- NOT REQUIRED. Column name for incremental sync watermark. Should be a TIMESTAMP, TIMESTAMP_NTZ, or DATE column that tracks row modifications. When set, only rows with {incremental_column} > last_sync_watermark are synced. Common values: updated_at, modified_at, last_updated, ingestion_timestamp. If omitted, full table scan on every sync (not recommended for large tables). examples: - updated_at - modified_at - last_updated - ingestion_timestamp primary_key_columns: anyOf: - items: type: string type: array - type: 'null' title: Primary Key Columns description: >- NOT REQUIRED. Column names forming the primary key for stable object IDs. Used to generate deterministic file_id for deduplication. If omitted, uses hash of entire row content (less stable). Recommendation: Always specify for production to ensure idempotent syncs. examples: - - id - - user_id - timestamp - - customer_id - order_id query_timeout_seconds: type: integer maximum: 3600 minimum: 30 title: Query Timeout Seconds description: >- Query timeout in seconds. Prevents long-running queries from blocking sync operations. Default: 300 seconds (5 minutes). Increase for large tables or complex queries. default: 300 fetch_size: type: integer maximum: 10000 minimum: 100 title: Fetch Size description: >- Number of rows to fetch per network round-trip. Higher values reduce network overhead but increase memory usage. Default: 1000 rows. Tune based on row size and available memory. default: 1000 type: object required: - credentials - account - warehouse title: SnowflakeConfig description: >- Snowflake data warehouse configuration for table-based sync. Enables syncing Snowflake table rows as JSON objects in Mixpeek buckets. Each row becomes one object, with incremental sync via watermark columns. Authentication Options: - Key Pair: Recommended for production (secure, password-less) - Username/Password: Fallback option (simpler setup) Requirements: - Snowflake account with read access to target tables - Warehouse with compute resources - SELECT permissions on target tables/schemas - USAGE permissions on database, schema, warehouse Use Cases: - Sync customer data tables for AI/ML pipelines - Ingest product catalog for search/recommendations - Process transaction logs for analytics - Mirror metadata tables for vector search Example: Production setup with key pair auth: ```python config = { "provider_type": "snowflake", "credentials": { "type": "key_pair", "username": "MIXPEEK_SYNC", "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n", }, "account": "xy12345.us-east-1", "warehouse": "MIXPEEK_SYNC_WH", "database": "PRODUCTION", "schema": "PUBLIC", "role": "SYNC_ROLE", "incremental_column": "updated_at", "primary_key_columns": ["id"], } ``` examples: - account: xy12345.us-east-1 credentials: private_key: | -----BEGIN PRIVATE KEY----- ... -----END PRIVATE KEY----- type: key_pair username: MIXPEEK_SYNC database: PRODUCTION description: Production setup with key pair auth and incremental sync incremental_column: updated_at primary_key_columns: - id provider_type: snowflake role: MIXPEEK_SYNC_ROLE schema: PUBLIC warehouse: MIXPEEK_SYNC_WH - account: myorg-analytics credentials: password: secure_password_here type: username_password username: SYNC_USER database: ANALYTICS description: Simple username/password setup incremental_column: modified_at primary_key_columns: - customer_id - event_id provider_type: snowflake schema: RAW warehouse: COMPUTE_WH SharePointConfig: properties: provider_type: type: string const: sharepoint title: Provider Type default: sharepoint credentials: oneOf: - $ref: '#/components/schemas/SharePointClientCredentials' - $ref: '#/components/schemas/SharePointDelegatedCredentials' title: Credentials description: >- REQUIRED. Azure AD authentication credentials. Choose client_credentials for production (app-only) or delegated for user-level access. The 'type' field determines which authentication flow is used. discriminator: propertyName: type mapping: client_credentials: $ref: '#/components/schemas/SharePointClientCredentials' delegated: $ref: '#/components/schemas/SharePointDelegatedCredentials' site_id: anyOf: - type: string - type: 'null' title: Site Id description: >- NOT REQUIRED if using personal OneDrive. SharePoint site identifier for targeting a specific site. Format: '{hostname},{site-collection-id},{web-id}' or site URL. Find via: Microsoft Graph API GET /sites?search={keyword} Example: 'contoso.sharepoint.com,12345678-...,87654321-...' examples: - >- contoso.sharepoint.com,12345678-1234-1234-1234-123456789abc,87654321-4321-4321-4321-cba987654321 - root drive_id: anyOf: - type: string - type: 'null' title: Drive Id description: >- NOT REQUIRED if you want to use the default document library. Specific drive (document library) ID within the site. Find via: GET /sites/{site-id}/drives Format: Base64-encoded ID starting with 'b!' examples: - b!abc123def456... folder_path: anyOf: - type: string - type: 'null' title: Folder Path description: >- NOT REQUIRED. Path within the drive to sync from. If omitted, syncs from the root of the drive. Format: Forward-slash separated path (e.g., '/Documents/Marketing'). Note: Leading slash is optional. examples: - /Shared Documents/Marketing - Projects/2024 - / type: object required: - credentials title: SharePointConfig description: |- Microsoft SharePoint and OneDrive for Business configuration. Enables Mixpeek to connect to SharePoint sites, document libraries, and OneDrive for Business for automated file ingestion and synchronization. Architecture: SharePoint hierarchy: Tenant → Sites → Drives (Document Libraries) → Items - site_id: Identifies the SharePoint site - drive_id: Identifies a specific document library within the site - folder_path: Path within the document library Authentication Methods: 1. Client Credentials (RECOMMENDED for production): - App-only access without user interaction - Requires admin consent for Graph API permissions - Access level determined by app permissions 2. Delegated: - User-level access via OAuth consent - Access limited to user's permissions - Requires refresh token management Requirements: - Azure AD application registration - Microsoft Graph API permissions (Sites.Read.All, Files.Read.All) - Network connectivity to graph.microsoft.com Use Cases: - Sync SharePoint document libraries - Ingest enterprise collaboration files - Monitor and process uploaded documents - Archive compliance-sensitive materials Example: ```python config = { "provider_type": "sharepoint", "credentials": { "type": "client_credentials", "tenant_id": "12345678-...", "client_id": "87654321-...", "client_secret": "your-secret", }, "site_id": "contoso.sharepoint.com,guid1,guid2", "drive_id": "b!abc123...", "folder_path": "/Shared Documents/Marketing", } ``` examples: - credentials: client_id: 87654321-4321-4321-4321-cba987654321 client_secret: your-client-secret-here tenant_id: 12345678-1234-1234-1234-123456789abc type: client_credentials description: Production SharePoint with client credentials drive_id: b!abc123def456ghi789 folder_path: /Shared Documents/Marketing provider_type: sharepoint site_id: >- contoso.sharepoint.com,12345678-1234-1234-1234-123456789abc,87654321-4321-4321-4321-cba987654321 - credentials: client_id: 87654321-4321-4321-4321-cba987654321 client_secret: your-client-secret refresh_token: 0.AAAA... tenant_id: common type: delegated description: Personal OneDrive with delegated auth folder_path: /Documents/Projects provider_type: sharepoint - credentials: client_id: 87654321-4321-4321-4321-cba987654321 client_secret: your-client-secret tenant_id: 12345678-1234-1234-1234-123456789abc type: client_credentials description: SharePoint root site, default document library provider_type: sharepoint site_id: root TigrisConfig: properties: provider_type: type: string const: tigris title: Provider Type default: tigris credentials: $ref: '#/components/schemas/TigrisAccessKeyCredentials' description: >- REQUIRED. Tigris access key credentials. Obtain from Tigris Console at https://console.tigris.dev region: type: string title: Region description: >- Region for Tigris. Typically 'auto' for automatic global distribution. Tigris automatically distributes data globally, so region is usually 'auto'. Default: auto default: auto examples: - auto endpoint_url: type: string title: Endpoint Url description: >- Tigris S3-compatible endpoint URL. Default: https://fly.storage.tigris.dev This is the standard Tigris endpoint and usually doesn't need to be changed. default: https://fly.storage.tigris.dev examples: - https://fly.storage.tigris.dev type: object required: - credentials title: TigrisConfig description: |- Tigris Data globally distributed object storage configuration. Tigris is an S3-compatible object storage service with automatic global distribution, zero egress fees, and built-in CDN capabilities. It uses the AWS S3 API, making integration straightforward. Key Features: - S3-compatible API (drop-in replacement) - Automatic global data distribution - Zero egress fees - Built-in CDN and caching - Strong consistency guarantees Authentication: - Access keys only (similar to S3 access keys) - No IAM role assumption (Tigris is not AWS) Requirements: - Tigris account with access keys - Bucket created in Tigris console - Network connectivity to fly.storage.tigris.dev Use Cases: - Globally distributed media assets - Low-latency content delivery worldwide - Cost-effective storage with no egress fees - S3-compatible workflows outside AWS examples: - credentials: access_key_id: tid_your_access_key secret_access_key: tsec_your_secret_key type: access_key description: Standard Tigris configuration endpoint_url: https://fly.storage.tigris.dev provider_type: tigris region: auto - credentials: access_key_id: tid_example_key secret_access_key: tsec_example_secret type: access_key description: Tigris with explicit endpoint (default) provider_type: tigris PostgreSQLConfig: properties: provider_type: type: string const: postgresql title: Provider Type default: postgresql credentials: $ref: '#/components/schemas/PostgreSQLCredentials' description: >- REQUIRED. PostgreSQL authentication credentials. Currently supports username/password authentication. host: type: string title: Host description: >- REQUIRED. PostgreSQL server hostname or IP address. Examples: 'localhost', 'db.example.com', '192.168.1.100' examples: - localhost - db.example.com - 192.168.1.100 port: type: integer maximum: 65535 minimum: 1 title: Port description: 'PostgreSQL server port. Default: 5432 (standard PostgreSQL port)' default: 5432 database: type: string title: Database description: >- REQUIRED. Database name to connect to. User must have CONNECT privilege on this database. examples: - production - analytics - customer_data schema: anyOf: - type: string - type: 'null' title: Schema description: >- Schema name for default context. Default: 'public'. User must have USAGE privilege on this schema. default: public examples: - public - raw_data - transformed ssl_mode: type: string title: Ssl Mode description: >- SSL/TLS connection mode. Options: 'disable', 'allow', 'prefer', 'require', 'verify-ca', 'verify-full'. Default: 'prefer'. RECOMMENDED: Use 'require' or stricter for production environments. default: prefer examples: - disable - prefer - require - verify-full incremental_column: anyOf: - type: string - type: 'null' title: Incremental Column description: >- NOT REQUIRED. Column name for incremental sync watermark. Should be a TIMESTAMP or DATE column that tracks row modifications. Common values: updated_at, modified_at, last_updated. If omitted, full table scan on every sync. examples: - updated_at - modified_at - last_updated primary_key_columns: anyOf: - items: type: string type: array - type: 'null' title: Primary Key Columns description: >- NOT REQUIRED. Column names forming the primary key for stable object IDs. Used to generate deterministic file_id for deduplication. If omitted, uses hash of entire row content. examples: - - id - - user_id - timestamp - - customer_id - order_id query_timeout_seconds: type: integer maximum: 3600 minimum: 1 title: Query Timeout Seconds description: >- Query timeout in seconds. Default: 300 seconds (5 minutes). Increase for large tables or complex queries. default: 300 fetch_size: type: integer maximum: 10000 minimum: 100 title: Fetch Size description: >- Number of rows to fetch per batch. Higher values reduce network overhead but increase memory usage. Default: 1000 rows. default: 1000 type: object required: - credentials - host - database title: PostgreSQLConfig description: >- PostgreSQL database configuration for table-based sync and SQL queries. Enables syncing PostgreSQL table rows as JSON objects and running SQL queries via the SQL Lookup retriever stage. Each row becomes one object, with incremental sync via watermark columns. Authentication: - Username/Password: Standard PostgreSQL authentication Requirements: - PostgreSQL 12+ recommended - Read access to target tables - Network connectivity to PostgreSQL server Use Cases: - Sync customer data tables for AI/ML pipelines - Run SQL lookups to enrich documents in retriever pipelines - Ingest product catalog for search/recommendations - Process transaction logs for analytics Example: ```python config = { "provider_type": "postgresql", "credentials": { "type": "username_password", "username": "mixpeek_sync", "password": "secure_password", }, "host": "db.example.com", "port": 5432, "database": "production", "schema": "public", "ssl_mode": "require", } ``` examples: - credentials: password: dev_password type: username_password username: mixpeek_user database: mixpeek_test description: Local PostgreSQL development setup host: localhost port: 5432 provider_type: postgresql schema: public ssl_mode: disable - credentials: password: secure_password_here type: username_password username: mixpeek_sync database: production description: Production PostgreSQL with SSL and incremental sync host: db.example.com incremental_column: updated_at port: 5432 primary_key_columns: - id provider_type: postgresql schema: public ssl_mode: require InstagramConfig: properties: provider_type: type: string const: instagram title: Provider Type default: instagram credentials: $ref: '#/components/schemas/InstagramOAuthCredentials' description: OAuth credentials for Instagram Graph API access. scopes: items: type: string type: array title: Scopes description: OAuth scopes granted during authorization. default: - instagram_basic - instagram_manage_insights - pages_show_list business_account_id: anyOf: - type: string - type: 'null' title: Business Account Id description: >- Instagram Business Account ID if different from the user ID. Auto-populated from OAuth flow. type: object required: - credentials title: InstagramConfig description: |- Instagram Business/Creator account configuration via Meta Graph API. Enables Mixpeek to sync media (posts, reels, carousels) from Instagram accounts using the official Instagram Graph API. Each media item becomes a bucket object with metadata (caption, likes, comments, permalink). Authentication: - OAuth 2.0 via Meta Developer Console - Long-lived tokens (60 days) with automatic refresh Requirements: - Instagram Business or Creator account - Facebook Page linked to the Instagram account - Meta app with instagram_basic permission Use Cases: - Sync creator content for talent analysis - Monitor competitor ad creatives - Build social media content libraries for multimodal search examples: - credentials: access_token: IGQ... client_id: '123456789' client_secret: app_secret_here instagram_user_id: '17841400000000000' instagram_username: mybrand token_expires_at: '2025-04-01T00:00:00Z' type: oauth description: Instagram Business account via OAuth provider_type: instagram scopes: - instagram_basic - instagram_manage_insights TikTokConfig: properties: provider_type: type: string const: tiktok title: Provider Type default: tiktok credentials: $ref: '#/components/schemas/TikTokOAuthCredentials' description: OAuth credentials for TikTok Content API access. scopes: items: type: string type: array title: Scopes description: OAuth scopes granted during authorization. default: - user.info.basic - video.list type: object required: - credentials title: TikTokConfig description: >- TikTok account configuration via TikTok Content API. Enables Mixpeek to sync videos from TikTok accounts using the official TikTok API. Each video becomes a bucket object with metadata (description, likes, views, shares, comments). Authentication: - OAuth 2.0 via TikTok Login Kit - Access tokens expire in 24h (refresh_token flow) Requirements: - TikTok Developer account with approved app - user.info.basic and video.list scopes Use Cases: - Sync creator video content for talent analysis - Monitor trending content and competitors - Build video content libraries for multimodal search examples: - credentials: access_token: act.xxx client_key: your_client_key client_secret: your_client_secret open_id: user_open_id refresh_token: rft.xxx token_expires_at: '2025-03-02T00:00:00Z' type: oauth description: TikTok account via OAuth provider_type: tiktok scopes: - user.info.basic - video.list RSSConfig: properties: provider_type: type: string const: rss title: Provider Type default: rss credentials: anyOf: - $ref: '#/components/schemas/RSSHttpHeaderCredentials' - type: 'null' description: Optional HTTP headers for private feeds. user_agent: type: string title: User Agent description: User-Agent header for feed requests. default: Mixpeek RSS Sync/1.0 request_timeout: type: integer maximum: 120 minimum: 5 title: Request Timeout description: HTTP request timeout in seconds. default: 30 type: object title: RSSConfig description: RSS/Atom feed configuration. source_path = feed URL. examples: - description: Public RSS feed provider_type: rss request_timeout: 30 user_agent: Mixpeek RSS Sync/1.0 - credentials: headers: Authorization: Bearer your-token-here type: http_headers description: Private RSS feed with auth header provider_type: rss request_timeout: 30 user_agent: Mixpeek RSS Sync/1.0 HTTPAPIConfig: properties: provider_type: type: string const: http_api title: Provider Type default: http_api credentials: anyOf: - $ref: '#/components/schemas/HTTPAPIHeaderCredentials' - type: 'null' description: Optional HTTP headers for authenticated API endpoints. http_method: type: string enum: - GET - POST title: Http Method description: HTTP method to use when calling the API. default: GET request_body: anyOf: - additionalProperties: true type: object - type: 'null' title: Request Body description: Optional JSON body for POST requests. items_path: type: string title: Items Path description: >- Dot-notation path to the array of items in the JSON response. Examples: 'hits' for {hits: [...]}, 'data.items' for {data: {items: [...]}}. Leave empty if the response is a top-level array. default: '' item_id_field: type: string title: Item Id Field description: >- REQUIRED. Field name in each item used as a unique ID for deduplication. Examples: 'id', 'objectID', 'uuid'. item_modified_field: anyOf: - type: string - type: 'null' title: Item Modified Field description: >- Optional field name for a timestamp used for incremental sync. Supports ISO 8601 strings and Unix timestamps. Examples: 'updated_at', 'created_at', 'modified'. response_content_type: type: string enum: - json - jsonl title: Response Content Type description: >- Response format: 'json' (default) or 'jsonl' (newline-delimited JSON). default: json user_agent: type: string title: User Agent description: User-Agent header for API requests. default: Mixpeek HTTP API Sync/1.0 request_timeout: type: integer maximum: 120 minimum: 5 title: Request Timeout description: HTTP request timeout in seconds. default: 30 type: object required: - item_id_field title: HTTPAPIConfig description: |- REST/HTTP JSON API configuration. source_path = API URL. Enables Mixpeek to sync items from any JSON REST API endpoint. Each item in the API response becomes a bucket object with its JSON body stored as the blob. Authentication: - Optional HTTP headers (API keys, Bearer tokens, etc.) Requirements: - API must return JSON (or JSONL) response - Response must contain an array of items (at root or nested path) - Each item must have a unique ID field for deduplication Use Cases: - Sync data from public APIs (Hacker News, GitHub, etc.) - Ingest records from internal REST services - Poll third-party APIs (Stripe, Shopify, etc.) - Monitor any JSON endpoint for new items examples: - description: Public API (Hacker News Algolia) item_id_field: objectID item_modified_field: created_at_i items_path: hits provider_type: http_api request_timeout: 30 - credentials: headers: Authorization: Bearer your-api-key type: http_headers description: Authenticated API with Bearer token item_id_field: id item_modified_field: updated_at items_path: data provider_type: http_api - description: POST API with request body http_method: POST item_id_field: id items_path: results provider_type: http_api request_body: limit: 10 query: mixpeek - description: JSONL endpoint item_id_field: id items_path: '' provider_type: http_api response_content_type: jsonl BoxConfig: properties: provider_type: type: string const: box title: Provider Type default: box credentials: oneOf: - $ref: '#/components/schemas/BoxOAuthCredentials' - $ref: '#/components/schemas/BoxCCGCredentials' - $ref: '#/components/schemas/BoxJWTCredentials' title: Credentials description: >- REQUIRED. Box authentication credentials. Choose 'oauth' for user-level access, 'ccg' for server-to-server (recommended), or 'jwt' for high-security enterprise. The 'type' field determines which authentication flow is used. discriminator: propertyName: type mapping: ccg: $ref: '#/components/schemas/BoxCCGCredentials' jwt: $ref: '#/components/schemas/BoxJWTCredentials' oauth: $ref: '#/components/schemas/BoxOAuthCredentials' folder_id: type: string title: Folder Id description: >- Box folder ID to sync from. Default '0' is the root folder. Find folder ID: Open folder in Box web UI, copy the numeric ID from the URL. Example URL: https://app.box.com/folder/123456789 → folder_id='123456789' default: '0' examples: - '0' - '123456789' - '987654321' type: object required: - credentials title: BoxConfig description: |- Box cloud content management and file sharing configuration. Enables Mixpeek to connect to Box for automated file ingestion and synchronization. Supports enterprise-grade content management features including folder sync, metadata, versioning, and retention policies. Authentication Methods: 1. OAuth 2.0 (for user-level access): - Standard OAuth flow with access/refresh tokens - Access scoped to the authorizing user's content 2. Client Credentials Grant (CCG) (RECOMMENDED for production): - Server-to-server without user interaction - Acts as service account or specific user - Requires admin authorization in Box Admin Console 3. JWT (for high-security enterprise): - RSA key pair for signing JWT assertions - No user interaction required - Highest security option Requirements: - Box Developer account with an application - Application authorized in Box Admin Console (for CCG/JWT) - Network connectivity to api.box.com Use Cases: - Sync enterprise document libraries - Ingest compliance and legal documents - Monitor collaboration folders for new content - Archive and search enterprise content examples: - credentials: client_id: abc123def456 client_secret: your-client-secret enterprise_id: '12345678' type: ccg description: Box with Client Credentials Grant (recommended) folder_id: '0' provider_type: box - credentials: access_token: your-access-token client_id: abc123def456 client_secret: your-client-secret refresh_token: your-refresh-token type: oauth description: Box with OAuth 2.0 user authentication folder_id: '987654321' provider_type: box - credentials: client_id: abc123def456 client_secret: your-client-secret enterprise_id: '12345678' jwt_key_id: abcdef12 private_key: | -----BEGIN ENCRYPTED PRIVATE KEY----- ... -----END ENCRYPTED PRIVATE KEY----- private_key_passphrase: passphrase type: jwt description: Box with JWT authentication folder_id: '0' provider_type: box BrightDataConfig: properties: provider_type: type: string const: brightdata title: Provider Type default: brightdata credentials: $ref: '#/components/schemas/BrightDataCredentials' description: REQUIRED. BrightData API token credentials. customer_id: anyOf: - type: string - type: 'null' title: Customer Id description: >- NOT REQUIRED. BrightData customer ID (zone-level auth). Required when using zone-specific API access. Find in: BrightData Dashboard > your zone settings. examples: - hl_abc12345 default_output_format: type: string enum: - jsonl - json - csv - ndjson title: Default Output Format description: >- Output format for downloaded dataset results. 'jsonl' (default) streams one JSON object per line. 'json' returns a full JSON array. 'csv' returns comma-separated values. 'ndjson' is equivalent to jsonl. default: jsonl country: anyOf: - type: string - type: 'null' title: Country description: >- NOT REQUIRED. ISO 3166-1 alpha-2 country code for geo-targeted data collection. When set, the dataset snapshot targets data from this country. Examples: 'us', 'gb', 'de', 'jp'. examples: - us - gb - de inputs: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Inputs description: >- NOT REQUIRED. Default seed inputs for datasets that require input URLs or records (e.g. LinkedIn company profiles). Each item is a JSON object passed to the BrightData trigger API. Overridden per-call when inputs are supplied at sync time. examples: - - url: https://www.linkedin.com/company/google type: object required: - credentials title: BrightDataConfig description: >- BrightData web data platform configuration for dataset and scraper syncs. Enables Mixpeek to collect data from BrightData's pre-built datasets (LinkedIn, Amazon, Google Maps, etc.) or custom web scrapers. Each sync triggers a new dataset snapshot, polls until ready, then downloads and ingests each row as a bucket object. Authentication: - API token from BrightData dashboard Requirements: - Active BrightData subscription with Datasets or Web Scraper access - API token with appropriate dataset permissions - Network connectivity to api.brightdata.com Use Cases: - Ingest LinkedIn company/people datasets for lead generation - Monitor e-commerce pricing via Amazon product datasets - Collect Google Maps business data for location intelligence - Run custom web scrapers and process results as documents - Aggregate public social media data for trend analysis examples: - credentials: api_token: your-brightdata-api-token type: api_token default_output_format: jsonl description: BrightData dataset sync provider_type: brightdata - country: us credentials: api_token: your-brightdata-api-token type: api_token customer_id: hl_abc12345 default_output_format: jsonl description: BrightData with geo-targeting provider_type: brightdata BackblazeConfig: properties: provider_type: type: string const: backblaze title: Provider Type default: backblaze credentials: $ref: '#/components/schemas/BackblazeCredentials' description: REQUIRED. Backblaze B2 application key credentials. type: object required: - credentials title: BackblazeConfig description: |- Backblaze B2 Cloud Storage configuration. Backblaze B2 is an S3-compatible object storage service. This provider auto-discovers the regional S3 endpoint via the B2 authorization API, so users only need to provide their key ID and application key. Authentication: - Application key ID + application key from Backblaze console Requirements: - Active Backblaze account with B2 Cloud Storage enabled - Application key with listBuckets, listFiles, and readFiles capabilities - (Optional) Restrict key to specific bucket for scoped access Use Cases: - Cold storage archive ingestion - Media file sync from Backblaze-hosted assets - Backup data processing - Cost-effective large-file storage sync examples: - credentials: application_key: K005xxxxxxxxxxxxxxxxxxxxxxxxxxxx key_id: 005870eff85b6c60000000001 type: application_key description: Backblaze B2 bucket sync provider_type: backblaze MuxConfig: properties: provider_type: type: string const: mux title: Provider Type default: mux credentials: $ref: '#/components/schemas/MuxCredentials' description: REQUIRED. Mux access token credentials. type: object required: - credentials title: MuxConfig description: |- Mux video infrastructure platform configuration. Mux is a video API platform that provides video hosting, encoding, and streaming. This provider syncs video assets from a Mux account into Mixpeek for processing and analysis. Authentication: - Access token ID + secret from Mux dashboard (HTTP Basic Auth) Requirements: - Active Mux account with video assets - Access token with Mux Video read permissions - Assets must have playback IDs for download Use Cases: - Video library ingestion and analysis - Media asset processing pipelines - Content moderation and classification - Video search and discovery examples: - credentials: access_token_id: abc123def456 access_token_secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx type: access_token description: Mux video sync provider_type: mux IconikConfig: properties: provider_type: type: string const: iconik title: Provider Type default: iconik credentials: $ref: '#/components/schemas/IconikCredentials' description: REQUIRED. Iconik App-ID + Auth-Token credentials. base_url: anyOf: - type: string - type: 'null' title: Base Url description: >- Iconik API base URL. Defaults to https://app.iconik.io. Override for on-premise Iconik installations. metadata_view_id: anyOf: - type: string - type: 'null' title: Metadata View Id description: >- Iconik metadata view ID for fetching custom metadata fields. Found in: Iconik Admin > Metadata > Views. Set to 'none' to skip metadata view fetches entirely. rate_limit_rps: anyOf: - type: number - type: 'null' title: Rate Limit Rps description: >- Max requests per second to the Iconik API (shared across all sync shards). rate_limit_redis_url: anyOf: - type: string - type: 'null' title: Rate Limit Redis Url description: Redis URL for distributed rate limiting across workers. type: object required: - credentials title: IconikConfig description: |- Iconik DAM platform configuration. Iconik is a cloud-native DAM (Digital Asset Management) platform. This provider syncs video/image/audio assets from an Iconik account into Mixpeek for processing and analysis. Authentication: - App-ID + Auth-Token (HTTP headers) Requirements: - Active Iconik account with API application configured - Auth token with asset read permissions Use Cases: - Video ad library ingestion and analysis - Creative asset search and discovery - Talent/cast identification across ad library examples: - credentials: app_id: e0f0ec84-... auth_token: eyJhbGciOi... type: app_token description: Iconik DAM sync provider_type: iconik shared__organizations__connections__provider_configs__EmailConfig: properties: provider_type: type: string const: email title: Provider Type default: email credentials: $ref: '#/components/schemas/EmailCredentials' description: Webhook signing credentials (auto-generated if omitted). inbound_address: anyOf: - type: string - type: 'null' title: Inbound Address description: >- System-assigned inbound email address. Auto-provisioned on connection creation. Read-only. allowed_senders: items: type: string type: array title: Allowed Senders description: >- Sender allowlist. Emails from senders not matching this list are rejected. Supports exact addresses (user@company.com) and domain wildcards (*@company.com). Empty list = accept all senders. store_raw_eml: type: boolean title: Store Raw Eml description: Store the raw .eml file as an additional blob for chain of custody. default: true type: object title: EmailConfig description: |- Inbound email connector configuration (push-based). Customers receive a dedicated inbound email address per connection. Emails forwarded to that address are parsed (MIME), attachments are extracted as blobs, and the whole message becomes a bucket object. Authentication: - Inbound webhook verified via HMAC-SHA256 signing secret Requirements: - External inbound email service (SES Inbound / Postmark / CloudMailin) configured to POST parsed emails to the webhook URL Use Cases: - Compliance document intake (legal, healthcare) - Customer support email ingestion - Secure document forwarding pipelines - Media collection via email attachments examples: - allowed_senders: - '*@lawfirm.com' - paralegal@partner.com credentials: type: webhook_secret webhook_secret: '' description: Legal document intake inbox provider_type: email store_raw_eml: true ErrorDetail: properties: message: type: string title: Message description: Human-readable error message type: type: string title: Type description: Stable error type identifier (machine-readable) code: anyOf: - type: string - type: 'null' title: Code description: >- Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition. details: anyOf: - additionalProperties: true type: object - type: 'null' title: Details description: >- Optional structured details to help debugging (validation errors, IDs, etc.) type: object required: - message - type title: ErrorDetail description: Error detail model. ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type type: object required: - loc - msg - type title: ValidationError GoogleDriveServiceAccountCredentials: properties: type: type: string const: service_account title: Type default: service_account project_id: type: string title: Project Id description: >- REQUIRED. Google Cloud project ID where the service account was created. Found in the JSON key file as 'project_id'. Format: lowercase alphanumeric with hyphens (e.g., 'my-project-123'). examples: - mixpeek-prod-456 - customer-ingestion private_key_id: type: string title: Private Key Id description: >- REQUIRED. Unique identifier for the private key. Found in the JSON key file as 'private_key_id'. Format: 40-character hexadecimal string. examples: - a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0 private_key: type: string title: Private Key description: >- REQUIRED. PEM-encoded RSA private key for authentication. Found in the JSON key file as 'private_key'. SECURITY: This field is encrypted at rest. Never log or expose this value. Format: Must include BEGIN/END PRIVATE KEY markers. client_email: type: string title: Client Email description: >- REQUIRED. Service account email address. Found in the JSON key file as 'client_email'. Share Drive files/folders with this email to grant access. Format: {account-name}@{project-id}.iam.gserviceaccount.com examples: - mixpeek-sync@my-project-123.iam.gserviceaccount.com - ingestion@customer-project.iam.gserviceaccount.com client_id: type: string title: Client Id description: >- REQUIRED. Numeric service account identifier. Found in the JSON key file as 'client_id'. Format: 21-digit numeric string. examples: - '123456789012345678901' type: object required: - project_id - private_key_id - private_key - client_email - client_id title: GoogleDriveServiceAccountCredentials description: >- Credentials for Google Drive service account authentication. Service accounts provide server-to-server authentication for Google Drive without requiring user interaction. They are ideal for automated sync operations. Prerequisites: - Create a service account in Google Cloud Console - Enable Google Drive API for the project - Download the JSON key file - Share target Drive files/folders with the service account email Security: - private_key field is encrypted at rest using MongoDB client-side field level encryption - Credentials never appear in logs or API responses - Use domain-wide delegation for G Suite environments Use Cases: - Automated ingestion pipelines from shared drives - Scheduled sync operations without user interaction - Service-to-service integration for enterprise deployments GoogleDriveOAuthCredentials: properties: type: type: string const: oauth title: Type default: oauth client_id: type: string title: Client Id description: >- REQUIRED. OAuth 2.0 client ID from Google Cloud Console. Found in the API credentials section. Format: {id}.apps.googleusercontent.com examples: - 123456789012-abcdefghijklmnop.apps.googleusercontent.com client_secret: type: string title: Client Secret description: >- REQUIRED. OAuth 2.0 client secret from Google Cloud Console. SECURITY: This field is encrypted at rest. Never log or expose this value. Format: Alphanumeric string from Google Cloud Console. examples: - GOCSPX-abc123def456ghi789jkl012 refresh_token: type: string title: Refresh Token description: >- REQUIRED. Long-lived refresh token obtained during OAuth consent flow. Used to automatically obtain new access tokens without user interaction. SECURITY: Encrypted at rest. Can be revoked by user at any time. Obtain via: Complete OAuth flow with drive.readonly or drive scope. type: object required: - client_id - client_secret - refresh_token title: GoogleDriveOAuthCredentials description: >- Credentials for Google Drive OAuth2 user authentication. OAuth2 credentials provide access to Google Drive on behalf of a specific user. This authentication method is suitable when accessing personal Drive files or when service account delegation is not available. Prerequisites: - Create OAuth 2.0 credentials in Google Cloud Console - Configure authorized redirect URIs - Complete OAuth consent flow to obtain refresh token - Ensure appropriate OAuth scopes are granted (drive.readonly or drive) Security: - client_secret and refresh_token are encrypted at rest - Access tokens are automatically refreshed and cached temporarily - Credentials are scoped to the user who granted consent Use Cases: - Personal Drive file access for individual users - Prototyping and development without service account setup - Environments where service account delegation is not feasible Limitations: - Requires user interaction during initial setup - Access is limited to files the consenting user can access - Refresh tokens can be revoked by the user S3AccessKeyCredentials: properties: type: type: string const: access_key title: Type default: access_key access_key_id: type: string maxLength: 128 minLength: 16 title: Access Key Id description: >- REQUIRED. AWS access key ID for authentication. Format: 20-character alphanumeric string starting with 'AKIA' (long-term) or 'ASIA' (temporary). Obtain from: AWS Console > IAM > Users > Security Credentials examples: - AKIAIOSFODNN7EXAMPLE - ASIAIOSFODNN7EXAMPLE secret_access_key: type: string minLength: 40 title: Secret Access Key description: >- REQUIRED. AWS secret access key for authentication. SECURITY: This field is encrypted at rest. Never log or expose this value. Format: 40-character base64-encoded string. Obtain from: AWS Console when creating/viewing access key (shown only once) session_token: anyOf: - type: string - type: 'null' title: Session Token description: >- NOT REQUIRED. Temporary session token for AWS STS credentials. REQUIRED when using temporary security credentials from AWS STS. NOT REQUIRED for long-term IAM user access keys. SECURITY: Encrypted at rest. Automatically expires after session duration. Format: Base64-encoded string, typically several hundred characters. Use case: Enhanced security with automatic credential rotation type: object required: - access_key_id - secret_access_key title: S3AccessKeyCredentials description: >- AWS S3 access key and secret credentials. Access keys provide programmatic access to S3 buckets using long-lived credentials. This authentication method is straightforward but less secure than IAM role assumption. Prerequisites: - IAM user or role with S3 access permissions - Access key and secret key generated in AWS Console - Appropriate bucket policies or IAM policies configured Security Considerations: - Access keys are long-lived and don't automatically expire - secret_access_key is encrypted at rest but should be rotated regularly - Consider using IAM role assumption (S3RoleCredentials) for production - Never commit access keys to version control Use Cases: - Quick prototyping and development - Testing S3 integrations - Temporary credentials with session_token for enhanced security - Accessing S3-compatible services (MinIO, DigitalOcean Spaces) Recommended Alternative: For production deployments, use S3RoleCredentials with IAM role assumption instead of access keys for better security and credential management. S3RoleCredentials: properties: type: type: string const: iam_role title: Type default: iam_role role_arn: type: string pattern: ^arn:aws:iam::\d{12}:role/[\w+=,.@-]+$ title: Role Arn description: >- REQUIRED. Amazon Resource Name (ARN) of the IAM role to assume. This role must exist in the customer's AWS account and have a trust relationship configured to allow Mixpeek to assume it. Format: arn:aws:iam::{account-id}:role/{role-name} Example trust policy should allow principal: arn:aws:iam::{mixpeek-account}:root Recommended role name: mixpeek-storage-sync-role examples: - arn:aws:iam::123456789012:role/mixpeek-storage-sync-role - arn:aws:iam::999888777666:role/external-ingestion-access external_id: type: string maxLength: 128 minLength: 8 title: External Id description: >- REQUIRED. External ID for secure role assumption (prevents confused deputy attacks). This value should be unique to your organization and kept confidential. Mixpeek provides this value during connection setup. Must match the ExternalId condition in the role's trust policy. Format: Recommended pattern is mixpeek-{organization_id} Security: Include this in the trust policy Condition statement examples: - mixpeek-org_abc123def456ghi - mixpeek-customer-enterprise-xyz type: object required: - role_arn - external_id title: S3RoleCredentials description: >- AWS S3 IAM role assumption credentials (RECOMMENDED for production). IAM role assumption provides secure, temporary credentials for accessing customer S3 buckets without sharing long-lived access keys. This is the recommended authentication method for production deployments. How It Works: 1. Customer creates an IAM role in their AWS account 2. Role trust policy allows Mixpeek AWS account to assume the role 3. External ID provides additional security against confused deputy attacks 4. Mixpeek assumes the role and receives temporary credentials (auto-renewed) 5. Temporary credentials are used to access the customer's S3 bucket Prerequisites: 1. Create IAM role in customer AWS account 2. Attach policy granting s3:GetObject, s3:ListBucket permissions 3. Configure trust relationship to allow Mixpeek account 4. Use organization-specific external_id for security 5. Share role ARN with Mixpeek Security Advantages: - No long-lived credentials shared with third parties - Temporary credentials automatically rotate (1-hour sessions by default) - Customer retains full control and can revoke access anytime - External ID prevents confused deputy attacks - Audit trail in CloudTrail for all access Use Cases: - Production deployments accessing customer S3 buckets - Enterprise integrations requiring strong security - Multi-tenant environments with customer-owned storage - Compliance-sensitive workloads (HIPAA, SOC 2, etc.) examples: - description: Production role assumption with external ID external_id: mixpeek-org_abc123def456ghi role_arn: arn:aws:iam::123456789012:role/mixpeek-storage-sync-role type: iam_role trust_policy_example: Statement: - Action: sts:AssumeRole Condition: StringEquals: sts:ExternalId: mixpeek-org_abc123def456ghi Effect: Allow Principal: AWS: arn:aws:iam::{mixpeek-account-id}:root Version: '2012-10-17' SnowflakeKeyPairCredentials: properties: type: type: string const: key_pair title: Type default: key_pair username: type: string title: Username description: Snowflake username for authentication examples: - MIXPEEK_SYNC_USER - data_ingestion private_key: type: string title: Private Key description: >- REQUIRED. PEM-encoded RSA private key for authentication. SECURITY: This field is encrypted at rest via CSFLE. Never log or expose. Format: -----BEGIN PRIVATE KEY-----...-----END PRIVATE KEY----- private_key_passphrase: anyOf: - type: string - type: 'null' title: Private Key Passphrase description: >- NOT REQUIRED. Passphrase for encrypted private key. SECURITY: Encrypted at rest if provided. Use only if private key is passphrase-protected. type: object required: - username - private_key title: SnowflakeKeyPairCredentials description: >- Snowflake key pair authentication (RECOMMENDED for production). Key pair authentication provides secure, password-less access to Snowflake. The private key is encrypted at rest using MongoDB CSFLE. Prerequisites: 1. Generate RSA key pair (2048-bit minimum) 2. Extract public key and assign to Snowflake user 3. Store private key securely (encrypted) Security: - Private key encrypted at rest via CSFLE - No password exposure - Key rotation supported - Recommended for production Example: Generate key pair: ```bash openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub ``` Assign public key to Snowflake user: ```sql ALTER USER mixpeek_sync SET RSA_PUBLIC_KEY='MIIBIjANBg...'; ``` SnowflakeUsernamePasswordCredentials: properties: type: type: string const: username_password title: Type default: username_password username: type: string title: Username description: REQUIRED. Snowflake username (case-insensitive). examples: - MIXPEEK_SYNC_USER - data_ingestion password: type: string title: Password description: >- REQUIRED. Snowflake password for authentication. SECURITY: This field is encrypted at rest via CSFLE. Never log or expose. type: object required: - username - password title: SnowflakeUsernamePasswordCredentials description: |- Snowflake username/password authentication (FALLBACK option). Traditional username/password authentication for Snowflake. Less secure than key pair authentication but simpler to set up. Security: - Password encrypted at rest via CSFLE - Consider using key pair auth for production - Enable MFA on Snowflake user account SharePointClientCredentials: properties: type: type: string const: client_credentials title: Type default: client_credentials tenant_id: type: string title: Tenant Id description: >- REQUIRED. Azure AD tenant ID (directory ID). Find in: Azure Portal > Azure Active Directory > Overview. Format: UUID (e.g., '12345678-1234-1234-1234-123456789abc') examples: - 12345678-1234-1234-1234-123456789abc client_id: type: string title: Client Id description: >- REQUIRED. Azure AD application (client) ID. Find in: Azure Portal > App Registrations > Your App > Overview. Format: UUID examples: - 87654321-4321-4321-4321-cba987654321 client_secret: type: string title: Client Secret description: >- REQUIRED. Azure AD client secret for authentication. SECURITY: This field is encrypted at rest via CSFLE. Never log or expose. Generate in: Azure Portal > App Registrations > Your App > Certificates & secrets. Note: Secrets expire; consider using certificates for production. type: object required: - tenant_id - client_id - client_secret title: SharePointClientCredentials description: >- SharePoint/OneDrive client credentials (app-only) authentication. Client credentials flow provides application-level access without user interaction. Recommended for automated sync operations in enterprise environments. Prerequisites: 1. Register an application in Azure AD (portal.azure.com) 2. Grant Microsoft Graph API permissions: - Sites.Read.All (Application) - for SharePoint site access - Files.Read.All (Application) - for file access 3. Admin consent granted for the permissions 4. Generate client secret Security: - client_secret encrypted at rest via CSFLE - Provides application-level access to all sites (based on permissions) - No user context - accesses files as the application itself - Consider using certificate-based auth for production Use Cases: - Automated enterprise-wide document ingestion - Background sync without user interaction - Multi-tenant applications with admin consent SharePointDelegatedCredentials: properties: type: type: string const: delegated title: Type default: delegated tenant_id: type: string title: Tenant Id description: >- REQUIRED. Azure AD tenant ID. Use 'common' for multi-tenant apps, or specific tenant ID for single-tenant. examples: - 12345678-1234-1234-1234-123456789abc - common client_id: type: string title: Client Id description: REQUIRED. Azure AD application (client) ID. examples: - 87654321-4321-4321-4321-cba987654321 client_secret: type: string title: Client Secret description: >- REQUIRED. Azure AD client secret. SECURITY: Encrypted at rest via CSFLE. refresh_token: type: string title: Refresh Token description: >- REQUIRED. OAuth2 refresh token obtained from consent flow. SECURITY: Encrypted at rest. Can be revoked by user. Obtain via: Complete OAuth flow with Files.Read.All scope. type: object required: - tenant_id - client_id - client_secret - refresh_token title: SharePointDelegatedCredentials description: |- SharePoint/OneDrive delegated (user) authentication via OAuth2. Delegated flow provides access on behalf of a specific user. Useful when you need to access files with user-level permissions. Prerequisites: 1. Register an application in Azure AD 2. Grant Microsoft Graph API permissions: - Sites.Read.All (Delegated) or Sites.Selected - Files.Read.All (Delegated) or Files.Read 3. Configure redirect URI for OAuth flow 4. Complete OAuth consent to obtain refresh token Security: - client_secret and refresh_token encrypted at rest - Access scoped to what the consenting user can access - Refresh tokens can be revoked by user or admin Use Cases: - Personal OneDrive access - User-specific SharePoint sites - Respecting per-user permissions TigrisAccessKeyCredentials: properties: type: type: string const: access_key title: Type default: access_key access_key_id: type: string minLength: 1 title: Access Key Id description: >- REQUIRED. Tigris access key ID for authentication. Obtain from: Tigris Console > Access Keys examples: - tid_AzureDiamond secret_access_key: type: string minLength: 1 title: Secret Access Key description: >- REQUIRED. Tigris secret access key for authentication. SECURITY: This field is encrypted at rest. Never log or expose this value. Obtain from: Tigris Console when creating access key (shown only once) type: object required: - access_key_id - secret_access_key title: TigrisAccessKeyCredentials description: >- Tigris Data access key credentials. Tigris uses S3-compatible authentication with access keys. Credentials can be obtained from the Tigris dashboard at https://console.tigris.dev. Prerequisites: - Create a Tigris account at https://www.tigrisdata.com - Create a bucket in the Tigris console - Generate access keys from the dashboard Security: - secret_access_key is encrypted at rest using MongoDB CSFLE - Rotate keys regularly via the Tigris dashboard - Use bucket-scoped keys when possible for least privilege Use Cases: - Globally distributed object storage - Low-latency content delivery - S3-compatible workflows with zero egress fees PostgreSQLCredentials: properties: type: type: string const: username_password title: Type default: username_password username: type: string title: Username description: REQUIRED. PostgreSQL username for authentication. examples: - mixpeek_sync - readonly_user password: type: string title: Password description: >- REQUIRED. PostgreSQL password for authentication. SECURITY: This field is encrypted at rest via CSFLE. Never log or expose. type: object required: - username - password title: PostgreSQLCredentials description: |- PostgreSQL username/password authentication. Standard username/password authentication for PostgreSQL databases. Password is encrypted at rest using MongoDB CSFLE. Security: - password field is encrypted at rest via CSFLE - Consider using SSL mode 'require' for production - Use dedicated read-only database user for sync operations InstagramOAuthCredentials: properties: type: type: string const: oauth title: Type default: oauth client_id: type: string title: Client Id description: Instagram App ID from Meta Developer Console. client_secret: type: string title: Client Secret description: >- SECURITY: Encrypted at rest via CSFLE. Instagram App Secret from Meta Developer Console. access_token: type: string title: Access Token description: >- SECURITY: Encrypted at rest. Long-lived Instagram access token (60-day validity). token_expires_at: anyOf: - type: string format: date-time - type: 'null' title: Token Expires At description: Token expiration timestamp. Used to trigger proactive refresh. instagram_user_id: type: string title: Instagram User Id description: Instagram User ID of the account that authorized access. instagram_username: anyOf: - type: string - type: 'null' title: Instagram Username description: Instagram username for display purposes. type: object required: - client_id - client_secret - access_token - instagram_user_id title: InstagramOAuthCredentials description: |- Credentials for Instagram Graph API OAuth2 authentication. Instagram uses Meta's OAuth2 flow to obtain access tokens. The flow produces a short-lived token which is exchanged for a long-lived token (valid for 60 days). Long-lived tokens can be refreshed before expiry. Prerequisites: - Meta Developer account with an Instagram app - Instagram Business or Creator account linked to a Facebook Page - App must have instagram_basic permission approved Security: - client_secret and access_token encrypted at rest via CSFLE - Long-lived tokens must be refreshed before 60-day expiry - Token refresh happens automatically during sync execution TikTokOAuthCredentials: properties: type: type: string const: oauth title: Type default: oauth client_key: type: string title: Client Key description: TikTok App Client Key from Developer Portal. client_secret: type: string title: Client Secret description: >- SECURITY: Encrypted at rest via CSFLE. TikTok App Client Secret from Developer Portal. access_token: type: string title: Access Token description: 'SECURITY: Encrypted at rest. TikTok access token (24h validity).' refresh_token: type: string title: Refresh Token description: 'SECURITY: Encrypted at rest. Used to obtain new access tokens.' token_expires_at: anyOf: - type: string format: date-time - type: 'null' title: Token Expires At description: Access token expiration timestamp. open_id: type: string title: Open Id description: TikTok user's open_id from the authorization response. type: object required: - client_key - client_secret - access_token - refresh_token - open_id title: TikTokOAuthCredentials description: |- Credentials for TikTok Content API OAuth2 authentication. TikTok uses OAuth2 via TikTok Login Kit. Access tokens are short-lived (24 hours) and must be refreshed using the refresh token. Prerequisites: - TikTok Developer account with an approved app - App must have user.info.basic and video.list scopes Security: - client_secret, access_token, and refresh_token encrypted at rest via CSFLE - Access tokens expire in 24h; refresh tokens used for renewal - Token refresh happens automatically during sync execution RSSHttpHeaderCredentials: properties: type: type: string const: http_headers title: Type default: http_headers headers: additionalProperties: type: string type: object title: Headers description: HTTP headers for feed requests (e.g., Authorization). type: object title: RSSHttpHeaderCredentials description: Optional HTTP credentials for private RSS feeds. HTTPAPIHeaderCredentials: properties: type: type: string const: http_headers title: Type default: http_headers headers: additionalProperties: type: string type: object title: Headers description: HTTP headers for API requests (e.g., Authorization, X-API-Key). type: object title: HTTPAPIHeaderCredentials description: >- Optional HTTP credentials for HTTP API endpoints. Supports arbitrary headers for authentication (API keys, Bearer tokens, etc.). Same pattern as RSS credentials but for REST API endpoints. Security: - Header values containing secrets are encrypted at rest via CSFLE - Common patterns: Authorization: Bearer , X-API-Key: BoxOAuthCredentials: properties: type: type: string const: oauth title: Type default: oauth client_id: type: string title: Client Id description: >- REQUIRED. Box application client ID. Found in: Box Developer Console > Your App > Configuration > OAuth 2.0 Credentials. examples: - abc123def456ghi789 client_secret: type: string title: Client Secret description: >- REQUIRED. Box application client secret. SECURITY: Encrypted at rest via CSFLE. Never log or expose. Found in: Box Developer Console > Your App > Configuration. access_token: type: string title: Access Token description: >- REQUIRED. Box OAuth 2.0 access token. SECURITY: Encrypted at rest. Expires in ~60 minutes. refresh_token: type: string title: Refresh Token description: >- REQUIRED. Box OAuth 2.0 refresh token for automatic token renewal. SECURITY: Encrypted at rest. Single-use; new one issued on each refresh. type: object required: - client_id - client_secret - access_token - refresh_token title: BoxOAuthCredentials description: >- Credentials for Box OAuth 2.0 authentication. Box supports OAuth 2.0 with access and refresh tokens. The refresh token is used to automatically obtain new access tokens without user interaction. Prerequisites: - Create a Box application at https://developer.box.com - Configure OAuth 2.0 with the appropriate scopes - Complete the OAuth consent flow to obtain tokens Security: - client_secret, access_token, and refresh_token encrypted at rest via CSFLE - Access tokens expire in ~60 minutes; refresh tokens used for renewal - Token refresh happens automatically during sync execution BoxCCGCredentials: properties: type: type: string const: ccg title: Type default: ccg client_id: type: string title: Client Id description: REQUIRED. Box application client ID from Developer Console. examples: - abc123def456ghi789 client_secret: type: string title: Client Secret description: >- REQUIRED. Box application client secret. SECURITY: Encrypted at rest via CSFLE. enterprise_id: anyOf: - type: string - type: 'null' title: Enterprise Id description: >- Enterprise ID to authenticate as. Required when using CCG to act as the enterprise (service account). Find in: Box Admin Console > Enterprise Settings. examples: - '12345678' user_id: anyOf: - type: string - type: 'null' title: User Id description: >- User ID to authenticate as. Used when the app needs to act as a specific managed user. Mutually exclusive with enterprise_id for token acquisition. examples: - '87654321' type: object required: - client_id - client_secret title: BoxCCGCredentials description: |- Credentials for Box Client Credentials Grant (CCG) authentication. CCG provides server-to-server authentication without user interaction. Recommended for enterprise and automated sync operations. Prerequisites: - Create a Box application with Server Authentication (Client Credentials Grant) - Authorize the application in the Box Admin Console - Optionally configure an enterprise or user ID to act as Security: - client_secret encrypted at rest via CSFLE - No user tokens involved; app authenticates as itself or as a user/enterprise BoxJWTCredentials: properties: type: type: string const: jwt title: Type default: jwt client_id: type: string title: Client Id description: REQUIRED. Box application client ID. examples: - abc123def456ghi789 client_secret: type: string title: Client Secret description: >- REQUIRED. Box application client secret. SECURITY: Encrypted at rest via CSFLE. enterprise_id: type: string title: Enterprise Id description: >- REQUIRED. Box enterprise ID for JWT authentication. Find in: Box Admin Console > Enterprise Settings. examples: - '12345678' jwt_key_id: type: string title: Jwt Key Id description: >- REQUIRED. Public key ID registered with Box. Found in the JSON config file as 'publicKeyID'. examples: - abcdef12 private_key: type: string title: Private Key description: >- REQUIRED. PEM-encoded RSA private key for JWT signing. SECURITY: Encrypted at rest via CSFLE. Never log or expose. Found in the JSON config file as 'privateKey'. private_key_passphrase: anyOf: - type: string - type: 'null' title: Private Key Passphrase description: >- Passphrase for the private key if it is encrypted. SECURITY: Encrypted at rest via CSFLE. type: object required: - client_id - client_secret - enterprise_id - jwt_key_id - private_key title: BoxJWTCredentials description: >- Credentials for Box JWT (JSON Web Token) authentication. JWT provides server-to-server authentication using a public/private key pair. Recommended for enterprise integrations requiring high security. Prerequisites: - Create a Box application with Server Authentication (with JWT) - Generate a public/private key pair in the Developer Console - Authorize the application in the Box Admin Console - Download the JSON configuration file Security: - private_key encrypted at rest via CSFLE - RSA key pair used for signing JWT assertions - No user interaction required BrightDataCredentials: properties: type: type: string const: api_token title: Type default: api_token api_token: type: string title: Api Token description: >- REQUIRED. BrightData API token. SECURITY: Encrypted at rest. Never log or expose. Find in: BrightData Dashboard > Account > API Token. examples: - xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx type: object required: - api_token title: BrightDataCredentials description: |- API token credentials for BrightData platform. Security: - api_token is encrypted at rest via CSFLE - Tokens can be created/rotated in the BrightData dashboard BackblazeCredentials: properties: type: type: string const: application_key title: Type default: application_key key_id: type: string title: Key Id description: >- REQUIRED. Backblaze B2 application key ID. Found in: Backblaze Console > App Keys > keyID column. examples: - 005870eff85b6c60000000001 application_key: type: string title: Application Key description: >- REQUIRED. Backblaze B2 application key (secret). SECURITY: Encrypted at rest. Shown only once when key is created. Found in: Backblaze Console > App Keys — copy immediately on creation. examples: - K005xxxxxxxxxxxxxxxxxxxxxxxxxxxx type: object required: - key_id - application_key title: BackblazeCredentials description: |- Application key credentials for Backblaze B2 Cloud Storage. Security: - application_key is encrypted at rest via CSFLE - Keys can be created/rotated in the Backblaze console MuxCredentials: properties: type: type: string const: access_token title: Type default: access_token access_token_id: type: string title: Access Token Id description: >- REQUIRED. Mux access token ID. Found in: Mux Dashboard > Settings > Access Tokens. examples: - abc123def456 access_token_secret: type: string title: Access Token Secret description: >- REQUIRED. Mux access token secret. SECURITY: Encrypted at rest. Shown only once when created. Found in: Mux Dashboard > Settings > Access Tokens — copy immediately on creation. examples: - xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx webhook_secret: anyOf: - type: string - type: 'null' title: Webhook Secret description: >- NOT REQUIRED. Mux webhook signing secret for verifying inbound webhook events (e.g. video.asset.deleted, which cascades deletes to Mixpeek bucket objects synced from this connection). Set this to enable cascade deletion when assets are removed from Mux. SECURITY: Encrypted at rest. Find in: Mux Dashboard > Settings > Webhooks > {webhook} > Signing Secret. After saving, register the URL https://api.mixpeek.com/v1/webhooks/mux/{connection_id} in your Mux dashboard. type: object required: - access_token_id - access_token_secret title: MuxCredentials description: |- Access token credentials for Mux video infrastructure platform. Security: - access_token_secret is encrypted at rest via CSFLE - Tokens can be created/rotated in the Mux dashboard IconikCredentials: properties: type: type: string const: app_token title: Type default: app_token app_id: type: string title: App Id description: >- REQUIRED. Iconik application ID. Found in: Iconik Admin > Settings > API Applications. examples: - e0f0ec84-4d63-11f1-99fa-b221c0ff30c8 auth_token: type: string title: Auth Token description: >- REQUIRED. Iconik auth token (JWT or API token). SECURITY: Encrypted at rest. Found in: Iconik Admin > Settings > API Applications > Token. webhook_secret: anyOf: - type: string - type: 'null' title: Webhook Secret description: >- NOT REQUIRED. Webhook secret for verifying inbound Iconik webhook events. Set this to enable webhook-driven sync (asset create/update/delete). SECURITY: Encrypted at rest. type: object required: - app_id - auth_token title: IconikCredentials description: |- App-ID + Auth-Token credentials for Iconik DAM platform. Security: - auth_token is encrypted at rest via CSFLE - Tokens are created in Iconik Admin > Settings > API Applications EmailCredentials: properties: type: type: string const: webhook_secret title: Type default: webhook_secret webhook_secret: type: string title: Webhook Secret description: >- Shared secret for verifying inbound email webhook signatures. Auto-generated on connection creation if left empty. SECURITY: Encrypted at rest via CSFLE. default: '' type: object title: EmailCredentials description: |- Webhook signing credentials for inbound email verification. Security: - webhook_secret is encrypted at rest via CSFLE - Used to verify HMAC-SHA256 signatures on inbound email webhooks ````