> ## Documentation Index > Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Create Upload > Generate a presigned URL for direct S3 upload. This endpoint validates all requirements BEFORE generating the presigned URL, ensuring immediate feedback if something is wrong (bucket inactive, quota exceeded, etc.). **Duplicate Detection (Enabled by Default)**: - If `file_hash` provided and `skip_duplicates=true`: Checks for existing upload - If duplicate found: Returns existing upload (200 OK) with `is_duplicate=true` - If new file: Returns presigned URL (201 Created) with `is_duplicate=false` **Two-Step Flow**: 1. Call this endpoint → Get presigned URL 2. PUT file to presigned URL → Upload directly to S3 3. Call confirm endpoint → Verify upload and create object ## OpenAPI ````yaml post /v1/buckets/{bucket_identifier}/uploads openapi: 3.1.0 info: title: Mixpeek API description: >- This is the Mixpeek API, providing access to various endpoints for data processing and retrieval. termsOfService: https://mixpeek.com/terms contact: name: Mixpeek Support url: https://mixpeek.com/contact email: info@mixpeek.com version: '0.82' servers: - url: https://api.mixpeek.com description: Production security: [] paths: /v1/buckets/{bucket_identifier}/uploads: post: tags: - Bucket Uploads summary: Create Upload description: |- Generate a presigned URL for direct S3 upload. This endpoint validates all requirements BEFORE generating the presigned URL, ensuring immediate feedback if something is wrong (bucket inactive, quota exceeded, etc.). **Duplicate Detection (Enabled by Default)**: - If `file_hash` provided and `skip_duplicates=true`: Checks for existing upload - If duplicate found: Returns existing upload (200 OK) with `is_duplicate=true` - If new file: Returns presigned URL (201 Created) with `is_duplicate=false` **Two-Step Flow**: 1. Call this endpoint → Get presigned URL 2. PUT file to presigned URL → Upload directly to S3 3. Call confirm endpoint → Verify upload and create object operationId: create_upload_v1_buckets__bucket_identifier__uploads_post parameters: - name: bucket_identifier in: path required: true schema: type: string description: The unique identifier of the bucket title: Bucket Identifier description: The unique identifier of the bucket requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateUploadRequest' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/UploadResponse' '400': description: Bad Request content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '401': description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '403': description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '404': description: Not Found content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' '500': description: Internal Server Error content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' components: schemas: CreateUploadRequest: properties: filename: type: string maxLength: 255 minLength: 1 title: Filename description: >- Name of the file to upload. REQUIRED. Must be a valid filename without path traversal characters (../, \). The filename is used to derive the blob_property if not explicitly provided. Examples: 'product_video.mp4', 'thumbnail.jpg', 'transcript.txt' examples: - product_video.mp4 - thumbnail.jpg - document.pdf content_type: type: string title: Content Type description: >- MIME type of the file. REQUIRED. Must be a valid MIME type (e.g., 'video/mp4', 'image/jpeg', 'application/pdf'). The presigned URL will enforce this content type during upload. Used to validate compatibility with bucket schema if create_object_on_confirm=true. examples: - video/mp4 - image/jpeg - image/png - application/pdf - text/plain file_size_bytes: anyOf: - type: integer minimum: 1 - type: 'null' title: File Size Bytes description: >- Expected file size in bytes. OPTIONAL but RECOMMENDED. If provided, will be validated against: 1. Tier-based file size limits (100MB free, 5GB pro, 50GB enterprise) 2. Storage quota availability 3. Actual uploaded file size during confirmation. If not provided, quota checking is skipped until confirmation. examples: - 10485760 - 104857600 presigned_url_expiration: type: integer maximum: 86400 minimum: 60 title: Presigned Url Expiration description: >- How long the presigned URL is valid, in seconds. OPTIONAL, defaults to 3600 (1 hour). Valid range: 60 seconds (1 minute) to 86400 seconds (24 hours). After expiration, the URL cannot be used and you must request a new one. Recommendation: Use shorter expiration (300-900 seconds) for security-sensitive files, longer expiration (3600-7200 seconds) for large files that take time to upload. default: 3600 examples: - 3600 - 7200 - 900 metadata: additionalProperties: true type: object title: Metadata description: >- Custom metadata for tracking purposes. OPTIONAL. Stored with the upload record for filtering and analytics. Does NOT affect the created bucket object (use object_metadata for that). Common uses: campaign tracking, user identification, upload source. examples: - campaign: summer_2024 source: mobile_app user_id: user_123 - department: marketing priority: high create_object_on_confirm: type: boolean title: Create Object On Confirm description: >- Whether to automatically create a bucket object when upload is confirmed. OPTIONAL, defaults to TRUE (object is created automatically). If true (default): - Bucket MUST have a schema defined - blob_property must exist in bucket schema - content_type must match schema field type - Validation happens BEFORE generating presigned URL - Object is created automatically on confirmation. If false: - Upload is confirmed but no object is created - Use this when combining multiple uploads into one object - Reference the upload_id later in POST /buckets/{id}/objects. default: true object_metadata: anyOf: - additionalProperties: true type: object - type: 'null' title: Object Metadata description: >- Metadata to attach to the created bucket object. OPTIONAL. Only used if create_object_on_confirm=true. This metadata will be: 1. Validated against bucket schema (if keys match schema fields) 2. Attached to the bucket object 3. Passed to downstream documents in connected collections. Example: {'priority': 'high', 'category': 'products', 'tags': ['featured']} examples: - category: products priority: high - campaign: q4_2024 status: draft tags: - review - featured blob_property: anyOf: - type: string pattern: ^[a-zA-Z0-9_]+$ - type: 'null' title: Blob Property description: >- Property name for the blob in the bucket object. OPTIONAL. Defaults to filename without extension (e.g., 'product_video.mp4' → 'product_video'). If create_object_on_confirm=true: - Must exist in bucket schema - Must be alphanumeric with underscores only - Will be validated BEFORE generating presigned URL. Common values: 'video', 'image', 'thumbnail', 'transcript', 'content'. examples: - video - thumbnail - transcript - content blob_type: anyOf: - type: string - type: 'null' title: Blob Type description: >- Type of blob. OPTIONAL. Defaults to type derived from content_type (e.g., 'video/mp4' → 'VIDEO'). Must be a valid BucketSchemaFieldType if provided. Valid values: IMAGE, VIDEO, AUDIO, TEXT, PDF, DOCUMENT, etc. If create_object_on_confirm=true, will be validated against bucket schema field type. examples: - VIDEO - IMAGE - AUDIO - TEXT - PDF file_hash: anyOf: - type: string maxLength: 64 minLength: 64 pattern: ^[a-f0-9]{64}$ - type: 'null' title: File Hash description: >- SHA256 hash of the file content for duplicate detection. OPTIONAL. If provided: - System checks for existing confirmed uploads with same hash - If duplicate found and skip_duplicates=true, returns existing upload - Hash will be validated against actual S3 ETag during confirmation. If not provided: - Hash is calculated from S3 ETag after upload - Duplicate detection only happens during confirmation. Use case: Pre-calculate hash client-side to avoid uploading duplicates. Format: 64-character hexadecimal string (SHA256). examples: - e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 - 2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae skip_duplicates: type: boolean title: Skip Duplicates description: >- Skip upload if a file with the same hash already exists. OPTIONAL, defaults to TRUE. If true (default): - If file_hash provided: System checks MongoDB for existing completed upload with same hash - If duplicate found: Returns existing upload details WITHOUT generating new presigned URL - If file_hash NOT provided: Duplicate check happens during confirmation using S3 ETag - Saves bandwidth, storage, and upload time by reusing existing files. If false: - Always generates new presigned URL even if file already uploaded - Creates separate upload record for same file content - Useful when you need distinct upload tracking for identical files. Recommendation: Keep default (true) unless you specifically need multiple upload records for same file. default: true type: object required: - filename - content_type title: CreateUploadRequest description: >- Request to generate a presigned URL for direct S3 upload. ⚠️ ⚠️ ⚠️ THIS IS THE PRESIGNED URL SYSTEM ⚠️ ⚠️ ⚠️ This endpoint (POST /buckets/{id}/uploads) is the COMPLETE presigned URL system. It handles: - ✅ Presigned URL generation (S3 PUT URLs) - ✅ Upload tracking and status management - ✅ Validation (quotas, file size, content type, schema) - ✅ Duplicate detection - ✅ Auto object creation on confirmation - ✅ Returns upload_id for later reference DO NOT CREATE A NEW PRESIGNED UPLOAD ENDPOINT! If you need presigned URLs, use this existing system. If you think you need a new endpoint: 1. Check if this system already does it (it probably does) 2. Extend this system instead of creating redundancy 3. See api/buckets/uploads/services.py for implementation Integration Points: - Object creation: Use upload_id in CreateBlobRequest.upload_id field - See: shared/buckets/objects/blobs/models.py::CreateBlobRequest - See: api/buckets/objects/canonicalization.py::resolve_upload_reference() Workflow: 1. POST /buckets/{id}/uploads → Returns presigned_url + upload_id 2. PUT presigned_url with file content (client uploads directly to S3) 3. POST /uploads/{upload_id}/confirm → REQUIRED to finalize upload 4. Object is created automatically (default behavior) ⚠️ IMPORTANT: Step 3 (confirm) is REQUIRED! S3 presigned URLs have no callback mechanism - the API cannot detect when your upload to S3 completes. You MUST call the confirm endpoint to: - Verify the file exists in S3 - Validate integrity (ETag/size) - Create the bucket object - Mark upload as COMPLETED If you don't confirm: - Upload stays in PENDING status forever - No object is created - File exists in S3 but is orphaned - Presigned URL expires (default: 1 hour) Use Cases: - Simple: Upload → confirm → object created automatically (default) - Advanced: Upload multiple files with create_object_on_confirm=false, then POST /buckets/{id}/objects with all upload_ids to create one object Requirements: - filename: REQUIRED, will be validated (no path traversal) - content_type: REQUIRED, must be valid MIME type - bucket_id: Comes from URL path parameter, not request body - All other fields: OPTIONAL with sensible defaults Note: The bucket_id comes from the URL path (/v1/buckets/{bucket_id}/uploads), not from the request body. The bucket is validated before generating presigned URL. examples: - content_type: video/mp4 description: Minimal upload - object created automatically on confirm (DEFAULT) file_size_bytes: 52428800 filename: product_demo.mp4 - blob_property: image content_type: image/jpeg description: Upload with metadata - object still created automatically file_size_bytes: 2097152 filename: profile.jpg object_metadata: category: avatars title: Profile Image - blob_property: raw_video content_type: video/quicktime description: Large file with extended expiration file_size_bytes: 5368709120 filename: 4k_footage.mov presigned_url_expiration: 7200 - content_type: video/mp4 create_object_on_confirm: false description: >- ADVANCED: Manual object creation - set create_object_on_confirm=false file_size_bytes: 104857600 filename: video.mp4 note: Only use this when combining multiple uploads into one object use_case: Combine multiple uploads into one object workflow: - 1. POST /uploads (video) with create_object_on_confirm=false - 2. POST /uploads (thumbnail) with create_object_on_confirm=false - 3. Upload both files to S3 - 4. Confirm both uploads - '5. POST /buckets/{id}/objects with both upload_ids:' - ' {blobs: [{property: ''video'', upload_id: ''upl_video''}, {property: ''thumb'', upload_id: ''upl_thumb''}]}' UploadResponse: properties: upload_id: type: string title: Upload Id description: >- Unique identifier for this upload. Auto-generated. ⚠️ NEXT STEP: After uploading to S3, you MUST confirm: POST /v1/uploads/{upload_id}/confirm Other operations: - Check status: GET /v1/uploads/{upload_id} - Cancel upload: DELETE /v1/uploads/{upload_id} Format: 'upl_' followed by 16 random characters. examples: - upl_a1b2c3d4e5f6 - upl_x9y8z7w6v5u4 bucket_id: type: string title: Bucket Id description: Target bucket ID where object will be created examples: - bkt_abc123 filename: type: string title: Filename description: Name of the file to upload examples: - product_video.mp4 content_type: type: string title: Content Type description: MIME type enforced by the presigned URL examples: - video/mp4 file_size_bytes: anyOf: - type: integer - type: 'null' title: File Size Bytes description: >- Expected file size in bytes if provided in request. Will be validated during confirmation. examples: - 10485760 presigned_url: anyOf: - type: string maxLength: 2083 minLength: 1 format: uri - type: 'null' title: Presigned Url description: >- Time-limited HTTPS URL for uploading directly to S3. **Step 1 - Upload to S3:** curl -X PUT '{presigned_url}' -H 'Content-Type: {content_type}' --upload-file {filename} **Step 2 - REQUIRED: Confirm the upload:** POST /v1/uploads/{upload_id}/confirm (S3 has no callback - you MUST call confirm to finalize) The URL includes authentication and expires after presigned_url_expiration seconds. S3 returns an ETag header on success - pass it to confirm for integrity validation. NOTE: This will be null if is_duplicate=true (duplicate found, no upload needed). presigned_url_expiration: type: integer title: Presigned Url Expiration description: How long the presigned URL is valid, in seconds examples: - 3600 s3_key: type: string title: S3 Key description: >- Full S3 object key where the file will be stored. Format: {internal_id}/{namespace_id}/api_buckets_uploads_create/{upload_id}/{filename}. Used internally for verification and object creation. examples: - int_xyz/ns_abc/api_buckets_uploads_create/upl_123/video.mp4 status: $ref: '#/components/schemas/TaskStatusEnum' description: >- Current upload status. After creation, always PENDING. Possible statuses: PENDING → IN_PROGRESS → PROCESSING → COMPLETED/FAILED/CANCELED examples: - PENDING metadata: additionalProperties: true type: object title: Metadata description: Custom metadata for tracking create_object_on_confirm: type: boolean title: Create Object On Confirm description: Whether bucket object will be auto-created on confirmation object_metadata: anyOf: - additionalProperties: true type: object - type: 'null' title: Object Metadata description: Metadata for the bucket object (if create_object_on_confirm=true) blob_property: anyOf: - type: string - type: 'null' title: Blob Property description: Property name for the blob in bucket object blob_type: anyOf: - type: string - type: 'null' title: Blob Type description: Type of blob (IMAGE, VIDEO, etc.) file_hash: anyOf: - type: string - type: 'null' title: File Hash description: >- SHA256 hash of the file content. Set during confirmation from S3 metadata or provided in request. Used for duplicate detection. examples: - e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 skip_duplicates: type: boolean title: Skip Duplicates description: Whether duplicate detection was enabled for this upload default: true is_duplicate: type: boolean title: Is Duplicate description: >- Whether this upload was identified as a duplicate of an existing file. If true: - duplicate_of_upload_id contains the original upload - presigned_url will be null (no upload needed) - You can use the original upload's S3 object. This saves bandwidth and storage costs. default: false duplicate_of_upload_id: anyOf: - type: string - type: 'null' title: Duplicate Of Upload Id description: >- If skip_duplicates=true and duplicate found, this is the original upload_id. The response will reference the existing upload instead of creating a new one. examples: - upl_original123 skipped_unique_key: type: boolean title: Skipped Unique Key description: >- Whether this upload was skipped because the unique key already exists in the bucket. If true: - existing_object_id contains the ID of the existing object - presigned_url will be null (no upload needed) - No S3 upload is required This saves bandwidth and prevents duplicate objects. default: false existing_object_id: anyOf: - type: string - type: 'null' title: Existing Object Id description: >- If skipped_unique_key=true, this is the object_id of the existing object that has the same unique key values. The upload was skipped to prevent duplicates. examples: - obj_abc123xyz789 message: anyOf: - type: string - type: 'null' title: Message description: >- Human-readable message about the upload. Provided when is_duplicate=true or other special conditions. Example: 'File already exists with the same content hash. No upload needed - returning existing upload.' created_at: type: string format: date-time title: Created At description: When this upload record was created (ISO 8601 format) examples: - '2024-01-15T10:30:00Z' expires_at: type: string format: date-time title: Expires At description: >- When the presigned URL expires (ISO 8601 format). After this time: - The presigned URL cannot be used - Upload status will be marked as FAILED if not completed - The upload record will be auto-deleted 30 days later (MongoDB TTL) examples: - '2024-01-15T11:30:00Z' completed_at: anyOf: - type: string format: date-time - type: 'null' title: Completed At description: When the upload was completed and verified (ISO 8601 format) examples: - '2024-01-15T10:35:00Z' verified_at: anyOf: - type: string format: date-time - type: 'null' title: Verified At description: When S3 object existence was verified (ISO 8601 format) examples: - '2024-01-15T10:35:00Z' etag: anyOf: - type: string - type: 'null' title: Etag description: S3 ETag from the uploaded object (set during confirmation) examples: - d41d8cd98f00b204e9800998ecf8427e object_id: anyOf: - type: string - type: 'null' title: Object Id description: Created bucket object ID (if create_object_on_confirm was true) examples: - obj_a1b2c3d4e5f6 task_id: anyOf: - type: string - type: 'null' title: Task Id description: Task ID for async confirmation (if processed asynchronously) examples: - task_abc123 type: object required: - upload_id - bucket_id - filename - content_type - presigned_url_expiration - s3_key - status - create_object_on_confirm - created_at - expires_at title: UploadResponse description: >- Response containing presigned URL and upload tracking information. This response includes everything needed to: 1. Upload your file to S3 using the presigned_url 2. Track the upload status using upload_id 3. Confirm the upload using the confirmation endpoint The presigned_url is time-limited and specific to this upload. After uploading to S3, call POST /v1/buckets/{bucket_id}/uploads/{upload_id}/confirm. examples: - blob_property: video blob_type: VIDEO bucket_id: bkt_prod_videos content_type: video/mp4 create_object_on_confirm: true created_at: '2024-01-15T10:30:00Z' description: >- New upload ready for S3 - object will be created on confirm (DEFAULT) expires_at: '2024-01-15T11:30:00Z' file_size_bytes: 52428800 filename: product_demo.mp4 is_duplicate: false object_metadata: category: tutorials presigned_url: https://s3.amazonaws.com/bucket/key?X-Amz-Signature=... presigned_url_expiration: 3600 s3_key: >- int_xyz/ns_abc/api_buckets_uploads_create/upl_abc123xyz789/product_demo.mp4 skip_duplicates: true status: PENDING upload_id: upl_abc123xyz789 - bucket_id: bkt_prod_videos completed_at: '2024-01-10T08:25:00Z' content_type: video/mp4 created_at: '2024-01-10T08:20:00Z' description: Duplicate detected - no upload needed, existing object returned duplicate_of_upload_id: upl_original456 file_hash: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 file_size_bytes: 52428800 filename: product_demo.mp4 is_duplicate: true message: >- File already exists with the same content hash. No upload needed - returning existing upload. object_id: obj_existing789 s3_key: >- int_xyz/ns_abc/api_buckets_uploads_create/upl_original456/product_demo.mp4 status: COMPLETED upload_id: upl_original456 - bucket_id: bkt_media content_type: video/mp4 create_object_on_confirm: false created_at: '2024-01-15T10:30:00Z' description: 'ADVANCED: Manual object creation (create_object_on_confirm=false)' expires_at: '2024-01-15T11:30:00Z' filename: video.mp4 note: Only use when combining multiple uploads into one object presigned_url: https://s3.amazonaws.com/bucket/key?X-Amz-Signature=... presigned_url_expiration: 3600 s3_key: int_xyz/ns_abc/api_buckets_uploads_create/upl_manual789/video.mp4 status: PENDING upload_id: upl_manual789 workflow: - 1. User uploads file to presigned_url - >- 2. POST /uploads/upl_manual789/confirm (verifies upload, no object created) - '3. POST /buckets/{id}/objects with upload_id reference:' - ' {blobs: [{property: ''video'', upload_id: ''upl_manual789''}]}' ErrorResponse: properties: success: type: boolean title: Success description: Always false for error responses default: false status: type: integer title: Status description: HTTP status code for this error error: $ref: '#/components/schemas/ErrorDetail' description: Error details payload type: object required: - status - error title: ErrorResponse description: Error response model. examples: - error: details: id: ns_123 resource: namespace message: Namespace not found type: NotFoundError status: 404 success: false HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError TaskStatusEnum: type: string enum: - PENDING - QUEUED - IN_PROGRESS - PROCESSING - COMPLETED - COMPLETED_WITH_ERRORS - FAILED - CANCELED - INTERRUPTED - UNKNOWN - SKIPPED - DRAFT - ACTIVE - ARCHIVED - SUSPENDED title: TaskStatusEnum description: |- Enumeration of task statuses for tracking asynchronous operations. Task statuses indicate the current state of asynchronous operations like batch processing, object ingestion, clustering, and taxonomy execution. Status Categories: Operation Statuses: Track progress of async operations Lifecycle Statuses: Track entity state (buckets, collections, namespaces) Values: PENDING: Task is queued but has not started processing yet IN_PROGRESS: Task is currently being executed PROCESSING: Task is actively processing data (similar to IN_PROGRESS) COMPLETED: Task finished successfully with no errors COMPLETED_WITH_ERRORS: Task finished but some items failed (partial success) FAILED: Task encountered an error and could not complete CANCELED: Task was manually canceled by a user or system UNKNOWN: Task status could not be determined SKIPPED: Task was intentionally skipped DRAFT: Task is in draft state and not yet submitted ACTIVE: Entity is active and operational (for buckets, collections, etc.) ARCHIVED: Entity has been archived SUSPENDED: Entity has been temporarily suspended Terminal Statuses: COMPLETED, COMPLETED_WITH_ERRORS, FAILED, CANCELED are terminal statuses. Once a task reaches these states, it will not transition to another state. Partial Success Handling: COMPLETED_WITH_ERRORS indicates that the operation completed but some documents/items failed. The task result includes: - List of successful items - List of failed items with error details - Success rate percentage This allows clients to handle partial success scenarios appropriately. Polling Guidance: - Poll tasks in PENDING, QUEUED, IN_PROGRESS, or PROCESSING states - Stop polling when task reaches COMPLETED, COMPLETED_WITH_ERRORS, FAILED, or CANCELED - Use exponential backoff (1s → 30s) when polling ErrorDetail: properties: message: type: string title: Message description: Human-readable error message type: type: string title: Type description: Stable error type identifier (machine-readable) code: anyOf: - type: string - type: 'null' title: Code description: >- Fine-grained error code for programmatic handling (e.g., namespace_name_taken, feature_extractor_not_found). Present only when consumers may need to branch on a specific error condition. details: anyOf: - additionalProperties: true type: object - type: 'null' title: Details description: >- Optional structured details to help debugging (validation errors, IDs, etc.) type: object required: - message - type title: ErrorDetail description: Error detail model. ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type type: object required: - loc - msg - type title: ValidationError ````