API Call

The API Call stage enriches documents by calling external HTTP APIs. This enables integration with third-party services like Stripe, GitHub, weather APIs, and more to augment documents with real-time data.

Stage Category: APPLY (1-1 Enrichment)Transformation: N documents → N documents (same count, expanded schema)

When to Use

Use Case	Description
Customer data lookup	Enrich with Stripe billing data, CRM info
Repository info	Fetch GitHub commit stats, stars
Real-time data	Add weather, stock prices, currency rates
Data validation	Verify addresses, phone numbers
External context	Lookup additional context from any API

When NOT to Use

Scenario	Recommended Alternative
Untrusted/user-provided URLs	Major security risk (SSRF)
API credentials can’t be secured	Use organization secrets vault
High-volume enrichment	Rate limits apply
Time-critical responses	Network latency adds 100-500ms
Internal-only APIs behind firewalls	Use `sql_lookup` for databases

Parameters

Required Parameters

Parameter	Type	Description
`url`	string	API endpoint URL. Supports `{DOC.field}` and `{INPUT.field}` templates.
`allowed_domains`	string[]	Domain allowlist for SSRF protection. *Never use ``**.
`output_field`	string	Dot-path where API response should be stored (e.g., `metadata.stripe_data`).

Optional Parameters

Parameter	Type	Default	Description
`method`	string	`GET`	HTTP method: `GET`, `POST`, `PUT`, `PATCH`, `DELETE`
`auth`	object	`null`	Authentication configuration (see below)
`headers`	object	`{}`	Additional HTTP headers (supports templates)
`body`	object	`null`	Request body for POST/PUT/PATCH (JSON)
`timeout`	integer	`10`	Request timeout in seconds (1-60)
`max_response_size`	integer	`10485760`	Maximum response size in bytes (default: 10MB)
`response_path`	string	`null`	JSONPath to extract specific field from response
`rate_limit`	object	`null`	Rate limiting per domain
`when`	object	`null`	Conditional filter for selective enrichment
`on_error`	string	`skip`	Error handling: `skip`, `remove`, or `raise`

Authentication Types

Bearer Token
API Key
Basic Auth
Custom Header

OAuth 2.0, JWT tokens - Most modern APIs (GitHub, OpenAI, Stripe)

{
  "auth": {
    "type": "bearer",
    "secret_ref": "stripe_api_key"
  }
}

Adds header: Authorization: Bearer {secret_value}

Header or query parameter - Weather APIs, Maps, etc.

{
  "auth": {
    "type": "api_key",
    "key": "X-API-Key",
    "location": "header",
    "secret_ref": "weather_api_key"
  }
}

Field	Description
`key`	Header name or query param name
`location`	`header` (recommended) or `query`
`secret_ref`	Secret name in vault

HTTP Basic Authentication - Legacy systems

{
  "auth": {
    "type": "basic",
    "secret_ref": "basic_credentials"
  }
}

Secret format: username:passwordAdds header: Authorization: Basic {base64(username:password)}

Non-standard auth headers

{
  "auth": {
    "type": "custom_header",
    "key": "X-Custom-Auth",
    "secret_ref": "custom_token"
  }
}

Adds header: {key}: {secret_value}

Error Handling

Strategy	Behavior	Best For
`skip`	Keep document unchanged	Optional enrichment
`remove`	Remove document from results	Mandatory enrichment
`raise`	Fail entire pipeline	Debugging, critical failures

Configuration Examples

{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.stripe.com/v1/customers/{DOC.metadata.stripe_id}",
      "method": "GET",
      "allowed_domains": ["api.stripe.com"],
      "auth": {
        "type": "bearer",
        "secret_ref": "stripe_api_key"
      },
      "output_field": "metadata.stripe_data",
      "timeout": 10,
      "on_error": "skip"
    }
  }
}

{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.github.com/repos/{INPUT.owner}/{INPUT.repo}",
      "method": "GET",
      "allowed_domains": ["api.github.com"],
      "output_field": "metadata.github_info",
      "response_path": "$.stargazers_count"
    }
  }
}

{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.example.com/v1/analyze",
      "method": "POST",
      "allowed_domains": ["api.example.com"],
      "auth": {
        "type": "api_key",
        "key": "X-API-Key",
        "location": "header",
        "secret_ref": "example_api_key"
      },
      "headers": {
        "Content-Type": "application/json"
      },
      "body": {
        "text": "{DOC.content}",
        "language": "en"
      },
      "output_field": "metadata.analysis"
    }
  }
}

{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.stripe.com/v1/customers/{DOC.metadata.customer_id}",
      "method": "GET",
      "allowed_domains": ["api.stripe.com"],
      "auth": {
        "type": "bearer",
        "secret_ref": "stripe_api_key"
      },
      "output_field": "metadata.billing",
      "when": {
        "field": "metadata.is_premium",
        "operator": "eq",
        "value": true
      },
      "on_error": "skip"
    }
  }
}

{
  "stage_name": "api_call",
  "stage_type": "apply",
  "config": {
    "stage_id": "api_call",
    "parameters": {
      "url": "https://api.weatherapi.com/v1/current.json?q={DOC.metadata.location}",
      "method": "GET",
      "allowed_domains": ["api.weatherapi.com"],
      "auth": {
        "type": "api_key",
        "key": "key",
        "location": "query",
        "secret_ref": "weather_api_key"
      },
      "output_field": "metadata.weather",
      "response_path": "$.current",
      "rate_limit": {
        "requests_per_minute": 60
      },
      "timeout": 5
    }
  }
}

Security

SSRF Protection RequiredThis stage makes external HTTP requests which can be exploited for Server-Side Request Forgery attacks. Always use allowed_domains to whitelist permitted domains.

Security Best Practices

Never use * in allowed_domains - Explicitly list each domain
Never store credentials in configuration - Always use auth.secret_ref to reference vault secrets
Set rate limits - Prevent abuse and excessive costs
Use HTTPS - HTTP URLs are automatically upgraded
Audit configurations - Review before deployment to prevent data exfiltration

Storing Secrets

# Create a secret in the organization vault
curl -X POST "$MP_API_URL/v1/organizations/secrets" \
  -H "Authorization: Bearer $MP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "stripe_api_key",
    "secret_value": "mxp_sk_..."
  }'

Then reference in configuration:

{
  "auth": {
    "type": "bearer",
    "secret_ref": "stripe_api_key"
  }
}

Performance

Metric	Value
Latency per request	100-500ms (network dependent)
Timeout range	1-60 seconds
Max response size	10MB (configurable)
Parallelization	Documents processed concurrently

Template Variables

URLs, headers, and body values support template variables:

Namespace	Description	Example
`DOC`	Current document fields	`{DOC.metadata.customer_id}`
`INPUT`	Query inputs	`{INPUT.api_version}`

SQL Lookup - For database enrichment
Document Enrich - For collection joins
Organization Secrets - Managing API credentials

Get started

Connect your data

Extract features

Build retrievers

Enrich & organize

Integrate & operate

Resources

When to Use

When NOT to Use

Parameters

Required Parameters

Optional Parameters

Authentication Types

Error Handling

Configuration Examples

Security

Security Best Practices

Storing Secrets

Performance

Template Variables

​When to Use

​When NOT to Use

​Parameters

​Required Parameters

​Optional Parameters

​Authentication Types

​Error Handling

​Configuration Examples

​Security

​Security Best Practices

​Storing Secrets

​Performance

​Template Variables

​Related

When to Use

When NOT to Use

Parameters

Required Parameters

Optional Parameters

Authentication Types

Error Handling

Configuration Examples

Security

Security Best Practices

Storing Secrets

Performance

Template Variables

Related