NEWVectors or files. Pick a path.Start →
    Search & Discovery
    11 min read
    Updated 2026-10-19

    Build an ARTF agent container that adds IAB content categories

    A worked ARTF v1 agent: it answers GetMutations with one ADD_CIDS mutation that puts IAB Content Taxonomy 3.1 category IDs on site.content.data or app.content.data, looked up in a reference file built from a Mixpeek collection, so nothing is called during the auction. Proto stubs, handler code, the manifest label, a grpcurl test with IAB Tech Lab's sample request, and a list of what was and was not tested.

    AdTech
    ARTF
    AAMP
    OpenRTB
    IAB Content Taxonomy
    gRPC
    Contextual

    What this guide builds



    An ARTF agent container answers one gRPC call, RTBExtensionPoint.GetMutations, and returns proposed changes to a bid request (ARTF v1.0 specification). The agent here adds content categories: for content it already knows, it returns one ADD_CIDS mutation that puts IAB Content Taxonomy 3.1 category IDs on site.content.data (or app.content.data for CTV and in-app). The categories come from a reference file built ahead of time from a Mixpeek collection, so the agent makes no network call during the auction.

    We wrote this agent in Python and ran it on 2026-10-19 against IAB Tech Lab's published protos and its samples/video-deals.json request. The section What was tested lists exactly what ran and what did not.

    Which intent carries content categories?



    ADD_CIDS. The ARTF protobuf defines eight intents (ARTF protobuf). ACTIVATE_SEGMENTS is limited to /user/data/segment, which describes the user. ADD_CIDS is the one whose eligible paths are /site/content/data and /app/content/data, and its payload is a list of OpenRTB Data objects with segments and ext.cids (ADD_CIDS intent doc). Content categories describe the content, so they belong there.

    The mutation the agent returns:
    {
      "intent": "ADD_CIDS",
      "op": "OPERATION_ADD",
      "path": "/site/content/data",
      "contentData": {
        "data": [{
          "id": "mixpeek",
          "name": "mixpeek-iab-content-3.1",
          "segment": [
            {"id": "210", "name": "Food & Drink"},
            {"id": "216", "name": "Cooking"}
          ],
          "ext": {"cids": ["content-12345"]}
        }]
      }
    }
    The segment IDs are IAB Content Taxonomy 3.1 unique IDs from IAB Tech Lab's published taxonomy file. If a downstream system also reads content.cat, it needs cattax set to 9 for Content Taxonomy 3.1; see why CTV bid requests lack content categories.

    Step 1: generate the gRPC stubs



    The protos live in the ARTF repository under AGPL-3.0. Download them at a pinned commit instead of copying them into your code:
    ARTF_SHA=9253072ec4bf0b469ce42370525fdd8e00d05384
    RAW=https://raw.githubusercontent.com/IABTechLab/agentic-real-time-framework/$ARTF_SHA/proto
    mkdir -p proto/com/iabtechlab/openrtb/v2 gen
    for p in agenticrtbframework.proto agenticrtbframeworkservices.proto com/iabtechlab/openrtb/v2/openrtb.proto; do
      curl -sfL "$RAW/$p" -o "proto/$p"
    done
    pip install grpcio==1.81.0 protobuf==6.33.6 grpcio-tools==1.81.0
    python3 -m grpc_tools.protoc -Iproto --python_out=gen --grpc_python_out=gen \
      agenticrtbframework.proto agenticrtbframeworkservices.proto com/iabtechlab/openrtb/v2/openrtb.proto

    Step 2: build the reference file from Mixpeek



    The agent reads one JSON file. Each key is the value the exchange will send (site.content.id, site.page or app.content.id), and each value lists IAB 3.1 categories:
    {
      "taxonomy": "IAB Content Taxonomy 3.1",
      "items": {
        "content-12345": {
          "content_id": "content-12345",
          "categories": [{"id": "210", "name": "Food & Drink"}, {"id": "216", "name": "Cooking"}]
        }
      }
    }
    To fill it, index the publisher's videos, pages or podcasts into a Mixpeek collection and attach an IAB taxonomy to it. The IAB taxonomy template writes iab_category_id, iab_category_name, iab_category_tier and iab_category_path onto each document (taxonomy templates, taxonomies). Video is classified per scene, so a long video can carry several categories. Then page through the collection and group by your content key:
    import json, os, urllib.parse, urllib.request
    
    API, KEY = "https://api.mixpeek.com", os.environ["MIXPEEK_API_KEY"]
    
    def documents(collection, namespace):
        cursor = None
        while True:
            q = {"page_size": "100", **({"cursor": cursor} if cursor else {})}
            req = urllib.request.Request(
                f"{API}/v1/collections/{collection}/documents?{urllib.parse.urlencode(q)}",
                headers={"Authorization": f"Bearer {KEY}", "X-Namespace": namespace})
            body = json.load(urllib.request.urlopen(req, timeout=60))
            yield from body.get("results") or []
            cursor = (body.get("pagination") or {}).get("next_cursor")
            if not cursor:
                return
    
    items = {}
    for doc in documents("col_publisher_video", "ns_publisher"):
        key = (doc.get("metadata") or {}).get("content_id")
        cat = doc.get("iab_category_id")
        if not key or cat is None:
            continue
        entry = items.setdefault(key, {"content_id": key, "categories": []})
        if all(c["id"] != str(cat) for c in entry["categories"]):
            entry["categories"].append({"id": str(cat), "name": doc.get("iab_category_name", "")})
    
    json.dump({"taxonomy": "IAB Content Taxonomy 3.1", "items": items},
              open("reference/iab_reference.json", "w"), indent=1)
    metadata.content_id is whatever field holds your content ID; use the page URL if the exchange only sends site.page. Rebuild the file when new content is indexed and ship it as a read-only volume. ARTF v1 has no standard way to deliver data like this to a container, so mount it the way your host allows.

    Step 3: the handler



    The whole decision is a dictionary lookup, which keeps it inside the auction's time budget and makes it easy to audit:
    import grpc
    import agenticrtbframework_pb2 as pb
    import agenticrtbframeworkservices_pb2_grpc as svc
    from com.iabtechlab.openrtb.v2 import openrtb_pb2 as ortb
    
    class ContextualAgent(svc.RTBExtensionPointServicer):
        def __init__(self, reference):
            self.reference = reference  # the "items" object from the reference file
    
        def lookup(self, br):
            if br.HasField("site"):
                for key in (br.site.content.id, br.site.page):
                    if key and key in self.reference:
                        return "/site/content/data", self.reference[key]
            if br.HasField("app") and br.app.content.id in self.reference:
                return "/app/content/data", self.reference[br.app.content.id]
            return None, None
    
        def GetMutations(self, request, context):
            mutations = []
            intents = list(request.applicable_intents)
            if not intents or pb.ADD_CIDS in intents:
                path, hit = self.lookup(request.bid_request)
                if hit and hit["categories"]:
                    data = ortb.BidRequest.Data(id="mixpeek", name="mixpeek-iab-content-3.1")
                    for c in hit["categories"]:
                        data.segment.add(id=c["id"], name=c["name"])
                    data.ext.cids.append(hit["content_id"])
                    mutations.append(pb.Mutation(
                        intent=pb.ADD_CIDS, op=pb.OPERATION_ADD, path=path,
                        content_data=pb.DataPayload(data=[data])))
            return pb.RTBResponse(id=request.id, mutations=mutations,
                                  metadata=pb.Metadata(api_version="1.0", model_version="iab-content-3.1"))
    Three rules from the spec shape it. The host accepts or rejects each mutation on its own, so the agent sends one complete mutation instead of several partial ones. Unknown content gets an empty response, never a guess. And the agent honours applicable_intents: if the host did not ask for ADD_CIDS, it returns nothing.

    Serve it with grpc.server(...) and add_RTBExtensionPointServicer_to_server on port 50051 (the reference implementation's gRPC port), plus /health/live and /health/ready on 8080, the probes in the spec's manifest example.

    Step 4: call it with IAB Tech Lab's sample request



    Add ADD_CIDS to applicable_intents in samples/video-deals.json from the ARTF repository (the sample asks only for segment and deal intents), then:
    grpcurl -plaintext -import-path proto -proto agenticrtbframeworkservices.proto \
      -d @ localhost:50051 \
      com.iabtechlab.bidstream.mutation.services.v1.RTBExtensionPoint/GetMutations < video-deals.json
    The response we got is the mutation shown above, with "id": "sample-video-001" and metadata.apiVersion "1.0". With the sample unchanged, the agent correctly returns no mutations.

    Step 5: manifest and image



    The manifest declares one intent from the spec's list, metadataEnhancement, no service dependencies, no egress, and a non-root user:
    {
      "name": "mixpeek-iab-contextual",
      "version": "0.1.0",
      "vendor": "Mixpeek",
      "resources": {"cpu": "250m", "memory": "128Mi"},
      "intents": [{"name": "metadataEnhancement",
                   "description": "ADD_CIDS with IAB Content Taxonomy 3.1 segments"}],
      "dependencies": {},
      "health": {
        "livenessProbe": {"httpGet": {"path": "/health/live", "port": 8080}},
        "readinessProbe": {"httpGet": {"path": "/health/ready", "port": 8080}}
      },
      "security": {"runAsNonRoot": true, "networkPolicies": {"ingress": [], "egress": []}}
    }
    Attach it Base64-encoded as the com.iabtechlab.artf.descriptor label, as the spec describes:
    FROM python:3.12-slim
    ARG MANIFEST_B64=""
    LABEL com.iabtechlab.artf.descriptor="${MANIFEST_B64}" \
          com.iabtechlab.artf.container-version="0.1.0"
    RUN pip install --no-cache-dir grpcio==1.81.0 protobuf==6.33.6
    WORKDIR /app
    COPY agent.py ./
    COPY gen ./gen
    RUN useradd --uid 10001 --no-create-home artf
    USER 10001
    EXPOSE 50051 8080
    ENTRYPOINT ["python3", "agent.py", "--reference", "/data/iab_reference.json"]
    docker build --build-arg MANIFEST_B64="$(base64 < manifest.json | tr -d '\n')" -t mixpeek-iab-contextual:0.1.0 .
    docker run --rm -p 50051:50051 -p 8080:8080 -v "$PWD/reference:/data:ro" mixpeek-iab-contextual:0.1.0

    What was tested



    On 2026-10-19 we ran:

  1. Stub generation from the ARTF protos at commit 9253072.
  2. Seven tests that start the gRPC server and call it over the wire: a known content ID gets one ADD_CIDS mutation with the right IAB IDs and ext.cids; the page URL works as a fallback key; app content uses /app/content/data; unknown content gets no mutations; a request without ADD_CIDS in applicable_intents gets no mutations; both health endpoints answer; the reference builder groups categories per content key and skips untagged documents.
  3. grpcurl against the running agent with the ARTF repository's samples/video-deals.json.


  4. Not tested yet: building and running the Docker image above, the image inside an exchange's ARTF host, image signing, OpenTelemetry export (the spec asks for it, and this sample does not emit it), MCP, and latency under load. We publish no latency number for this agent. A host should measure it against its own tmax.

    Where Mixpeek fits



    Mixpeek does the expensive part ahead of time: indexing the video, audio and pages, and classifying them against IAB Content Taxonomy 3.1, including per-scene classification for long video. The agent above only reads the result. Mixpeek does not run ARTF containers or operate as an ARTF host. IAB Tech Lab says AAMP 2.3 integrates contextual enrichment technology that Mixpeek donated, to match context to the taxonomy standards used in programmatic (IAB Tech Lab, AAMP 2.3), and IAB Tech Lab's reference buyer agent includes a Mixpeek client for content classification.

    For how ARTF relates to AdCP, AAMP and Agentic Audiences, see AdCP, AAMP, ARTF and Agentic Audiences: what each one does. For the taxonomy itself, see IAB Content Taxonomy classification.

    Changelog



  5. 2026-10-19: first version.


  6. Frequently Asked Questions



    Can an ARTF container add content categories to a bid request?



    Yes, with the ADD_CIDS intent on /site/content/data or /app/content/data. Its payload is a list of OpenRTB Data objects, so categories go in as segments with the provider in id and name (ADD_CIDS intent doc).

    Why not use ACTIVATE_SEGMENTS for contextual categories?



    ACTIVATE_SEGMENTS only accepts the path /user/data/segment, which describes the user (ACTIVATE_SEGMENTS intent doc). Content categories describe the page, video or episode, so ADD_CIDS on the content object is the right fit.

    Should an ARTF agent call an API during the auction?



    Avoid it where you can. The host gives each request a tmax budget and sets the container's network policies, so a lookup against data already inside the container is simpler to approve and to keep inside the budget. This agent declares no egress in its manifest.

    Does the Go reference implementation support ADD_CIDS?



    Not yet at commit 9253072. Its ProcessContentData handler is a placeholder that returns nothing until the Go code is regenerated from the current protos, which already define ADD_CIDS as intent 8 (ARTF repository). The Python stubs generated in step 1 include it.

    How does the reference file stay current?



    Rebuild it from the Mixpeek collection whenever new content is indexed and roll the new file out as a read-only volume. The agent loads it at start-up, so a restart or a new pod picks up the change.

    Sources



  7. ARTF v1.0 specification
  8. ARTF repository
  9. ARTF protobuf
  10. ADD_CIDS intent doc
  11. ACTIVATE_SEGMENTS intent doc
  12. IAB Tech Lab taxonomy files
  13. IAB Tech Lab, AAMP 2.3
  14. IAB Tech Lab buyer agent
  15. Mixpeek docs: taxonomies
  16. Managed Mixpeek

    Put multimodal search to work

    Connect a bucket and Mixpeek runs the whole multimodal search pipeline for you: extraction, indexing, and search over your own objects. No models to wire up, nothing to host.

    Start with Managed
    MVS · bring your own

    Already have vectors?

    Keep your embeddings on your own cloud and run dense, sparse, and BM25 search directly on object storage. From $25/mo.

    Start with MVS

    Run this on your own data

    Point Mixpeek at the storage you already have and search your video, images, audio, and documents the way this guide describes. Build starts at $25/mo for up to 1M vectors.

    Search your own archiveRead Docs

    Related guides

    Search & Discovery

    OpenProposal (AAMP 3.0) explained: the object, its 71 fields and what buyer agents read

    OpenProposal is the core object of IAB Tech Lab's AAMP 3.0: a machine-readable ad offer that a buying agent parses, configures within seller-declared limits and commits to, landing in OpenDirect 2.1, the Deals API or OpenRTB 2.6. How the proposal and line items are shaped, the seven mutability markers, a CTV line item read field by field, where content categories are missing, and how to comment before the October 22, 2026 deadline.

    Read guide →
    Search & Discovery

    AdCP, AAMP, ARTF and Agentic Audiences: what each one does

    AdCP and AAMP are two separate efforts to standardize AI agents in advertising. AdCP is one protocol from AgenticAdvertising.org, carried over MCP or A2A, with tasks like get_signals and create_media_buy. AAMP is IAB Tech Lab's umbrella, which includes ARTF (agent containers inside an exchange that return bid request mutations) and Agentic Audiences (embeddings carried in OpenRTB). A dated comparison table, spec links, and what to do when a contextual embedding model changes.

    Read guide →
    Search & Discovery

    Brand Safety vs Brand Suitability: How AI Classifies Video for Advertisers

    Safety is a universal floor; suitability is a per-brand tolerance curve over graded risk tiers: and the GARM-style taxonomy remains the reference for both even after the organization wound down. How multimodal classification actually works on video (visual + speech + on-screen text + adjacency), why treatment beats topic, why moderation APIs can't express per-brand tiers, and the query-time reclassification pattern that avoids re-paying analysis.

    Read guide →