Pegasus 1.5 on Bedrock expands video metadata generation and on‑the‑fly processing

Amazon Bedrock now hosts TwelveLabs’ Pegasus 1.5, a video language model that can ingest a full‑length video and emit timestamped JSON metadata. The update introduces time‑based segment extraction, visual reference prompting, direct S3 or base64 input, and support for videos up to two hours, removing the need for a pre‑processing ingest step.

Key Changes in Pegasus 1.5

• Time‑based metadata generation: Users supply segment definitions and receive a segments array with start/end times and any requested fields.
• Multimodal prompting: Segment definitions can combine textual descriptions with reference images, enabling searches for specific logos, products, or people.
• On‑the‑fly processing: Videos are sent directly from an S3 URI (s3Location) or as a base64 payload (max 25 MB) without an indexing pipeline.
• Extended video length: Single‑request analysis now supports up to two hours, double the previous one‑hour limit.
• Same core capabilities: Video‑to‑text, granular visual comprehension, temporal grounding, multimodal understanding and multilingual support remain.

Practical Implications for Engineers

AI engineers can replace custom frame‑extraction scripts with a single API call that returns structured JSON, simplifying downstream analytics or content moderation pipelines. The multimodal prompt format reduces the need for separate image‑search services.

Cloud and platform engineers must provision Bedrock access in the us-west-2 region and ensure the S3 bucket policy allows Bedrock to read the video object. Because the model runs on‑demand, scaling concerns shift to request throttling and cost monitoring rather than managing a dedicated inference cluster.

DevOps / SRE practitioners should instrument the InvokeModel and InvokeModelWithResponseStream calls with standard observability tools. Tagging the inference profile ARN enables cost attribution per application. Streaming responses can be used to reduce latency for large payloads, but they also require handling partial JSON fragments.

Security engineers need to verify that only authorized principals can call the Bedrock runtime and that the S3 bucket owner ID is correctly supplied in bucketOwner. When using base64 payloads, the video data travels in the request body, so IAM policies and network controls must protect that path. All processing stays within the AWS data boundary, which simplifies compliance checks for data residency.

Architecture and Operational Considerations

Typical integration points include:

{
  "modelId": "twelvelabs.pegasus-1-5-v1:0",
  "input": {
    "s3Location": {"uri": "s3://my‑bucket/video.mp4", "bucketOwner": "123456789012"},
    "analysisMode": "time_based_metadata",
    "responseFormat": {"segments": [{"label": "logo", "imageReference": "..."}]}
  }
}

Deployments should treat the model as a stateless service: no local caching of video frames is required, but request payload size limits (25 MB for base64) may dictate when to prefer S3 references. Monitoring should capture request latency, error rates, and token usage to detect throttling or unexpected cost spikes.

Related CloudNinjas coverage: AWS.

What This Means For Practitioners

Adopt Pegasus 1.5 when you need structured, searchable video insights without building a custom pipeline. Verify IAM permissions for Bedrock and S3, tag inference profiles for cost tracking, and design your workflow to handle JSON segment output. Keep an eye on region availability, request size limits, and any future Bedrock model updates that could affect limits or pricing.