Video understanding

Supported models

• Capability note: Video understanding relies on a multimodal model's vision capability, using extracted frames or video content parts. Currently offered models mainly focus on text chat. Refer to the Model Catalog (Models in the console) for video-understanding availability and performance.

Input methods

In content, use a video_url part in the content array to provide video:

• Video URL: Provide a publicly accessible video address. Common formats include mp4, mov, mpeg and avi.
• Base64 encoding: Encode a local video file as a data URL in the form data:video/mp4;base64,....

Code examples

Python

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.yomiapi.com/v1"
)
response = client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Summarize this video and list the key scenes in chronological order"},
            {"type": "video_url", "video_url": {"url": "https://example.com/demo.mp4"}}
        ]
    }]
)
print(response.choices[0].message.content)

cURL

curl https://api.yomiapi.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "What happens in this video?"},
        {"type": "video_url", "video_url": {"url": "https://example.com/demo.mp4"}}
      ]
    }]
  }'

Use cases

• Content summaries: Generate video summaries, chapter divisions and material for subtitles.
• Educational video analysis: Extract lesson points and create exercises or notes.
• Surveillance clip analysis: Identify objects, actions and unusual events in footage.
• Marketing video review: Analyze visual pacing, communication and compliance risks.

Recommendations

• Limit file size: Longer videos and higher bitrates increase request duration and token usage. Compress the video or extract key clips before uploading.
• Split long videos: Analyze shorter segments individually, then ask the model to combine their conclusions for more stable results.
• Ask focused questions: Specify the subject or time range, such as "Which people appear in the first minute?", for more accurate results than open-ended questions.