Multimodal Understanding

Ling-3.0-flash-VL is a vision-language model that supports multimodal inputs and understands text, images, and video. Ming-3.0-audio-flash is a speech understanding model — unlike the VL model, it only supports speech understanding and does not support images or video. This guide covers their capabilities, calling patterns, and limits.

Image Understanding

The image understanding model answers based on the image you pass in. Inputs are passed via Base64 encoding only.

• Image format: JPEG, PNG.
• Image size: each image must not exceed 16384 * 784 pixels.
  • Base64 input: each Base64-encoded image string must not exceed 32 MB.
• Number of images: a single request can include up to 40 images.
• Maximum request body: 32 MB.

Base64 Input

Convert the image into a Base64-encoded string before passing it in. The encoded string must not exceed 32 MB.

OpenAI Chat Completions API

Prefix the encoded data with data:{MIME_TYPE};base64,$BASE64_IMAGE.

• {MIME_TYPE}: the MIME type that identifies the image format, for example image/jpeg or image/png.
• $BASE64_IMAGE: the raw Base64-encoded image data without any prefix.

curl --location --request POST 'https://api.ant-ling.com/v1/chat/completions' \
--header 'Authorization: Bearer sk-studio-xx' \
--header 'content-Type: application/json' \
--data-raw '{
  "model": "Ling-3.0-flash-VL",
  "stream": true,
  "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "data:{MIME_TYPE};base64,$BASE64_IMAGE"
                    }
                },
                {
                    "type": "text",
                    "text": "please describe the content of the image"
                }
            ]
        }
    ],
    "max_completion_tokens": 102400
}'

Anthropic Messages API

• {MIME_TYPE}: the MIME type that identifies the image format, for example image/jpeg or image/png.
• $BASE64_IMAGE: the raw Base64-encoded image data without any prefix.

curl --location --request POST 'https://api.ant-ling.com/anthropic/v1/messages' \
--header 'Authorization: Bearer sk-studio-xx' \
--header 'content-type: application/json' \
--data-raw '{
  "model": "Ling-3.0-flash-VL",
  "stream": true,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image",
          "source": {
            "type": "base64",
            "media_type": "{MIME_TYPE}",
            "data": "$BASE64_IMAGE"
          }
        },
        {
          "type": "text",
          "text": "Describe the content of the image."
        }
      ]
    }
  ]
}'

Unsupported Properties

OpenAI Chat Completions API

• messages.content.image_url.detail: controls the image processing resolution; the inference engine defaults to default.

Video Understanding

The video understanding model answers based on the video you pass in. Inputs are passed via Base64 encoding only.

• Video format: MP4, MOV, WMV.
• Video size: duration ≤ 30s.
  • Base64 input: each Base64-encoded video string must not exceed 32 MB.
• Number of videos: limited to 1 per request.
• Frame sampling: fixed 2fps, capped at 32 frames per video.

OpenAI Chat Completions API

curl --location --request POST 'https://api.ant-ling.com/v1/chat/completions' \
--header 'Authorization: Bearer sk-studio-xx' \
--header 'content-Type: application/json' \
--data-raw '{
  "model": "Ling-3.0-flash-VL",
  "stream": true,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "video_url",
          "video_url": {
            "url": "data:{MIME_TYPE};base64,$BASE64_VIDEO"
          }
        },
        {
          "type": "text",
          "text": "please describe the content of the image"
        }
      ]
    }
  ],
  "max_completion_tokens": 102400
}'

Audio Understanding

The audio understanding model answers based on the audio you pass in. Inputs are passed via Base64 encoding only.

• Audio format: mp3, wav.
• Audio size: a single audio file must not exceed 100 MB and must be no longer than 3600s.
  • Base64 input: a single Base64-encoded audio string must not exceed 32 MB.
• Number of audios: limited to 1 per request.
• Maximum request body: 32 MB.

Base64 Input

Convert the audio file into a Base64-encoded string before passing it in. The encoded string must not exceed 32 MB.

OpenAI Chat Completions API

Prefix the encoded data with data:{MIME_TYPE};base64,$BASE64_AUDIO.

• {MIME_TYPE}: the MIME type that identifies the audio format; replace it with the MIME value of the actual audio, for example audio/mpeg or audio/wav.
• $BASE64_AUDIO: the raw Base64-encoded audio data without any prefix.

curl --location --request POST 'https://api.ant-ling.com/v1/chat/completions' \
--header 'Authorization: Bearer sk-studio-xx' \
--header 'content-Type: application/json' \
--data-raw '{
  "model": "Ming-3.0-audio-flash",
  "stream": true,
  "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "audio_url",
                    "audio_url": {
                        "url": "data:{MIME_TYPE};base64,$BASE64_AUDIO"
                    }
                },
                {
                    "type": "text",
                    "text": "please describe the content of the audio"
                }
            ]
        }
    ],
    "max_completion_tokens": 32768
}'

Unsupported Properties

OpenAI Chat Completions API

• messages.content.input_audio: the URL or Base64-encoded audio data.