calute.core.multimodal

calute.core.multimodal#

Multimodal content handling utilities for Calute.

This module provides utilities for handling multimodal content, specifically images, within the Calute framework. It includes functions for downloading, loading, and serializing images in various formats, with support for base64 encoding and PIL Image objects.

Key Features:

  • Download images from URLs with proper error handling

  • Load images from bytes, base64 strings, or PIL Image objects

  • Serialize images to base64-encoded strings for API transmission

  • Pydantic-compatible SerializableImage type for model integration

Example

>>> from calute.core.multimodal import download_image, SerializableImage
>>>
>>> # Download an image from URL
>>> image = download_image("https://example.com/image.png")
>>>
>>> # Use SerializableImage in Pydantic models
>>> from pydantic import BaseModel
>>> class ImageRequest(BaseModel):
...     image: SerializableImage
calute.core.multimodal.SerializableImage#

Pydantic-compatible type for serializable PIL Images.

A type alias for PIL.Image.Image that includes Pydantic validators and serializers. This type can accept PIL Images, raw bytes, or base64-encoded strings as input, and serializes to base64-encoded strings for JSON/API compatibility.

The type uses:

  • BeforeValidator: Converts bytes/base64 strings to PIL Images on input

  • PlainSerializer: Converts PIL Images to base64 strings on serialization

Example

>>> from pydantic import BaseModel
>>> from calute.core.multimodal import SerializableImage
>>>
>>> class ImageMessage(BaseModel):
...     content: str
...     image: SerializableImage
>>>
>>> # Create with PIL Image
>>> from PIL import Image
>>> img = Image.new("RGB", (100, 100))
>>> msg = ImageMessage(content="Hello", image=img)
>>> # Serializes to base64 when converting to dict/JSON
>>> msg.model_dump()  # image will be base64 string

alias of Image[Image]

calute.core.multimodal.download_image(url: str) → Image[source]#

Download an image from a URL and return it as a PIL Image.

Fetches an image from the specified URL using HTTP GET request with a custom User-Agent header and returns it as a PIL Image object.

Parameters

url – The URL of the image to download.

Returns

The downloaded image as a PIL Image object.

Raises

RuntimeError – If there is an error downloading the image from the URL or if the downloaded content cannot be converted to a PIL Image.

Example

>>> image = download_image("https://example.com/photo.jpg")
>>> print(image.size)
(800, 600)
calute.core.multimodal.maybe_load_image_from_str_or_bytes(x: PIL.Image.Image | str | bytes) → Image[source]#

Load an image from various input formats.

Converts the input to a PIL Image object. If the input is already a PIL Image, it is returned as-is. If it’s bytes, it’s decoded directly. If it’s a string, it’s assumed to be base64-encoded image data.

This function is used as a Pydantic BeforeValidator for the SerializableImage type.

Parameters

x – The input to load the image from. Can be: - PIL.Image.Image: Returned as-is - bytes: Raw image bytes to be decoded - str: Base64-encoded string of image bytes

Returns

The loaded image as a PIL Image object.

Raises

RuntimeError – If the bytes or base64 string cannot be decoded into a valid image.

Example

>>> import base64
>>> # Load from PIL Image (passthrough)
>>> from PIL import Image
>>> img = Image.new("RGB", (100, 100))
>>> result = maybe_load_image_from_str_or_bytes(img)
>>> result is img
True
calute.core.multimodal.serialize_image_to_byte_str(im: Image, info: SerializationInfo) → str[source]#

Serialize a PIL Image to a base64-encoded string.

Converts a PIL Image to its base64-encoded string representation. The output format can be customized through the serialization context.

This function is used as a Pydantic PlainSerializer for the SerializableImage type.

Parameters

  • im – The PIL Image to serialize.

  • info – Pydantic serialization info containing optional context with: - max_image_b64_len: Maximum length for the base64 string (truncates with “…”) - add_format_prefix: If True, adds “data:image/{format};base64,” prefix

Returns

The serialized image as a base64-encoded ASCII string. If add_format_prefix is set in context, includes a data URI prefix. If max_image_b64_len is set, the string is truncated to that length with “…” appended.

Example

>>> from PIL import Image
>>> img = Image.new("RGB", (10, 10), color="red")
>>> # Basic serialization (without Pydantic context)
>>> class MockInfo:
...     context = None
>>> b64_str = serialize_image_to_byte_str(img, MockInfo())
>>> b64_str.startswith("iVBOR") or len(b64_str) > 0
True
Contents