describe-image

Pi extension — describes images using a vision model when your active model cannot see images directly.

Why?

Most coding models are text-only. When a user shares a screenshot, UI mockup, or photo, the model sees:

(tool image omitted: model does not support images)

This extension gives blind models eyes — it registers the describe_image tool that calls MiniMax-M3 (or any vision model you configure) and returns a text description.

How it works

• Registers a tool called describe_image
• The tool calls a vision model API and returns the description as text
• Auto-manages itself: activates when the active model can't see images, deactivates when using a multimodal model
• Listens to model_select events — switches on/off automatically when you change models mid-session

Installation

# from npm
pi install npm:@maulanashalihin/describe-image

# or from GitHub
pi install git:github.com/maulanashalihin/describe-image

Or clone manually into ~/.pi/agent/extensions/:

git clone https://github.com/maulanashalihin/describe-image.git \
  ~/.pi/agent/extensions/describe-image

Usage

Just share an image with your model. If the model is text-only, it will automatically call describe_image to "see" it.

User: [screenshot of error message]
Model: *calls describe_image, gets description, helps debug*

Or call it directly:

describe_image path="./screenshot.png" prompt="What does the error say?"

Configuration

Edit config.json next to the extension:

{
  "model": "minimax-m3",
  "apiUrl": "https://opencode.ai/zen/go/v1/messages",
  "authKey": "opencode-go",
  "apiFormat": "anthropic",
  "maxTokens": 2048,
  "temperature": 0
}

Field	Default	Description
model	minimax-m3	Vision model ID
apiUrl	https://opencode.ai/zen/go/v1/messages	API endpoint
authKey	opencode-go	Key name in ~/.pi/agent/auth.json
apiFormat	anthropic	API format: "openai" or "anthropic"
maxTokens	2048	Max response tokens
temperature	0	Model temperature

apiFormat

The extension supports two API formats for image input:

Format	Header	Image block	Endpoint example
"openai"	Authorization: Bearer	{ type: "image_url", image_url: { url: "data:..." } }	/chat/completions
"anthropic"	x-api-key	{ type: "image", source: { type: "base64", media_type: "...", data: "..." } }	/messages

Provider examples

OpenCode API (Anthropic-format models — MiniMax, Qwen):

{
  "model": "minimax-m3",
  "apiUrl": "https://opencode.ai/zen/go/v1/messages",
  "authKey": "opencode-go",
  "apiFormat": "anthropic"
}

OpenCode API (OpenAI-format models — Kimi, DeepSeek, MiMo, GLM):

{
  "model": "mimo-v2.5-pro",
  "apiUrl": "https://opencode.ai/zen/go/v1/chat/completions",
  "authKey": "opencode-go",
  "apiFormat": "openai"
}

Direct MiniMax API:

{
  "model": "MiniMax-M3",
  "apiUrl": "https://api.minimax.io/v1/chat/completions",
  "authKey": "minimax",
  "apiFormat": "openai"
}

The API key is read from ~/.pi/agent/auth.json under the key specified by authKey:

{
  "opencode-go": {
    "type": "api_key",
    "key": "sk-..."
  }
}

Files

~/.pi/agent/extensions/describe-image/
├── config.json    ← vision model settings
├── index.ts       ← extension code
└── package.json   ← npm package metadata

License

MIT