pi-gemini-multimodal

Multimodal perception for Pi: image / audio / video / document understanding, transcription, and image generation — delegated to Gemini.

English · 中文

Two providers (pick one)

provider	needs	when
gemini_api	Gemini API key from aistudio.google.com (AIza... / AQ.)	default; direct REST, fast
antigravity_cli	local agy CLI, signed in	no key to manage

Install

pi pkg install pi-gemini-multimodal

Create ~/.pi/agent/extensions/pi-gemini-multimodal/config.json:

{
  "provider": "gemini_api",
  "apiKey": "<your gemini key>",
  "outputDir": "/path/for/images"
}

Key: aistudio.google.com → sign in → Get API key → Create API key. Free tier: understanding is generous, image generation is rate-limited (429s possible).

Tools

tool	does
media_understand	analyze image/audio/video/URL (OCR, charts, UI, speech)
media_transcribe	audio/video → verbatim text with timestamps
image_generate	prompt → PNG saved to outputDir (low free-tier quota)
read_document	summarize / answer over PDF, Office, text

Security (antigravity_cli)

Headless agy -p cannot prompt, so the plugin passes --dangerously-skip-permissions by default. Set "skipPermissions": false and add allow-rules in ~/.gemini/antigravity-cli/settings.json to tighten. Prefer gemini_api for a narrower trust boundary.

Development

npm install
npm run typecheck
npm test

License

MIT

Changelog

• 0.1.1 — bump for CI publish validation; zero-config default provider (antigravity_cli).
• 0.1.0 — initial release: dual providers, 4 multimodal tools.