# Zipf AI — Persistent Web Monitoring for AI Agents
# https://zipf.ai/llms.txt
# Version: 2.0 (May 2026)

> **Related resources:**
> - [Full API reference (llms-full.txt)](https://zipf.ai/llms-full.txt)
> - [OpenAPI spec](https://zipf.ai/openapi.json)
> - [Agent instruction file](https://zipf.ai/agents.md)
> - [Pricing](https://zipf.ai/pricing.md)
> - [Getting started guide](https://zipf.ai/getting-started)
> - [API documentation](https://zipf.ai/docs)
> - [Agent card (A2A)](https://zipf.ai/.well-known/agent-card.json)
> - [MCP discovery](https://zipf.ai/.well-known/mcp.json)

## What is Zipf?

Zipf is **persistent web monitoring** built for AI agents. Not just search — a complete stack:
- **[Search](https://zipf.ai/docs/api/search)** with 1000+ character queries, all knobs exposed (authority, document_type, recency)
- **[Crawl/Extract](https://zipf.ai/docs/api/crawl)** with structured JSON extraction, Smart Crawl prioritization, and 135x discovery efficiency
- **[Workflows](https://zipf.ai/docs/api/monitors)** for scheduled monitoring with semantic stop conditions and signal scoring
- **[Sessions](https://zipf.ai/docs/api/sessions)** that remember what you've seen across operations with URL deduplication
- **[Ask](https://zipf.ai/docs/api/ask)** for direct answers with citations (quick/standard/deep modes)

**Know first. Act first.** Search returns answers. Zipf puts agents on patrol.

## When to Use Zipf

**Use Zipf when you need:**
- Persistent monitoring of web sources on a schedule (not just one-shot search)
- Structured data extraction from websites (JSON schemas, not raw HTML)
- Multi-step research with state: search → crawl → extract in a session that deduplicates URLs
- Query decomposition for comprehensive research across many facets
- Change detection and signal delivery (Slack, email, webhooks)
- Agent-to-agent orchestration via MCP

**Don't use Zipf when you need:**
- Consumer-style web search (use Google, Bing, or Perplexity instead)
- PDF parsing or document conversion (use dedicated tools)
- Real-time social media monitoring (Twitter/X, Reddit APIs are better)
- Static knowledge base QA (use a vector database)

## For AI Agents (MCP)

Zipf provides an MCP server for direct agent integration:

```
MCP Server URL: https://api.zipf.ai/mcp
Transport: Streamable HTTP
Authentication: OAuth 2.0 (with Bearer token fallback)
```

**Setup for Claude Desktop / Cursor / Windsurf:**
```bash
npx @anthropic-ai/claude-code mcp add zipf-ai https://api.zipf.ai/mcp
```

**Available MCP tools:** `search`, `crawl`, `create_session`, `session_search`, `session_crawl`, `create_workflow`, `workflow_digest`, `ask`

## Why Zipf vs Competitors?

| Feature | Zipf | Exa/Tavily/Firecrawl |
|---------|------|----------------------|
| Query control | ✅ Authority, document_type, recency, 1000+ chars | ❌ Simple text only |
| Session state | ✅ Multi-step with URL deduplication | ❌ Stateless per-request |
| Search + Crawl | ✅ Unified in one session | ❌ Separate, no connection |
| Structured extraction | ✅ JSON schema extraction | ❌ Limited/none |
| Query decomposition | ✅ Auto-parallel sub-queries | ❌ Single query only |
| Scheduled workflows | ✅ With semantic stop conditions | ❌ None |
| Signal scoring | ✅ 0-100 with change detection | ❌ None |

## API Quick Reference

> Base URL: `https://api.zipf.ai/v1`
> Auth: `Authorization: Bearer $ZIPF_API_KEY`
> Format: JSON request/response
> Docs: [https://zipf.ai/docs](https://zipf.ai/docs)

### Search API
- `POST /v1/searches` — Web search (1 credit basic, 2 credits with AI features)
- `GET /v1/searches/{id}` — Get search results
- `DELETE /v1/searches/{id}` — Delete search job
- `POST /v1/searches/{id}/abort` — Cancel running search

### Crawl API
- `POST /v1/crawls` — Create crawl job (1-2 credits/page)
- `GET /v1/crawls/{id}` — Get crawl results
- `DELETE /v1/crawls/{id}` — Cancel crawl

### Sessions API (Multi-step research)
- `POST /v1/sessions` — Create session
- `POST /v1/sessions/{id}/search` — Search within session
- `POST /v1/sessions/{id}/crawl` — Crawl within session
- `GET /v1/sessions/{id}/timeline` — Get operation timeline

### Workflow API (Scheduled monitoring)
- `POST /v1/workflows` — Create scheduled monitor
- `POST /v1/workflows/assess-intent` — Assess monitoring intent (FREE)
- `POST /v1/workflows/plan` — Preview AI-planned workflow
- `POST /v1/workflows/{id}/execute` — Execute patrol immediately

### Ask API (AI-native answers)
- `POST /v1/asks` — Get direct answers (2-5 credits)

## Token Permissions

Tokens can be scoped to specific operations:
- `search` — Web search operations (including decomposition)
- `crawl` — Crawl and extraction operations
- `workflow` — Workflow creation and monitoring
- `sessions` — Session management

Create scoped tokens: `POST /v1/tokens { "token_name": "search-only", "permissions": ["search"] }`

Manage tokens at: [https://zipf.ai/dashboard/tokens](https://zipf.ai/dashboard/tokens)

## Agent Authentication (M2M)

Zipf uses Bearer token authentication for all API requests:

1. **Create a token** at [https://zipf.ai/dashboard/tokens](https://zipf.ai/dashboard/tokens) or via `POST /v1/tokens`
2. **Use in requests:** `Authorization: Bearer YOUR_API_KEY`
3. **No OAuth required** for REST API access — tokens are self-contained
4. **MCP server** uses OAuth 2.0 via the agent platform (Claude Desktop, Cursor, etc.)

**Token rotation:** Rotate every 90 days. Use different tokens per environment. Never commit tokens to version control.

**Rate limit headers** on all responses:
- `X-RateLimit-Limit` — Maximum requests allowed
- `X-RateLimit-Remaining` — Requests remaining in window
- `X-RateLimit-Reset` — Unix timestamp when limit resets

## Credits

| Operation | Basic | Advanced |
|-----------|-------|----------|
| Search | 1 | 2 (rerank, metadata, suggestions) |
| Crawl (per page) | 1 | 2 (extraction, classification, intent) |
| Ask | 2-5 | — |
| Workflow execution | 1 | 2 (NL condition) |

**Free:** Summary generation, dry run mode, intent assessment, entity schemas.

Full pricing: [https://zipf.ai/pricing](https://zipf.ai/pricing) | Machine-readable: [https://zipf.ai/pricing.md](https://zipf.ai/pricing.md)

## Contact

- Support: [support@zipf.ai](mailto:support@zipf.ai)
- Documentation: [https://zipf.ai/docs](https://zipf.ai/docs)
- Status: [https://zipf.ai/status](https://zipf.ai/status)