# Krawl — Full Agent Manual

> Autonomous deep-research agent API. One call → 50+ verified sources checked →
> a cited markdown report in ~90 seconds. Searches the web, X/Twitter, GitHub,
> academic papers, crypto/DeFi data, and community sources in parallel.

This is the complete machine-readable manual for agents. For the short index,
see [llms.txt](https://krawl.sh/llms.txt). For auth specifically, see
[auth.md](https://krawl.sh/auth.md).

## When to use Krawl

- You need comprehensive, **cited** research on any topic (not just links).
- You want to analyze crypto tokens, DeFi protocols, or on-chain activity.
- You want structured data extraction with per-field citations and confidence.
- You need scheduled/recurring research (lookouts).
- You want a research API your agents can call programmatically (REST or MCP).

## When NOT to use Krawl

- Simple factual lookups (use a search engine).
- Real-time price ticks (use CoinGecko/CoinMarketCap directly).
- Image/video search or personal-data lookups.

## How agents should interact

1. **Discover**: read this file, [agent.json](https://krawl.sh/agent.json)
   (or `https://krawl.sh/?mode=agent`), and the
   [OpenAPI spec](https://api.krawl.sh/openapi.json).
2. **Authenticate**: get a free key via `POST /auth/demo-key`, or pay per call
   with x402 (no account). See [auth.md](https://krawl.sh/auth.md).
3. **Call**: prefer the [MCP server](https://api.krawl.sh/mcp) (`research`,
   `quick_search`, `ask_krawl` tools) if your platform speaks MCP; otherwise use
   the REST endpoints below.
4. **Stream**: `POST /research` returns Server-Sent Events; read to the terminal
   `result` event for the final markdown + `sources` array.

## Authentication

Three paths, increasing setup cost:

1. **Demo key** (free, instant, no signup):

   ```bash
   curl -X POST https://api.krawl.sh/auth/demo-key
   # → {"api_key":"krawl_demo_xxx","quota":{...},"expires_in":"~24h"}
   ```

2. **Managed key** (`krawl_live_...`) tied to a billing plan — get one at
   [krawl.sh](https://krawl.sh). Send as a header:

   ```
   X-API-Key: krawl_live_xxx
   ```

3. **x402 pay-per-request** (keyless, USDC on Base): call a research endpoint
   with no key → `HTTP 402` with `{x402Version:1, accepts:[PaymentRequirements]}`
   and a `WWW-Authenticate: Payment` header → sign an EIP-3009 USDC
   authorization → retry with a base64 `X-PAYMENT` header. Discovery:
   [/discovery/resources](https://api.krawl.sh/discovery/resources).

## Primary REST endpoints

- `POST /research` — deep research, SSE stream (accepts `X-API-Key` or x402).
- `POST /research/upload` — research with uploaded documents (PDF/DOCX/TXT).
- `POST /research/async` — submit a background job, returns `202` + task id.
- `GET /tasks/{id}` — poll an async task.
- `POST /search` — single-source search (web/x/github/academic/crypto).
- `POST /ask` — NLWeb natural-language routing endpoint.
- `GET /results` / `GET /results/{id}` — list/fetch past reports.
- `GET /results/{id}/export?format=pdf|markdown|html|docx|slides` — export.
- `POST /research/compare` — compare multiple results.
- `GET /templates` / `POST /templates/{id}/research` — templated research.
- `POST /lookouts` — schedule recurring research.
- `GET /usage` — monthly usage rollup (tokens, cost, quota).
- `POST /v1/chat/completions` — OpenAI-compatible shim (point any OpenAI SDK here).
- `GET /health` — health check.

## MCP server

Streamable HTTP transport at `https://api.krawl.sh/mcp` (JSON-RPC 2.0). Tools:

- `research` — deep multi-source research → cited markdown (`query`, `mode`, `depth`).
- `quick_search` — single-source raw results (`query`, `source`, `num_results`).
- `ask_krawl` — natural-language capability routing (`query`).

Manifest: [/.well-known/mcp/manifest.json](https://krawl.sh/.well-known/mcp/manifest.json).
Server card: [/.well-known/mcp/server-card.json](https://krawl.sh/.well-known/mcp/server-card.json).

```bash
# MCP initialize (discovery, no auth required)
curl -X POST https://api.krawl.sh/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}'

# List tools
curl -X POST https://api.krawl.sh/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}'
```

## Example — run research over REST

```bash
curl -N -X POST https://api.krawl.sh/research \
  -H "X-API-Key: krawl_demo_xxx" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{"query": "Compare DeFi lending protocols by risk-adjusted yield", "mode": "crypto"}'
```

The terminal `result` SSE event carries:

```json
{
  "markdown": "# Report ...",
  "sources": [{"url": "...", "title": "...", "source_type": "web", "credibility_tier": 1}],
  "learnings": [{"text": "...", "citations": ["..."]}],
  "total_sources": 53,
  "review_score": 0.91
}
```

## Research modes

`deep` (default), `quick`, `crypto`, `token-analysis`, `protocol-research`,
`whale-tracking`, `narrative`, `risk-assessment`, `yield-strategy`.

## Source types

`web` (Exa, Tavily, Serper, Brave, DuckDuckGo), `x` (xAI), `github`,
`academic` (PubMed, arXiv), `crypto` (CoinGecko, DeFiLlama), `media`,
`community` (Reddit, Hacker News).

## Plans & rate limits

| Plan | Deep krawls/mo | Search credits/mo | Rate limit |
|------|----------------|-------------------|------------|
| demo | 5 | 100 | 5/hour (~24h key) |
| free | 25 | 1,000 | 10/min |
| payg | metered | metered | 60/min |
| pro | unlimited | unlimited | 120/min |

Quota exhausted → `HTTP 402 {"error":"quota_exceeded"}`. Per-request LLM cost
ceiling: $2.50 deep / $0.40 quick.

## Error handling

All errors are structured JSON:

```json
{"error_code": "NOT_FOUND", "message": "Resource not found", "detail": "...", "retry_after": null}
```

Codes: `NOT_FOUND`, `RATE_LIMITED`, `VALIDATION_ERROR`, `INTERNAL_ERROR`,
`quota_exceeded`. HTTP `402` means payment required (x402) or quota exhausted.

## Agent commerce (optional)

- x402 discovery: [/discovery/resources](https://api.krawl.sh/discovery/resources)
- UCP profile: [/.well-known/ucp](https://krawl.sh/.well-known/ucp)
- ACP checkout: `POST /checkout_sessions`; delegate payment:
  `POST /agentic_commerce/delegate_payment`.

## SDKs

- Python: `pip install krawl`
- Node.js: `npm install @krawl/sdk`
- CLI: `pip install krawl-cli`

## Links

- Website: [https://krawl.sh](https://krawl.sh)
- Docs: [https://docs.krawl.sh](https://docs.krawl.sh)
- OpenAPI: [https://api.krawl.sh/openapi.json](https://api.krawl.sh/openapi.json)
- GitHub: [https://github.com/ravidsrk/krawl](https://github.com/ravidsrk/krawl)
- Auth guide: [https://krawl.sh/auth.md](https://krawl.sh/auth.md)
- Agent profile: [https://krawl.sh/agent.json](https://krawl.sh/agent.json)
- Comparison: [https://krawl.sh/compare](https://krawl.sh/compare)