Web Search and Scraping

Find current sources with Venice Web Search and extract public pages as model-ready Markdown with Web Scrape before running model inference.

Venice provides separate APIs for discovering web pages and extracting their content:

• Web Search returns structured results with titles, URLs, snippets, and dates.
• Web Scrape converts a selected public page into Markdown.

Use them together when your application needs current, inspectable source material before inference.

Search the Web

Send a query to the Search API:

curl https://api.venice.ai/api/v1/augment/search \
  -H "Authorization: Bearer $VENICE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"latest developments in decentralized AI"}'

Brave is the default provider and provides Zero Data Retention. Google is also available with anonymized queries proxied through Venice. See the Web Search API reference for provider selection and the complete response schema.

Scrape a Page

After selecting a result, send its public URL to the Scrape API:

curl https://api.venice.ai/api/v1/augment/scrape \
  -H "Authorization: Bearer $VENICE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com/article"}'

The endpoint returns the page content as Markdown. It first requests the site’s native Markdown representation when available and otherwise uses browser-based extraction.

Search-to-Scrape Workflow

1. Search

   Submit a focused query and collect the structured results.

2. Select sources

   Filter results by relevance, domain, and date before fetching page content.

3. Scrape

   Extract Markdown only from the pages your application intends to use.

4. Use or store the content

   Include the Markdown in a prompt, create embeddings, cache it, or preserve source URLs for citations.

Standalone Search or Model Web Search?

Use the standalone APIs when your application needs control over source selection or wants to reuse retrieved content. Enable web search on a model request when you want Venice to handle retrieval as part of a single inference request.

Need	Approach
Inspect or filter results before inference	Standalone Web Search
Extract a specific known URL	Standalone Web Scrape
Cache or reuse retrieved content	Standalone APIs
Let the model retrieve current context automatically	Model web search

Web Search and Web Scrape are experimental APIs. Their request and response formats may change without notice. Web Scrape only accepts publicly accessible pages, and sites that block automated access may be rejected.

Related Resources

Building a search-to-scrape pipeline? Cited Answers with Web Search walks through the whole workflow step by step, including source selection, handling pages that fail to extract, and producing an answer where every claim resolves to a URL.

• Web Search API Reference
• Web Scrape API Reference
• Cited Answers with Web Search
• Private Research Agent
• Embeddings