Search API

AI-powered web search, content extraction, and deep research capabilities. The Oblien Search API enables you to search the web, extract specific content from pages, and perform comprehensive website research with real-time streaming results.

What is Search API?

Oblien Search API provides three powerful capabilities for gathering and processing web information:

Features

AI-Powered Search - Get relevant web results with optional AI-generated answers
Smart Extraction - Extract specific content from web pages using natural language instructions
Deep Research - Crawl and analyze websites with real-time streaming results
Batch Processing - Process multiple queries or pages simultaneously
Flexible Options - Control result quality, format, and filtering

Architecture

┌─────────────────────────────────────────┐
│         Your Application                │
│  (AI Agent, Research Tool, Crawler)     │
└────────────────┬────────────────────────┘
                 │
                 │ HTTPS API
                 ↓
┌─────────────────────────────────────────┐
│         Oblien Search Service           │
│                                         │
│  • Web Search with AI Answers           │
│  • Content Extraction                   │
│  • Website Crawling & Research          │
│  • Real-Time Streaming                  │
└────────────────┬────────────────────────┘
                 │
                 ↓
┌─────────────────────────────────────────┐
│          Web Content                    │
│  • Search Engines                       │
│  • Target Websites                      │
│  • Online Resources                     │
└─────────────────────────────────────────┘

Key Features

AI-Powered Intelligence

Every search can include AI-generated answers:

Fast, basic answers with 'low' summary level
Balanced quality with 'medium' summary level
Highest quality insights with 'intelligent' summary level
Context-aware responses based on search results

Batch Processing

Efficient handling of multiple requests:

Process multiple search queries in one API call
Extract content from multiple pages simultaneously
Automatic parallelization for best performance
Consolidated results and error handling

Smart Extraction

Natural language instructions for content extraction:

Specify exactly what to extract from each page
Support for structured data extraction
Multiple output formats (markdown, HTML, text)
Quality levels for different use cases

Real-Time Streaming

Get results as they become available:

Server-Sent Events (SSE) for crawl operations
Live progress updates during research
AI thinking process streaming
Content chunks as they're discovered

Use Cases

AI Research Assistant

Gather and synthesize information automatically:

import { SearchClient } from 'search-agent';

const client = new SearchClient({
  clientId: process.env.OBLIEN_CLIENT_ID,
  clientSecret: process.env.OBLIEN_CLIENT_SECRET
});

// Search with AI-generated answers
const results = await client.search(
  ['What is quantum computing?', 'Latest quantum computing breakthroughs'],
  { summaryLevel: 'intelligent', includeAnswers: true }
);

console.log(results[0].answer);
console.log(results[0].results);

const response = await fetch('https://api.oblien.com/search', {
  method: 'POST',
  headers: {
    'X-Client-ID': process.env.OBLIEN_CLIENT_ID,
    'X-Client-Secret': process.env.OBLIEN_CLIENT_SECRET,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    queries: ['What is quantum computing?', 'Latest quantum computing breakthroughs'],
    options: {
      summaryLevel: 'intelligent',
      includeAnswers: true
    }
  })
});

const results = await response.json();
console.log(results[0].answer);

curl -X POST https://api.oblien.com/search \
  -H "X-Client-ID: $OBLIEN_CLIENT_ID" \
  -H "X-Client-Secret: $OBLIEN_CLIENT_SECRET" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["What is quantum computing?", "Latest quantum computing breakthroughs"],
    "options": {
      "summaryLevel": "intelligent"و
      "includeAnswers": true,
    }
  }'

Data Extraction

Extract structured information from web pages:

// Extract specific content from pages
const extracted = await client.extract([
  {
    url: 'https://example.com/product',
    details: [
      'Extract product name',
      'Extract price',
      'Extract features list',
      'Extract customer rating'
    ],
    summaryLevel: 'medium'
  }
], {
  format: 'markdown'
});

console.log(extracted.data[0].result);

const response = await fetch('https://api.oblien.com/search/extract', {
  method: 'POST',
  headers: {
    'X-Client-ID': process.env.OBLIEN_CLIENT_ID,
    'X-Client-Secret': process.env.OBLIEN_CLIENT_SECRET,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    pages: [
      {
        url: 'https://example.com/product',
        details: [
          'Extract product name',
          'Extract price',
          'Extract features list',
          'Extract customer rating'
        ],
        summaryLevel: 'medium'
      }
    ],
    options: {
      format: 'markdown'
    }
  })
});

const extracted = await response.json();

curl -X POST https://api.oblien.com/search/extract \
  -H "X-Client-ID: $OBLIEN_CLIENT_ID" \
  -H "X-Client-Secret: $OBLIEN_CLIENT_SECRET" \
  -H "Content-Type: application/json" \
  -d '{
    "pages": [
      {
        "url": "https://example.com/product",
        "details": [
          "Extract product name",
          "Extract price",
          "Extract features list",
          "Extract customer rating"
        ],
        "summaryLevel": "medium"
      }
    ],
    "options": {
      "format": "markdown"
    }
  }'

Competitive Intelligence

Research websites and gather comprehensive insights:

// Deep crawl with real-time updates
const result = await client.crawl(
  'Research https://competitor.com and summarize their product offerings, pricing, and key features',
  (event) => {
    if (event.type === 'page_crawled') {
      console.log('Crawled:', event.url);
    } else if (event.type === 'content') {
      console.log('Found:', event.text);
    }
  },
  { type: 'deep', maxPages: 20 }
);

const response = await fetch('https://api.oblien.com/search/crawl', {
  method: 'POST',
  headers: {
    'X-Client-ID': process.env.OBLIEN_CLIENT_ID,
    'X-Client-Secret': process.env.OBLIEN_CLIENT_SECRET,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    instructions: 'Research https://competitor.com and summarize their product offerings, pricing, and key features',
    options: {
      type: 'deep',
      maxPages: 20
    }
  })
});

// Handle SSE stream
const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  
  const chunk = decoder.decode(value);
  // Process SSE events
}

curl -X POST https://api.oblien.com/search/crawl \
  -H "X-Client-ID: $OBLIEN_CLIENT_ID" \
  -H "X-Client-Secret: $OBLIEN_CLIENT_SECRET" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{
    "instructions": "Research https://competitor.com and summarize their product offerings, pricing, and key features",
    "options": {
      "type": "deep",
      "maxPages": 20
    }
  }'

API Overview

The Search API provides three main endpoints:

Endpoint	Purpose	Response Type
POST /search	Web search with AI answers	JSON (immediate)
POST /search/extract	Content extraction	JSON (immediate)
POST /search/crawl	Website research	SSE (streaming)

Authentication

All endpoints require client authentication:

Headers:
  X-Client-ID: your_client_id
  X-Client-Secret: your_client_secret
  Content-Type: application/json

Get your credentials from oblien.com/dashboard/api.

Quick Comparison

Feature	Search	Extract	Crawl
Speed	Fast (< 3s)	Medium (3-10s)	Slow (10s-minutes)
Use Case	Find information	Get specific data	Research deeply
AI Answers	Optional	No	Integrated
Batch Support	Yes	Yes	No
Streaming	No	No	Yes
Max Input	Multiple queries	Multiple pages	Single instruction

Getting Started

Ready to use Search API? Check out these guides:

Web Search - Search the web with AI-powered answers
Content Extraction - Extract specific content from pages
Deep Research - Crawl and analyze websites

SDK Installation

npm install search-agent

import { SearchClient } from 'search-agent';

const client = new SearchClient({
  clientId: process.env.OBLIEN_CLIENT_ID,
  clientSecret: process.env.OBLIEN_CLIENT_SECRET
});

Response Formats

All successful responses include:

{
  "success": true,
  ...
}

All error responses include:

{
  "success": false,
  "error": "Error message"
}

Rate Limits

Search: 100 requests per minute
Extract: 50 requests per minute
Crawl: 10 concurrent crawls per account

Rate limit headers are included in all responses:

X-RateLimit-Limit - Total requests allowed
X-RateLimit-Remaining - Requests remaining
X-RateLimit-Reset - Reset time (Unix timestamp)

Best Practices

Use batch processing - Combine related searches or extractions in one request
Choose appropriate summary levels - Use 'low' for speed, 'intelligent' for quality
Set reasonable limits - Use maxPages and maxDepth to control crawl scope
Handle streaming properly - Process crawl events as they arrive
Implement error handling - Always check success status and handle errors

On this page