Affordable Bing SERP scraping after API shutdown in 2025

Microsoft’s decision to retire the Bing Search API on August 11, 2025, has left developers, SEO analysts, and AI teams in a bind. The official endpoint that once powered rank trackers, brand monitoring tools, and retrieval-augmented generation (RAG) pipelines now returns nothing. Teams that relied on structured Bing SERP data found themselves forced to either migrate to Azure AI’s "Grounding with Bing Search"—a service tailored for large language model workflows—or seek alternatives. Yet for many use cases, scraping Bing’s public search results remains the most practical and cost-effective solution.

Why Bing SERP results still matter in 2025

Bing remains a distinct search ecosystem with ranking behaviors that differ from Google. Its algorithm weights on-page keyword density and anchor text more heavily, often surfacing niche forums, community discussions, and long-tail content that Google’s index may overlook. For AI applications, Bing’s index can complement Google’s blind spots, especially when building RAG pipelines that require diverse source coverage. Bing’s crawler, bingbot, operates independently, with its own crawl frequency and indexing priorities—making its SERP data valuable for comprehensive market or competitor analysis.

The core data—titles, URLs, snippets, and positions—is embedded in the HTML of www.bing.com/search?q=, the same endpoint that browsers use. While Microsoft removed its dedicated API, the public SERP interface remains accessible. The challenge lies in reliably extracting and formatting that data at scale, without triggering anti-bot defenses or handling brittle pagination logic.

No official Bing search API exists anymore

As of August 11, 2025, Microsoft no longer provides an official API for retrieving Bing’s organic search results. The longstanding Bing Web Search API v7 was discontinued, and its replacement—"Grounding with Bing Search"—is tightly coupled to Azure AI workflows. It is not available as a standalone SERP endpoint for arbitrary queries, making it unsuitable for most developers seeking raw Bing data.

The only viable path to programmatic Bing SERP access is through scraping the public HTML endpoint, just as a browser would. This approach offers full parity with the retired API while avoiding the cost and complexity of Azure subscriptions or third-party providers charging hundreds of dollars per month.

How the scraper delivers clean, structured data

Each extracted result is normalized into a single structured row, validated against a strict schema before output. The dataset includes nine consistent fields, eliminating guesswork and silent data corruption:

• query: The original search term
• position: 1-indexed rank across all paginated results (e.g., position 11 is the first result on page 2)
• title: The result’s heading as displayed
• url: The canonical destination URL (decoded from Bing’s redirect wrapper)
• displayed_url: The breadcrumb Bing renders on screen
• snippet: The search result description
• country: The target country code (e.g., US, GB)
• language: The query language (e.g., en, fr)
• scraped_at: Timestamp of extraction

The url field is always the final destination—never a tracking redirect like bing.com/ck/a. The scraper automatically decodes these wrappers, ensuring downstream compatibility with analytics tools, crawlers, or RAG pipelines.

Common pitfalls in DIY Bing scraping—and how to avoid them

A basic scrape script using Python and a few open-source libraries might look straightforward:

import requests
from parsel import Selector

resp = requests.get(
  "
  params={"q": "bing serp scraper"}
)
sel = Selector(text=resp.text)

for li in sel.css("li.b_algo"):
    title = li.css("h2 a::text").get()

While functional in testing, this approach fails consistently in production for three critical reasons:

• TLS fingerprint mismatches: Bing inspects the TLS handshake signature (JA3/JA4) of incoming connections. Standard Python HTTPS clients emit signatures that don’t match real browsers, resulting in 403 errors before the query is even processed. The solution is to impersonate modern browser TLS stacks using tools like curl-cffi with rotating profiles (e.g., Chrome 131, Firefox 147) at the socket level.

• Tracking redirect wrappers: Bing wraps every organic result URL in a click-tracking redirect (). Parsing these raw URLs leads to broken links downstream. The scraper must decode the u=a1` parameter to extract the canonical destination.

• Proxy and session instability: Rotating proxies mid-query disrupts session continuity, causing inconsistent results—fewer organic blocks, more ads, or empty pages. The solution is to maintain a stable session per query burst and rotate proxies only between queries, with exponential backoff on rate limits (2s → 4s → ... max 30s, 5 retries). Partial success is captured and surfaced via status messages.

Without addressing these issues, even a well-written scraper will fail under load or return incomplete or corrupted data.

Introducing a turnkey Bing Search Scraper on Apify

To eliminate these challenges, the Bing Search Scraper Actor was developed and published on Apify. It handles TLS impersonation, redirect decoding, pagination, proxy rotation, and rate limiting automatically. The Actor costs approximately $0.001 per result—around $1.05 per 1,000 queries—when using Apify’s residential proxy pool.

You can run it directly from the Apify Console by pasting your search terms, or call it via API:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("DevilScrapes/bing-search-scraper").call(
  run_input={
    "queries": [
      "bing search api replacement",
      "serp scraping 2025",
      "ai data sources beyond Google"
    ],
    "country": "US",
    "language": "en",
    "maxResultsPerQuery": 30,
    "useProxy": True
  }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["position"], item["url"], item["snippet"])

Key input fields include:

• queries: List of search terms (required)
• country: Target country code (e.g., US, DE)
• language: Query language (e.g., en, fr)
• maxResultsPerQuery: Maximum results to return per query (default: 10)
• useProxy: Enable residential proxy routing (recommended)

With this setup, teams can replicate the functionality of the retired Bing Search API—without Azure costs, subscription tiers, or brittle third-party services—while staying within budget and maintaining data integrity.