Diffbot Global Index
The Diffbot Global Index lets you search the web like a database. Our Automatic APIs regularly scour the web, indexing articles, comments, images, products and more—all available for realtime search across any structured field.
Search for...
News |
Tens of thousands of news sources are monitored constantly for new content. Search millions of articles and filter by author, date, topic, region, sentiment, language and more. |
Comments |
Article and blog post comments are automatically extracted and structured alongside article content. Find comments by author, search by topic or simply query for all comments matching your specific term. |
Images and Video |
The Global Index indexes all images and videos featured in articles, allowing you to filter by caption, source, image tags or image similarity. |
More coming soon |
Discussions, products, videos and more page types are on their way. |
Features
New API, Same Great JSON
The Global Index API returns JSON objects in the same format as our Automatic APIs. No refactoring necessary.
Realtime indexing
Powered by Crawlbot, the Global Index is always up-to-date, regularly monitoring tens of thousands of sites.
Faceted Searching
Powerful faceting engine lets you apply filters to any search: facet by date, site, topic, author and practically any other field.
Add custom sources
Need to include content from an obscure source? Contact Support to add the site to our regularly monitored sources.