Bulk Extract API
Bulk Extract lets you send a large quantity of URLs through any Diffbot Extract API for fast, asynchronous processing.
The Bulk Extract API sends all submitted page URLs to an Extract API (either automatic or custom). All structured page results are then compiled into a single "collection," which can be downloaded in full or searched.
Note: Bulk Extract is not a crawler: it does not spider a site for additional links. You must supply each URL you wish to process. For crawling/spidering, see the Crawl API. For the bulk version of the Knowledge Graph Enhance API, see Bulk Enhance — a distinct service on a different endpoint.
Access to Bulk Extract API is limited to Plus plans and up
Upgrade to a Plus plan anytime at diffbot.com/pricing, or contact sales@diffbot.com for more information.
Endpoints
- Create a Bulk Job — submit a batch of URLs for asynchronous extraction.
- Manage a Bulk Job — pause, resume, restart, or delete an existing job.
- Retrieve Bulk Job Data — download the structured results of a completed job.
Data Retention
Inactive bulk jobs will be deleted within ten days of completion. This includes the extracted data as well as the job meta information (name, settings, etc.).
"Active" jobs are those that are not in a permanently "paused" state. Currently active jobs will not be deleted or removed from your account. After a job finishes, it will be subject to regular deletion policies.