2015 Changelog ​

December 2015

2015-12-21

  • Crawlbot and Bulk jobs pending delete (per your Diffbot plan) are now identified in the Crawlbot and Bulk interfaces.
  • The API Toolkit now uses Diffbot's custom rendering engine for live web page previews. This should reduce inaccuracies when creating custom rules.

2015-12-18

  • Fixed an issue where plain-text POSTed to the Article API would not perform text analysis (tags, sentiment, language-detection).
  • Improved Crawlbot behavior on Ajax-heavy sites so that pages with the exact same HTML source are no longer deduplicated.
  • Fixed an issue within the Crawlbot and Bulk interfaces where the "Last 500" URL Report was incorrectly returning the first 500.
  • Improved author detection within the Article API.

2015-12-07

The Analyze API now supports POSTed content.

November 2015

2015-11-27

The Account API now returns a list of child or sub-tokens.

2015-11-19

  • Fixed an issue in the Analyze API where products with an API-Toolkit-overridden price field would not reflect changes in the "details" field (offerPriceDetails, regularPriceDetails, etc.).
  • Fixed an Article API issue for certain top-level domains where articles dated in the near future (e.g., tomorrow) would incorrectly be returned with a date from the prior year.

2015-11-11

  • Crawlbot will now successfully spider URLs that contain (invalid) UTF-8 characters.
  • Global Index API: search-by-tag can optionally be performed using a tag-match shorthand.

October 2015

2015-10-16

  • Fixed an issue where Crawlbot and Bulk API data downloads did not include a filename.
  • The breadcrumb element is now a default field in the Article API.

2015-10-22

  • We now offer an Account API for tracking token API usage and billing history.
  • Global Index API: negative search queries (diffbot AND -"machine learning") are now functioning as documented;

2015-10-08

  • APIs no longer ignore "format characters"—invisible characters that may have an effect on neighboring characters. For example, ‌.
  • Crawlbot and Bulk Service URL Reports now offer an option to download the last 500 URLs crawled.
  • Global Index API: Faceted date queries will no longer return a min value of 0.

2015-10-01

Hey, as of today we're publishing a changelog. It's visible... here.

September 2015

2015-09-23

  • Additional token support within a single account has been added. Additional tokens are available on a case-by-case basis to paying customers. Please contact support@diffbot.com if you would like to discuss additional tokens.
  • API Toolkit now allows direct update of URL pattern / regular expression without having to create a new ruleset.
  • API Toolkit rule output automatically trims fields to remove leading or trailing blank spaces.
  • The diffbotUri field is now computed based on rule-based output, if a custom rule is used to override default output.
  • The resolvedPageUrl is correctly returned in Custom APIs (if a submitted page is redirected).

2015-09-16

  • Each tag in the tags element now returns a list of all matching rdfTypes.
  • Email invoices now return both dollars and cents.

August 2015

2015-08-23

  • Performance improvements to Article API to prevent intermittent extra-long responses.

2015-08-13

  • Semantria output updated to include additional fields.
  • Fix for timeout parameter when sending data to Semantria.

2015-08-10

Invoices are now visible and printable within the Developer Dashboard (under "Account > Billing History").

2015-08-06

Tokens are now case-insensitive across all Diffbot APIs.

July 2015

2015-07-16

  • Article API now returns the siteName, publisherCountry and publisherRegion of an article, if it can be determined or if already known.
  • If price data is overridden with a custom rule, the corresponding "details" field (offerPriceDetails) will be computed from the overridden value.

June 2015

2015-06-28

  • Spotify embeds are properly maintained/returned with Article API html.
  • Multipage articles, when concatenated, will no longer return duplicate images (that appear on multiple pages).

2015-06-11

Article and discussion tagging now supports Spanish and Chinese language tags, in addition to English and French.

February 2015

2015-10-02

  • Analyze API now supports a "fallback" API via the argument fallback. By passing an API value, any otherwise unsupported pages will be forcibly processed via that API. E.g., passing fallback=article will result in any "other" pages being processed by the Article API.