2020 Changelog ​

December 2020

KG Entity Identifiers - Updated

We've rolled out an update to the Knowledge Graph. Entity identifiers are now preceded by an ‘E’ for each entity regardless of the entity type. Previously, Diffbot Ids were preceded by a letter representing the entity type, i.e. ‘P’ for Person, ‘O’ for Organization, ‘C’ for Corporation, etc. The new format enables us to extend and scale the Diffbot Knowledge Graph to a wider and richer spectrum of entity types over time. Please note: all KG queries will continue to work as before. You do not need to modify the way you search the graph. Below is an example of the new entity id format you will see in results:

Example: IBM
"type": "Corporation", "Organization"
"diffbotUri": "http://diffbot.com/entity/EPdsrDmLiMQCskvBLp_dloQ"
"id": "EPdsrDmLiMQCskvBLp_dloQ"

Example: IBM Board Member & CEO, Virginia 'Ginni' Rometty
"type": "Person"
"diffbotUri": "http://diffbot.com/entity/EFCXA8DGjPMq5oTjl9RESEw",
"id": "EFCXA8DGjPMq5oTjl9RESEw"

October 2020

Knowledge Graph Data - annual and quarterly revenue data: integer vs. string

quarterlyRevenues and yearlyRevenues with integer are now live:

the field yearlyRevenues.year (previously a String such as “2019”) is transformed into an Integer representing the year (2019)
the field quarterlyRevenues.quarter (previously a String such as “Q1-2019") is split into: quarterlyRevenues.quarter which is an Integer representing the quarter (1) and quarterlyRevenues.year which is an Integer representing the year (2019)

https://app.diffbot.com/search/?query=type%3AOrganization+quarterlyRevenues.revenue.value%3E10000000000

Eg.: Apple, Inc.

json
"quarterlyRevenues": [{"revenue": {"currency": "USD","value": 91818999808},"isCurrent": false,"year": 2020,
json
"filingDate": {"str": "d2020-01-29","precision": 3,"timestamp": 1580256000000},
json
"revenueDate": {"str": "d2019-12-31","precision": 3,"timestamp": 1577750400000},"quarter": 1}

Excel Add-in Updated - v1.8.0.0

Excel version 1.8.0.0

  • Enhance V3 - we added an option to configure the number of matches/outputs returned per input, e.g. org name + homepage as input: specify return 1 match or 3 matches.
  • Enhance V3 - we added an option to configure the acceptable match threshold. You can now specify a minimum threshold for the inclusion of an entity match/output.
  • View a Diffbot entity regardless of the Excel Add-in status.
  • Saved Searches - you can now save and load DQL queries from the Add-In. These queries are synchronized with the Diffbot Dashboard.

Enhance API Updates

EnhanceAPI version v3

As part of API version v3, the following functionality is now available:

  1. Fetch more than 1 result for an enhance query with&size=n query parameter:
    https://kg.diffbot.com/kg/v3/enhance_endpoint?token=DIFFBOT-TOKEN&type=organization&url=www.ibm.com&size=3
  2. Specify the threshold score with&threshold=d query parameter: https://kg.diffbot.com/kg/v3/enhance_endpoint?token=DIFFBOT-TOKEN&type=organization&name=DummyName&url=www.ibm.com&threshold=0.1&size=3
  3. Head top-n bulkjob records for preview with&head=n query parameter: https://kg.diffbot.com/kg/v3/enhance_endpoint/bulk/BULKJOB-ID?token=DIFFBOT-TOKEN&head=10
  4. Bulkjob response format changed to jsonl from json (Content-Type: application/json-lines). Each enhanced record is on a separate line in the response. This, coupled with the head parameter enables clients to scale up more fluidly for large bulkjobs.

The API spec is available at docs.diffbot.com

Bulkjob retention

Bulkjob retention period has been increased from 7 days to 30 days.

Deprecation of API versions v1 and v2

API versions v1 and v2 are now deprecated and will be removed in future versions of the product. Please plan to migrate.

Enhance API Updates

EnhanceAPI version v3

As part of API version v3, the following functionality is now available:

  1. Fetch more than 1 result for an enhance query with&size=n query parameter:
    https://kg.diffbot.com/kg/v3/enhance_endpoint?token=DIFFBOT-TOKEN&type=organization&url=www.ibm.com&size=3
  2. Specify the threshold score with&threshold=d query parameter: https://kg.diffbot.com/kg/v3/enhance_endpoint?token=DIFFBOT-TOKEN&type=organization&name=DummyName&url=www.ibm.com&threshold=0.1&size=3
  3. Head top-n bulkjob records for preview with&head=n query parameter: https://kg.diffbot.com/kg/v3/enhance_endpoint/bulk/BULKJOB-ID?token=DIFFBOT-TOKEN&head=10
  4. Bulkjob response format changed to jsonl from json (Content-Type: application/json-lines). Each enhanced record is on a separate line in the response. This, coupled with the head parameter enables clients to scale up more fluidly for large bulkjobs.

The API spec is available at docs.diffbot.com

Bulkjob retention

Bulkjob retention period has been increased from 7 days to 30 days.

Deprecation of API versions v1 and v2

API versions v1 and v2 are now deprecated and will be removed in future versions of the product. Please plan to migrate.

September 2020

September 2020

Industries/Category Updates

We've improved mapping to NAICS (93% industries covered) classifications for Org entities and updated our industries categories structure and label set.

Main updates (267 industries):

Added Retailers as main industry sector (+30 sub-industries)
Grouped the new industries categories list by major industry groups
Differentiated some specific cases.

E.g.: Furniture Companies --> (1) Furniture Manufacturers, (2) Furniture Retailers

     Jewelry Companies -->   (1) Jewelry Manufacturers, (2) Jewelry Retailers

     Toy Companies  -->   (1) Toy Manufacturers, (2) Toy And Video Game Retailers

     Vehicle Parts -->   (1) Vehicle Parts Manufacturers, (2) Automotive Part Retailers
        ...

     Motor Vehicle Manufacturers -->   (1) Motor Vehicle Manufacturers, (2) Vehicle Retailers And Dealership
    Sporting Goods Manufacturers -->   (1) Sporting Goods Manufacturers, (2) Sporting Goods Retailers
        ...

August 2020

AccountAPI: childUsage param

The Diffbot AccountAPI returns the usage for the supplied token, and a list of child tokens. However, it previously did not return a usage array for each child token. We added support for a parameter, childUsage, that will return the usage of the child tokens.

Excel Add-In v1.6

A new Excel add-in version (1.6.x) is available in production (available at office.com & desktop)

What's new?

  • Added to Organization output: NAICs, zip codes, and phone numbers.
  • Minor design changes and bug fixes.
  • New 'Help' tab, tutorials, docs, and getting started videos.

August 2020

We've launched the Diffbot Event API BETA designed to facilitate the extraction of online and in-person event data for individual events. Support for multi-day, multi-track events, i.e. full conferences and festivals, is planned for a future release.

To try out the API directly, insert your token into this API call:
http://api.diffbot.com/v3/event?token=YOURTOKEN&url=https://www.meetup.com/BAS-G1Outdoors/events/271921536/
or use the Analyze API.

The BETA program enables you to extract data for

July 2020

July 2020

A new Excel add-in version (1.5.0.0) is available in production (available at office.com & for your desktop ). What's new?

  • Match score is part of Enhance output (data enrichment)
  • Search articles by sentiment (Article-level filter)
  • Pull query parameter values from a designated spreadsheet
  • Drag and drop in the outputs tab to reorder columns returned

Support for Deep Pagination: streaming KG search results. You are now able to remove the 1000 result limit on KG queries. For Example:

curl
curl -s "https://kg.diffbot.com/kg/v2/dql_endpoint/stream?token=TOKEN&size=1500&query=type:Organization%20nbEmployeesMax%3E2%20nbEmployeesMax%3C10%20location.country.name:%22Venezuela%22" > stream.es.jsonl

or

curl
curl -s "https://kg.diffbot.com/kg/v2/dql_endpoint/stream?token=TOKEN&size=1500&query=type:Article%20tags.uri:%22https://diffbot.com/entity/BEAlC2Xn3M16WkrunMNhqvw%22%20publisherRegion:'North%20America'" > stream.gb.jsonl

May 2020

May 2020

  • Natural Language API (BETA) & Demo: https://demo.nl.diffbot.com/
    • The demo combines graph visualization and table visualization (see the tabs 'Entities' and 'Facts') to show how the natural language API can be used to build knowledge graphs from text and the web. Double-click on a node to expand it with facts from the Diffbot KG.
    • NL API Documentation: https://docs.diffbot.com/docs/en/nl-index
  • Support for querying Discussions by sentiment (e.g. type:Article title:"Coronavirus" discussion.posts.sentiment>0.5 sortBy:date)

April 2020

April 2020

Now you can track the sentiment for a company, brand, or industry over time. For example, the following query will show only articles that are negative towards Apple Inc, sorted by date:
type:Article tags.{uri:"http://diffbot.com/entity/CHb0_0NEcMwyY8b083taTTw" sentiment<0.0} sortBy:date.timestamp

Or, count the number of negative/positive articles by day/week/month for the last 365 days/52 weeks/12 months. For example, the following query shows the number of articles that are positive towards Apple Inc grouped by week:
type:Article tags.{uri:"http://diffbot.com/entity/CHb0_0NEcMwyY8b083taTTw" sentiment>0.0} facet[week]:date

Article Search has also been added to the Diffbot Excel Add-In. If you haven't installed the Add-In yet, try it out in Office365 or on your desktop. Download it here.

March 2020

March 2020

Announcing the Diffbot Google Sheets integration for the Diffbot Knowledge Graph! Now you can search the graph and enrich your data from within a GSheet (you may need to coordinate with your IT team if you are using a corporate/office-sponsored Google Suite account vs. a personal one).

We also made an update to the AccountAPI to ease management and tracking of the usage of your Diffbot services including entities downloaded and credits consumed.

February 2020

February 2020

  • We added support for a quarterly revenues attribute sourced from SEC filings for US corporate entities: type:Organization quarterlyRevenues.quarter:"Q1-2020"
  • You can now facet on NAICs code names: type:Organization nbEmployeesMax>100000 facet:naicsClassification.name
  • We added over 15k open-source academic journals to the list of Diffbot Knowledge Graph article sources.
  • You can now search the Knowledge Graph and enhance firmographic data profiles from the Diffbot Excel Add-In.
  • We added Diffbot Knowledge Graph ontology reference documentation.
  • We expanded Diffbot Query Language (DQL) docs.

January 2020

January 2020

  • Improved handling of tables and lists in Article data to better support Entity tagging and linking.
  • Optimized entity tagging in the Article Title. It now occurs when the same entity is mentioned in the title and text of the Article.
  • Improved location data extraction in the AnalyzeAPI for events.
  • Added a Diffbot Excel Plug-in to enable our clients to TestDrive Diffbot's data enrichment API. The beta service currently supports organization firmographic profile data enrichment.