If you're here, you're probably aware that an API is an Application Programming Interface, or a way for one software application to talk to another. If that's news to you, this guide may be a hard slog, but you might still be interested in checking out labs.wikirate.org to see some of the cool things being developed by WikiRate's community of API users.
A little context
WikiRate's API, like its underlying data structure, is a little unconventional. But if you understand why and how the structure is unusual, the API design will make sense. So let's start there.
First, WikiRate is, as it's name implies, a wiki. It is built with an open-sourced wiki platform called Decko, which uses wiki-inspired building blocks called cards to create complex web systems. Pretty much everything you interact with on WikiRate is a card: Company, Metric, Answer, Project, Data Set, Topic, etc. –– they're all cards. What's particularly unusual is that the substructures of these pages are cards, too. The logo of a company, the formula of a metric, the topic tags on a data set... it's cards all the way down.
The embrace of this wiki card concept helps WikiRate live up to its values of transparency. Since everything is a card, and every card stored has a revertible revision history, the WikiRate platform is deeply transparent, and it exposes not just current data but also data histories to its API.
The API docs make frequent references to marks. "Mark" is our general term for any of the ways for uniquely identifying cards.
Every card on WikiRate, across all types, has a unique name. You cannot, for example, have a project and a company with the same name. This helps WikiRate adhere to an old wiki principle that you should be able to link to any content knowing only its name, and (as we will see) it also makes for very user-friendly urls. This, too, applies to nested cards, which are typically organized using structured names. For example, the logo on a company card is the name of the card plus "image". Here, for example, is the "Apple Inc+image" card:
While the standard form of a card's name retains punctuation and capitalization, these things are stripped in the key that governs WikiRate's namespace. So "Apple Inc", "apple inc", "APPLE INC", and "Apple@#$#$%INC" all refer to the same card (which has the default form of "Apple Inc.").
Every real card on WikiRate also has a unique numerical ID. These IDs are seldom seen in the UI but can easily be exposed by adding "/id" (or, for a text response, "/id.txt" or "/id.json") to a standard card url.
The above shows that Apple Inc., and Apple Inc.+Image are indeed two different cards with different ids. Now that we have figured out their ids, we can use the id in place of the name in any request by preceding the id with a tilde, eg:
Those two links are functionally equivalent. But note that IDs are more permanent than names; if a card's name is edited, its ID remains the same. So a best practice for coders using the API is to use friendly variable names for companies (or metrics or data sets or company groups or whatever else) to hold IDs for use in API calls.
Wait, back up. What do you mean every "real" card has an id? Some cards aren't real??
Indeed, some patterned cards are virtual, meaning that they aren't actually stored in the database. Instead, they are dynamically constructed on the fly using rules. Here are some examples:
Examples of virtual cards on WikiRate
- The metric Scope 2 Emissions is tagged with the topic Climate Change. The real card that stores this connection is the +topic card on the metric card: CDP+Scope 2 Emissions+Topic. (If you don't believe me, you can confirm that it's real using the id trick above). It's also possible to see all the metrics tagged with a given topic using +metric cards on topic cards, eg Climate Change+Metric. But there's no need to store a duplicate copy of the tagging there; that information can be found with a simple search. So this card is a virtual card dynamically generated based on rules, and (as you can confirm) it has no id.
- The metric Company Category is a (fairly complex) formula metric that generates tens of thousands of answers. Maintaining a revision history of all these answers would be extremely computationally expensive, not to mention silly, since they're all governed by one formula. However, each of these answers can still be accessed by name, eg Commons+Company Category+Adage Capital Management LP+2019
You may sometimes encounter a third identifier, the "codename". Most cards don't have these, only cards that are referred to directly in code. These permanent identifiers are more developer friendly than IDs, and unlike IDs they are constant across different Decko sites that share the same code. To use a codename in a request, preface it with a colon (:).
WikiRate makes heavy use of compound names – that is, names separated by a plus sign (+). For example, metrics names follow the pattern METRIC_DESIGNER+METRIC_TITLE (this means different designers can use the same title for different metrics). Records follow the pattern METRIC+COMPANY. Answers follow the pattern METRIC+COMPANY+YEAR. The list of companies covered by a Data Set follows the pattern DATA_SET+companies. And so on.
When using the API, it is possible to use different combinations of marks to identify the same card. An answer can be identified as:
...to list just a few permutations.
Formats and views
Since WikiRate's data is structured around a single namespace, so is its API. Whereas most APIs have different endpoints for different datatypes, all WikiRate data is in cards, so we focus on a single card endpoint with many formats and views.
WikiRate's API provides data views in HTML, JSON, CSV, TXT, and other formats. Because images are also cards, image formats like PNG, JPEG, and SVG follow the same patterns. In all cases, you can use any one of the following three methods to specify the format:
- via filename extension, eg GET /cardname.json
- via query variable, eg GET /cardname?format=json
- via http access header, eg Accept: application/json
For any card on WikiRate, it's possible to add ".json" to its name to see a default JSON view for that card, but there are also many other views available. In the "Card marks" section above, we demonstrated on very simple view: id. There are two ways to specify a view:
- in the path as /CARD_MARK/VIEW, eg GET /Apple_Inc/id
- in the query as /CARD_MARK?view=VIEW, eg GET Apple_Inc?view=id
Handy JSON views
The API docs list some views for different types, but these JSON views are implemented for all cards:
- nucleus - a card's marks, its type, and a default url
- atom - nucleus details plus a card's content, where applicable.
- molecule - atom details plus metadata and connections to closely related cards (this is the default json view)
(Each item links to the described view of the *logo card.)
Sorting and filtering
If you've explored WikiRate's UI, you've likely noticed the prominent sorting and filtering features. All of this functionality is available via the API. And in fact, one handy pattern when you're getting started with WikiRate's API is to being by using the UI, and then to click on the JSON export link below the results.
Authentication and authorization
WikiRate data is and always will be free and open. There are, however, more restrictions on API usage for unauthenticated users. For example, many answer searches will return up to 5000 results for authenticated users but only 500 for anonymous users. Moving forward, the relative benefits to authenticated users will only increase, so it's generally well worth your while to sign up for a free account and authenticate.
To authenticate you will need an API key, which you can generate from your profile page. To get there, click your username on the right side of the nav bar at the top of the page, go to the accounts tab, and click "Generate Key"
Once you have your API Key, you can use it to authenticate in one of two ways: directly or via JWTs.
The advantage of using the API Key directly in your requests is that it's a little quicker to get started. The disadvantages are that it's less secure and a little bit slower.
To use your API key directly, you have two options:
- add the api key as a regular query parameter, eg /cardname?api_key=MY_API_KEY, or
- use the X-API-Key header, eg curl -H 'X-API-Key MY_API_KEY' https://wikirate.org/cardname
If you have generated an API Key, then you will see text below your key that includes a link to generate a JWT. You can use the link directly or call it via the API. (The best practice pattern is to use your API key to retrieve a JWT and then authenticate with that JWT until it expires).
Similarly, there are two ways to use JSON Web Tokens:
- add the token as a regular query parameter, eg GET /cardname?token=TOKEN
- use the Authorization header with a bearer token, eg curl -v -H 'Authorization: Bearer TOKEN' https://wikirate.org/cardname
Python, Java & Ruby API Wrappers
API wrappers are available in Python, Java and Ruby thanks to NLnet’s support. They are easy to install, and they make it easy to get rolling with WikiRate API calls. All three wrappers are open-source and available on GitHub, where anyone can contribute, give feedback, and request changes.
Each wrapper has detailed documentation, and all necessary links to additional resources are provided in the README file on each repository, along with details on how you can contribute and give feedback:
Folks are building cooler and cooler things with WikiRate's API. At labs.wikirate.org we're gathering the best experiments and making them easy to explore, study, and extend.