Consuming a JSON webservice
This article explains how to consume a JSON webservice from within your WEM project. At the end of this article we will have built a simple application that uses the Open Movie Database API to search for a movie and show the details. You can already see the final application working at https://omdb.live.wem.io.
This part of this article explores a new feature in WEM that allows you to perform any HTTP web request. The second part will focus on JSON messages and how to handle them in WEM.
HTTP (Hypertext Transfer Protocol) is the protocol that is used to communicate with web servers. Your browser sends an HTTP request to a web server any time you request a new web page, while webservers use HTTP to communicate with each other through webservices.
We recently added a new node type to the flowchart editor that can perform any HTTP web request, and conveniently called it the “web request node“. When you drag this node in your flowchart, you can specify all the aspects that make up an HTTP request message:
The url (this includes the scheme, which is either http or https).
The request method (GET and POST are the most common methods, but WEM supports all common methods).
Request headers – The header fields Accept, Connection and Content-Type are predefined. You can specify any number of additional headers.
Query fields – You can manually put the queryfields in the url yourself, but I recommend to specify them separately. This way WEM will take care of proper url encoding.
The request body, if applicable. This can either be plain text (for instance, json or xml) or a document such as an image or a pdf file.
When it comes to security, WEM supports basic authentication and client certificates out of the box. Other authentication schemes are often possible through custom HTTP headers.
The web request node also lets you specify how to handle the web response that you get back from the server. You can map each response header to a data field, and the response body can be stored in either a text field or a file field. The node has an error exit that is followed when the server is not available or when the server responds with a 50x Server Error. You can also add exits to handle specific server responses, such as 404 Not Found or 401 Unauthorized.
Searching the Open Movie Database is very easy. It consists of a simple GET request to https://www.omdbapi.com/with some query fields:
s – The movie title to search for.
y – The year of release (_optional_).
r – The message format of the response (json or xml).
There are more optional query fields, but these are the three that are used in the demo application. You can even try the API in your browser, and inspect the JSON response:
http://www.omdbapi.com?s=Matrix&y=1999&r=json
For our example I created a simple flowchart:
The first node contains a simple form where the user can enter (a part of) the title and year of the movie. I created two data fields named [Search.Title] and [Search.Year] for this form.
The second node is the new web request node. It performs a GET request to the url "http://www.omdbapi.com/". No authentication is needed. The response body is stored in an other data field [Response.Json]. This is a plain text field. The following three query fields are added for this request:
Note that the query field r contains the literal text "json". This instructs this particular webservice to send the response in json format.
The last node in the flowchart is an overlay that shows the JSON message. When I searched for “The Godfather”, I received the following json response:
The first part of this article described how to use the web request node to send an HTTP web request to a json webservice. The response was a JSON message. This part shows how to import the JSON message into WEM data fields.
WEM has data import and export nodes, a convenient way to import or export excel or CSV files. We recently added the option to import JSON data.
Importing JSON data is different from importing tabular data.
JSON stands for JavasScript Object Notation. It describes data in a text-based and human-readable format. JSON is often used in conjunction with REST APIs and web-based services. It is more powerfull than for example CSV, because it isn’t restricted to tabular data.
The following example is a JSON message that contains information about a person “John Smith”:
The basic construct for JSON messages is the so called object. An object is a list of “key/value” pairs surrounded by curly brackets ({}). The message itself is an object. In the example above, the line "firstName": "John" is a key/value pair. The key in this case is firstName, and the value is "John".
Key is always surrounded by quotes. The value can be any of the following type:
A string (e.g. text). For example, the value "John" is a string. Strings are always surrounded by quotes.
A number. In the example above, the age and height are numbers. Numbers are not surrounded by quotes.
A boolean (e.g. true or false). In the example, a boolean is used for the key "isMember".
The value null can be used when the actual value is unknown or missing.
A value can again be an object. This allows for nesting of objects. In the example above, the address is an object that contains the keys "street", "postcode" and "place".
A value can also be a list (called an _array_) of values. This list is surrounded by square brackets ([]). In the example above "phoneNumbers" is a list of objects.
An important aspect is that JSON messages can be deeply nested. It isn’t unusual to see a message that contains a list of objects, where each object again contains a list of objects, etc…
In part one of this blog we used the Open Movie Database API. This API contains a method to search for movies, given a title and optionally a year. The following query searches for the Jurassic Park movies: http://www.omdbapi.com/?s=Jurassic+Park&r=json
We will now import this JSON response using the data import node. A new drop down is added to the data import node, that allows you to choose the file format. It this case we choose JSON. Next, we have to specify the data source. This sould be the data field that was used by the web request node to store the response body.
If we click Edit Mappings, an overlay appears that allows you to describe the structure of the JSON message, and map the fields and arrays to data fields in WEM. For CSV, you could specify a flat list of columns. JSON has a nested structure however, so you define this structure by building a tree. This tree consists of objects, arrays, text fields, number fields or “true/false” fields, which corresponds to the different JSON value types described above. The tree that describes the response of the OMDB query looks as follows:
Now, instead of building this tree structure manually, I used the feature “Create mapping from example”. This allows you to copy & paste the JSON result and let WEM infer the JSON structure based on this result. This saves a lot of time, and you can always refine the structure if WEM wasn’t able to infer everything from one example.
The next step is to map the the JSON fields to data fields in WEM. For this, I created the following data model in WEM:
For the next step, we have to map the JSON fields to the corresponding WEM data fields. You do this simply by clicking on the field, and choosing the corresponding data field by clicking on the “Map to” button. After all the relevant fields are mapped, the tree should look like this:
Now that we have imported the data, we can simply build a page that shows the results. In this example, we used a repeater and a panel to list all the movies:
