Inspecting the MarkLogic REST API in action

by Evan Lenz

Did you know that, as of MarkLogic 6, the application generated by Application Builder is a JavaScript-based application making direct use of the MarkLogic REST API? This gives us a great way to see an example of the REST API in action with nothing more than a browser's development tools. In this article, we're going to look under the covers using the inspection tools that come with the Chrome browser (but you could also use another browser's development tools, such as the Firebug plugin for Firefox).

First of all, we need to generate the app. To do this, we go to http://localhost:8000/appservices/ (assuming MarkLogic's running on our local machine). Next, we'll click the "New Example Application" button:

Machine generated alternative text: ¿ Application Builder Applications Create a search application from a database of your content Name Database Port The re are currently no applications in the system. + NewApplication pIeAppIication

Let's call the app "Oscars" using a new database called "oscars":

Machine generated alternative text: New Example Application Create an application using sample Oscar award content. Application name Content Source New includes sample data and indexes (recommended) New database oscars Existing Oscars database Cancel eAppliætion

Next, we get to choose all the details of our app. Since all we really want to do is inspect the result, let's use all the defaults for this example and skip straight to deployment.

Machine generated alternative text: I Search Assemble Results Sort Content Appearance

Let's have App Builder create a new REST API instance for us, using whatever defaults it suggests:

Machine generated alternative text: Compile and Launch Application Existing REST API Instance Overwrite this RESTAPI instance’s configuration and its modules database with the compiled application. ( New REST API Instance Create a new RESTAPI instance and modules database. Name Oscars Port 8003 Ioy

After waiting a few moments for the app to get generated, we get automatically redirected to our new app!

Machine generated alternative text: Oscar® Explorer Award lead—actor (9) V Decade 193 Os (4) 1940s (5) V Winners false (6) true (3) Welcome, admin Search 1 to 9 of 9 Gregory Peck” Gentleman’s Agreement” Gregory Peck (5 April 1916 - 12 June 2003) was an American film actor. He was one of 20th Century Fox s most popular film stars, from the 1940s... lead—actor— 1947 Charles Laughton “The Private Life of Henry VIII” Charles Laughton (1 July 1899 - 15 December 1962) was an English Academy Award —winning stage and film actor, screenwriter producer and one—time... lead-actor — 1932 Awards lead -actor = = = People ‘dictor McLaglen  \ - Jam es Stewart Orson Welles / \ John Garfield - [— Charles Laughtor z,, -. James Cagney - Fred nc March Gregory Peck ‘ SORT V

Now let's start taking a look under the covers. In Chrome, open View->Developer->Developer Tools. Click the "Network" tab in the pane that pops open:

Machine generated alternative text: Q Elements Resources Sources Timeline ij Profiles Audits Console

Now let's run a search. Type "stewart" in the search box and click the "Search" button:

Machine generated alternative text: Stewart Search

The results are narrowed down to the five Oscar nominations (out of 9 sample documents total) that mention the name "stewart". Now let's look at the HTTP request & response underlying this search. In Chrome's developer pane, we see a POST request was made to the /v1/search endpoint:

Machine generated alternative text: Q Elements Resources Network Sources 3Timeline Profiles Audits flConsole 1 Status Size Time Met.. Text Type Initiator Timeline ISGms 234ms 312ms 200 iauerv-1.7.1 3.97KE 390ms POST OK appl. Script 3.83KE 388ms

If you click into the entry, you'll see the details of the HTTP request. First we see the request URL itself, which accesses the same HTTP server as our app (at port 8003). The one server is thus doubling as both the REST server for our data and the Web server for our HTML pages. We see that the desired result is JSON format (given format=json in the query string) and that we want to get the first 10 results (start=1&pageLength=10):

Machine generated alternative text: Headers Preview Response Cookies Timing Request URL: http:/ø Request Method: POST Status Code: 2øø OK V Request Headers view source Accept appUcation/json, text/javascript, *1*; q=ø.ø1

If this were a GET request, we'd also probably include a q parameter containing our search text. But it's a POST request, which means the query is provided as the POST payload. Scrolling down a bit, we see the query itself:

Machine generated alternative text: V Request Payload (“query : {“qtext” : “stewart})

Now let's look at the response. Click the "Response" tab to see the raw JSON that resulted from the request:

Machine generated alternative text: ‘ Headers Preview Response Cookies Timing 1 {“snippet—format” : “snippet”, “total” :5, ‘start”: 1, page—length”: lB, results:

For a friendlier view, click the "Preview" tab. Here we see the JSON expressed as an object tree which we can drill down into at will:

Machine generated alternative text: Headers Preview Response Cookies liming V {snippet—format:snippet, totaLS, start:1, page—1.ength:1ø,...} facets: {award:{type:xs:string, facetValjjes:[{name:tead—actor, count:5}]},...} metrics: {query—resotution—time:PTG.139997S, facet—resotution—time:PTG.034269S,... page—tength: 10 query: {word—query:{text:stewart, option:punctuation—insensitive}} report: “(cts:search(fn:couectiono), cts:and—queryC(cts:word—query(”stewart”,... resutts: [{index:1, uri:/oscars/435156173430018770.xml., path:fn:doc(”/oscars/435 snippet—format: “snippet” start: 1 total.: 5

If you look back and forth between your app's search results page and this JSON object structure, you'll see exactly where all the data came from. Let's add one more search constraint. Let's say you want to narrow your results down to James Stewart's nominations. You can do this by clicking the applicable pie chart wedge:

Machine generated alternative text: Orson Welles James Cagney - People — James Stewart L Gregory Peck

Now we just have two results total. Let's look at the underlying request that was made by clicking on the applicable "search" request entry in Chrome tools Network pane. Inspecting the "Headers" tab of this request, we see the full JSON query request payload:

Machine generated alternative text: Preview Response Cookies Timing stdrL: L pageLength: 10 V ecuest Payload (“query”: (“qtext” : “stewart” ,“and—query”: (“queries”: [(“range—constraint—query”: (“constraint—name”: “name”, “val.ue”: [“James Stewart” J }}] }}} V esponse Headers view source Connection: Keep—AUve Content—Length: 2528 Content—type: appUcation/j son Keep—Alive tirreout=1, rax=96 Server: MarkLogic

Here we see that, in addition to the "stewart" search string, we also are constraining the results using the constraint named "name" with the value "James Stewart".

For some reason, when I looked at the JSON response in Chrome for this request, it didn't show me the expected JSON results (I suspect this is a bug in the Chrome developer tools, since I'm seeing the correct results on the web page itself). So I decided to use curl to test the REST server directly (which you can do also):

This gives us the expected results (2 nominations). Let's see how the data maps to what's shown on the screen, breaking it into two parts: first the facets and then the results themselves. These are combined into the same JSON result, but we'll look at them separately. First, here's the JSON for the facets:

The above data contains five facets, four of which are string-valued (win, film, award, and name) and one of which uses bucketed values (decade). The award, decade, and win facets are displayed in the sidebar of the app:

Machine generated alternative text: V Award lead-actor Ø VDecade 19305 (1) 1940s (1) V Winners faIseØ

And the award and name facets are displayed as charts:

Machine generated alternative text: Orson Welles John Garfield — Jam es Cagney Gregory Peck Awards lead -actor People ., -‘ s Charles Laughtor Fredric March

Clicking any of the facet values (such as "1930s" or "Gregory Peck") effectively narrows down the current result set. Here's the portion of the JSON that represents the list of matching documents:

The above data underlies the list of two results we see in the app:

Machine generated alternative text: = = = = 1 to 2 of 2 James Stewart” Mr. Smith Goes to Washington” james Stewart lead—actor — 1939 James Stewart” It’s a Wonderful Life” james Stewart lead-actor - 1946

If you want to dig deeper and see how the JSON is processed in JavaScript, you can view the HTML source and start inspecting the JavaScript files themselves. That sounds like a good topic for another post someday...