Web Services with MarkLogic Server

Gary Katz
Last updated September 20, 2010

Note: Since this article was written, MarkLogic has added a function library for designing custom REST services. See:

In addition, as of release 6, MarkLogic now includes a built-in REST API. For more information, see:

Summary

It is often the case that you'll want MarkLogic Server to interact with other applications in a Services Oriented Architecture (SOA). Happily, MarkLogic Server natively supports web services and working within SOA environments can easily be accomplished. This tutorial will walk the reader through the process of developing a simple XML-RPC web service wrapper to exemplify how a query might be passed to MarkLogic Server. At the end of this tutorial, the reader will also have a solid foundation for how to extend this model to support Simple Object Access Protocol (SOAP) messages.

Prerequisites

This tutorial assumes that the reader has downloaded the latest version of MarkLogic Server, is familiar with how to create and populate a MarkLogic database, and how to create an HTTP application server. An excellent tutorial for how to quickly accomplish this can be found here: Understanding Applications in MarkLogic Server: Part 1

Content

The content set used in this tutorial is a subset of The New York Times Annotated Corpus, a rich set of highly tagged NITF formatted news articles spanning a 20 year period. As such, screen captures shown below may reference snippets of this content. You are, of course, free to use whatever XML content you like.

Creating an HTTP Server

After creating a database and loading some content into it, the very first thing you'll need to do it set up an HTTP Server that will act as the gateway between MarkLogic Server and the outside world. For your convenience, a snapshot of an HTTP Server configuration from the MarkLogic Administrative Interface is provided.


HTTP Server Setup

Note several items:

  • Server Name: For server name, the convention of "database-port" is used. This is not required but a matter of convenience so that you can easily see what ports you've used from multiple panels in the MarkLogic Administrative Interface
  • Root: In this field you should specify the root directory where your HTTP application server will look for XQuery files to process. In the above screen capture, observe that we point our HTTP server to the XQuery directory in our development environment. For rapid development purposes, this makes debugging/testing activites convenient before moving code to a purely production location (outside of your dev environment.)
  • Port: Set your HTTP application server to any available port number.
  • Database: Set this entry to the name of the database that you created for this tutorial. In the screen capture above, this is set to the "nyt" database that holds our New York Times content.

Writing Your Web Service

Consider the following code:

(:
This web service issues a query against the ML Search API and returns the results

Example Request:
<request>
<query>hello world</query>

</request>

Example Response:
<response>
<status>SUCCESS</status>
<message>Query {$query} has been has been issued against the repository</message>
<search:response total="1" start="1" page-length="10"
xmlns:search="http://marklogic.com/appservices/search">
<search:result index="1" uri="/hello.xml" path="doc("/hello.xml")" score="136"
confidence="0.67393" fitness="0.67393">
<search:snippet>
<search:match path="doc("/hello.xml")/hello">This is where you say "<search:highlight>Hello</search:highlight><search:highlight>World</search:highlight>".
</search:match>
</search:snippet>
</search:result>
<search:qtext>hello world</search:qtext>
<search:metrics>
<search:query-resolution-time>PT0.328S </search:query-resolution-time>
 <search:total-time>PT0.352S</search:total-time>
</search:metrics>
</search:response>
</response>
:)

declare namespace error="http://marklogic.com/xdmp/error";
import module namespace search = "http://marklogic.com/appservices/search"
at "/MarkLogic/appservices/search/search.xqy";

let $request := xdmp:get-request-field("request", "")
return
if ($request eq "")
then
<response>
<status>FAILURE</status>
<message>No REQUEST found</message>
</response>
else
try
{
(: parse request, issue query to MarkLogic Server :)
let $xml-request := xdmp:unquote($request)
let $query:= data($xml-request//query)[1]
let $results := search:search($query)
return
<response>
<status>SUCCESS</status>
<message>Query '{$query}' has been has been issued against the Server</message>

{$results}
</response>
}
catch($e)
{
<response>
<status>FAILURE</status>
<message>{$e/error:format-string}</message>

</response>
}

Copy and paste this code into a new file called "wbsvc-simple-query.xqy" and save this code in the directory specified as the root of your HTTP Server. So, as per the HTTP Server screen capture, this would be saved to the "C:\eclipse\workspace\nyt\src\xquery" directory. Now let's explore this code in further detail...

First, observe the helpful documentation that precedes the actual code. This documentation desribes your standard input/output specifications as would (should!) be provided for any code that is written.

Your web service can issue queries to MarkLogic Server in a variety of ways. For the purposes of this tutorial, we've elected to use the MarkLogic Search API that supports a "Google-esque" query grammar. Thus, observe that the outputs in the above code comments reflect the results being from the MarkLogic Search API. Should you require a greater degree of query flexibility or prefer not to use the Search API, you are certainly welcome use other MarkLogic libraries such as cts:search() and cts:query() (or perhaps custom methods you've authored) to construct more complicated queries as needed.

Okay, so lets get into the actual code. The very first thing we do is declare our error namespace for our web service module to use, as well as import the search API library for use:

declare namespace error="http://marklogic.com/xdmp/error";
import module namespace search = "http://marklogic.com/appservices/search"
at "/MarkLogic/appservices/search/search.xqy";

Next, we make a call to xdmp:get-request-field() to parse the input stream from the calling application for a variable called "request" that we store in a variable called $request. Later in this tutorial, we will create an HTML form utilizing an input field called "request" where our query will be stored.

let $request := xdmp:get-request-field("request", "")

The next piece of code handles the real logic.

return
if ($request eq "")
then
<response>
<status>FAILURE</status>
<message>No REQUEST found</message>
</response>
else
try
{
(: parse request, issue query to MarkLogic Server :)
let $xml-request := xdmp:unquote($request)
let $query:= data($xml-request//query)[1]
let $results := search:search($query)
return
<response>
<status>SUCCESS</status>
<message>Query '{$query}' has been has been issued against the Server</message>
{$results}
</response>
}
catch($e)
{
<response>
<status>FAILURE</status>
<message>{$e/error:format-string}</message>
</response>
}

We first implement a basic form of error-checking such that if the $request variable is empty, then we return a fail state to the calling application with an informative error message ("No REQUEST found").

However, should the $request variable not be empty, we proceed to parse the request so that we may issue a query against the database. Note that this piece of code is encapsulated in a try-catch statement. In this case, should an unexpected error occur during the transaction, the calling application can be provided an informative MarkLogic Server error message that can then be used for debugging or otherwise be handled by the calling application. Observe that it is in our catch{} code where we utilze the MarkLogic error library that we first declared at the top of our web service code.

In parsing the $request, the first step is to recognize that the value contained in the variable is currently a string. However, we anticipate that this string is really an XML request, so we'd therefore like to unquote the string so that it can be interpretted as XML. As such, we call the MarkLogic Server function xdmp:unquote() to remove the quotes from the ends of the string and thus allowing the request to be interpretted as XML:

let $xml-request := xdmp:unquote($request)

Based on our input specification, we anticipate only one <query> element to be present in our $xml-request, and as such issue the following code to retrieve the data value of the first (and coincidentally, only) <query> element:

let $query:= data($xml-request//query)[1]

Now that we've isolated the Search API query string that was in the payload from the calling application, we can issue this query against the MarkLogic Search API and store the output into a variable we call $results

let $results := search:search($query)

The very last step of writing our web service is to return the results of the query back to the calling application in the expected <response> element, compliant with our output specification. Observe how we embed XQuery statements into our XML constructors by use of curly braces {}:

return
<response>
<status>SUCCESS</status>
<message>Query '{$query}' has been has been issued against the Server</message>
{$results}
</response>

Writing an HTML Test Interface

To recap thus far, we created an HTTP Server pointing to a location on disk where we've saved off an XQuery web service called wbsvc-simple-query.xqy. This service accepts a request input containing query syntax that can be issued against the MarkLogic Search API. Lets now create a basic HTML form which was can use to test this web service.

Consider the following code:

<html>
<head>
<title>Web Services Tester</title>
</head>

<body>
<h3>Web Services Tester</h3>
<form name="requestform" action="http://localhost:8008/wbsvc-simple-query.xqy" method="post">
Request XML:
<input type="text" size="120" name="request"
value="<request><query>equity AND company</query></request>">

<br/>
<input type="submit" value="Submit">
</form>

</body>
</html>

Copy and paste this code into a new file called "WebServicesTester.html" and save this file anywhere on disk, then open the file using your favorite brower. You should see an interface that looks like this:

HTTP Server Setup

The interface for the web services tester implements a simple HTML form that allows you to enter request XML with a <query> element containing MarkLogic Search API grammar syntax. By submitting this query via our Web Services Tester interface, the request will be sent to MarkLogic Server, query parsed, and results rendered in the browser. The HTML form's action attribute points to the specific web service we wrote on our HTTP server. The query itself is wrapped in the web services <request> element as though being received from an external application (in this case, a simple HTTP post.) Also observe that the only <input> field in this form is named request to coincide with the call to xdmp:get-request-field("request", "") made inside our web service.

The web service we've written will display the results back to the browser. Also note that the tester file, when opened in your browser, will default to a sample query we used on our sample content -- you’ll want to change this to something more specific to the content you've loaded into your MarkLogic database. Below is a screen capture of what some sample output might look like when pressing the Submit button on the Web Services Tester:

HTTP Server Setup

You've now written your first web service using MarkLogic Server.

SOAP

We've seen that MarkLogic Server lends itself easily facilitating web services. However, many applications rely on the SOAP standard for SOA environments. Can MarkLogic Server accept SOAP messages?

Yes, absolutely. To handle SOAP messages, a web service can be written (similarly to the one we've just authored) that recongizes SOAP envelope instead of request. The web services would then parse the SOAP message for its payload, while remaining cognizant of any WSDL information.

Conclusions

We've walked through a pretty thorough code analyis of how to set up an XML-RPC web service in MarkLogic. To summarize, all that was needed to do this was:

  • A MarkLogic database with some content loaded
  • An HTTP Server
  • An XQuery web service wrapper
  • A simple testing interface

We hope this tutorial was helpful, and that perhaps you picked up a few hints, tricks, and best practices along the way. As your requirements to interact with MarkLogic Server expand, you may need to write several web services to meet these different needs. Hopefully this sample gives you a good idea of just how easily this can be accomplished.

Comments