Server-side JavaScript

Important concepts

Server-Side JavaScript allows developers to access the powerful query and data manipulation capabilities of MarkLogic in a language and with tools that they’re already familiar with. Combined with JSON as a native data format, MarkLogic provides an ideal platform for building JSON-based services with JavaScript.

Uses

Clients of MarkLogic can be a browser, Node.js, or other platform -- the term "Server-side JavaScript" SJS distinguishes JavaScript running inside MarkLogic from these uses.

How you think about SJS may depend on your architectural choice. A common model is to use a three-tiered architecture, with JavaScript as the database's stored procedure language. MarkLogic also enables developers to build two-tier applications, with MarkLogic acting as the application server and hosting HTML, client-side JavaScript, CSS, and images. In this case, server-side endpoints can be written in Server-side JavaScript or in XQuery.

JSON Nodes vs. JavaScript Objects

When working with Server-Side JavaScript, it’s important to distinguish between instances of JSON nodes (Node and its derived types) and JavaScript objects (plain old Objects). JSON nodes are what are persisted in the database. They map one-to-one with the JSON data model and are immutable. When you get a document out the database it’s an instance of a node, literally instanceof Node in JavaScript. You can also create a JSON node from a JavaScript object using xdmp.toJSON(obj) or from a string using xdmp.unquote(str). You can read properties of nodes, but you cannot update nodes in-place, like you would a plain JavaScript object. This is a performance optimization. See Declare Updates below for best practices around updating JSON nodes in JavaScript.

JavaScript objects, on the other hand, exist only in memory in an E-node. They are mutable and represent the JavaScript data model, including things like functions, references to other objects, undefined, and NaN, that can’t be represented in JSON. Functions like xdmp.documentInsert() automatically convert JavaScript objects to nodes, similar to the result of calling xdmp.toJSON(obj). To get an object from a node, use the Node instance’s .toObject() function.

To see an example of this, let's first put a document into our database:

declareUpdate();

var user = {
  "name": "Dave Cassel"
};
    
xdmp.documentInsert(
  '/dave-cassel.json',
  user,
  xdmp.defaultPermissions(), 
  ["fake data"]
);

When we retrieve the document, we can can convert it from a node to an object, allowing us to make changes to it in memory. We can then persist those changes by writing them back to the database.

// We need to call declareUpdate() to persist changes in the database. 
declareUpdate();

var uri = "/n81ff186a6c5290db.json";

// cts.doc() returns a Document node or null, given a unique id (URI)
var user = cts.doc(uri) 
  .root        // The root Node instance of the Document container. For a JSON document, likely an ObjectNode
  .toObject(); // Turn any Node instance into JavaScript object. 

// Update the "name" property to upper-case.  Again, you don't need the preceding .toObject() 
// if you just want to read the value. However, to update it you need to first convert it 
// to a plain old JavaScript object with .toObject().
user.name = user.name.toUpperCase(); 

// Convert the plain old Object instance back into a Document node.
var node = xdmp.toJSON(user);

// You can read properties on Node instances, but you must convert to an object to update any aspect.
// Remember, xdmp.toJSON() creates a Document node, so you have to use .root to get the root ObjectNode.
node.root.name;

// …but you don't have to convert an object to a Node to persist it.
// Functions like xdmp.documentInsert() will automatically do the equivalent of xdmp.toJSON(user), above.

xdmp.documentInsert(uri, user, xdmp.defaultPermissions(), ["fake data"]);

To summarize, the typical pattern for updating a document in the database is:

Call declareUpdate() at the top of your module.
Get a Document from something like cts:doc() or looping over cts:search() (see "Iterators" below).
Convert the Document to a plain old JavaScript object using the document’s .toObject() function.
Change the object instance like you would any other object, e.g. obj.prop = "new" or obj.list.push("new").
Write the updated object back to the database using xdmp.documentInsert(). It handles the conversion of the object instance to a Node just as if you’d explicitly called xdmp.toJSON() on the object.

Iterators

Most functions that read data from the database return an instance of a ValueIterator, rather than a full instantiation of the data. This allows the evaluation environment to lazily and asynchronously load data as it’s required, rather than up-front. Use the .toArray() function on a ValueIterator instance to eagerly load the entire contents into a new array.

The ValueIterator interface implements the ECMAScript 6 Iterators interface. Like any iterator, you can loop through a ValueIterator using a for…of loop. Note that the for…of loop is not at all the same as a for…in loop, which loops over an object’s properties.

// fn.collection(), and most things that interact with the database, returns a ValueIterator
// The ES6 for…of loop automatically iterates.
for (var doc of fn.collection()) {
  var user = doc.toObject();
  // Do something with the object...
}

That’s a shorter way of explicitly using the Iterator interface:

var itr = fn.collection();
while (true) {          // Loop until break
  var item = itr.next(); // Advance the iterator
  if(item.done) break;   // If we're at the end, exit the loop
  var user = item.value.toObject();
  // Do something with the object...
}

Generators

Generators are closely related to Iterators and are also part of ECMAScript 6. Generators are a special kind of function, which may be paused in the middle, once or many times, and resumed later, allowing other code to run during these paused periods. This is an incredibly powerful concept, especially paired with iterators, allowing you to lazily work with sequences of indeterminate length, such as those returned from cts.search().

function* asObjects(itr) {
  for(var doc of itr) {
    yield doc.toObject(); 
  }
}

var a = [];
for(var obj of asObjects(fn.collection())) {
  // Assumes you have JSON documents in the database with .name properties, as in the examples below.
  a.push(obj.name = "Generated: " + obj.name);
}
a;

For more details on generators, take a look at "The Basics Of ES6 Generators".

Declare Updates

When a JavaScript module makes changes to the database, for example calling xdmp.documentInsert(), those changes are not executed immediately. Rather, the changes are recorded over the life of the request and applied atomically at the end. In XQuery, the fact that a module was writing to the database can be determined with static analysis of the code. In JavaScript any module that needs to update the database needs to state its intentions using the global declareUpdate() function. This function must be called before the first update. In general, putting declareUpdate() as the first statement is best practice, making it clear the reader that this module will make updates.

Modules

Server-Side JavaScript main modules can import library modules using a syntax based on CommonJS, similar to Node.js. At its core, a library module declares the variables and functions that it exports. A main module or another library module can import those into its scope using the require() function and assigning the exports to module-level global variables, which behave like namespaces.

A main module imports a library module into a variable.

// Main module
var util = require("utilities.sjs");
util.doSomething();

The library module declares the functions and variables that it exports.

// utilities.sjs

// Library modules can require other library modules.
var o = require("../other.sjs");

module.exports = {
  // Maps the internal implementation to a public name.
  doSomething: internalSomething
};

// "Protected" function not accessible outside of the current module.
function internalSomething() {
  // Do something…

The syntax of MarkLogic’s JavaScript modules will be familiar to Node.js users. However, the implementation is different in several important ways.

Import paths and precedence: The global require() function looks up module paths according to a well-defined set of rules, coincidently shared by the XQuery engine. When a module starts with a slash (/ or C:\ on Windows), MarkLogic will look for a file on the specified path relative to the Modules file-system directory of the MarkLogic installation. If the module is not found there, it will look relative to the app server root for both file-system- and modules database-backed configurations. If the import path does not start with a slash, the path is resolved relative to the location of the module calling the require().
XQuery imports: Server-Side JavaScript modules can import XQuery library modules. The importing JavaScript can access public functions and variables from the XQuery module as if they were JavaScript. (An example is shown on the Working Together page of this tutorial.) If an imported module—Server-Side JavaScript or XQuery—does not end with a file extension, require() will first look for the module with the user-specified name appended with the configured extensions for JavaScript module, and then appended with the configured extensions for XQuery module. The file extensions are defined in the mimetype configuration. Server-Side JavaScript (application/vnd.marklogic-javascript) defaults to .sjs and XQuery (application/vnd.marklogic-xdmp) defaults to any one of .xqy, .xqe, .xq, or .xquery.
Export scope: It is possible to change the state of an exported variable in MarkLogic. However, that change only persists for the request in which it was made. Thus subsequent requests and other hosts won’t have access to modified exports. Imported modules are cached, but their state does not persist beyond the scope of an individual request. In general, changing exports at runtime is not a best practice because it makes code more difficult to reason and to be reflected globally would have to happen in every importing module. This is different than how exports work in Node.js. In Node.js, export state is global and changes are reflected in subsequent requests to the same Node process.

Contents