This tutorial is a part of a series pertaining to the MarkLogic Connector for Mulesoft. This is the “Access Flow” of the series, where we discuss the bottom flow that hosts a second REST endpoint, which triggers querying and retrieving inspection JSON documents back out of MarkLogic. The content is sent back as a JSON-based HTTP response to your web client. Here are the other steps of the series:
- Getting Started with the MarkLogic Connector for Mulesoft — Walk through installing the quickstart, configuring credentials and properties, and running the quickstart. Dive into a discussion of what each flow means and learn more about the MuleSoft ecosystem. Start here if you haven’t used the quickstart before; this tutorial assumes you have the quickstart installed and running.
- Ingest Data into MarkLogic using the Connector for MuleSoft — Discussion of the top flow, or the Ingest Flow, of our application, which hosts a REST endpoint, and when called, triggers fetching, transforming, logging, and ingesting the inspections CSV content as JSON into MarkLogic.
HTTP Listener
Just like the Ingest flow, our Egress flow also starts with an HTTP listener. It reuses the same listener as the Ingest flow, but implements a different endpoint. On this flow, the listener includes a Get Documents by Business Name endpoint at “/nyc-inspections/documents/businessName”.
The flow is expecting a GET request to the endpoint. It accepts an HTTP query parameter called “name,” whose value is set to the name of the business we want to retrieve out of MarkLogic on the “businessName” property in our JSON documents. For example, if we want to find the outcome of Target stores in New York City, we’ll use: http://localhost:8081/nyc-inspections/documents/businessName?name=”Target Corporation”
Figure 1: Egress listener
Structured Query Generation
Figure 2: Structured Query Generation
Our Egress flow application is now listening for an HTTP request, with a businessName value provided. This can serve as the basis of constructing our query into MarkLogic. The queryDocs operation in the MarkLogic Connector can accept either a Structured Query or a serialized CTS query as a String. No matter the query strategy, you can opt to express the query in either JSON or XML. Here, we’ll choose a JSON Structured Query. In terms of a type of MarkLogic query, we’re building a Structured Query representation of a cts.jsonPropertyValueQuery().
In order to build up the JSON Structured Query, we’ll set a variable using the built-in MuleSoft Set Variable transformer, and directly build the JSON using Dataweave. Call it “myStructuredQueryJSON”. Notice on the left in our input/source pane, we get to work with an Object output from the HTTP listener, which is addressable with “attributes.*”. Since the HTTP request query parameter “name” contains the value we want to query in MarkLogic, we can grab it directly from “attributes.queryParams.name,” and put it directly into our Structured Query JSON.
In order to use the JSON as the query string in MarkLogic queryDocs, we need to convert the JSON to string. To do this, we can simply use the Set Variable transformer as a follow-on operation and as a new variable. Refer to “Set StructuredQuery as String” in the flow, and in the “MIME Type” tab, set the MIME Type as “text/plain.” Doing so achieves a casting the JSON query object to a String representation of the JSON object. We’ll call the new variable “myStructuredQueryString.”
One could also easily allow another version of this endpoint accepting a POST, whereby the user could send in the JSON or XML Structured Query as the request payload, feeding that directly (with some guards against injection) to MarkLogic queryDocs.
MarkLogic Query Docs by Business Name
Now turn attention to the MarkLogic “Query Docs by businessName” part of the flow, which uses the MarkLogic Connector queryDocs operation. This operation instance simply reuses the same established MarkLogic connection from the import flow. We set the Serialized Query String parameter to use our variable from “Set StructuredQuery as String” with a bit of Dataweave notation: “#[vars.myStructuredQueryString]”. Since we used a JSON-structured Query, we want to set the Search Strategy dropdown to “RawStructuredQueryDefinition,” and set the “Serialized Query Format” to “JSON.”
Figure 4: Query docs by businessName
Notice on the right in the “Input” tab, our variables are preserved and referenced for ease of access. The red exclamation at top right is not an indicator of error– rather, it’s an indicator that neither MarkLogic nor MuleSoft cannot predict the output format of the data returned by MarkLogic, since MarkLogic can return zero to many of XML, JSON, text, or binary back as a response. The operator still proceeds as expected.
Transform JSON Response and Finish
Finally, once our Structured Query has been sent to MarkLogic and has completed, we can simply transform the output and send it back. Once again, we’ll use the Transform operation with a bit of Dataweave code. This time, however, we won’t perform any mapping. We’ll just author a straight JSON response. The response will consist of:
- A count of result documents from MarkLogic (using the
sizeOf()function) - The result documents themselves (just using “payload”)
- An echo back of the input Structured Query JSON variable
To conclude, the output of the transform gets routed back to the HTTP listener response, and the response gets sent back to the user.
Figure 5: Egress response
Our HTTP GET call to return documents detailing inspections at Target stores in New York City returns 16 results:
Wrapping Up
As you can see from this series, MarkLogic and MuleSoft make for an impactful duo. MuleSoft provides a robust set of connectors and operations to ease any integration solution, allowing users of the ecosystem streamlined capabilities to work with data and services within MarkLogic. MarkLogic and its Data Hub Framework provides the single source of truth, enterprise persistence layer, complete with Common Criteria certified security. We look forward to your feedback and assistance in continuing to develop the MarkLogic Connector for MuleSoft.
- Getting Started with the MarkLogic Connector for Mulesoft — Walk through installing the quickstart, configuring credentials and properties, and running the quickstart. Dive into a discussion of what each flow means and learn more about the MuleSoft Ecosystem.
- Ingest Data into MarkLogic using the Connector for MuleSoft — Discussion of the top flow, or the Ingest Flow, of our application, which hosts a REST endpoint, and when called, triggers fetching, transforming, logging, and ingesting the CSV content as JSON into MarkLogic.
- Access Data in MarkLogic using the Connector for MuleSoft — Walk through the bottom flow, or the Egress Flow of the series, that hosts a second REST endpoint, which triggers querying and retrieving JSON documents back out of MarkLogic. The content is sent back as a JSON-based HTTP response to your web client.