Apache NiFi is a data flow platform which helps automate the movement of data between disparate systems. Flexible and secure from inception, NiFi started life as an internal project for the NSA before becoming a part of the Apache community.
Built on Java, NiFi was a great candidate for integration with the MarkLogic Java Client API and DMSDK. Its focus on security and provenance align with our core principles. NiFi’s collection of processors, easy to use GUI, and production monitoring capabilities make it even more attractive as a strategy for data orchestration.
MarkLogic is proud to announce its officially supported NiFi processors. As a member of the NiFi community, MarkLogic is leveraging the proven capabilities of NiFi as part of a data flow orchestration ecosystem. You can now ingest data from multiple source systems into MarkLogic using Apache NiFi which can also act as spokes for your data hub.
Currently, we support two processors built for MarkLogic– PutMarkLogic processor, for ingesting data into MarkLogic, and the QueryMarkLogic processor, used for querying documents in MarkLogic. There is also a growing collection of community developed processors, templates, and recipes.
Getting started with NiFi and MarkLogic could not be any easier. Here are some simple directions:
.narfiles provided by MarkLogic in the zip folder into the lib folder (nifi-1.7.0/lib) of the unpacked NiFi distribution.
https://localhost:8080/nifi/to run NiFi. Make sure you’re on the latest MarkLogic version (9.0+).
More instructions for getting started are available on GitHub. We also have a video tutorial on MarkLogic and NiFi. We have submitted our processors to the NiFi Community for inclusion in an upcoming release.
In this lightning talk delivered by Ganesh Vaideeswaran, VP of Product Development at MarkLogic, he demonstrates how MarkLogic’s DMSDK-based NiFi processors allow you to create an ingest dataflow in just a few clicks.