Inside MarkLogic Server

This paper describes the MarkLogic Server internals: its data model, indexing system, update model, and operational behaviors. It's intended for a technical audience — either someone new to MarkLogic wanting to understand its capabilities, or someone already familiar with MarkLogic who wants to understand what's going on under the hood.


This paper is not an introduction to using MarkLogic Server. For that you can read the official product documentation. Instead, this paper explains the principles on which MarkLogic is built. The goal isn't to teach you to write code, but to help you understand what's going on behind your code, and thus help you write better and more robust applications.


The paper is organized into sections. The first section provides a high-level overview of MarkLogic Server. The next few sections explain MarkLogic's core indexes. The sections after that explain the transactional storage system, multi-host clustering, and the various connection options. At this point there's a natural stopping point for the casual reader, but those who read on will find sections covering advanced indexing features as well as topics like replication and failover. The final section discusses the ecosystem built up around MarkLogic.


The latest version of the paper is available here.


The paper is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. For permissions beyond this license, send requests to community-requests@marklogic.com.


Table of Contents

What Is MarkLogic Server?

  • Document-Centric
  • Transactional
  • Search-Centric
  • Structure-Aware
  • Schema-Agnostic
  • Programmatic
  • High Performance
  • Clustered
  • Database Server

Core Topics

  • Indexing Text and Structure
    • Indexing Words
    • Indexing Phrases
    • Indexing Longer Phrases
    • Indexing Structure
    • Indexing Values
    • Indexing Text Values
    • Indexing Text with Structure
    • Special Phrase Handling
    • Index Size
    • Reindexing
    • Relevance
    • Lifecycle of a Query
  • Indexing Document Metadata
    • Collection Indexes
    • Directory Indexes
    • Security Indexes
    • Properties Indexes
  • Fragmentation
    • Fragment vs. Document
    • Estimate and Count
    • Unfiltered
  • The Range Index
    • Range Queries
    • Data-type Aware Equality Queries
    • Extracting Values
    • Optimized "Order By"
    • Using Range Indexes for Joins
    • Using Path Range Indexes for Extra Optimization
    • Lexicons
  • Data Management
    • What's on Disk: Databases, Forests, and Stands
    • Ingesting Data
    • Modifying Data
    • Multi-Version Concurrency Control
    • Time Travel
    • Locking
    • Updates
    • Documents are Like Rows
    • Lifecycle of a Document
    • Multi-Statement Transactions
    • XA Transactions
  • Tiered Storage
    • Fast Data Directory on SSDs
    • Large Data Directory for Binaries
  • Clustering and Caching
    • Cluster Management
    • Caching
    • Caching Binary Documents
    • Cache Partitions
    • No Need for Global Cache Invalidation
    • Locks and Timestamps in a Cluster
    • Lifecycle of a Query in a Cluster
    • Lifecycle of an Update in a Cluster
  • Coding and Connecting to MarkLogic
    • XQuery and XSLT
    • XDBC/XCC for Java and .NET Access
    • REST API
    • WebDAV: Remote Filesystem Access
    • SQL/ODBC Access for Business Intelligence
    • QC for Remote Coding

Advanced Topics

  • Advanced Text Handling
    • Text Sensitivity Options
    • Stemmed Indexes
    • Relevance Scoring
    • Stop Words
  • Fields
    • More with Fields
  • Registered Queries
  • The Geospatial Index
  • The Reverse Index
    • Reverse Query Use Cases
    • A Reverse Query Carpool Match
    • The Reverse Index
    • Range Queries in Reverse Indexes
  • Managing Backups
    • Typical Backup
    • Flash Backup
    • Journal Archiving and Point-in-Time Recovery
  • Failover and Replication
    • Shared-Disk Failover
    • Local-Disk Failover
    • Database Replication
    • Flexible Replication
  • Hadoop
  • Aggregate Functions and UDFs in C++
  • Low-Level System Control
  • Outside the Core
    • Application Services
    • Content Pump (mlcp)
    • Content Processing Framework
    • Office Toolkits
    • Connector for SharePoint
    • Unofficial Tools, Libraries, and Connectors
  • But Wait, There's More...
Back to Top
 

Comments