At a high level, the purpose of Packaging is to answer the questions “how do these two applications differ and how can I make them the same?”
Natural uses for packaging include:
- Maintaining a certified master configuration. By comparing a packaged version of the certified master with the current configuration of a cluster, any deviations can be found.
- Bringing a new developer online. If a standard set of applications is maintained in a package, perhaps checked into a version control system, then new developers can “spin up” a system simply by installing MarkLogic Server then installing the necessary package.
- Migrating the configuration of one system to another: a development cluster to the testing cluster, testing to staging, or staging to production.
For the purposes of this posting, consider a common scenario. We’ve rolled out an application and it has been very successful. Our team has been resolving issues and implementing new features. Now we’re ready to test it out on our staging cluster.
Moving it to staging means copying the code and making sure that all of the databases and application servers have the correct configurations for the application. A potentially tedious process!
Not so with packaging! Step one is to fire up Configuration Manager on port 8002. If you’re a developer, you’re probably already familiar with configuration manager as a useful “read only” view of the system configuration.
What’s new in MarkLogic 7 are the “Export” and “Import” tabs.
Configuration Manager
Starting on our development machine, we’ll click on the “Export” tab.
Configuration Manager Export
The export view adds check boxes to the databases and application servers. These check boxes allow us to select which artifacts are going to be stored in the package.
Selecting Databases
The example application in our scenario uses two databases (AppContent and AppModules) and two servers (AppHttp and AppWebDAV). On the screenshot above, you can see that we’ve selected the two databases used by our application. This will add the configuration of those databases to the package.
Packaging cannot be used for copying the content of ordinary databases. For that, you can use mlcp
Configuration Manager uses the standard “three state” check boxes to summarize what you’ve selected. An empty check box means nothing is selected, a check box with a check in it means that entire item is selected, a check box that’s shaded blue indicates partial selection.
Unselecting Memory Settings
One of the important new features of packaging is the ability to construct a “sparse” package. A sparse package is one that contains only a subset of the configuration about an artifact. In the screenshot above, I’ve clicked on the AppContentdatabase and scrolled down to the memory settings. My development laptop and the staging cluster are significantly different machines; it makes no sense to copy my memory settings to the cluster.
Any item that you “uncheck” in a database or application server will not be part of the package. That means installing the package will not change that setting. In the event that the package is creating an entirely new artifact, default values will be used for any unspecified properties.
Selecting Application Servers
On the servers page, I’ve selected both the AppHttp and AppWebDAV servers. I’ve also checked the “Include Modules” check box. Also new in packaging is the ability to move code between clusters. If your application stores its code in a modules database, you can include the content of that database in the package.
Constructing the Package
When I click “Export”, the package will be created. Packages in MarkLogic 7 are ZIP files that contain all of the artifacts selected in the package.
Package files can then be transfered over the network, on a USB key, or any other way you wish. Now that we have the configuration of our development machine saved in a package, we can move that over to the staging cluster and compare the new configuration with the current configuration.
Importing a Package
Start on the “Import” tab, browse to or type the name of the package ZIP file, and click “Compare”. Compare gives you a visual summary of the differences between the package and the current cluster configuration.
Package Comparison Overview
The initial compare screen summarizes the changes in the package. On the upper left, we see the number of databases and servers in the package and the kinds of changes they make. Refer to the key on the lower left for the meanings of the symbols. On the right hand side, we see each of the databases (or servers if we click “Servers” on the left) with a summary of the changes, additions, and deletions to that artifact.
Database Changes
By clicking on the artifact, we can drill down and see precisely what changes are being proposed by the package. On this screen in the AppContent database we can see that the setting for “fast phrase searches” and “triple index” are being changed.
The package installer has the option of excluding changes from the package. Here, by unchecking the box next to “fast phrase searches” we’ve excluded that change from the package.
Installed Package
Finally, by clicking “Apply”, we make the changes. After package installation finishes, the summary page highlights what has been changed and offers a link to the package ticket and “rollback” page. Rolling back a package undoes the changes just made.
That’s a quick tour through packaging in MarkLogic 7. Everything avialable in the browser is also available through REST and native XQuery APIs.
There are still things that are not covered by packaging in MarkLogic 7. Packaging cannot be used to:
- Move security settings (users, roles, privileges).
- Move scheduled tasks, triggers, and other artifacts not contained in application servers and databases.
- Move code not stored in a modules database.
- Move “cluster wide” configuration items, such as the distribution of forests across D and E nodes in a cluster.
These remain possible enhancements for the future, but in the meantime, we think the packaging changes in MarkLogic 7 will make application development easier, faster, and more reliable!