10 Years Later, A Dream Come True

by Evan Lenz

(This is the first in a series of posts from guest developer, Evan Lenz)

I started my first XML job in the spring of 2000, working for an XML database/search engine company. This was around the time that XQuery was just being conceived of. I was (and still am) an avid XSLT user, and the rise of XQuery represented a sort of threat to my turf of XML processing languages—to the point that I even wrote an impassioned argument against "Reinventing the Wheel." Why do we need XQuery when we already have XSLT? Well, I've since done some growing up, and now I recognize the value of both. In fact, I joined the efforts in the W3C to help ensure that XPath 2.0 fully met the needs of both XSLT 2.0 and XQuery 1.0 users.

The received wisdom is that XQuery is great for querying data and XSLT is great for styling documents. And that's undoubtedly true. But I've always wanted to be able to apply the core power of XSLT (template rules, which are absent in XQuery) to large datasets. Traditionally, XSLT processors have required their input to be loaded into memory. This puts obvious limits on how much data you can process/query/transform in a single go. My dream back at the beginning of the century was to overcome that limitation, implementing XSLT as a query language over thousands, or millions, of documents. What if XSLT users could have large data sets instantly at their fingertips, without having to worry about memory constraints or the overhead of parsing all that XML at the time they run their stylesheet?

10 years later, my dream has come true. Over the past few months, I've had the great fortune of consulting on a project for MarkLogic, using an internal version of their upcoming software release: MarkLogic Server 4.2. This has truly been a joy to work with. Everything just works as I'd expect it. I haven't done extensive conformance testing, but I've definitely been kicking the tires a lot. I've been able to implement all of my architectural ideas without any significant issues. I don't know how they did it (and I'm slowly learning, each time I get a chance to talk to their engineers), but it's doing just what I want it to do.

I think MarkLogic has done the right thing. There are a lot of XSLT users out there. By supporting XSLT as a first-class query language, they have instantly expanded their market. To tell you the truth, before the XSLT support came along, I was nervous to dive in. Now, I'm chomping at the bit.

I'm going to be writing more about how you can use XSLT in MarkLogic. Until then, you can take a look at the result of my first interaction with MarkLogic's XSLT support. I helped write the code that runs the Developer Community website. Take a peek under the hood at the XSLT-heavy source code for "RunDMC."