[MarkLogic Dev General] lib-parser

Michael Blakeley michael.blakeley at marklogic.com
Wed Oct 8 14:29:29 PDT 2008

Today, lib-parser calls cts:tokenize() without the language argument, so 
it always uses the database default language. So the tokenization is 
language-aware, but there's no per-query control over which language it 

If per-query control over language awareness would be useful, how would 
you like to express it? As another (optional) argument to 

I'm a little concerned about maintaining a distinction between cts:query 
term-level language, vs the language passed to cts:tokenize() in 
lp:get-cts-query-element(). But if it's useful functionality, let's 
figure out how to add it.

-- Mike

Shannon wrote:
> Hi,
> Does anyone know whether lib-parser has support for language-aware  
> tokenization, for lp:get-cts-query specifically?
> Thanks,
> __________________________________________________
> Shannon Scott Shiflett, programmer/analyst with ROTUNDA,
> The University of Virginia Press, Charlottesville, VA  USA
> http://rotunda.upress.virginia.edu
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://xqzone.com/mailman/listinfo/general

More information about the General mailing list