[MarkLogic Dev General] lib-parser

Shannon Shiflett at virginia.edu
Thu Oct 9 09:40:53 PDT 2008


Thank you, Mike--that's so very agreeable--yes, per-query control  
language awareness would be most useful!  Given a form that accepts a  
query string input and a language selector that includes an "all"  
option, the desired behavior is language-specific tokenization, in  
this case, for English and French; Danny demystified the search recall  
logic, but lib-parser doesn't provide the full support, yet, to get  
the most out of the French language module--currently I'm using the  
overloaded lp:get-cts-query() that grabs $options at the 3rd argument;  
maybe another overload with a 4th argument, or perhaps take the hint  
from the lang option if supplied?
Thanks,
Shannon

On Oct 8, 2008, at 5:29 PM, Michael Blakeley wrote:

> Today, lib-parser calls cts:tokenize() without the language  
> argument, so it always uses the database default language. So the  
> tokenization is language-aware, but there's no per-query control  
> over which language it uses.
>
> If per-query control over language awareness would be useful, how  
> would you like to express it? As another (optional) argument to  
> lp:get-cts-query()?
>
> I'm a little concerned about maintaining a distinction between  
> cts:query term-level language, vs the language passed to  
> cts:tokenize() in lp:get-cts-query-element(). But if it's useful  
> functionality, let's figure out how to add it.
>
> -- Mike
>
> Shannon wrote:
>> Hi,
>> Does anyone know whether lib-parser has support for language-aware   
>> tokenization, for lp:get-cts-query specifically?
>> Thanks,
>> __________________________________________________
>> Shannon Scott Shiflett, programmer/analyst with ROTUNDA,
>> The University of Virginia Press, Charlottesville, VA  USA
>> http://rotunda.upress.virginia.edu
>> _______________________________________________
>> General mailing list
>> General at developer.marklogic.com
>> http://xqzone.com/mailman/listinfo/general
>
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://xqzone.com/mailman/listinfo/general

__________________________________________________
Shannon Scott Shiflett, programmer/analyst with ROTUNDA,
The University of Virginia Press, Charlottesville, VA  USA
http://rotunda.upress.virginia.edu



More information about the General mailing list