[XQZone General] Guaranteed unique?

[Stewart, Joel (LNG-DAY)]

I have been wondering how to accomplish this when needing to call it a
number of times from within an xquery function, e.g.:

define function next-id ($doc-uri) as xs:integer {
         let $id := doc($doc-uri)/id
         let $next-val :=
                 if ($id castable as xs:integer) then xs:integer
($id) + 1
                 else 1
         let $next-node := <id>{$next-val}</id>
         let $insert :=
                 if ($id) then xdmp:node-replace ($id, $next-node)
                 else xdmp:document-insert ($doc-uri, $next-node)

         return $next-val
}

let $seq := (1,2,3,4,5,6,7,8,9)
return 
  for $x in $seq
     return next-id("tempuri")

running this will result in
XDMP-CONFLICTINGUPDATES: Conflicting updates
xdmp:document-insert("tempuri", /id, (), (), 0) and
xdmp:document-insert("tempuri", /id, (), (), 0) at line 18


If I was to call this from a Java layer that iterates the nodes to be
inserted, I recognize this method would work such that you either get a
number or you get an exception.  Retry logic would need to be written.  

We put a lot of work in XQuery modules, and iterate large numbers of
nodes and do many inserts within the xquery function for performance.
To my knowledge, we have not needed an auto-increment function in all
this, but I have often wondered what we should do should that need
arise.  Hopefully, the number of ids that will be needed can be
determined easily and quickly before invoking the functions that do the
inserts.  In this way, we can pass in a sequence or range of ids.  But
if the count of nodes is hard or expensive to determine - I was trying
to think of a work around.

One idea is to use eval-in.  I have been wondering if using that to put
your ids in another database would work around the above issue.
Apparently it does.  By using eval-in, the "next-id" document is
actually committed.  So trying the following will work if you crate a
new DB to actually execute the "next-id" function.  The code could be
more pretty by putting next-id in a module in that db, but it gets the
idea across.

define function next-id ($doc-uri) {
  let $func := 
fn:concat("
         let $doc-uri :=""",$doc-uri,"""
         let $id := doc($doc-uri)/id
         let $next-val :=
                 if ($id castable as xs:integer) then xs:integer
($id) + 1
                 else 1
         let $next-node := <id>{$next-val}</id>
         let $insert :=
                 if ($id) then xdmp:node-replace ($id, $next-node)
                 else xdmp:document-insert ($doc-uri, $next-node)

         return $next-val
         ")

   return xdmp:eval-in($func, xdmp:database("ANOTHER-DB")) 
                   
}

let $seq := (1,2,3,4,5,6,7,8,9)
return 
  for $x in $seq
     return next-id("tempuri")

So, running this will work.  The hard part will be putting error
handling and retry logic in the function calling it - if it so happens
two threads get on that node at the same time and cause conflicting
updates.  But I think this goes a long way toward what is needed to be
implemented natively in marklogic - some "autoincrement" function that
is not bound to the same rollback, and is atomic such that you will not
get the conflicting updates exception.  

Other ideas are welcomed.

Thanks.
joel

-----Original Message-----
From: general-bounces at xqzone.marklogic.com
[mailto:general-bounces at xqzone.marklogic.com] On Behalf Of Ron Hitchens
Sent: Tuesday, November 22, 2005 7:40 PM
To: General XQZone Discussion
Subject: Re: [XQZone General] Guaranteed unique?


    Here is a function to do what Ian describes, lifted
from the xfaqtor code available on xqzone:


(: Return a monotonically increasing integer (xs:integer) value.
    Store the value in the root node of doc ($doc-uri).
    You could enhance this file to store IDs for multiple uses.
    Logic is to fetch the current id, add one if it exists or assign the
    value to one if it doesn't yet exist, then replace or insert the new
    value before returning it.
:)
define function next-id ($doc-uri) as xs:integer
{
         let $id := doc($doc-uri)/id
         let $next-val :=
                 if ($id castable as xs:integer) then xs:integer  
($id) + 1
                 else 1
         let $next-node := <id>{$next-val}</id>
         let $insert :=
                 if ($id) then xdmp:node-replace ($id, $next-node)
                 else xdmp:document-insert ($doc-uri, $next-node)

         return $next-val
}


On Nov 22, 2005, at 4:15 PM, Ian Small wrote:

> Andy -
>
> Just create a document that contains the serial number in  
> question.  Write some simple XQuery to retrieve the id from the  
> document, increment it, and update the document (using xdmp:node- 
> replace(), etc.).
>
> Because your XQuery module is performing an "update", it will block  
> all other requests trying to update that same document, thereby  
> guaranteeing unique IDs.
>
> Only "queries" (ie. requests which are statically analyzed to be  
> "read-only" requests) run lock-free per your description.  Updates  
> (ie. requests which are statically analyzed to potentially change  
> the state of the database) lock documents and consequently block  
> each other as appropriate.
>
> ian
>
> -----Original Message-----
> From: general-bounces at xqzone.marklogic.com [mailto:general- 
> bounces at xqzone.marklogic.com] On Behalf Of Andy Townsend
> Sent: Tuesday, November 22, 2005 1:43 PM
> To: general at xqzone.marklogic.com
> Subject: [XQZone General] Guaranteed unique?
>
>
>
>
> Anyone know a nice way to generate guaranteed unique sequential IDs  
> with XQuery in Mark Logic?
>
> So I can for example create the values for:  id="1" , id="2", etc...
> and be sure that two users cannot ever pick up the same number.   
> Just having a document in the database with the current number  
> doesn't seem enough to me because I believe there would be a race  
> condition where two users could start at about the same time and so  
> see the same initial 'state' of the database which I presume the  
> XQuery would insist they both saw throughout their queries.
>
> I thought that document-locks might get me there (though in  
> probably contravention of the standard) but have not had any joy  
> with those as yet
>
> In Java it would be easy - just a serialised block - equally easy  
> in C++, but what about XQuery?
>
> Someone must have done this already!   :-)
>
> Thanks
>
> Andy
>
> ######################################################################
> The information contained in this e-mail and any subsequent  
> correspondence is private and confidential and intended solely for  
> the named recipient(s).  If you are not a named recipient, you must  
> not copy, distribute, or disseminate the information, open any  
> attachment, or take any action in reliance on it.  If you have  
> received the e-mail in error, please notify the sender and delete  
> the e-mail.
>
> Any views or opinions expressed in this e-mail are those of the  
> individual sender, unless otherwise stated.  Although this e-mail  
> has been scanned for viruses you should rely on your own virus  
> check, as the sender accepts no liability for any damage arising  
> out of any bug or virus infection.
> ######################################################################
> _______________________________________________
> General mailing list
> General at xqzone.marklogic.com
> http://xqzone.com/mailman/listinfo/general
> _______________________________________________
> General mailing list
> General at xqzone.marklogic.com
> http://xqzone.com/mailman/listinfo/general

---
Ron Hitchens {ron.hitchens at marklogic.com}  650-655-2351


_______________________________________________
General mailing list
General at xqzone.marklogic.com
http://xqzone.com/mailman/listinfo/general