[MarkLogic Dev General] Delete URI from a directory which contains more file

Geert Josten geert.josten at dayon.nl
Fri Oct 11 13:32:58 PDT 2013


Hi Gurbeer,



If you change ‘maintain last modified’ to false, you’d best also set
‘maintain directory last modified’ to false as well if that is not yet the
case. Once you have done that, you can also consider getting rid of
properties fragments, if you are not using properties on documents or
directories. And while at it, you can also check the ‘directory creation’
setting. You usually want it set to manual. After that, you might want to
get rid of directory fragments as well. In MarkLogic 6, these are the
‘factory’ defaults, but I thought that wasn’t yet the case in MarkLogic 5.



How to get rid of all those properties fragments? Deleting and recreating
the docs would work, but you might not want to do that. This thread
provides an alternative:
http://marklogic.markmail.org/thread/pg7t56v6b2xxbbbw



Note: if you are processing a lot of document at once, you might want to
process things in batches of 500 to 1000 documents. Spawn tasks in the
background to do so..



HTH!



Kind regards,

Geert



*Van:* general-bounces at developer.marklogic.com [mailto:
general-bounces at developer.marklogic.com] *Namens *Danny Sokolsky
*Verzonden:* vrijdag 11 oktober 2013 19:12
*Aan:* MarkLogic Developer Discussion
*Onderwerp:* Re: [MarkLogic Dev General] Delete URI from a directory which
contains more file



Glad you got past the first problem J



That probably means there are merges going on.  This is A Good Thing.  When
they finish, your forest size should be smaller.  This is why you need the
extra space, so merges can combine stands and remove deleted documents,
ending up using less disk space.



Also, you should consider upgrading at least to the latest 5.0 release.



-Danny



*From:* general-bounces at developer.marklogic.com [
mailto:general-bounces at developer.marklogic.com<general-bounces at developer.marklogic.com>]
*On Behalf Of *Singh, Gurbeer
*Sent:* Friday, October 11, 2013 10:05 AM
*To:* MarkLogic Developer Discussion
*Subject:* Re: [MarkLogic Dev General] Delete URI from a directory which
contains more file



Ok , that worked , but new problem



I made this change “*maintain last modified*” à false

We are using 5.01.2



After making above change , we are trying to delete all URI from “docsIn”
folder ,  we are doing this through REST API





Running below CODE ..

let $docURI:=  xdmp:get-request-field("uri", "")

return

if (fn:exists($docURI)) then

xdmp:document-delete($docURI)

else ()



We are doing this so that we can clean some space , when I start this
process at that time forest size was  54267 MB

After deleting some thousand of records its now 63697  MB



I am reading this number from Monitoring Dashboard àDisk Space à Mouse
hover on Forest.



~Gurbeer



*From:* general-bounces at developer.marklogic.com [
mailto:general-bounces at developer.marklogic.com<general-bounces at developer.marklogic.com>]
*On Behalf Of *Danny Sokolsky
*Sent:* Friday, October 11, 2013 12:28 PM
*To:* MarkLogic Developer Discussion
*Subject:* Re: [MarkLogic Dev General] Delete URI from a directory which
contains more file



Hi Gurbeer,



Do you have maintain directory last modified set on the database?  That
setting will cause a lot of locks to occur.  If it is true, true, try
setting it to false and see what happens.



Also, what version of MarkLogic ( xdmp:version() )?



-Danny

*From:* general-bounces at developer.marklogic.com [
mailto:general-bounces at developer.marklogic.com<general-bounces at developer.marklogic.com>]
*On Behalf Of *Singh, Gurbeer
*Sent:* Friday, October 11, 2013 9:18 AM
*To:* MarkLogic Developer Discussion
*Subject:* Re: [MarkLogic Dev General] Delete URI from a directory which
contains more file

62850

60910

54267

Actually this directory is very big 54267 MB

I tried to delete  all URI , but its hanging , so I thought to try one by
one. But no luck



~Gurbeer



*From:* general-bounces at developer.marklogic.com [
mailto:general-bounces at developer.marklogic.com<general-bounces at developer.marklogic.com>]
*On Behalf Of *Harry B.
*Sent:* Thursday, October 10, 2013 8:27 AM
*To:* MarkLogic Developer Discussion
*Subject:* Re: [MarkLogic Dev General] Delete URI from a directory which
contains more file



That's odd. I've never had xdmp:document-delete() work slowly when there's
hundreds of thousands of documents in a directory. You can have problems
with directory-based operations, but a single URI is not affected by this.
Is this document abnormally large or have some other problem that causes it
to not delete?



On Wed, Oct 9, 2013 at 3:20 PM, Singh, Gurbeer <
Gurbeer.Singh at morganstanley.com> wrote:

Delete URI from a directory which contains file count ‘304528’



I am running simple query



let $docURI:="/docsIn/1034916_meta.xml"

(:return fn:doc($docURI):)

return

xdmp:document-delete($docURI)





Its taking so much time  not working .





~Gurbeer


------------------------------


NOTICE: Morgan Stanley is not acting as a municipal advisor and the
opinions or views contained herein are not intended to be, and do not
constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall
Street Reform and Consumer Protection Act. If you have received this
communication in error, please destroy all electronic and paper copies and
notify the sender immediately. Mistransmission is not intended to waive
confidentiality or privilege. Morgan Stanley reserves the right, to the
extent permitted under applicable law, to monitor electronic
communications. This message is subject to terms available at the following
link: http://www.morganstanley.com/disclaimers If you cannot access these
links, please notify us by reply message and we will send the contents to
you. By messaging with Morgan Stanley you consent to the foregoing.


_______________________________________________
General mailing list
General at developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general




------------------------------


NOTICE: Morgan Stanley is not acting as a municipal advisor and the
opinions or views contained herein are not intended to be, and do not
constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall
Street Reform and Consumer Protection Act. If you have received this
communication in error, please destroy all electronic and paper copies and
notify the sender immediately. Mistransmission is not intended to waive
confidentiality or privilege. Morgan Stanley reserves the right, to the
extent permitted under applicable law, to monitor electronic
communications. This message is subject to terms available at the following
link: http://www.morganstanley.com/disclaimers If you cannot access these
links, please notify us by reply message and we will send the contents to
you. By messaging with Morgan Stanley you consent to the foregoing.


------------------------------


NOTICE: Morgan Stanley is not acting as a municipal advisor and the
opinions or views contained herein are not intended to be, and do not
constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall
Street Reform and Consumer Protection Act. If you have received this
communication in error, please destroy all electronic and paper copies and
notify the sender immediately. Mistransmission is not intended to waive
confidentiality or privilege. Morgan Stanley reserves the right, to the
extent permitted under applicable law, to monitor electronic
communications. This message is subject to terms available at the following
link: http://www.morganstanley.com/disclaimers If you cannot access these
links, please notify us by reply message and we will send the contents to
you. By messaging with Morgan Stanley you consent to the foregoing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://developer.marklogic.com/pipermail/general/attachments/20131011/30b6617b/attachment-0001.html 


More information about the General mailing list