[MarkLogic Dev General] RecordLoader READ_ROLES

[Michael Blakeley]

Lee,

Thanks, that's a good problem report.

I believe the problem is that any permissions set in XCC (or 
xdmp:document-insert) will override, rather than overlay, the default 
permissions. So the server sees a new document that has the permission 
(public,read) and nothing else. Since the inserting user doesn't have 
the admin role, the server then throws XDMP-MUSTHAVEUPDATE. You probably 
don't want to grant admin to content-loader, because the resulting 
documents will *only* have (public,read) permissions.

I have a couple of suggestions, ranging from quick hacks to complex code 
changes.

a) Create a new user 'content-loader-public' that looks just like 
content-loader and also has the (public,read) default permission. Use 
that login for this recordloader run, but drop the READ_ROLES property 
so that the default permissions will be used. This is a hack: clearly 
you wouldn't want to create an unbounded number of special-purpose users 
like this.

b) Extend recordloader with UPDATE_ROLES, INSERT_ROLES, and 
EXECUTE_ROLES. This would be much more straightforward than (b): mostly 
copy-paste-modify from existing code, with no XQuery coding required.

c) Extend recordloader so that, during its initialization, it copies any 
default permissions for the active user. I'm not sure how practical this 
is, but I think it would be a good feature so I'll look into it.

Of these, (c) is my favorite but also the most work to implement. The 
only one that qualifies as a code-free workaround is (a). I will look 
into both (b) and (c), but can't promise to have either done on a 
particular schedule. So if you are on a deadline and can't live with 
(a), I would recommend trying (b) as it's the easier modification. As 
always, I welcome patches.

-- Mike

On 2010-03-10 03:15, Lee Pollington wrote:
> I'm trying to use READ_ROLES with RecordLoader to load some documents and give permissions to a role called public from a user called content-loader. content-loader is assigned a single role called backoffice. backoffice has various insert/update related permissions and the following capabilities:
>
> backoffice (update)
> backoffice (insert)
> backoffice (read)
> backoffice (execute)
>
> When I run RecordLoader with the following command (specific names changed...):
>
> java -cp xpp3-1.1.4c.jar:/marklogic-4.1.3.jar:recordloader-1.0.jar -DCONNECTION_STRING=xcc://content-loader:content-loader@myserver:51058 -DID_NAME=#AUTO -DRECORD_NAME=record -DOUTPUT_COLLECTIONS="urn://myhost.com/content/published<http://myhost.com/content/published>  urn://myhost.com/content/document<http://myhost.com/content/document>"  -DREAD_ROLES=public com.marklogic.ps.RecordLoader /home/agile/mydoc.xml
>
> I get:
>
> RecordLoader starting, version 2009-11-15.1 on 1.6.0_16 (Java(TM) SE Runtime Environment) UTF-8
> 10-Mar-2010 11:08:21 com.marklogic.ps.RecordLoader initConfiguration
> INFO: Configuration is com.marklogic.recordloader.xcc.XccConfiguration
> logging to CONSOLE
> logging to file simplelogger-%u-%g.log
> 10-Mar-2010 11:08:21 com.marklogic.ps.SimpleLogger configureLogger
> INFO: setting up logging for: com.marklogic.ps<http://com.marklogic.ps>
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.AbstractConfiguration validateProperties
> INFO: using CONNECTION_STRING=xcc://content-loader:content-loader@myserver:51058
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.AbstractConfiguration validateProperties
> INFO: using READ_ROLES=public
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.AbstractConfiguration validateProperties
> INFO: using OUTPUT_COLLECTIONS=urn:/myhost.com/content/published<http://myhost.com/content/published>  urn://myhost.com/content/document<http://myhost.com/content/document>
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.AbstractConfiguration validateProperties
> INFO: using RECORD_NAME=record
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.AbstractConfiguration validateProperties
> INFO: using ID_NAME=#AUTO
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Configuration setUseAutomaticIds
> INFO: generating automatic ids
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Configuration configureOptions
> INFO: using input encoding UTF-8
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Configuration configureOptions
> INFO: using malformed input action REPORT
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Configuration configureCollections
> INFO: adding extra collection: com.marklogic.ps.RecordLoader.1268219301575
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Configuration configure
> INFO: connecting to xcc://content-loader:content-loader@myserver:51058
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.xcc.XccConfiguration configure
> INFO: configuring XCC-specific options
> 10-Mar-2010 11:08:21 com.marklogic.ps.RecordLoader<init>
> INFO: client hostname = AGILE
> 10-Mar-2010 11:08:21 com.marklogic.ps.RecordLoader<init>
> INFO: RecordLoader starting, version 2009-11-15.1 on 1.6.0_16 (Java(TM) SE Runtime Environment) UTF-8
> 10-Mar-2010 11:08:21 com.marklogic.ps.RecordLoader run
> INFO: thread count = 1
> 10-Mar-2010 11:08:21 com.marklogic.ps.RecordLoader initInputHandlerConstructor
> INFO: input handler = com.marklogic.recordloader.DefaultInputHandler
> 10-Mar-2010 11:08:21 com.marklogic.ps.RecordLoader runInputHandler
> INFO: inputs.size = 1
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.LoaderFactory<init>
> INFO: Loader is com.marklogic.recordloader.Loader
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Loader getXppVersion
> INFO: XPP3 version = 1.1.4c
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.DefaultInputHandler run
> INFO: populating queue
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Configuration getContentFactoryConstructor
> INFO: ContentFactory is com.marklogic.recordloader.xcc.XccContentFactory
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Configuration getContentFactoryConstructor
> INFO: client = 4.1-3, server = 4.1-3
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.DefaultInputHandler run
> INFO: queued 1 loader(s)
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.ProducerFactory<init>
> INFO: Producer is com.marklogic.recordloader.Producer
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Loader processRecords
> WARNING: error in asist-thesaurus at /home/agile/Desktop/asist-thesaurus.xml
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Loader processRecords
> WARNING: current URI = mydoc/2
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Loader processRecords
> WARNING: producer = 420 in empty byteBuffer
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Loader processRecords
> WARNING: buffer = null
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Loader processRecords
> WARNING: pos =  END_TAG seen ...<ttm>activities and operations</ttm>\n\n</record>... @29:12
> 10-Mar-2010 11:08:21 com.marklogic.recordloader.Loader processRecords
> WARNING: text =</record>
> 10-Mar-2010 11:08:21 com.marklogic.ps.SimpleLogger logException
> SEVERE: exception
> com.marklogic.xcc.exceptions.XQueryException: XDMP-MUSTHAVEUPDATE: Document must have at least one update permission
> in /insert
>      at com.marklogic.xcc.impl.handlers.ServerExceptionHandler.handleResponse(ServerExceptionHandler.java:30)
>      at com.marklogic.xcc.impl.handlers.ContentInsertController.serverDialog(ContentInsertController.java:122)
>      at com.marklogic.xcc.impl.handlers.AbstractRequestController.runRequest(AbstractRequestController.java:76)
>      at com.marklogic.xcc.impl.SessionImpl.insertContent(SessionImpl.java:194)
>      at com.marklogic.xcc.impl.SessionImpl.insertContent(SessionImpl.java:222)
>      at com.marklogic.recordloader.xcc.XccContent.insert(XccContent.java:74)
>      at com.marklogic.recordloader.AbstractLoader.insert(AbstractLoader.java:326)
>      at com.marklogic.recordloader.Loader.processStartElement(Loader.java:294)
>      at com.marklogic.recordloader.Loader.processRecords(Loader.java:154)
>      at com.marklogic.recordloader.Loader.process(Loader.java:103)
>      at com.marklogic.recordloader.AbstractLoader.call(AbstractLoader.java:96)
>      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>      at java.lang.Thread.run(Thread.java:619)
>
>
> All help appreciated,
> Thanks,
> Lee
>