[MarkLogic Dev General] Logging queries

Sat Apr 18 03:25:03 PDT 2009

Mike,

There is another alternative, but I am not sure it suits your purpose.

It is the trace function, which causes 'event' messages to be written to the ErrorLog.txt (not the acceslog though). You can turn trace logging on and off at group level (Diagnostics configuration), but you can a) define your own events from application side giving you both full control on granularity and content, and b) select which events are to be logged whenever you feel like it.

Using the xdmp:query-meters(), you can add performance numbers to the trace messages.

On the other hand, if you are just interested in plain performance statistics (not as continuous monitoring, but as single time tests), you might want to take a look at the performance meters project on xqzone. You can write various test queries and this tool will run them a large number of times, multi-threaded if you like, giving you all kind of statistics..

Kind regards,
Geert

>

Drs. G.P.H. Josten
Consultant

http://www.daidalos.nl/
Daidalos BV
Source of Innovation
Hoekeindsehof 1-4
2665 JZ Bleiswijk
Tel.: +31 (0) 10 850 1200
Fax: +31 (0) 10 850 1199
http://www.daidalos.nl/
KvK 27164984
De informatie - verzonden in of met dit emailbericht - is afkomstig van Daidalos BV en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onbedoeld hebt ontvangen, verzoeken wij u het te verwijderen. Aan dit bericht kunnen geen rechten worden ontleend.

> From: general-bounces at developer.marklogic.com
> [mailto:general-bounces at developer.marklogic.com] On Behalf Of
> Danny Sokolsky
> Sent: vrijdag 17 april 2009 18:47
> To: General Mark Logic Developer Discussion
> Cc: Scott Cipriano
> Subject: RE: [MarkLogic Dev General] Logging queries
>
> Hi David and Mike,
>
> The file log level specifies the log level for the
> ErrorLog.txt file, and it does that on a group setting, so it
> is not a one-time thing.
>
> The only way I know of to log queries is to do so at the
> application level, which it sounds like you are already
> doing.  The access logs are really for the http and xdbc
> requests, and do not have other information about the content
> of the requests.  For many application, however, you might be
> able to reverse-engineer the queries based on the request URL
> in the access log.  For example, if you have a request
> parameter called query and its value is what the user entered
> in the text field, then you could probably figure out what
> that query is.
>
> Logging the actual XQuery text of every query can get
> expensive in production, and make your logs get very large
> very quickly.  If you want to get metrics like those from
> query-meters, that also does not come for free (and can
> generate a lot of output).  This might be acceptable
> depending on your workload, though, you would have to
> experiment to see how it does.  If you log via xdmp:log, you
> can be very specific about what you log, making it easy for
> some script to pick up bad stuff you might be looking for.
>
> -Danny
>
> -----Original Message-----
> From: general-bounces at developer.marklogic.com
> [mailto:general-bounces at developer.marklogic.com] On Behalf Of
> David Sewell
> Sent: Friday, April 17, 2009 7:04 AM
> To: General Mark Logic Developer Discussion
> Cc: Scott Cipriano
> Subject: Re: [MarkLogic Dev General] Logging queries
>
> On a server-wide basis you can configure "system log level"
> in the Admin setup under Groups - Default - Configure. And it
> appears that on a per-query basis you could use
> admin:group-set-file-log-level():
>
> http://xqzone.com/pubs/4.0/apidocs/AdminLibraryGroup.html#admi
> n:group-set-file-log-level
>
> but I don't have any experience actually using it.
>
> On Fri, 17 Apr 2009, Mike Sokolov wrote:
>
> > Does anybody know whether there is any way to configure
> Mark Logic to
> > save more information to its access logs?  In particular
> we'd like to
> > be able to see query metrics; at a bare minimum, we need to
> know how long a query took.
> > Ideally we would also like to be able to get detailed query logging
> > showing the queries that are being performed, and all their
> arguments.
> > Any additional metrics, like # fragments, memory required,
> etc would be gravy.
> >
> > We've built support for something this in our application,
> but need to
> > support multiple products administered by different vendors and it
> > would be great if we could control logging centrally using
> MarkLogic
> > admin tools.  We want this to be turned on and off without
> modifying
> > the queries themselves or the applications that invoke them.
> >
> > So - is this possible?  In 3.2? In 4.0?  Is it coming?
> >
> > any suggestions appreciated
> >
> > -Mike Sokolov
> > _______________________________________________
> > General mailing list
> > General at developer.marklogic.com
> > http://xqzone.com/mailman/listinfo/general
> >
>
> --
> David Sewell, Editorial and Technical Manager ROTUNDA, The
> University of Virginia Press PO Box 801079, Charlottesville,
> VA 22904-4318 USA
> Courier: 310 Old Ivy Way, Suite 302, Charlottesville VA 22903
> Email: dsewell at virginia.edu   Tel: +1 434 924 9973
> Web: http://rotunda.upress.virginia.edu/
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://xqzone.com/mailman/listinfo/general
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://xqzone.com/mailman/listinfo/general
>