[MarkLogic Dev General] What does the ML Journal contain ?

[Michael Blakeley]

Danny, http://en.wikipedia.org/wiki/Transaction_log might be useful background reading. A similar technique is used by every ACID-compliant database that I am familiar with.

-- Mike

On 20 Apr 2012, at 06:37 , Ron Hitchens wrote:

> 
>   The journal certainly does contain the data.  It
> records the deltas to the database.  That means that
> when an update request runs, all the changes it made
> to the state of the database are recorded in the journal.
> Those changes can be applied again from the journal,
> without running the request again.
> 
>   Updates can be additions, replacements or deletions
> of documents (fragments actually, which are not always
> 1-1 with documents).
> 
>   It's the journal that protects you from outages.
> Writing journal frames is as efficient as it possibly
> can be.  There is some overhead to write to the journal,
> but that's the cost of insuring that when a transaction
> commit completes, it is guaranteed to survive a system
> crash thereafter.
> 
>   Journal creation efficiency is not something you need
> to be concerned about.  It's rarely significant as overhead
> in most cases.
> 
>   For those cases where you might be making large updates
> on a regular basis, it's possible to tell MarkLogic to
> pre-allocate journal space so that it's even more efficient.
> 
> On Apr 20, 2012, at 2:23 PM, Danny Sinang wrote:
> 
>> Hi Ron,
>> 
>> Thanks for the info.
>> 
>> So in effect, the journal entries somehow store the actions to be taken, not the new data to be applied ?
>> 
>> I'm asking this to somehow assure myself that journal entry creation would be fast, even for large transactions. The faster the journal creation, the less chances of it failing due to an outage.
>> 
>> Regards,
>> Danny
>> 
>> On Fri, Apr 20, 2012 at 8:58 AM, Ron Hitchens <ron at ronsoft.com> wrote:
>> 
>>  No user serviceable parts inside.
>> 
>>  The journal contains information that describes
>> sets of updates done by transactions.  In case of a
>> crash, the journal can be replayed to re-apply any
>> updates that may not yet have been written to the
>> on-disk forests.
>> 
>>  A transaction does not complete until the journal
>> entry is written to disk, but the corresponding updates
>> to the forest(s) are written later.  The database view
>> your queries see are the sum of the on-disk in in-memory
>> stands that comprise forests which make up a database.
>> 
>>  Journal entries are in a compact binary format that
>> is only used internally by the server to assure data
>> consistency.
>> 
>> On Apr 20, 2012, at 1:43 PM, Danny Sinang wrote:
>> 
>>> Hi,
>>> 
>>> I'm told that for document inserts and updates, an ML journal entry gets written.
>>> 
>>> Does anyone know what gets written to the journal ? Is it just the xquery getting executed ? Or is it the new data to be appended to the database ?
>>> 
>>> Regards,
>>> Danny
>>> 
>>> _______________________________________________
>>> General mailing list
>>> General at developer.marklogic.com
>>> http://developer.marklogic.com/mailman/listinfo/general
>> 
>> ---
>> Ron Hitchens {mailto:ron at ronsoft.com}   Ronsoft Technologies
>>    +44 7879 358 212 (voice)          http://www.ronsoft.com
>>    +1 707 924 3878 (fax)              Bit Twiddling At Its Finest
>> "No amount of belief establishes any fact." -Unknown
>> 
>> 
>> 
>> 
>> _______________________________________________
>> General mailing list
>> General at developer.marklogic.com
>> http://developer.marklogic.com/mailman/listinfo/general
>> 
>> _______________________________________________
>> General mailing list
>> General at developer.marklogic.com
>> http://developer.marklogic.com/mailman/listinfo/general
> 
> ---
> Ron Hitchens {mailto:ron at ronsoft.com}   Ronsoft Technologies
>     +44 7879 358 212 (voice)          http://www.ronsoft.com
>     +1 707 924 3878 (fax)              Bit Twiddling At Its Finest
> "No amount of belief establishes any fact." -Unknown
> 
> 
> 
> 
> _______________________________________________
> General mailing list
> General at developer.marklogic.com
> http://developer.marklogic.com/mailman/listinfo/general
>