[MarkLogic Dev General] SVC-MAPBIG: Mapped file too large to map: 5825151760 bytes error
john at yuxipacific.com
Fri Nov 4 08:10:28 PDT 2011
Thank you, Mike. And yes, I knew the server was overloaded. I just want to
know if there is another suggestion except upgrading the server. It is
On Fri, Nov 4, 2011 at 11:00 PM, Michael Blakeley <mike at blakeley.com> wrote:
> To be blunt: that host is out of memory and over capacity.
> Stop ingestion right away, and revert the configuration changes you have
> made. Turning off merges is almost always a mistake: there ought to be a
> pop quiz built into the configuration screen. You can easily paint yourself
> into a corner. The other changes were almost certainly counterproductive,
> How can I tell that you are out of memory? The error message states that
> MarkLogic was unable to map a 5-GiB index. That is only one of many indexes
> that need to memory-map, so I would guess that the system is badly
> overloaded. The rest of the data confirms it: on a 16-GiB host your total
> index memory (aka forest memory) is about 12-GiB. You can't use all the
> host memory for forest indexes: you'll need about that much again for
> group-level caches, and about 25% of RAM for queries, merges, and the OS.
> For your data so far, you ought to have at least 32-GB RAM.
> Do you know where you are in your total load? Are you about 10% done, or
> closer to 75% done?
> With the system in its current state, it will probably use a lot of
> virtual memory for paging, and performance will probably be unacceptable.
> Am I right in thinking this is a VM? If so, you may be able to upgrade the
> VM memory without much trouble. Virtual or not, if a memory upgrade isn't
> practical then I would recommend clearing the database and starting over,
> and this time stop before you are out of memory.
> -- Mike
> On 4 Nov 2011, at 00:33 , John Zhong wrote:
> > Hi all,
> > I am running MarkLogic 4.1-6 version and encountered the below error
> > XDMP-FORESTERR: Error in merge of forest: SVC-MAPBIG: Mapped file too
> large to map: 5825151760 bytes: 'D:\Program
> > The MarkLogic instance is running on a EC2 server has 1 CUP with 2
> cores, 17.1GB RAM, 64 bit windows server 2008. We have created only one
> database with one forest, now the forest has 6 stands, total is 86GB, and
> there is still 300GB disk space available.
> > active fragments deleted fragments on disk size
> in memory size
> > stand 1: 2,922,206 3,320,621 36,246 MB
> 4,162 MB
> > stand 2: 1,111,868 1,795,830 39,987 MB
> 6,836 MB
> > stand 3: 63,545 107,616 3,588
> MB 606 MB
> > stand 4: 7,620 24,492
> 925 MB 163 MB
> > stand 5: 5,638 6,882
> 479 MB 79 MB
> > stand 6: 11,757 28,626
> 1,761 MB 319 MB
> > TOTAL: 4,122,634 5,284,067 82,986 MB
> 12,165 MB
> > We enable the default merge setting and tried to modify the merge max
> size to 2048 then merge manually, but still failed.
> > Now, we disable the merge.
> > Any sugestion will be very appreciate!!
> > Thank you in advance.
> > John
> > _______________________________________________
> > General mailing list
> > General at developer.marklogic.com
> > http://developer.marklogic.com/mailman/listinfo/general
> General mailing list
> General at developer.marklogic.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the General