Overpass API developpement

Text archives Help


Re: [overpass] import of Belgium extract failed flush-size <> ram


Chronological Thread 
  • From: mmd < >
  • To:
  • Subject: Re: [overpass] import of Belgium extract failed flush-size <> ram
  • Date: Thu, 21 Sep 2017 19:14:27 +0200

Am 20.09.2017 um 23:45 schrieb marc marc:

> is it possible that flush-size uses the actual size of the malloc
> instead of the number of objects ?

That's not so easy due to the automatic memory management by the C++ STL
containers. Memory is usually allocated in much larger chunks rather
than each time you add an entry to a container for obvious performance
reasons.

Roland did in fact implement a memory consumption check to determine the
minimum required memory for all data in the inputsets. That's part of a
health check during query execution. Actual memory allocation towards
the OS may still be a bit larger depending on the STL memory management
strategy.

I guess it wasn't worth the effort to do so for the initial import as
well, as it never turned out to be an issue on 32GB machines. :)

--







Archive powered by MHonArc 2.6.19+.

Top of page