Overpass API developpement

[marc marc <>]

Le 20. 09. 17 à 22:28, mmd a écrit :
>> I tried with --flush-size=6 and the import succeeded
>>
>> what is the exact meaning of --flush-size = 1 ? a flush after each
>> modified node/way/relation ? after each tag ? each changeset ?
>>
>> Is it a "formula" between flush-size and maximum consumed ram ?
>> I imagine something like x Mo + x Mo x flush-size = max malloc ram
> 
> update_database.cc has the following definitions:
> 
> Default value: flush_limit = 16*1024*1024
> 
> When setting --flush-size=6 as a parameter, flush_limit will be set to
> 6*1024*1024.
> 
> Very roughly speaking, when importing nodes/ways/relations, the number
> of objects will be counted, and once the flush-size threshold is
> reached, objects will be flushed to disk and the counter reset to zero.
> As objects have varying sizes, this is a rough approximation of the
> actual memory requirements.
> 
> Setting it close to the maximum value your memory permits reduces 
> the overall runtime of the initial load.

yes of course, it is what I 'm trying todo : found the hightest value to 
speedup the initiale load AND having a value that avoid a out of memory.

But due to that flush-size is based on the number of objects and not 
their size, it is difficult to prevent an out of memory when
a change occurs on a lot of "bigger than average" objects.
is it possible that flush-size uses the actual size of the malloc 
instead of the number of objects ?
or is this option too little used to make it important?

Regards,
Marc