Overpass API developpement

[Igor Brejc <>]

Hi Roland,

On Mon, Apr 9, 2018 at 5:04 PM, Roland Olbricht <> wrote:

The Overpass dispatcher uses two internal limits, "space" and "time" to determine whether it has load. Please run

dispatcher --osm-base --status

Thank you for the useful tip!

I will post a sample of queries I'm running on my private instance and what I discovered when trying to speed it up.

 

I expect that separating for CPU kernels will not work. The queries are distinct processes, and virtually any operating system is good in distributing processes on CPUs. Opposed to that I have run into two other systematic bottlenecks:
- super slow motion of swap: The OS will not warn if the RAM is short and instead start to swap if too many processes perform actual work in parallel. Swap is horribly slow in such a setting.
- disk dead lock: Similarly, the OS lets the disk jump so often between many concurrent processes that most time is spent seeking. I have not systematically ivestigated whether the problem persists with a SSD.

In my case, the bottleneck is mostly CPU. top shows the interpreter process running at 100% of CPU most of the time, while the disk is mostly not doing anything or not doing much (I have SSD, by the way). RAM is also not a problem, at least not until I reach a certain size of the query area.

Regarding the disk: is it possible to separate Overpass DB tables to different disks (nodes on one disk and the rest on the other, for example)? Would this speed up things in cases when the disk is the bottleneck?

Igor