Overpass API developpement

Text archives Help


[overpass] Dispatcher... and scalability


Chronological Thread 
  • From: mmd <mmd.osm@gmail.com>
  • To: overpass@osm101.openstreetmap.fr
  • Subject: [overpass] Dispatcher... and scalability
  • Date: Sat, 3 Dec 2016 11:14:30 +0100

Hi,

I was always a bit puzzled that I could never achieve more than about 30
requests/second even on quite beefy hardware, regardless how small the
query was.

Investigating a bit further I found an issue related to a design
decision, which affects scalability of the dispatcher.

To validate assumptions I did some prototyping and switched to event
based processing. In addition, adding a bit of caching for index files
and replacing CGI now produces a nice throughput of up to 500-550
requests/s end-to-end for amenity nodes in a small bbox.

Mobile scenarios in particular should benefit from the increased number
of concurrent users. And there's quite a number of them out there,
requesting e.g. maxspeed tags at the current location.

Due to the changes some tiny queries now also execute in 15-80ms:

- http://overpass-turbo.eu/s/krk
-
http://umap.openstreetmap.fr/map/postbox-overpass-api-dev-server-demo_112819#14/43.5503/7.0156

(runs on a busy hard disk, re-run query for SSD like experience)

As usual, more details can be found on Github.

Cheers,
mmd



  • [overpass] Dispatcher... and scalability, mmd, 12/03/2016

Archive powered by MHonArc 2.6.18.

Top of page