Overpass API developpement

Text archives Help


Re: [overpass] [Overpass] Is there a default limit on concurrency?


Chronological Thread 
  • From: mmd <mmd.osm@gmail.com>
  • To: overpass@listes.openstreetmap.fr
  • Subject: Re: [overpass] [Overpass] Is there a default limit on concurrency?
  • Date: Mon, 22 Feb 2016 19:24:59 +0100

Hi,

Am 17.02.2016 um 05:18 schrieb arungowtham.a@zohocorp.com:
> Hi ,
> I have local setup of overpass-api, i face concurrency issues, for 100
> threads with each one query it takes around 18-19 seconds, though
> individually
> each query takes less than a second.
> Can this be altered, lots of requests reach my server at a time, and this is
> causing a lot of time delay :( .Also i came across this field
> max_num_processes
> (https://github.com/drolbr/Overpass-API/blob/a0b334e8fcbee40f05cf74c0a744584c18f4d711/src/overpass_api/core/settings.cc#L137
> )
> does this have anything to do with concurrency.
>


I think max_num_processes sets the maximum number of connections the
dispatcher unix domain socket would accept. You can try to bump up that
value.

In addition, I would suggest to reduce the sleep times in dispatcher.cc
around line 662 to have faster dispatcher responses. With 100 threads
you just spend too much time waiting otherwise.

if (command == 0)
{
++counter;
++idle_counter;
millisleep(idle_counter < 10 ? idle_counter*10 : 100); <<< replace
continue;
}

Right now, this is at least 10ms or a multiple of 10ms. At the expense
of having a higher CPU consumption of your dispatcher process, you could
try to decrease that value to just 1 ms:


millisleep(1);

The current polling approach unfortunately doesn't scale perfectly well
to a higher number of concurrent clients. Maybe we'll discuss something
like libevent and an epoll based approach as an alternative.


hth,
mmd



Archive powered by MHonArc 2.6.18.

Top of page