Overpass API developpement

Text archives Help


[overpass] overpass-api.de : slots available but answer too many requests


Chronological Thread 
  • From: < >
  • To:
  • Subject: [overpass] overpass-api.de : slots available but answer too many requests
  • Date: Fri, 13 Jul 2018 08:20:38 +0000

Hi!

I have a python script which make some requests to
https://overpass-api.de/api/interpreter
Before sending the query, the script check if there is an available slot, but
sometime, even though slots are available, I receive a http error 429 (too
many requests).

For exemple, a small part of my yesterday's log:
2 slots available now.
Currently running queries (pid, space limit, time limit, start time):
2018-07-12 22:55:38,695 :: INFO :: Starting new HTTPS connection (1):
overpass-api.de
2018-07-12 22:55:54,171 :: DEBUG :: "POST /api/interpreter HTTP/1.1" 429 488
2018-07-12 22:55:54,173 :: WARNING :: HTTP error 429, retrying...

The script pause for 20 secondes then retry

How is it possible ? And how could I correct this problem ?

Another problem: With this retry, I got a HTTP 200 answer:
2018-07-12 22:56:14,188 :: DEBUG :: Starting query for Communauté de communes
du Val d'Essonne
2018-07-12 22:56:14,190 :: INFO :: Starting new HTTP connection (1): overpass-
api.de
2018-07-12 22:56:14,275 :: DEBUG :: "GET /api/status HTTP/1.1" 200 150
2018-07-12 22:56:14,276 :: DEBUG :: Connected as: 1324341004

but the osm file says :
runtime error: open64: 0 Success /osm3s_v0.7.55_osm_base
Dispatcher_Client::request_read_and_idx::rate_limited. Please check /api/
status for the quota of your IP address.

So sometime I get a HTTP 429 error, and sometime a error inside the osm file.

Here is the function:

def send_query(osm_id, template):
"""
Send the main query to the overpass api
@osm_id : the relation id used as the area for the query
@template : the query with some placeholders
@return : the requests answer (status code, data, ...)
"""
query = template.replace("{_START_DATE_}", start_date.strftime("%Y-%m-
%dT00:00:00Z"))
query = query.replace("{_END_DATE_}", end_date.strftime("%Y-%m-
%dT00:00:00Z"))
query = query.replace("{_EPCI_ID_}", str(osm_id))

# check if a slot is available then send the request
try:
check_status = requests.get("http://overpass-api.de/api/status";).text
while "slots available now" not in check_status:
logger.warning("All slots already in use")
time.sleep(15)
check_status =
requests.get("http://overpass-api.de/api/status";).text

logger.debug(check_status)
#logger.debug("requests : {0}".format(query.strip("\n")))
return requests.post(overpass_url, data=query, timeout = 1200)

except Exception as e:
logger.error(e)


Stéphane


  • [overpass] overpass-api.de : slots available but answer too many requests, stephane.peneau, 07/13/2018

Archive powered by MHonArc 2.6.19+.

Top of page