Overpass API developpement

Text archives Help


Re: [overpass] std::bad_alloc runtime error


Chronological Thread 
  • From: mmd <mmd.osm@gmail.com>
  • To: public-overpass-L2GxOb7Hh0gVQHylE0JAcSxnlxUeqk/I@plane.gmane.org
  • Subject: Re: [overpass] std::bad_alloc runtime error
  • Date: Mon, 25 Apr 2016 23:32:46 +0200



Hi Roland,

Am 24.04.2016 um 23:04 schrieb Roland Olbricht:
> Hi mmd,
>
>
> Thank you for forwarding the report.
>
> Investigating this just yielded the fix
> 4abe454a5460e77a4cb32b97ec9ab165f554a6bf resp.
> e1efbf1112c83fbaba1aea83471c000400891903.
>

I have another attic testcase, which currently fails with 2 GB limit.

Fortunately, I have already created a fix for it, which reduces memory
consumption down to 77 MB.

Here's the link to the pull request. It includes some background, which
hopefully makes some sense:

https://github.com/drolbr/Overpass-API/pull/174

And the respective attic query:

[date:"2015-09-25T00:00:00Z"]
[timeout:600];
area["name:de"="Deutschland"]
[boundary=administrative];
node(area)[railway=switch];
out count;


BTW: Regarding the 2 GB limit: There seems to be a tiny difference of
4096 bytes (or 1 page), which causes the query to get stuck with just 1
GB (+ 1 page).

* setrlimit will set the limit to 2147483648 bytes
* mmap will however try to allocate 2147487744 bytes -> fail!

I've never really seen a query move up to 2 GB. Could you investigate
this a bit further? Probably memory is requested this way by the
underlying C++ memory management functions.

I could reproduce this via:

strace -e signal,setrlimit,mmap ./osm3s_query

...


setrlimit(RLIMIT_CPU, {rlim_cur=62, rlim_max=62}) = 0
setrlimit(RLIMIT_AS, {rlim_cur=2097152*1024, rlim_max=2097152*1024}) = 0


mmap(NULL, 2147487744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = -1 ENOMEM (Cannot allocate memory)
mmap(NULL, 2147618816, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = -1 ENOMEM (Cannot allocate memory)
mmap(NULL, 134217728, PROT_NONE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7fb692b17000
mmap(NULL, 2147487744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = -1 ENOMEM (Cannot allocate memory)


]
}
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
= 0x7fb69c7e0000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
= 0x7fb69c7e0000
runtime error: Query run out of memory using about 2048 MB of RAM.
+++ exited with 0 +++





Best,
mmd





Archive powered by MHonArc 2.6.18.

Top of page