Overpass API developpement

[Donal Diamond <>]

areas still increasing:

overpass/db$ du -shc area*.bin
1.9G    area_blocks.bin
32M     areas.bin
228M    area_tags_global.bin
246M    area_tags_local.bin
2.4G    total

No active requests except for rules update

### osm-base

>osm3s/bin$  ./dispatcher --osm-base --status
Number of not yet opened connections: 0
Number of connected clients: 2
Rate limit: 0
Total available space: 12884901888
Total claimed space: 4294967296
Average claimed space: 4294967296
Total available time units: 262144
Total claimed time units: 0
Average claimed time units: 0
Counter of started requests: 150
Counter of finished requests: 149
202 29494 0 4294967296 0 1473111184
pending 30212

### areas
>osm3s/bin$  ./dispatcher --areas --status
Number of not yet opened connections: 0
Number of connected clients: 2
Rate limit: 0
Total available space: 4294967296
Total claimed space: 0
Average claimed space: 536870912
Total available time units: 262144
Total claimed time units: 0
Average claimed time units: 180
Counter of started requests: 12
Counter of finished requests: 12
pending 29494
pending 30244


>osm3s/bin$ ps uwax | grep 29494
overpass 29494 98.6  8.7 381360 356016 pts/6   R    22:33   4:04 ./osm3s_query --progress --rules

D

On 5 September 2016 at 22:26, Roland Olbricht <> wrote:

Hi Donal,

overpass/db$ ls -al area*
-rw-rw-r-- 1 overpass overpass 657981440 Sep  5 15:39 area_blocks.bin
-rw-rw-r-- 1 overpass overpass      3720 Sep  5 15:37 area_blocks.bin.idx
-rw-rw-r-- 1 overpass overpass      3720 Sep  5 15:37
area_blocks.bin.idx.shadow
-rw-rw-r-- 1 overpass overpass         8 Sep  5 15:37 area_blocks.bin.shadow
-rw-rw-r-- 1 overpass overpass  11010048 Sep  5 15:39 areas.bin
-rw-rw-r-- 1 overpass overpass        24 Sep  5 15:37 areas.bin.idx
-rw-rw-r-- 1 overpass overpass        24 Sep  5 15:37 areas.bin.idx.shadow
-rw-rw-r-- 1 overpass overpass         8 Sep  5 15:37 areas.bin.shadow
-rw-rw-r-- 1 overpass overpass         5 Sep  5 15:37 areas_shadow.lock
-rw-rw-r-- 1 overpass overpass  73465856 Sep  5 15:39 area_tags_global.bin
-rw-rw-r-- 1 overpass overpass       901 Sep  5 15:37
area_tags_global.bin.idx
-rw-rw-r-- 1 overpass overpass       901 Sep  5 15:37
area_tags_global.bin.idx.shadow
-rw-rw-r-- 1 overpass overpass         8 Sep  5 15:37
area_tags_global.bin.shadow
-rw-rw-r-- 1 overpass overpass  80805888 Sep  5 15:39 area_tags_local.bin
-rw-rw-r-- 1 overpass overpass      2777 Sep  5 15:37
area_tags_local.bin.idx
-rw-rw-r-- 1 overpass overpass      2777 Sep  5 15:37
area_tags_local.bin.idx.shadow
-rw-rw-r-- 1 overpass overpass         8 Sep  5 15:37
area_tags_local.bin.shadow
-rw-rw-r-- 1 overpass overpass        21 Sep  5 15:30 area_version
-rw-rw-r-- 1 overpass overpass        21 Sep  5 15:37 area_version.shadow

Thank you for the file sizes. The area_blocks.bin.idx has the same size as that of mmd. This means that there is the same amount of fresh data. What keeps piling up in your case is data that is intended to serve long running processes.

For that reason I would like to see the output of two status commands.
These commands tells us what the dispatcher thinks about the files.
I have appended the output of these commands on the overpass-api.de server to give you an idea how it can look.

The first one is

roland@overpass-api:$ ./dispatcher --osm-base --status
Number of not yet opened connections: 0
Number of connected clients: 8
Rate limit: 3
Total available space: 12884901888
Total claimed space: 4831838208
Average claimed space: 5280466291
Total available time units: 262144
Total claimed time units: 900
Average claimed time units: 1005
Counter of started requests: 9308011
Counter of finished requests: 9299214
202 18505 0 2147483648 0 1473024936
202 14689 1247845089 536870912 180 1473110370
202 14800 749452308 536870912 180 1473110378
202 14851 2953392293 536870912 180 1473110380
202 14883 3049346194 536870912 180 1473110383
202 14915 1191229123 536870912 180 1473110385
pending 14517
pending 14916
quota   3166169494 1473111826
quota   1494216340 1473111223
quota   542131069 1473110398
quota   1247845089 1473110391
quota   874857634 1473110385
quota   2604202762 1473110385
quota   3763371776 1473110394
quota   872864295 1473110385
quota   2535829806 1473110386
quota   2604202762 1473110385
quota   2522546160 1473110385
quota   2534191394 1473110385

The second one is

roland@overpass-api:$ ./dispatcher --areas --status
Number of not yet opened connections: 0
Number of connected clients: 4
Rate limit: 0
Total available space: 4294967296
Total claimed space: 1073741824
Average claimed space: 536870912
Total available time units: 262144
Total claimed time units: 190
Average claimed time units: 123
Counter of started requests: 418274
Counter of finished requests: 418050
202 20474 3047464522 536870912 180 1473110648
202 20602 2889518028 536870912 10 1473110654
pending 14982
pending 20629

Best regards,

Roland