Overpass API developpement

Text archives Help


Re: [overpass] Overpass API Area Creation in a Custom Server


Chronological Thread 
  • From: Sameera Bandara < >
  • To:
  • Subject: Re: [overpass] Overpass API Area Creation in a Custom Server
  • Date: Thu, 25 Jul 2019 09:41:26 +0200
  • Authentication-results: postout.lrz.de (amavisd-new); dkim=pass (2048-bit key) reason="pass (just generated, assumed good)" header.d=tum.de

Hey Yann,

I have a dedicated server only for hosting OSM with 2 GHz 8 cores and 16 GB RAM. But I didn't prioritize down the processes as still I am not using the server and thought the processes would finish fast. Below are the consumption stats. It consumes 100% CPU :O

  1  [                                       0.0%]   5  [                                       0.0%]

  2  [                                       0.0%]   6  [                                       0.0%]

  3  [||                                     1.3%]   7  [                                       0.0%]

  4  [||||||||||||||||||||||||||||||||||||||99.3%]   8  [                                       0.0%]

  Mem[||||||||||||||||||||||||||||||||4.99G/15.7G]   Tasks: 41, 103 thr; 2 running

  Swp[||                               33.2M/861M]   Load average: 1.00 1.00 1.00 

                                                     Uptime: 9 days, 22:19:58


  PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command

30013 sameera    20   0 4929M 4903M  2620 R 100. 30.6 10h24:01 ./osm3s_query --progress --rules

31945 sameera    20   0 29544  4012  3320 R  1.3  0.0  0:03.52 htop

 8737 sameera    20   0 19240  1680   680 S  0.0  0.0  2:19.47 /data/osm/osm-3s_v0.7.55/bin/dispatcher --ar

 8692 sameera    20   0 33476 14168     4 S  0.0  0.1  2:22.61 /data/osm/osm-3s_v0.7.55/bin/dispatcher --os

 1757 root       20   0  283M   804     0 S  0.0  0.0  0:12.68 /usr/lib/accountsservice/accounts-daemon

 8546 root       20   0 69960  4240  3036 S  0.0  0.0  0:11.26 /usr/sbin/apache2 -k start

 1779 root       20   0  283M   804     0 S  0.0  0.0  0:12.58 /usr/lib/accountsservice/accounts-daemon




On Thu, 25 Jul 2019 at 09:01, yann Guillerm < > wrote:
whhooo 24H ! 
i use rules_loop on 2 server where i have priorize the process down and it always make it (for world db) within a few hours.
Actually for France only db, but the server is use for a lot more processes using sometime 100% of the processor, and it take 1h20 to finish the loop. The server specification : intel Xeon E3 1230 v6, 16go Ram.

in the pre-requirement, the documentation said to have at least 4Go ram, a lot of hard disk and that processor is not revelant.

are you sure that processor and ram are free to use ? 

sincerly, 

Yann.




Le jeu. 25 juil. 2019 à 08:12, Sameera Bandara < > a écrit :
Hi Roland,

Thank you very much for the reply. I actually already sorted out the issue after checkin nohup.out. As you have mentioned the issue was something with absolute paths. I specified the relative path to the database_dir and it worked. Below is the error I got initially.

/data/osm/osm-3s_v0.7.55/bin/rules_loop.sh: line 40: /data/osm/osm-3s_v0.7.55/bin//data/osm/osm-3s_v0.7.55/bin/database_dir/rules_loop.log: No such file or directory

/data/osm/osm-3s_v0.7.55/bin/rules_loop.sh: line 41: /data/osm/osm-3s_v0.7.55/bin//data/osm/osm-3s_v0.7.55/bin/database_dir/rules/areas.osm3s: No such file or directory

/data/osm/osm-3s_v0.7.55/bin/rules_loop.sh: line 42: /data/osm/osm-3s_v0.7.55/bin//data/osm/osm-3s_v0.7.55/bin/database_dir/rules_loop.log: No such file or directory


Now the areas are populated and I can issue queries for areas and get results, however the log says the query was timed-out. And it ran exactly 24 hours.


After 23h59m3s: in "make-area", part 0, on line 257. Stack: 0 of 0 8442756 of 0

After 23h59m18s: in "recurse", part 0, on line 255. Stack: 0 of 0 8445531 of 0 0 of 0

After 23h59m33s: in "recurse", part 0, on line 255. Stack: 0 of 0 8447843 of 0 0 of 0

After 23h59m48s: in "make-area", part 0, on line 257. Stack: 0 of 0 8450335 of 0

After 24h0m1s: in "make-area", part 0, on line 257. Stack: 0 of 0 8452832 of 0


</osm>

runtime error: Query timed out in "make-area" at line 257 after 86401 seconds.

<?xml version="1.0" encoding="UTF-8"?>

<osm version="0.6" generator="Overpass API 0.7.55.7 8b86ff77">

<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>

<meta osm_base=""/>


kind regards,
Sameera

On Wed, 24 Jul 2019 at 21:55, Roland Olbricht < > wrote:
Hi Sameera,

could you please check what the file nohup.out contains. There should be error messages there.

As a blind guess, please try the following:

- Stop the rules_loop.sh process, e.g. by "kill 9990".

- Edit the file /data/osm/osm-3s_v0.7.55/bin/rules_loop.sh:
   replace there the line
     DB_DIR="`pwd`/$1"
   with
     DB_DIR="$1"

- Restart the process, i.e. run the command again please.

I coincidentally ran on the Russian instance into the problem that the script fails if its argument is an absolute path because of that line.

Kind regards,
Roland




Archive powered by MHonArc 2.6.19+.

Top of page