Overpass API developpement

[Igor Brejc <>]

Hi,

I'm posting this in a new thread since I don't know whether it is related to the previous report (about problems with suspend), but it shows the problematic behavior of the fetch_osc_and_apply.sh script (not detecting that something is wrong and continuing applying diffs).

I've attached the apply_osc_to_db.log and transactions.log files. BTW: apply_osc_to_db.log uses local time (CET in my case) while transactions.log uses UTC, which is a bit confusing.

This is the situation:

• I brought back my Overpass server from suspend yesterday at 19:37. I did not really use it after that, just left it running until this morning.
  
• The diff service seems to have been running OK until this moment (from apply_osc_to_db.log): 
  

2017-09-07 22:23:54: update complete of 2600274

• The transaction log does not have any entries after that moment
• After that, all updates take just 1 second, until this happens, then it starts to behave differently:
  

2017-09-07 23:35:00: trying to apply 2612575

2017-09-07 23:35:10: trying to apply 2612575

2017-09-07 23:35:10: update complete of 2612575

2017-09-07 23:35:10: trying to apply 2612576

2017-09-07 23:35:21: trying to apply 2612576

2017-09-07 23:35:31: trying to apply 2612576

2017-09-07 23:35:41: trying to apply 2612576

2017-09-07 23:35:51: trying to apply 2612576

2017-09-07 23:36:01: trying to apply 2612576

2017-09-07 23:36:11: trying to apply 2612576

2017-09-07 23:36:12: update complete of 2612576

I've run some commands (including the ones Roland suggested, below are the results). I've left the machine running as it is, in case you need any more diagnostics.

Cheers,

Igor

igor@jazz:~/overpass$ sudo systemctl status overpass

● overpass.service - Overpass init agent

   Loaded: loaded (/lib/systemd/system/overpass.service; enabled; vendor preset: enabled)

   Active: active (running) since tor 2017-09-05 08:25:31 CEST; 2 days ago

 Main PID: 11318 (dispatcher)

   CGroup: /system.slice/overpass.service

           └─11318 /home/igor/overpass/osm-3s_v0.7.54/bin/dispatcher --osm-base --db-dir=/home/igor/overpass/db

sep 05 08:25:31 jazz systemd[1]: Starting Overpass init agent...

sep 05 08:25:31 jazz systemd[1]: Started Overpass init agent.

----------

igor@jazz:~/overpass$ sudo systemctl status overpass-diff

● overpass-diff.service - Overpass diff init agent

   Loaded: loaded (/lib/systemd/system/overpass-diff.service; enabled; vendor preset: enabled)

   Active: active (running) since sre 2017-09-06 15:41:57 CEST; 1 day 14h ago

 Main PID: 12908 (bash)

   CGroup: /system.slice/overpass-diff.service

           ├─ 3591 sleep 10

           └─12908 bash /home/igor/overpass/osm-3s_v0.7.54/bin/fetch_osc_and_apply.sh http://planet.osm.org/replication/minut

sep 08 06:00:09 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:01:10 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:02:11 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:03:13 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:04:04 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:05:05 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:06:07 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:08:09 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:09:11 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

sep 08 06:10:12 jazz fetch_osc_and_apply.sh[12908]: cat: write error: Broken pipe

-----------

igor@jazz:~/overpass$ ls -l db/osm*

srw-rw-rw- 1 igor igor  0 sep  5 08:25 db/osm3s_v0.7.54_osm_base

-rw-r--r-- 1 igor igor 23 sep  8 06:11 db/osm_base_version

-rw-r--r-- 1 igor igor 23 sep  8 06:11 db/osm_base_version_munin

-----------

igor@jazz:~/overpass$ ls -l /dev/shm/ | grep osm

- returns empty result

----------

igor@jazz:~/overpass$ ps -ef | grep dispatcher

igor      3689  3106  0 06:12 pts/3    00:00:00 grep --color=auto dispatcher

igor     11318     1  0 sep05 ?        00:00:50 /home/igor/overpass/osm-3s_v0.7.54/bin/dispatcher --osm-base --db-dir=/home/igor/overpass/db

Attachment: apply_osc_to_db.log
Description: Binary data

Attachment: transactions.log
Description: Binary data