Overpass API developpement

[<>]

Hello,

I have an overpass (version 0.7.56.7) server running on Ubuntu 18.04-LTS.
After installation I run:

$ ./download_clone.sh --db-dir=/data --
source=http://dev.overpass-api.de/api_drolbr/ --meta=attic
$ nohup /home/overpass/osm3s/bin/dispatcher --osm-base --attic --rate-limit=5
--space=18000000000 --db-dir=/data/ >> /home/overpass/dispatcher.out &
$ nohup /home/overpass/osm3s/bin/fetch_osc_and_apply.sh
https://planet.osm.org/replication/minute --meta=attic >> /home/overpass/
fetch_osc_and_apply.out &

it runs fine for some time but within a week or so will eventually halt
processing diffs. The dispatcher will give an error:

File_Error 17 File exists /data/osm_base_shadow.lock write_start:1

The fetch_osc_and_apply logs show:

$ tail fetch_osc_and_apply.out
2020-09-17 18:59:07
URL:https://planet.osm.org/replication/minute/004/199/858.osc.gz [97717/97717]
-> "/tmp/osm-3s_update_ZIPCfV/004199858.osc.gz" [1]
2020-09-17 18:59:07
URL:https://planet.osm.org/replication/minute/004/199/859.state.txt [158/158]
-> "/tmp/osm-3s_update_ZIPCfV/004199859.state.txt" [1]
2020-09-17 18:59:09
URL:https://planet.osm.org/replication/minute/004/199/859.osc.gz
[132706/132706] -> "/tmp/osm-3s_update_ZIPCfV/004199859.osc.gz" [1]
https://planet.osm.org/replication/minute/004/199/860.state.txt:
2020-09-17 18:59:09 ERROR 404: Not Found.
https://planet.osm.org/replication/minute/004/199/860.osc.gz:
2020-09-17 18:59:10 ERROR 404: Not Found.
Reading XML file ... finished reading nodes. Flushing to database ...... done.
Reading XML file ... finished reading ways. Flushing to
database ......terminate called after throwing an instance of
'std::logic_error'
  what():  Bad geometry for way 637700045: nds <truncated>
/home/overpass/osm3s/bin/fetch_osc_and_apply.sh: line 101:  8272 Aborted
(core dumped) ./update_from_dir --osc-dir=$1 --version=$DATA_VERSION $META --
flush-size=0

$ tail /data/fetch_osc_and_apply.log
tail /data/fetch_osc_and_apply.log
2020-09-17 17:59:46: updating to 4199799
2020-09-17 18:16:45: update complete 4199799
2020-09-17 18:16:45: updating from 4199799
2020-09-17 18:17:15: updating to 4199817
2020-09-17 18:35:47: update complete 4199817
2020-09-17 18:35:47: updating from 4199817
2020-09-17 18:36:19: updating to 4199836
2020-09-17 18:58:30: update complete 4199836
2020-09-17 18:58:30: updating from 4199836
2020-09-17 18:59:10: updating to 4199859


Which I have not figured out how to fix except to clone the database from
scratch and start over, which takes a good deal of time. How to I prevent this
failure and are there ways to fix it that are faster than a full re-clone?

Best,
Dakota
HOTOSM Devops Manager