Overpass API developpement

[michael spreng <>]

Hello

So I found more time to look into the issue, why I can't import the cut
planet history into overpass. And after a weekend full of testing, I
think I found a minimal example of what is going wrong. I put the
example at https://osm.ch/share/way30913536.tar.gz for anyone to test.

Things I learned: The first call to overpass with a fresh db directory
has to be with --meta, --keep-attic (with and without
--flush-size=0)will fail without clear explanation (lots of node x not
found in way etc. when they clearly are present in the file). This might
be an improvement for overpass, returning an error for --keep-attic when
there is no db yet.

So I first import a single node to create the db: update_database
--db-dir=/db/ --meta < way30913536.osm then I import the history of this
way with update_database --db-dir=/db/ --keep-attic < way30913536.osc
which runs without any errors. But making any call to overpass with a
date will result in following errors:

call: [date:"2008-01-01T00:00:00Z"][out:xml];way(30913536);out;
http://overpass.urlaun.osm.ch/api/interpreter?data=%5Bdate%3A%222008-01-01T00%3A00%3A00Z%22%5D%5Bout%3Axml%5D%3B%0Away%2830913536%29%3B%0Aout%3B%0A

result:
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:27Z: Bad geometry for way 30913536 </remark>
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:26Z: Bad geometry for way 30913536 </remark>
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:25Z: Bad geometry for way 30913536 </remark>
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:24Z: Bad geometry for way 30913536 </remark>
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:23Z: Bad geometry for way 30913536 </remark>
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:22Z: Bad geometry for way 30913536 </remark>
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:21Z: Bad geometry for way 30913536 </remark>
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:20Z: Bad geometry for way 30913536 </remark>
<remark> runtime error: Way 30913536 cannot be expanded at timestamp
2009-02-09T12:16:12Z: Bad geometry for way 30913536 </remark>

So it seems there are edits in OSM history that are not chronological
(node gets deleted before way)

Interestingly, the .eu instance does not return this way, and I tried a
few possible dates during which the way should exist.

Another thing I observed: Most OSM tools are tolerant to replays, in
case something went wrong with a diff, one can go back a few diffs and
re apply them crhonologically, and everything is fine again. Overpass
sems to have problems, complaining about duplicate objects. This replay
tolerance seems to me important, as usually, when something goes wrong
with the diffs, the OWG recommends to replay from a ceartain point in
time to fix the data. What is behind overpasses different behaviour?
Missing feature or or some reason reason that makes replays undesirable?

Have a nice weekend
Michael