Overpass API developpement

[mmd <>]

Hi,

Am 18.09.2016 um 14:41 schrieb Michael Larsen:
> Hi,
> 
> Last time I tried consuming augmented diffs on a minutely basis, I hit the 
> load-limitations which meant that I could not consume augmented diffs for 
> some 
> time afterwards, i.e. this will lead to black holes in your history.

the value returned by augmented_diff_status corresponds to current
database timestamp (as number of minutes since the license change). It
does not need to increase one-by-one(!), e.g. the database may process
several minutely diffs in one go, due to some backlog. If you always
download the number returned by augmented_diff_stauts, you will indeed
get some holes!

That situation can be avoided, if you keep your own internal sequence
id, and fetch augmented diffs up to and including to the value returned
by augmented_diff_status.

If augmented_diff_status does not return any value (due to overload),
just wait some time and try again. The same applies to downloading
augmented_diffs: you may get HTTP 429 or HTTP 504 in case of overload,
or if you exceed your quota (see /api/status for details). In that case,
don't increase your internal sequence id yet, but try downloading the
same augmented diff again.

> 
> Also, using timestamp start/end to fetch diffs for a given timestamp (like 
> avachi) is problematic with some changesets that stay open for > 1 hours 
> (this 
> happens quite ofter). The live service running on osm.expandable.dk use the 
> API as described previously to get the augmented diff for a changeset. If 
> there where a better way I'm all ears!
> 

I hope the situation will improve a bit once the database has moved to
version 0.7.53 and a different compression. If you're at SotM next week,
you could maybe ask Roland for a current status.

--