Overpass API developpement

[Igor Brejc <>]

Hi, 

Thank you very much for the answers. I will update the OSM wiki page with the information you provided, if you don't mind. My comments are below:

On Sat, Jul 29, 2017 at 12:13 AM, mmd <> wrote:

that depends a bit on your set up. Some update extracts may only provide
hourly or daily updates. For the full planet, I tried minutely, hourly
and daily diffs before w/o major issues.

One thing to keep in mind here is that in very rare cases (=server side
osmosis crashes), some changes contained in minutely diffs wouldn't show
up in hourly/daily diffs, leading to loss of data. That's quite nasty in
case of an attic database.

So basically minutely diffs are the way to go.

 

If you don't care about storing minutely diffs,
fetch_osc_and_apply may be a better option.

Since I only intend to have a single database, this is fine for my case.

 

I always use the "auto" option rather than providing an actual
replicate_id number in case of minutely updates. apply_osc_to_db.sh
assumes that you have fetch_osc.sh running in parallel, so it should
find some new diff files once in a while and start applying those.

I don't know how I missed the "auto" option in the script's source code. I'll add the description to the wiki page.

>
> By the way, my server has 16 GB of RAM, 512 GB SSD and an 8-core CPU.
> Running Ubuntu 16.04.2

At one point you mentioned that you're using a 32 bit desktop image
version. Any reason why you don't run a 64bit server image?

Frankly I didn't even know a 64-bit image existed :-). When you Google for "ubuntu 16.04.2", the first link is Ubuntu's official download page that only has 32-bit images listed for Intels (and I didn't notice the bittorrent link below). Come to think of it, it's was silly of me to assume only 32-bit version existed, perhaps I'll reinstall the machine with the 64-bit one, thanks for the tip.

As for the desktop image, I plan to use this machine for other Linux-related stuff, too, and having a GUI helps a newbie like me. 

 

Depending on
your load 16 GB may be a bit small for a full planet. The servers I know
have at least 32 GB.

It will be a private instance used only by me for occasional fetching of larger datasets or more complex queries (the ones that can't be done on the public Overpass instances due to the rate limits), so I think 16 GB should be fine. I've done some testing and it basically uses around 3-4 GB of RAM for the kinds of queries I need.

 

cheers,
mmd

Thanks again ;-)

Igor