Overpass API developpement

[mmd <>]

A few additional remarks:

Am 03.03.2015 um 18:25 schrieb mmd:
> 
> Other queries [2] involving the 'foreach' statement seemed to be even
> slower with backend compression.

<speculation>
I have the impression that OS buffer cache keeps uncompressed blocks in
case of file system compression. Compression backend on the other hand
seems decompress the same blocks again, if they're needed (which is not
unlikely in a 'foreach' scenario).

This could explain at least some of the differences. Further
measurements are needed in any case...
</speculation>

> Based on the good experience I have with LZO compression on btrfs, I
> would also highly welcome seeing LZO or even LZ4 as additional
> compression algorithms in the compression backend (optional: as
> configurable option on a per-file type basis). As you pointed out
> before, the design is already open for different algorithms.


I've created a new branch with LZ4 compression enabled by default [2],
to facilitate further performance testing. Compression will not be as
good as with ZLIB, but CPU consumption should be quite a bit lower.

To compile this branch you need to manually install LZ4 r127 [3], as
r127 is not included in Ubuntu 14.04.

The lz4 branch should be benchmarked against branch [1]. Reason being
that both branches include a few unrelated performance tweaks, which
Roland hasn't merged into the official version yet.

(Both branches are not meant to be used in production.)

[1] https://github.com/mmd-osm/Overpass-API/tree/test753_compression
[2] https://github.com/mmd-osm/Overpass-API/tree/test753_lz4
[3] https://github.com/Cyan4973/lz4/releases/tag/r127