wiki:DataStorageEN

Version 13 (modified by Martin Kolman, 14 years ago) (diff)

--

Data storage

This article describes where modRana stores data.

Map data

Map data are by default stored in the modRana directory in cache/images in per layer folders. When using the sqlite database for storing map data, each layer has its own database files (lookup database and 1+ storage databases) stored in its own folder.

Maemo5@N900

The default path for the map data folder on Maemo is /home/user/MyDocs/.maps, bacause MyDocs resides on a 27GB FAT32 partition, which is the largest continuous partition in default Maemo 5 install on the N900.

NOTE: Because of the FAT32 file-system used in MyDocs, you should consider enabling sqlite tile storage (in options->map->tile storage). FAT32 uses very large per-file clusters, which together with map data that consists of thousands of individual tiles causes huge waste of space. Sqlite can store thousands of tiles in a single file and thus avoids this issue.

sqlite tile storage

available since V0.16-1

What is this good for ?
As mentioned earlier in this thread and also in a related Mappero thread, tile take up more space than expected because each tile, even if only 500B in size, takes a whole 64kB cluster. When using sqlite for tile storage, there are basically just 2 files per layer, not the usual tens of thousands of files and folders.

How to enable sqlite tile storage ?
Go to options->map and switch tile storage from files to sqlite.

Is it stable ?
It seems to work OK for both normal automatic tile download (including overlay) and batch download. But some errors can still can show up, so please report any unusual behavior when using this ! :)

What about the 4GB maximum file size limit on FAT32 ?
When a size of the storage database reaches 3.7 GB, a new one is added. The is no limit on the number of storage databases.

How does it work ?
The tiles are stored in a sqlite database as blobs. There are two types of database files, lookup.sqlite and store.sqlite. The lookup file stores an database that indicates in which store the requested tile is. The store file has the actual data. Multiple stores should be numbered in ascending order, starting from 0:

store.sqlite.0
store.sqlite.1
store.sqlite.2
etc.

The storage database looks like this:

table tiles (z integer, x integer, y integer, store_filename string, extension varchar(10), unix_epoch_timestamp integer, primary key (z, x, y, extension))


The store databases look like this:

table tiles (z integer, x integer, y integer, tile blob, extension varchar(10), unix_epoch_timestamp integer, primary key (z, x, y, extension))

The only difference in the structure is that the lookup databases only stores the name of the store for given coordinates and the store database stores the actual blob. Both also have a table called called version which has an integer column called v. There is a single 1 inserted, which indicates the current version of the table.

These database files are stored in the corresponding layer folders.

When looking for a tile in the database, modRana first asks the lookup database and when it gets an answer, it asks the store described in the store_filename for the given coordinates.

Could this be used by other navigation apps ?
Why do you think I just roughly described how it works ? :D

POI

Tracklogs

Configuration files

Persistent setting database

Main configuration file

Map layers configuration file