Introduction
OSM data model and OSM history
https://www.openstreetmap.org/node/5207201516
(where 5207201516
is the node id) currently represents the Notting Hill subway station in London; similarly, the way accessible at
https://www.openstreetmap.org/way/32965412
, (where 32965412
is the way id) currently represents the Statue of Liberty in New York; and the relation accessible at
https://www.openstreetmap.org/relation/7515426
(where 7515426
is the relation id) currently represents the Louvre Museum in Paris. It is worth clarifying that the ids of nodes, ways and relations cannot be considered as permanent ids of real-world features [31]. In other words, a real-world feature which is currently represented by e.g. a node might in the future be represented by another node or by a way or a relation. As an example, the Louvre Museum was in the past represented by another relation with id 3262297
and might be represented by new relations in the future. This fundamental difference between the permanent ids of real-world features and the OSM ids of nodes, ways and relations should be properly considered when analyzing OSM history data.
https://www.openstreetmap.org/changeset/68221957
, where 68221957
is the changeset id. Since each changeset carries information about the edits made to the OSM database (i.e. which nodes, ways, relations and/or tags were added, modified and/or deleted, and how), the contributor who made the edits, and the timestamp of the edits, the availability of all the changesets ensures the availability of the whole history of each OSM node, way and relation, and - on a larger scale - of the full OSM database.Applications based on OSM history
Scientific applications
Software applications
Aim
|
Name
|
Reference
|
Type
|
History source
|
License
|
---|---|---|---|---|---|
Visualization | openstreetmap.org | Web app | OSM API | GPL 2 | |
Achavi | Web app | Overpass API | MIT | ||
OSM Deep History | Web app | OSM API | BSD | ||
OSM History Explorer | [68] | Web app | OSM Planet | Closed | |
OSM History Renderer* | [75] | Application | OSM planet | BSD 2 | |
OSM History Viewer (osmrmhv) | Web app | OSM API | AGPL 3 + MIT/PD | ||
OSM History Viewer (PeWu version) | Web app | OSM planet | Apache 2 | ||
OSMCha | Web app | OSM planet | GPL 3 | ||
OSMatrix* | Web app | OSM planet | MIT | ||
OSMvis | Web app | OSM planet | GPL-3 | ||
Show me the way | Web app | Overpass API | MIT | ||
Visualize Change | Web app | OSM planet | BSD 3 clause | ||
Who did it? | Web app | OSM Planet | WTFPL | ||
OSM Latest Changes | Web app | OSM API | Public domain | ||
Is OSM up-to-date? | Web app | OSM API | AGPL 3 | ||
Statistics | OSM wiki - Stats | [82] | Web page | OSM planet | Closed |
OSMstats | [83] | Web app | OSM planet | Closed | |
How did you contribute to OpenStreetMap? | [84] | Web app | OSM planet | Closed | |
osmstats.stevecoast.com | Web app | OSM planet | Source available | ||
OSM Tag History | Web app | OSM planet | ISC | ||
qxosm | Web app | OSM planet | Closed | ||
Brave Mappers | Web app | OSM planet | MIT | ||
iOSMAnalyzer* | Application | OSM planet | GPL 3 | ||
Visualization and statistics | OSM Analytics | Web app | OSM planet | BSD 3 clause | |
OSM Live Changes | Web app | OSM planet | Apache 2 | ||
Oshome Dashboard | [92] | Web app | OSM Planet | Closed | |
Analysis | EPIC-OSM* | Framework | OSM planet | MIT | |
Ohsome/OSHDB | Framework | OSM planet | GPL 3 | ||
OSM Data Classification | [99] | Tool | OSM planet | MIT | |
OSMesa | Framework | OSM planet | Apache 2 | ||
Conversion | OSM History Importer* | [75] | Tool | OSM planet | BSD 2 clause |
postgres_osm_pbf_fdw | [96] | Tool | OSM planet | PostgreSQL | |
OSM Parquetizer | [97] | Tool | OSM planet | Apache 2 | |
Osmium | Tool | OSM planet | GPL 3 |
Implementation: Is OSM up-to-date?
-
Simple setup: being designed for the general (non-technical) public, the application shall be primarily accessed as a web app on an existing server; to use it on a personal server, a Docker container [104] shall be provided in order to reduce the amount of time spent in setting up and maintaining the system; finally, users shall be able to use the application as a regular command line tool with minimal external dependencies.
-
Ease of use: the web application shall not require user registration and shall have an intuitive user interface, well-known icons and a color palette which facilitates qualitative quality assessments.
-
Low requirements: the application shall not require to download big databases nor shall require additional computing power than the one provided by a regular desktop or smartphone device.
-
Compatibility with existing systems: the application shall produce outputs compatible with existing Geographic Information Systems (GIS) software using well-known formats.
Technical overview
requirements.txt
file) or inside a Docker container (which can be built from Dockerfile
or downloaded from [105]). The application makes use of CircleCI [106] to build the Docker image and run a basic test. The backend and frontend of the application are separately presented in the following.
Backend
Fetching data from the OSM API
/api/0.6/map
(using the parameter bbox
) and N is the number of requests to be done to /api/0.6/{osm_type}/{feature_id}/history
, which allows to retrieve the history of a specific OSM node or way identified by its id. Since the official openstreetmap.org
server is used, the high number of requests to the OSM API (which can exceed one hundred even for a 1 km2 area) may require a lot of time to be processed and may be automatically blocked by the OSM server due to violation of the fair-usage policy [107]. This problem was solved using Dugong [108], a Python library which takes advantage of the HTTP 1.1 pipelining feature1 to open a single HTTP connection with the server and reuse it for many different requests without closing it every time, thus reducing overhead and the time for fetching the data. The maximum number of requests allowed over a single connection is 100. This value was found empirically, since the OSM web server was found unable to reply to any additional request after the 100th has been made.Converting data into a SQLite database
Producing the data
Accessing the data
Frontend
getData
5. The command line interface allows to use the software locally and save the GeoJSON output (i.e. OSM nodes and ways with their history-based quality parameters) for further GIS analysis. The way the web interface and the command line interface can be used is detailed in the next section.Application
Web interface
-
date of creation (first edit)×××××
-
date of last edit
-
number of versions (revisions)
-
number of different contributors who edited that node or way
-
frequency of update
-
timestamp of the last edit and OSM contributor responsible for it
-
timestamp of the first edit and OSM contributor responsible for it
-
number of versions
-
number of different contributors who edited that node or way
-
list of currently available tags (with clickable keys and values linked to the specific wiki pages of OSM keys and tags)
-
links to the OSM iD editor (in edit mode on that node or way), history and details of that node or way (both linked to the OSM website)
Command line interface
-r
internal option allowing to be identified by the OSM server. The GeoJSON output includes all the OSM nodes and ways available in the selected bounding box, with the following attributes: id and username of the OSM user who made the last edit on the node or way; id of the node or way; timestamps of the first and the last edit on the node or way; number of versions of the node or way; current tags of the node or way; total number of different OSM contributors who edited the node or way and their usernames; frequency of update of the node or way.result.geojson
, an extract of which (retrieved on March 10, 2019) is the following: