Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Import OpenStreetMap-Data (OSM-File) to a MySQL-Database / SQL-File

Intro

I downloaded and extracted the OpenStreetMap Source File (See here). I have a 400GB XML-sourcefile now, and i would like to (1) insert the data in a MySQL Table OR (2) export this to an SQL file, doesn't matter.

What I already did / tried

  1. I have the XML File^^ and the MySQL Server (5.6.14)
  2. I found the tool Osmosis, to convert OSM data and I installed it
  3. I read this post: Read cities and streets into a database without api
  4. I also read this post: Is this config is capable to import complete osm planet file? (It is.)
  5. I tried to insert the data with osmosis, but it throws an error after a few seconds, or is doing nothing for hours, without inserting one single row. I also don't understand how this should work, because I don't have the OSM table structure in MySQL.

The Question

Is it generally possible to import the whole File in a Database / SQL-File, and if yes, how? Where do I find the table/database-structure? Has anyone already tried?

Thank you very much!

like image 248
user2896540 Avatar asked Nov 21 '13 15:11

user2896540


People also ask

What database does OpenStreetMap use?

One reason why OpenStreetMap uses Postgres, that with PostGIS (a GIS extension for Postgres) it offers probably one of the best GIS support in the relational world (=SQL databases). About the only part of Postgres that is object orientated is the TABLE INHERITS in CREATE TABLE .

How do I import data from OSM to PostGIS?

You will need to add . osm to the map file in order to import it to PostgreSQL. To import OSM data, we'll use the command line tool osm2pgsql. osm2pgsql is a command line tool that transforms OSM data into a format that can be saved into PostgreSQL.

In what format is OSM data?

The most important formats are: PBF Format – highly compressed, optimized binary format similar to the API, recommended for data processing. OSM XML – XML format provided by the API. Please use PBF if you can.


2 Answers

I believe the answer to this question has now changed. As of MySQL 5.6, there is now a fully-featured OpenGIS geospatial feature set in MySQL (it's not even an extension; it's built in). Thus, from a purely theoretical standpoint, there's no reason you can't import the OSM data into MySQL using the standard, supported database schema that you'd use for PostgreSQL (with syntax changed as necessary, of course). In fact, this presentation (which still refers to an older, incomplete version of MySQL's OpenGIS support) suggests that, with proper indexing, some MySQL geospatial actions may actually perform better than PostgreSQL PostGIS, though I'm sure that's up for debate. So the remaining problem is import tooling...

Currently, there's no official osm2mysql equivalent to osm2pgsql. There are a couple of projects out there named osm2mysql, but I've seen people have only moderate—if any—success using those projects. However, GDAL's open source ogr2ogr appears to be a very viable solution.

I downloaded, compiled, and installed ogr2ogr and was able to import a shapefile into a geospatial schema in MySQL, and successfully (and apparently accurately) query against the data using things like ST_Contains(...). I have not yet tried OSM (that's my next step), but ogr2ogr claims to have full support for OSM (XML and binary), so given my success with shapefiles, I imagine I'll have similar success with OSM.

I did have some initial problems, one of which was likely related to my not using make clean between configure attempts, and the others which were related to the large queries involved (some inserts > 1MB) and some MySQL settings that needed adjusting to compensate. You can view the details of my solutions in this mailing list thread.

like image 172
Nick Williams Avatar answered Sep 22 '22 19:09

Nick Williams


Well, indeed it is possible: http://wiki.openstreetmap.org/wiki/MySQL
But be warned, this is a not well supported solution:

  • has no geospatial extension
  • custom DB schema
  • not supported by major tools from OSM toolchain

I highly recommend to switch to postgreSQL with PostGIS extension, that is a semi-standard at the GIS world.

In the end it depends on your usecase (prototyping ... highly available DB server with live OSM data updates). You will find further informations at http://switch2osm.org

like image 38
MaM Avatar answered Sep 24 '22 19:09

MaM