OSMAlchemy-0.1.3/0000755000175000017500000000000013042151631013034 5ustar niknik00000000000000OSMAlchemy-0.1.3/OSMAlchemy.egg-info/0000755000175000017500000000000013042151631016467 5ustar niknik00000000000000OSMAlchemy-0.1.3/OSMAlchemy.egg-info/PKG-INFO0000644000175000017500000002150213042151631017564 0ustar niknik00000000000000Metadata-Version: 1.1 Name: OSMAlchemy Version: 0.1.3 Summary: OpenStreetMap to SQLAlchemy bridge Home-page: https://github.com/Veripeditus/OSMAlchemy Author: Dominik George, Eike Tim Jesinghaus Author-email: osmalchemy@veripeditus.org License: UNKNOWN Description: OSMAlchemy ========== OSMAlchemy is a bridge between SQLAlchemy and the OpenStreetMap API. Goals ----- OSMAlchemy's goal is to provide completely transparent integration of the real-world OpenStreetMap data within projects using SQLAlchemy. It provides two things: 1. Model declaratives resembling the structure of the main OpenStreetMap database, with some limitations, usable wherever SQLAlchemy is used, and 2. Transparent proxying and data-fetching from OpenStreetMap data. The idea is that the model can be queried using SQLAlchemy, and OSMAlchemy will either satisfy the query from the database directly or fetch data from OpenStreetMap. That way, projects already using SQLAlchemy do not need another database framework to use OpenStreetMap data, and the necessity to keep a local copy of planet.osm is relaxed. If, for example, a node with a certain id is queried, OSMAlchemy will… - …try to get the node from the database/ORM directly, then… - …if it is available, check its caching age, and… - …if it is too old, refresh it from OSM, or… - …else, fetch it from OSM, and… - …finally create a real, ORM-mapped database object. That's the rough idea, and it counts for all kinds of OSM elements and queries. OSMAlchemy uses Overpass to satisfy complex queries. Non-goals ~~~~~~~~~ OSMAlchemy does not aim to replace large-scale OSM data frameworks like PostGIS, Osmosis or whatever. In fact, in terms of performance and otherwise, it cannot keep up with them. If you are running a huge project that handles massive amounts of map data, has millions of requests or users, then OSMAlchemy is not for you (YMMV). OSMAlchemy fills a niche for projects that have limited resources and cannot handle a full copy of planet.osm and an own API backend and expect to handle limited amounts of map data. It might, however, be cool to use OSMAlchemy as ORM proxy with an own API backend. Who knows? It might, as well, turn out that OSMAlchemy is an incredibly silly idea under all circumstances. Usage ----- Here are a few tiny examples of how to basically use OSMAlchemy: Installation ~~~~~~~~~~~~ OSMAlchemy can be installed just like any other standard Python package by one of… .. code-block:: console # pip3 install OSMAlchemy # python3 setup.py install …or what ever kind of distribution and install system you prefer. Using plain SQLAlchemy ~~~~~~~~~~~~~~~~~~~~~~ Make sure to get at least an engine from SQLAlchemy. Even better, get a declarative base and a scoped session: .. code-block:: python from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker, scoped_session engine = create_engine("sqlite:////tmp/foo.db") base = declarative_base(bind=engine) session = scoped_session(sessionmaker(bind=engine)) You can then initialise OSMAlchemy like so: .. code-block:: python osmalchemy = OSMAlchemy((engine, base, session), overpass=True) And probably install the databases: .. code-block:: python base.metadata.create_all() Using Flask-SQLAlchemy and Flask-Restless ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Imagine you have an SQLAlchemy object from Flask-SQLAlchemy bound to your Flask application. called db, and a Flask-Restless API manager as manager: .. code-block:: python from osmalchemy import OSMAlchemy osm = OSMAlchemy(db, overpass=True) db.create_all() osm.create_api(manager) You should now magically be able to query OSM via the REST API. Keep in mind that, with no filter provided, OSMAlchemy refuses to do automatic updates from Overpass. However, providing a query in the default JSON query way in Flask-Restless will give you live data and cache it in the database. Limitations ~~~~~~~~~~~ Only some basic SQL queries are supported by the online update code. This is because compiling SQLAlchemy's queries to OverpassQL is very complex. If you are very good at algorithms and building compilers, feel free to help us out! The following kinds of queries are fully supported: .. code-block:: python # A node with a specific id session.query(osmalchemy.node).filter_by(id=12345).one() # All nodes within a bounding box session.query(osmalchemy.node).filter( and_(latitude>51.0, latitude<51.1, longitude>7.0, longitude<7.1) ).all() # All nodes having a specific tag session.query(osmalchemy.node).filter( osmalchemy.node.tags.any(key="name", value="Schwarzrheindorf Kirche") ).all() You can go mad combining the two with and\_() and or\_(). You can also query for tags of ways and relations and for ways and relations by id. Not supported (yet) are queries for ways or relations by coordinates. You also cannot query for nodes related to a way or anything related to a relation - having a way or a relation, accessing it will, however, magically pull and update the nodes and members and add them to the database: .. code-block:: python # Get all nodes that are members of a (unique) named way session.query(osmalchemy.way).filter( osmalchemy.way.tags.any(key="name", value="My Unique Way") ).one().nodes This should, in reality, cover most use cases. If you encounter a use case that is not supported, please open an issue asking whether it can be supported (if you have an idea how it can be, please add it or even implement it and open a pull request). Projects using OSMAlchemy ~~~~~~~~~~~~~~~~~~~~~~~~~ OSMAlchemy was designed for use in the Veripeditus Augmented Reality framework. Development and standards ------------------------- Albeit taking the above into account, OSMAlchemy is developed with quality and good support in mind. That means code shall be well-tested and well-documented. OSMAlchemy is tested against the following SQLAlchemy backends: - SQLite - PostgreSQL - MySQL However, we recommend PostgreSQL. MySQL acts strangely with some data and is incredibly slow, and SQLite just doesn't scale too well (however, it is incredibly fast, in comparison). Authors and credits ------------------- :Authors: Dominik George, Eike Tim Jesinghaus :Credits: Special thanks to Mike Bayer from SQLAlchemy for his help with some SQLAlchemy bugs and pitfalls, and also some heads-up. :Contact: E-mail to osmalchemy@veripeditus.org License ------- OSMAlchemy is licensed under the MIT license. Alternatively, you are free to use OSMAlchemy under Simplified BSD, The MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python itself. Keywords: osm,openstreetmap,proxy,caching,orm Platform: UNKNOWN Classifier: Development Status :: 3 - Alpha Classifier: Environment :: Plugins Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python :: 3 :: Only Classifier: Topic :: Database Classifier: Topic :: Scientific/Engineering :: GIS Classifier: Topic :: Software Development :: Libraries :: Python Modules OSMAlchemy-0.1.3/OSMAlchemy.egg-info/SOURCES.txt0000644000175000017500000000110613042151631020351 0ustar niknik00000000000000AUTHORS CHANGELOG LICENSE MANIFEST.in ODBL README.rst setup.py OSMAlchemy.egg-info/PKG-INFO OSMAlchemy.egg-info/SOURCES.txt OSMAlchemy.egg-info/dependency_links.txt OSMAlchemy.egg-info/requires.txt OSMAlchemy.egg-info/top_level.txt OSMAlchemy.egg-info/zip-safe osmalchemy/__init__.py osmalchemy/model.py osmalchemy/osmalchemy.py osmalchemy/triggers.py osmalchemy/util/__init__.py osmalchemy/util/db.py osmalchemy/util/online.py osmalchemy/util/patch.py test/__init__.py test/test_api.py test/test_model.py test/test_util_db.py test/test_util_online.py test/data/schwarzrheindorf.osmOSMAlchemy-0.1.3/OSMAlchemy.egg-info/dependency_links.txt0000644000175000017500000000000113042151631022535 0ustar niknik00000000000000 OSMAlchemy-0.1.3/OSMAlchemy.egg-info/requires.txt0000644000175000017500000000012113042151631021061 0ustar niknik00000000000000SQLAlchemy>=1.0.0 python-dateutil overpass [Flask] Flask>=0.10 Flask-SQLAlchemy OSMAlchemy-0.1.3/OSMAlchemy.egg-info/top_level.txt0000644000175000017500000000001313042151631021213 0ustar niknik00000000000000osmalchemy OSMAlchemy-0.1.3/OSMAlchemy.egg-info/zip-safe0000644000175000017500000000000113042143656020127 0ustar niknik00000000000000 OSMAlchemy-0.1.3/osmalchemy/0000755000175000017500000000000013042151631015175 5ustar niknik00000000000000OSMAlchemy-0.1.3/osmalchemy/util/0000755000175000017500000000000013042151631016152 5ustar niknik00000000000000OSMAlchemy-0.1.3/osmalchemy/util/__init__.py0000644000175000017500000000000012752414163020262 0ustar niknik00000000000000OSMAlchemy-0.1.3/osmalchemy/util/db.py0000644000175000017500000002044013000162473017111 0ustar niknik00000000000000# ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Utility code for the OSMAlchemy database OSMAlchemy. """ import operator import xml.dom.minidom as minidom import dateutil.parser from sqlalchemy.sql.elements import BinaryExpression, BooleanClauseList, BindParameter from sqlalchemy.sql.annotation import AnnotatedColumn def _import_osm_dom(oa, dom, session=None): """ Import a DOM tree from OSM XML into an OSMAlchemy model. Not called directly; used by _import_osm_xml and _import_osm_file. """ # Get session if session is None: session = oa.session oa.logger.debug("Started import of data from DOM.") def _dom_attrs_to_any(xml_element, osm_element): if "version" in xml_element.attributes.keys(): osm_element.version = int(xml_element.attributes["version"].value) if "changeset" in xml_element.attributes.keys(): osm_element.changeset = int(xml_element.attributes["changeset"].value) if "user" in xml_element.attributes.keys(): osm_element.user = xml_element.attributes["user"].value if "uid" in xml_element.attributes.keys(): osm_element.uid = int(xml_element.attributes["uid"].value) if "visible" in xml_element.attributes.keys(): osm_element.visible = True if xml_element.attributes["visible"].value == "true" else False if "timestamp" in xml_element.attributes.keys(): osm_element.timestamp = dateutil.parser.parse(xml_element.attributes["timestamp"].value) def _dom_tags_to_any(xml_element, osm_element): # Remove all tags previously associated with element for key in list(osm_element.tags.keys()): del osm_element.tags[key] # Iterate over all nodes in the DOM element for xml_tag in xml_element.getElementsByTagName("tag"): # Append data to tags osm_element.tags[xml_tag.attributes["k"].value] = xml_tag.attributes["v"].value def _dom_to_node(xml_element): with oa.session.no_autoflush: # Get mandatory node id id_ = int(xml_element.attributes["id"].value) oa.logger.debug("Importing OSM node with id %i." % id_) # Find object in database and create if non-existent node = oa.session.query(oa.node).filter_by(id=id_).scalar() if node is None: node = oa.node(id=id_) # Store mandatory latitude and longitude node.latitude = xml_element.attributes["lat"].value node.longitude = xml_element.attributes["lon"].value # Store other attributes and tags _dom_attrs_to_any(xml_element, node) _dom_tags_to_any(xml_element, node) # Add to session oa.session.add(node) oa.session.commit() def _dom_to_way(xml_element): with oa.session.no_autoflush: # Get mandatory way id id_ = int(xml_element.attributes["id"].value) oa.logger.debug("Importing OSM way with id %i." % id_) # Find object in database and create if non-existent way = oa.session.query(oa.way).filter_by(id=id_).scalar() if way is None: way = oa.way(id=id_) # Find all related nodes for node in xml_element.getElementsByTagName("nd"): # Get node id and find object ref = int(node.attributes["ref"].value) new_node = oa.session.query(oa.node).filter_by(id=ref).one() # Append to nodes in way way.nodes.append(new_node) # Store other attributes and tags _dom_attrs_to_any(xml_element, way) _dom_tags_to_any(xml_element, way) # Add to session oa.session.add(way) oa.session.commit() def _dom_to_relation(xml_element): with oa.session.no_autoflush: # Get mandatory way id id_ = int(xml_element.attributes["id"].value) oa.logger.debug("Importing OSM relation with id %i." % id_) # Find object in database and create if non-existent relation = oa.session.query(oa.relation).filter_by(id=id_).scalar() if relation is None: relation = oa.relation(id=id_) # Find all members for member in xml_element.getElementsByTagName("member"): # Get member attributes ref = int(member.attributes["ref"].value) type_ = member.attributes["type"].value if "role" in member.attributes.keys(): role = member.attributes["role"].value else: role = "" element = oa.session.query(oa.element).filter_by(id=ref, type=type_).scalar() if element is None: # We do not know the member yet, create a stub if type_ == "node": element = oa.node(id=ref) elif type_ == "way": element = oa.way(id=ref) elif type_ == "relation": element = oa.relation(id=ref) # We need to commit here because element could be repeated oa.session.add(element) oa.session.commit() # Append to members relation.members.append((element, role)) # Store other attributes and tags _dom_attrs_to_any(xml_element, relation) _dom_tags_to_any(xml_element, relation) # Add to session oa.session.add(relation) oa.session.commit() # Get root element osm = dom.documentElement # Iterate over children to find nodes, ways and relations for xml_element in osm.childNodes: # Determine element type if xml_element.nodeName == "node": _dom_to_node(xml_element) elif xml_element.nodeName == "way": _dom_to_way(xml_element) elif xml_element.nodeName == "relation": _dom_to_relation(xml_element) # Rmove children xml_element.unlink() def _import_osm_xml(oa, xml, session=None): """ Import a string in OSM XML format into an OSMAlchemy model. oa - reference to the OSMAlchemy model instance xml - string containing the XML data """ # Get session if session is None: session = oa.session # Parse string into DOM structure dom = minidom.parseString(xml) return _import_osm_dom(oa, dom, session=session) def _import_osm_file(oa, file, session=None): """ Import a file in OSM XML format into an OSMAlchemy model. oa - reference to the OSMAlchemy model instance file - path to the file to import or open file object """ # Get session if session is None: session = oa.session # Parse file if isinstance(file, str): with open(file, "r", encoding="utf-8") as f: dom = minidom.parse(f) else: dom = minidom.parse(file) return _import_osm_dom(oa, dom, session=session) OSMAlchemy-0.1.3/osmalchemy/util/online.py0000644000175000017500000004161513042147050020017 0ustar niknik00000000000000## ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016, 2017 Dominik George # Copyright (c) 2016 Eike Tim Jesinghaus # Copyright (c) 2017 mirabilos # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Utility code for OSMAlchemy's overpass code. """ import operator import overpass import re from sqlalchemy.sql.elements import BinaryExpression, BooleanClauseList, BindParameter, Grouping from sqlalchemy.sql.annotation import AnnotatedColumn from sqlalchemy.sql.selectable import Exists def _generate_overpass_api(endpoint=None): """ Create and initialise the Overpass API object. Passing the endpoint argument will override the default endpoint URL. """ # Create API object with default settings api = overpass.API() # Change endpoint if desired if not endpoint is None: api.endpoint = endpoint return api def _get_single_element_by_id(api, type_, id_, recurse_down=True): """ Retrieves a single OpenStreetMap element by its id. api - an initialised Overpass API object type_ - the element type to query, one of node, way or relation id_ - the id of the element to retrieve recurse_down - whether to get child nodes of ways and relations """ # Construct query query = "%s(%d);%s" % (type_, id_, "(._;>;);" if recurse_down else "") # Run query result = api.Get(query, responseformat="xml") # Return data return result def _get_elements_by_query(api, query, recurse_down=True): """ Runs a query and returns the resulting OSM XML. api - an initialised Overpass API object query - the OverpassQL query recurse_down - whether to get child nodes of ways and relations """ # Run query result = api.Get("%s%s" % (query, "(._;>;);" if recurse_down else ""), responseformat="xml") # Return data return result # Define operator to string mapping _OPS = {operator.eq: "==", operator.ne: "!=", operator.lt: "<", operator.gt: ">", operator.le: "<=", operator.ge: ">=", operator.and_: "&&", operator.or_: "||"} def _where_to_tree(clause, target): """ Transform an SQLAlchemy whereclause to an expression tree. This function analyses a Query.whereclause object and turns it into a more general data structure. """ if isinstance(clause, BinaryExpression): # This is something like "latitude >= 51.0" left = clause.left right = clause.right op = clause.operator # Left part should be a column if isinstance(left, AnnotatedColumn): # Get table class and field model = left._annotations["parentmapper"].class_ field = left # Only use if we are looking for this model if model is target: # Store field name left = field.name else: return None else: # Right now, we cannot cope with anything but a column on the left return None # Right part should be a literal value if isinstance(right, BindParameter): # Extract literal value right = right.value else: # Right now, we cannot cope with something else here return None # Look for a known operator if op in _OPS.keys(): # Get string representation op = _OPS[op] else: # Right now, we cannot cope with other operators return None # Return polish notation tuple of this clause return (op, left, right) elif isinstance(clause, BooleanClauseList): # This is an AND or OR operation op = clause.operator clauses = [] # Iterate over all the clauses in this operation for clause in clause.clauses: # Recursively analyse clauses res = _where_to_tree(clause, target) # None is returned for unsupported clauses or operations if res is not None: # Append polish notation result to clauses list clauses.append(res) # Look for a known operator if op in _OPS.keys(): # Get string representation op = _OPS[op] else: # Right now, we cannot cope with anything else return None # Return polish notation tuple of this clause return (op, clauses) elif isinstance(clause, Exists): # This case is a bit hard to verify. # We expect this to be the EXISTS sub-clause of something like: # # session.query(osmalchemy.node).filter( # osmalchemy.node.tags.any(key="name", value="Schwarzrheindorf Kirche") # ).all() # # For now, we stick with simply expecting that until someone # rewrites this entire code. try: # Try to get the real conditionals from this weird statement conditionals = (clause.get_children()[0]._whereclause.clauses[1]. get_children()[0].get_children()[0]._whereclause.clauses[1].clauses) except: # Simply return None if we got something unexpected return None key = "" value = "" for clause in conditionals: if clause.left.name == "key": key = clause.right.value elif clause.left.name == "value": value = clause.right.value # Check if we got only a key, a key and a value or neither if key and not value: return ("has", key, "") elif key and value: return ("==", key, value) else: return None elif isinstance(clause, Grouping): # Ungroup by simply taking the first element of the group # This is not correct in general, but correct for all documented # use cases. return _where_to_tree(clause.get_children()[0], target) else: # We hit an unsupported type of clause return None def _trees_to_overpassql(tree_dict): """ Transform an expression tree (from _where_to_tree) into OverpassQL. """ # Called recursively on all subtrees def _tree_to_overpassql_recursive(tree, type_): # Empty result string result = "" # Test if we got a tree or an atom if isinstance(tree[1], list): # We are in a subtree # Store operation of subtree (conjunction/disjunction) op = tree[0] # Empty bounding box bbox = [None, None, None, None] # List of genrated set names set_names = [] # Iterate over all elements in the conjunction/disjunction for element in tree[1]: # Check if element is a tree or an atom if isinstance(element[1], list): # Recurse into inner tree result_inner_tree = _tree_to_overpassql_recursive(tree[1]) # Store resulting query and its name result += "%s" % result_inner_tree[1] set_names.append(result_inner_tree[0]) else: # Parse atom # latitude and longitude comparisons form a bounding box if element[1] == "latitude" and element[0] in [">", ">="]: # South edge if bbox[0] is None: bbox[0] = float(element[2]) elif op == "&&" and bbox[0] <= element[2]: bbox[0] = float(element[2]) elif op == "||" and bbox[0] >= element[2]: bbox[0] = float(element[2]) elif element[1] == "latitude" and element[0] in ["<", "<="]: # North edge if bbox[2] is None: bbox[2] = float(element[2]) elif op == "&&" and bbox[2] >= element[2]: bbox[2] = float(element[2]) elif op == "||" and bbox[2] <= element[2]: bbox[2] = float(element[2]) elif element[1] == "longitude" and element[0] in [">", ">="]: # West edge if bbox[1] is None: bbox[1] = float(element[2]) elif op == "&&" and bbox[1] <= element[2]: bbox[1] = float(element[2]) elif op == "||" and bbox[1] >= element[2]: bbox[1] = float(element[2]) elif element[1] == "longitude" and element[0] in ["<", "<="]: # East edge if bbox[3] is None: bbox[3] = float(element[2]) elif op == "&&" and bbox[3] >= element[2]: bbox[3] = float(element[2]) elif op == "||" and bbox[3] <= element[2]: bbox[3] = float(element[2]) # Query for an element with specific id elif element[1] == "id" and element[0] == "==": # Build query if op == "||": idquery = "%s(%i)" % (type_, element[2]) # Store resulting query and its name set_name = "s%i" % id(idquery) result += "%s->.%s;" % (idquery, set_name) set_names.append(set_name) elif op == "&&": idquery = "(%i)" % (element[2]) # Store resulting query and its name set_name = "s%i" % id(idquery) result += idquery set_names.append(set_name) elif element[1] == "id": # We got an id query, but not with equality comparison raise ValueError("id can only be queried with equality") # Everything else must be a tag query else: # Check whether it is a comparison or a query for existence if element[0] == "==": # Build query for tag comparison if op == "||": tagquery = "%s[\"%s\"=\"%s\"]" % (type_, _escape_tag(element[1]), _escape_tag(element[2])) elif op == "&&": tagquery = "[\"%s\"=\"%s\"]" % (_escape_tag(element[1]), _escape_tag(element[2])) elif element[0] == "has": if op == "||": tagquery = "%s[\"%s\"]" % (type_, _escape_tag(element[1])) elif op == "&&": tagquery = "[\"%s\"]" % (_escape_tag(element[1])) # Store resulting query and its name set_name = "s%i" % id(tagquery) if op == "||": result += "%s->.%s;" % (tagquery, set_name) elif op == "&&": result += tagquery set_names.append(set_name) # Check if any component of the bounding box was set if bbox != [None, None, None, None]: # Amend minima/maxima if bbox[0] is None: bbox[0] = -90.0 if bbox[1] is None: bbox[1] = -180.0 if bbox[2] is None: bbox[2] = 90.0 if bbox[3] is None: bbox[3] = 180.0 # Build query if op == "||": bboxquery = "%s(%.7f,%.7f,%.7f,%.7f)" % (type_, bbox[0], bbox[1], bbox[2], bbox[3]) elif op == "&&": bboxquery = "(%.7f,%.7f,%.7f,%.7f)" % (bbox[0], bbox[1], bbox[2], bbox[3]) # Store resulting query and its name set_name = "s%i" % id(bboxquery) if op == "||": result += "%s->.%s;" % (bboxquery, set_name) elif op == "&&": result += bboxquery set_names.append(set_name) # Build conjunction or disjunction according to current operation if len(set_names) > 1: if op == "&&": # Conjunction, build an intersection result = "%s%s" % (type_, result) elif op == "||": # Disjunction, build a union result += "(" for set_name in set_names: result += ".%s;" % set_name result += ")" else: if op == "||": result += "(.%s;)" % set_names[0] elif op == "&&": result = "%s%s" % (type_, result) else: # We got a bare atom # latitude and longitude are components of a bounding box query if tree[1] == "latitude" and tree[0] == ">": # South edge result = "%s(%.7f,-180.0,90.0,180.0)" % (type_, float(tree[2])) elif tree[1] == "latitude" and tree[0] == "<": # West edge result = "%s(-90.0,-180.0,%.7f,180.0)" % (type_, float(tree[2])) elif tree[1] == "longitude" and tree[0] == ">": # North edge result = "%s(-90.0,%.7f,-90.0,180.0)" % (type_, float(tree[2])) elif tree[1] == "longitude" and tree[0] == "<": # East edge result = "%s(-90.0,-180.0,-90.0,%.7f)" % (type_, float(tree[2])) # Query for an id elif tree[1] == "id" and tree[0] == "==": result = "%s(%i)" % (type_, tree[2]) elif tree[1] == "id": # We got an id query, but not with equality comparison raise ValueError("id can only be queried with equality") # Everything else must be a tag query else: result = "%s[\"%s\"=\"%s\"]" % (type_, _escape_tag(tree[1]), _escape_tag(tree[2])) # generate a name for the complete set and return it, along with the query set_name = id(result) result += "->.s%i;" % set_name return (set_name, result) # Run tree transformation for each type in the input tree results = [] for type_ in tree_dict.keys(): # Get real type name (OSMNode→node,…) real_type = type_[3:].lower() # Do transformation and store query and name results.append(_tree_to_overpassql_recursive(tree_dict[type_], real_type)) # Build finally resulting query in a union full_result = "" set_names = "(" for result in results: full_result += result[1] set_names += ".s%s; " % result[0] set_names = "%s);" % set_names.strip() full_result += set_names # Return final query return full_result def _normalise_overpassql(oql): """ Normalise an OverpassQL expression. Takes an OverpassQL string as argument and strips all set names of the form \.s[0-9]+ """ def replace_setname(match): if not match.group().startswith('"'): return re.sub(r"\.s[0-9]+", ".s", match.group()) else: return match.group() return re.sub(r'"[^"]*"|([^"]*)', replace_setname, oql) def _escape_tag(tag): """ Escapes special characters in a tag query component. """ res = tag res = res.replace('\\', '\\\\') res = res.replace('"', '\\"') return res OSMAlchemy-0.1.3/osmalchemy/util/patch.py0000644000175000017500000001002512764502275017637 0ustar niknik00000000000000# ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Michael Bayer # Copyright (c) 2016 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. def monkey_patch_sqlalchemy(): from sqlalchemy.ext.associationproxy import AssociationProxy from sqlalchemy.util import memoized_property # Monkey patch support for chained association proxy queries into SQLAlchemy # https://bitbucket.org/zzzeek/sqlalchemy/issues/3769/chained-any-has-with-association-proxy if not hasattr(AssociationProxy, "_unwrap_target_assoc_proxy"): def _unwrap_target_assoc_proxy(self): attr = getattr(self.target_class, self.value_attr) if isinstance(attr, AssociationProxy): return attr, getattr(self.target_class, attr.target_collection) return None, None AssociationProxy._unwrap_target_assoc_proxy = memoized_property(_unwrap_target_assoc_proxy) orig_any = AssociationProxy.any def any_(self, criterion=None, **kwargs): target_assoc, inner = self._unwrap_target_assoc_proxy if target_assoc is not None: if target_assoc._target_is_object and target_assoc._uselist: inner = inner.any(criterion=criterion, **kwargs) else: inner = inner.has(criterion=criterion, **kwargs) return self._comparator.any(inner) orig_any(self, criterion, **kwargs) AssociationProxy.any = any_ orig_has = AssociationProxy.has def has(self, criterion=None, **kwargs): target_assoc, inner = self._unwrap_target_assoc_proxy if target_assoc is not None: if target_assoc._target_is_object and target_assoc._uselist: inner = inner.any(criterion=criterion, **kwargs) else: inner = inner.has(criterion=criterion, **kwargs) return self._comparator.has(inner) orig_has(self, criterion, **kwargs) AssociationProxy.has = has def monkey_patch_flask_restless(): try: import flask_restless.helpers except: return # Monkey patch support for chained association proxy into Flask-Restless # https://github.com/jfinkels/flask-restless/issues/578 if hasattr(flask_restless.helpers, "get_related_association_proxy_model"): from sqlalchemy.ext.associationproxy import AssociationProxy def _get_related_association_proxy_model(attr): if isinstance(attr.remote_attr, AssociationProxy): return _get_related_association_proxy_model(attr.remote_attr) else: prop = attr.remote_attr.property for attribute in ('mapper', 'parent'): if hasattr(prop, attribute): return getattr(prop, attribute).class_ return None flask_restless.helpers.get_related_association_proxy_model = _get_related_association_proxy_model OSMAlchemy-0.1.3/osmalchemy/__init__.py0000644000175000017500000000047013042150521017304 0ustar niknik00000000000000# ~*~~ coding: utf-8 ~*~ # Define the version of OSMAlchemy __version__ = "0.1.3" # Monkey patch SQLAlchemy to support some query constructs from .util.patch import monkey_patch_sqlalchemy, monkey_patch_flask_restless monkey_patch_sqlalchemy() monkey_patch_flask_restless() from .osmalchemy import OSMAlchemy OSMAlchemy-0.1.3/osmalchemy/model.py0000644000175000017500000003263713042143132016657 0ustar niknik00000000000000# ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Simple representation of OpenStreetMap's conceptual data model. (cf. http://wiki.openstreetmap.org/wiki/Elements) This implementation of the model assumes that only current data is used, not historic data. """ import datetime from sqlalchemy import (Column, ForeignKey, Integer, BigInteger, Numeric, String, Unicode, DateTime, Boolean, UniqueConstraint) from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.ext.associationproxy import association_proxy from sqlalchemy.ext.orderinglist import ordering_list from sqlalchemy.orm import relationship, backref from sqlalchemy.orm.collections import attribute_mapped_collection def _generate_model(oa): """ Generates the data model. The model classes are generated dynamically to allow passing in a declarative base and a prefix. """ class OSMTag(oa.base): """ An OSM tag element. Simple key/value pair. """ # Name of the table in the database, prefix provided by user __tablename__ = oa.prefix + "tags" # The internal ID of the element, only for structural use tag_id = Column(BigInteger().with_variant(Integer, "sqlite"), primary_key=True) # Key/value pair key = Column(Unicode(256)) value = Column(Unicode(256)) def __init__(self, key="", value="", **kwargs): """ Initialisation with two main positional arguments. Shorthand for OSMTag(key, value) """ self.key = key self.value = value # Pass rest on to default constructor oa.base.__init__(self, **kwargs) class OSMElement(oa.base): """ Base class for all the conceptual OSM elements. """ # Name of the table in the database, prefix provided by user __tablename__ = oa.prefix + "elements" # The internal ID of the element, only for structural use element_id = Column(BigInteger().with_variant(Integer, "sqlite"), primary_key=True) # Track element modification for OSMAlchemy caching osmalchemy_updated = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now) # The type of the element, used by SQLAlchemy for polymorphism type = Column(String(256)) # ID of the element in OSM, not to be confused with the primary key element_id id = Column(BigInteger) # Tags belonging to the element # Accessed as a dictionary like {'name': 'value', 'name2': 'value2',…} # Uses proxying across several tables to OSMTag tags = association_proxy(oa.prefix+"elements_tags", "tag_value", creator=lambda k, v: OSMElementsTags(tag=OSMTag(key=k, value=v))) # Metadata shared by all element types version = Column(Integer) changeset = Column(BigInteger) user = Column(Unicode(256)) uid = Column(BigInteger) visible = Column(Boolean) timestamp = Column(DateTime) # OSM ids are unique per type _u_type_id = UniqueConstraint("type", "id") # Configure polymorphism __mapper_args__ = { 'polymorphic_identity': 'element', 'polymorphic_on': type, 'with_polymorphic': '*' } class OSMElementsTags(oa.base): """ Secondary mapping table for elements and tags """ # Name of the table in the database, prefix provided by user __tablename__ = oa.prefix + "elements_tags" # Internal ID of the mapping, only for structural use map_id = Column(BigInteger().with_variant(Integer, "sqlite"), primary_key=True) # Foreign key columns for the element and tag of the mapping element_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'elements.element_id')) tag_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'tags.tag_id')) # Relationship with all the tags mapped to the element # The backref is the counter-part to the tags association proxy # in OSMElement to form the dictionary element = relationship(OSMElement, foreign_keys=[element_id], backref=backref(oa.prefix+"elements_tags", collection_class=attribute_mapped_collection( "tag_key" ), cascade="all, delete-orphan")) # Relationship to the tag object and short-hand for its key and value # for use in the association proxy tag = relationship(OSMTag, foreign_keys=[tag_id]) tag_key = association_proxy("tag", "key") tag_value = association_proxy("tag", "value") class OSMNode(OSMElement): """ An OSM node element. A node hast a latitude and longitude, which are non-optional, and a list of zero or more tags. """ # Name of the table in the database, prefix provided by user __tablename__ = oa.prefix + "nodes" # The internal ID of the element, only for structural use # Synchronised with the id of the parent table OSMElement through polymorphism element_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'elements.element_id'), primary_key=True) # Track element modification for OSMAlchemy caching osmalchemy_updated = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now) # Geographical coordinates of the node latitude = Column(Numeric(precision="9,7", asdecimal=False)) longitude = Column(Numeric(precision="10,7", asdecimal=False)) # Configure polymorphism with OSMElement __mapper_args__ = { 'polymorphic_identity': 'node', } def __init__(self, latitude=None, longitude=None, **kwargs): """ Initialisation with two main positional arguments. Shorthand for OSMNode(lat, lon). """ self.latitude = latitude self.longitude = longitude # Pass rest on to default constructor OSMElement.__init__(self, **kwargs) class OSMWaysNodes(oa.base): """ Secondary mapping table for ways and nodes """ # Name of the table in the database, prefix provided by user __tablename__ = oa.prefix + "ways_nodes" # Internal ID of the mapping, only for structural use map_id = Column(BigInteger().with_variant(Integer, "sqlite"), primary_key=True) # Foreign key columns for the connected way and node way_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'ways.element_id')) node_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'nodes.element_id')) # Relationships for proxy access node = relationship(OSMNode, foreign_keys=[node_id]) # Index of the node in the way to maintain ordered list, structural use only position = Column(Integer) class OSMWay(OSMElement): """ An OSM way element (also area). Contains a list of two or more nodes and a list of zero or more tags. """ # Name of the table in the database, prefix provided by user __tablename__ = oa.prefix + "ways" # The internal ID of the element, only for structural use # Synchronised with the id of the parent table OSMElement through polymorphism element_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'elements.element_id'), primary_key=True) # Track element modification for OSMAlchemy caching osmalchemy_updated = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now) # Relationship with all nodes in the way # Uses association proxy and a collection class to maintain an ordered list, # synchronised with the position field of OSMWaysNodes _nodes = relationship(OSMWaysNodes, order_by="OSMWaysNodes.position", collection_class=ordering_list("position")) nodes = association_proxy("_nodes", "node", creator=lambda _n: OSMWaysNodes(node=_n)) # Configure polymorphism with OSMElement __mapper_args__ = { 'polymorphic_identity': 'way', } class OSMRelationsElements(oa.base): """ Secondary mapping table for relation members """ # Name of the table in the database, prefix provided by user __tablename__ = oa.prefix + "relations_elements" # Internal ID of the mapping, only for structural use map_id = Column(BigInteger().with_variant(Integer, "sqlite"), primary_key=True) # Foreign ley columns for the relation and other element of the mapping relation_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'relations.element_id')) element_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'elements.element_id')) # Relationships for proxy access element = relationship(OSMElement, foreign_keys=[element_id]) # Role of the element in the relationship role = Column(Unicode(256)) # Index of element in the relationship to maintain ordered list, structural use only position = Column(Integer) # Produce (element, role) tuple for proxy access in OSMRelation @property def role_tuple(self): return (self.element, self.role) class OSMRelation(OSMElement): """ An OSM relation element. Contains zero or more members (ways, nodes or other relations) with associated, optional roles and zero or more tags. """ # Name of the table in the database, prefix provided by user __tablename__ = oa.prefix + "relations" # The internal ID of the element, only for structural use # Synchronised with the id of the parent table OSMElement through polymorphism element_id = Column(BigInteger().with_variant(Integer, "sqlite"), ForeignKey(oa.prefix + 'elements.element_id'), primary_key=True) # Track element modification for OSMAlchemy caching osmalchemy_updated = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now) # Relationship to the members of the relationship, proxied across OSMRelationsElements _members = relationship(OSMRelationsElements, order_by="OSMRelationsElements.position", collection_class=ordering_list("position")) # Accessed as a list like [(element, "role"), (element2, "role2")] members = association_proxy("_members", "role_tuple", creator=lambda _m: OSMRelationsElements(element=_m[0], role=_m[1])) # Configure polymorphism with OSMElement __mapper_args__ = { 'polymorphic_identity': 'relation', } class OSMCachedQuery(oa.base): """ Table for caching queries run by the OSMAlchemy triggers. """ __tablename__ = oa.prefix + "cached_queries" # The internal ID of the element, only for structural use query_id = Column(BigInteger().with_variant(Integer, "sqlite"), primary_key=True) # Hash sum of the query to identify it oql_hash = Column(Unicode(32), unique=True) # Last time this query was run oql_queried = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now) # Return the relevant generated objects return (OSMNode, OSMWay, OSMRelation, OSMElement, OSMCachedQuery) OSMAlchemy-0.1.3/osmalchemy/osmalchemy.py0000644000175000017500000002030412765063533017725 0ustar niknik00000000000000# ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Module that holds the main OSMAlchemy class. The classe encapsulates the model and accompanying logic. """ from sqlalchemy.engine import Engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker, scoped_session try: from flask_sqlalchemy import SQLAlchemy as FlaskSQLAlchemy except ImportError: # non-fatal, Flask-SQLAlchemy support is optional # Create stub to avoid bad code later on class FlaskSQLAlchemy(object): pass from .model import _generate_model from .util.db import _import_osm_file from .util.online import _generate_overpass_api from .triggers import _generate_triggers class OSMAlchemy(object): """ Wrapper class for the OSMAlchemy model and logic This class holds all the SQLAlchemy classes and logic that make up OSMAlchemy. It is contained in a separate class because it is a template that can be modified as needed by users, e.g. by using a different table prefix or a different declarative base. """ def __init__(self, sa, prefix="osm_", overpass=None, maxage=60*60*24): """ Initialise the table definitions in the wrapper object This function generates the OSM element classes as SQLAlchemy table declaratives. Positional arguments: sa - reference to SQLAlchemy stuff; can be either of… …an Engine instance, or… …a tuple of (Engine, Base), or… …a tuple of (Engine, Base, ScopedSession), or… …a Flask-SQLAlchemy instance. prefix - optional; prefix for table names, defaults to "osm_" overpass - optional; API endpoint URL for Overpass API. Can be… …None to disable loading data from Overpass (the default), or… …True to enable the default endpoint URL, or… …a string with a custom endpoint URL. maxage - optional; the maximum age after which elements are refreshed from Overpass, in seconds, defaults to 86400s (1d) """ # Store logger or create mock import logging self.logger = logging.getLogger('osmalchemy') self.logger.addHandler(logging.NullHandler()) # Create fields for SQLAlchemy stuff self.base = None self.engine = None self.session = None # Inspect sa argument if isinstance(sa, tuple): # Got tuple of (Engine, Base) or (Engine, Base, ScopedSession) self.engine = sa[0] self.base = sa[1] if len(sa) == 3: self.session = sa[2] else: self.session = scoped_session(sessionmaker(bind=self.engine)) self.logger.debug("Called with (engine, base, session) tuple.") elif isinstance(sa, Engine): # Got a plain engine, so derive the rest from it self.engine = sa self.base = declarative_base(bind=self.engine) self.session = scoped_session(sessionmaker(bind=self.engine)) self.logger.debug("Called with a plain SQLAlchemy engine.") elif isinstance(sa, FlaskSQLAlchemy): # Got a Flask-SQLAlchemy instance, extract everything from it self.engine = sa.engine self.base = sa.Model self.session = sa.session self.logger.debug("Called with a Flask-SQLAlchemy wrapper.") else: # Something was passed, but none of the expected argument types raise TypeError("Invalid argument passed to sa parameter.") # Store prefix self.prefix = prefix # Store maxage self.maxage = maxage # Store API endpoint for Overpass if not overpass is None: if overpass is True: # Use default endpoint URL from overpass module self.overpass = _generate_overpass_api() self.logger.debug("Overpass API enabled with default endpoint.") elif isinstance(overpass, str): # Pass given argument as custom URL self.overpass = _generate_overpass_api(overpass) self.logger.debug("Overpass API enabled with endpoint %s." % overpass) else: # We got something unknown passed, bail out raise TypeError("Invalid argument passed to overpass parameter.") else: # Do not use overpass self.overpass = None # Generate model and store as instance members self.node, self.way, self.relation, self.element, self.cached_query = _generate_model(self) self.logger.debug("Generated OSMAlchemy model with prefix %s." % prefix) # Add triggers if online functionality is enabled if not self.overpass is None: _generate_triggers(self) self.logger.debug("Triggers generated and activated.") def import_osm_file(self, path): """ Import data from an OSM XML file into this model. path - path to the file to import """ # Call utility funtion with own reference and session _import_osm_file(self, path) def create_api(self, api_manager): """ Create Flask-Restless API endpoints. """ def _expand_tags(obj): # Type name to object mapping _types = { "node": self.node, "way": self.way, "relation": self.relation } # Get tags dictionary from ORM object instance = self.session.query(_types[obj["type"]]).get(obj["element_id"]) # Fill a tag dictionary res = {} for key in obj["tags"]: res[key] = instance.tags[key] # Replace tags list with generated dictionary obj["tags"] = res def _cleanup(obj): # Remove unnecessary entries from dict del obj["osm_elements_tags"] del obj["type"] def _post_get(result, **_): # Post-processor for GET # Work-around for strange bug in Flask-Restless preventing detection # of dictionary-like association proxies if "objects" in result: # This is a GET_MANY call for obj in result["objects"]: _expand_tags(obj) _cleanup(obj) else: # This is a GET_SINGLE call _expand_tags(result) _cleanup(result) # Define post-processors for all collections postprocessors = {"GET_SINGLE": [_post_get], "GET_MANY": [_post_get]} # Define collections for all object types api_manager.create_api(self.node, postprocessors=postprocessors) api_manager.create_api(self.way, postprocessors=postprocessors) api_manager.create_api(self.relation, postprocessors=postprocessors) OSMAlchemy-0.1.3/osmalchemy/triggers.py0000644000175000017500000001464113042143132017400 0ustar niknik00000000000000# ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Dominik George # Copyright (c) 2016 Eike Tim Jesinghaus # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Trigger code for live OSMAlchemy/Overpass integration. """ import datetime from hashlib import md5 from sqlalchemy import inspect from sqlalchemy.event import listens_for from sqlalchemy.orm import Query from weakref import WeakSet from .util.db import _import_osm_xml from .util.online import (_get_elements_by_query, _where_to_tree, _trees_to_overpassql, _normalise_overpassql, _get_single_element_by_id) def _generate_triggers(oa): """ Generates the triggers for online functionality. oa - reference to the OSMAlchemy instance to be configured """ _visited_queries = WeakSet() @listens_for(oa.node, "load") @listens_for(oa.way, "load") @listens_for(oa.relation, "load") def _instance_loading(target, context, _=None): # Get query session session = context.session # Skip if the session was in a trigger before # Prevents recursion in import code if hasattr(session, "_osmalchemy_in_trigger"): return # Determine OSM element type and id type_ = target.__class__.__name__[3:].lower() id_ = target.id # Guard against broken objects without an id if id_ is None: return oa.logger.debug("Loading instance of type %s with id %i." % (type_, id_)) # Check whether object needs to be refreshed updated = target.osmalchemy_updated timediff = datetime.datetime.now() - updated if timediff.total_seconds() < oa.maxage: oa.logger.debug("Instance is only %i seconds old, not refreshing online." % timediff.total_seconds()) return oa.logger.debug("Refreshing instance online.") # Get object by id as XML xml = _get_single_element_by_id(oa.overpass, type_, id_) # Import data session._osmalchemy_in_trigger = True _import_osm_xml(oa, xml, session=session) del session._osmalchemy_in_trigger @listens_for(Query, "before_compile") def _query_compiling(query): # Get the session associated with the query: session = query.session # Skip if the session was in a trigger before # Prevents recursion in import code if hasattr(session, "_osmalchemy_in_trigger"): return # Prevent recursion by skipping already-seen queries if query in _visited_queries: return else: _visited_queries.add(query) oa.logger.debug("Analysing new ORM query.") # Check whether this query affects our model affected_models = set([c["type"] for c in query.column_descriptions]) our_models = set([oa.node, oa.way, oa.relation, oa.element]) if affected_models.isdisjoint(our_models): # None of our models is affected oa.logger.debug("None of our models are affected.") return # Check whether this query filters elements # Online update will only run on a specified set, not all data if query.whereclause is None: # No filters oa.logger.debug("No filters found in query.") return oa.logger.debug("Building query tree from ORM query structure.") # Analyse where clause looking for all looked-up fields trees = {} for target in our_models.intersection(affected_models): # Build expression trees first tree = _where_to_tree(query.whereclause, target) if not tree is None: trees[target.__name__] = tree # Bail out if no relevant trees were built if not trees: oa.logger.debug("No relevant query trees built.") return # Compile to OverpassQL oql = _trees_to_overpassql(trees) oa.logger.debug("Compiled OverpassQL: %s" % oql) # Look up query in cache hashed_oql = md5(_normalise_overpassql(oql).encode()).hexdigest() cached_query = session.query(oa.cached_query).filter_by(oql_hash=hashed_oql).scalar() # Check age if cached query was found if cached_query: timediff = datetime.datetime.now() - cached_query.oql_queried if timediff.total_seconds() < oa.maxage: # Return and do nothing if query was run recently oa.logger.debug("Query was run online only %i seconds ago, not running." % timediff.total_seconds()) return # Run query online oa.logger.debug("Running Overpass query.") xml = _get_elements_by_query(oa.overpass, oql) # Import data session._osmalchemy_in_trigger = True _import_osm_xml(oa, xml, session=session) del session._osmalchemy_in_trigger # Store or update query time if not cached_query: cached_query = oa.cached_query() cached_query.oql_hash = hashed_oql cached_query.oql_queried = datetime.datetime.now() session.add(cached_query) session.commit() oa.logger.debug("Query cached.") OSMAlchemy-0.1.3/test/0000755000175000017500000000000013042151631014013 5ustar niknik00000000000000OSMAlchemy-0.1.3/test/data/0000755000175000017500000000000013042151631014724 5ustar niknik00000000000000OSMAlchemy-0.1.3/test/data/schwarzrheindorf.osm0000644000175000017500001110126012752414163021041 0ustar niknik00000000000000 OSMAlchemy-0.1.3/test/__init__.py0000644000175000017500000000000012750447612016126 0ustar niknik00000000000000OSMAlchemy-0.1.3/test/test_api.py0000644000175000017500000000663612765023113016214 0ustar niknik00000000000000#!/usr/bin/env python3 # ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Tests concerning the OSMAlchemy object. """ # Standard unit testing framework import unittest # Module to be tested from osmalchemy import OSMAlchemy # SQLAlchemy for working with model and data from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker, scoped_session # Flask for testing integration with Flask from flask import Flask from flask_sqlalchemy import SQLAlchemy as FlaskSQLAlchemy class OSMAlchemyAPITests(unittest.TestCase): def test_instantiate_with_engine(self): engine = create_engine("sqlite:///:memory:") osmalchemy = OSMAlchemy(engine) self.assertIsInstance(osmalchemy, OSMAlchemy) self.assertIs(osmalchemy.engine, engine) engine.dispose() def test_instantiate_with_engine_and_base(self): engine = create_engine("sqlite:///:memory:") base = declarative_base(bind=engine) osmalchemy = OSMAlchemy((engine, base)) self.assertIsInstance(osmalchemy, OSMAlchemy) self.assertIs(osmalchemy.engine, engine) self.assertIs(osmalchemy.base, base) engine.dispose() def test_instantiate_with_engine_and_base_and_session(self): engine = create_engine("sqlite:///:memory:") base = declarative_base(bind=engine) session = scoped_session(sessionmaker(bind=engine)) osmalchemy = OSMAlchemy((engine, base, session)) self.assertIsInstance(osmalchemy, OSMAlchemy) self.assertIs(osmalchemy.engine, engine) self.assertIs(osmalchemy.base, base) self.assertIs(osmalchemy.session, session) engine.dispose() def test_instantiate_with_flask_sqlalchemy(self): app = Flask("test") db = FlaskSQLAlchemy(app) osmalchemy = OSMAlchemy(db) self.assertIsInstance(osmalchemy, OSMAlchemy) self.assertIs(osmalchemy.engine, db.engine) self.assertIs(osmalchemy.base, db.Model) self.assertIs(osmalchemy.session, db.session) # Make runnable as standalone script if __name__ == "__main__": unittest.main() OSMAlchemy-0.1.3/test/test_model.py0000644000175000017500000004050413000163001016513 0ustar niknik00000000000000#!/usr/bin/env python3 # ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Tests concerning the OSM data model implementation. """ # Standard unit testing framework import unittest # We want to profile test cases import time # Helper libraries for different database engines from testing.mysqld import MysqldFactory from testing.postgresql import PostgresqlFactory # Module to be tested from osmalchemy import OSMAlchemy # SQLAlchemy for working with model and data from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker, scoped_session # Flask for testing integration with Flask from flask import Flask from flask_sqlalchemy import SQLAlchemy as FlaskSQLAlchemy # Create database engine factories to enable caching Postgresql = PostgresqlFactory(cache_initialized_db=True) Mysqld = MysqldFactory(cache_initialized_db=True) # Dictionary to store profiling information about tests profile = {} def tearDownModule(): """ Global test suite tear down code """ # Purge caches of database engines Postgresql.clear_cache() Mysqld.clear_cache() # Print profiling info print("Database model test times") print("=========================") print("") for suite in profile: print("%8.3f s\t%s" % (sum([profile[suite][test] for test in profile[suite]]), suite)) for test in profile[suite]: print("\t%8.3f s\t%s" % (profile[suite][test], test)) class OSMAlchemyModelTests(object): """ Incomplete base class for common test routines. Subclassed in engine-dependent test classes. """ def setUp(self): if not self.__class__.__name__ in profile: profile[self.__class__.__name__] = {} profile[self.__class__.__name__][self.id().split(".")[-1]] = time.time() def tearDown(self): profile[self.__class__.__name__][self.id().split(".")[-1]] -= time.time() profile[self.__class__.__name__][self.id().split(".")[-1]] *= -1 def test_create_node(self): # Create node node = self.osmalchemy.node() node.latitude = 51.0 node.longitude = 7.0 # Store node self.session.add(node) self.session.commit() # Ensure removal from ORM self.session.remove() # Query for node and check node = self.session.query(self.osmalchemy.node).filter_by(latitude=51.0).first() self.assertEqual(node.latitude, 51.0) self.assertEqual(node.longitude, 7.0) self.assertEqual(len(node.tags), 0) def test_create_node_with_tags(self): # Create node and tags node = self.osmalchemy.node(51.0, 7.0) node.tags[u"name"] = u"test" node.tags[u"foo"] = u"bar" # Store everything self.session.add(node) self.session.commit() # Ensure removal from ORM self.session.remove() # Query for node and check node = self.session.query(self.osmalchemy.node).filter_by(latitude=51.0).first() self.assertEqual(node.latitude, 51.0) self.assertEqual(node.longitude, 7.0) self.assertEqual(len(node.tags), 2) self.assertEqual(node.tags[u"name"], u"test") self.assertEqual(node.tags[u"foo"], u"bar") def test_create_way_with_nodes(self): # Create way and nodes way = self.osmalchemy.way() way.nodes = [self.osmalchemy.node(51.0, 7.0), self.osmalchemy.node(51.1, 7.1), self.osmalchemy.node(51.2, 7.2), self.osmalchemy.node(51.3, 7.3), self.osmalchemy.node(51.4, 7.4)] # Store everything self.session.add(way) self.session.commit() # Ensure removal from ORM self.session.remove() # Query for way and check way = self.session.query(self.osmalchemy.way).first() self.assertEqual(len(way.nodes), 5) self.assertEqual((way.nodes[0].latitude, way.nodes[0].longitude), (51.0, 7.0)) self.assertEqual((way.nodes[1].latitude, way.nodes[1].longitude), (51.1, 7.1)) self.assertEqual((way.nodes[2].latitude, way.nodes[2].longitude), (51.2, 7.2)) self.assertEqual((way.nodes[3].latitude, way.nodes[3].longitude), (51.3, 7.3)) self.assertEqual((way.nodes[4].latitude, way.nodes[4].longitude), (51.4, 7.4)) def test_create_way_with_nodes_and_tags(self): # Create way and nodes way = self.osmalchemy.way() way.nodes = [self.osmalchemy.node(51.0, 7.0), self.osmalchemy.node(51.1, 7.1), self.osmalchemy.node(51.2, 7.2), self.osmalchemy.node(51.3, 7.3), self.osmalchemy.node(51.4, 7.4)] way.tags[u"name"] = u"Testway" way.tags[u"foo"] = u"bar" # Store everything self.session.add(way) self.session.commit() # Ensure removal from ORM self.session.remove() # Query for way and check way = self.session.query(self.osmalchemy.way).first() self.assertEqual(len(way.nodes), 5) self.assertEqual((way.nodes[0].latitude, way.nodes[0].longitude), (51.0, 7.0)) self.assertEqual((way.nodes[1].latitude, way.nodes[1].longitude), (51.1, 7.1)) self.assertEqual((way.nodes[2].latitude, way.nodes[2].longitude), (51.2, 7.2)) self.assertEqual((way.nodes[3].latitude, way.nodes[3].longitude), (51.3, 7.3)) self.assertEqual((way.nodes[4].latitude, way.nodes[4].longitude), (51.4, 7.4)) self.assertEqual(len(way.tags), 2) self.assertEqual(way.tags[u"name"], u"Testway") self.assertEqual(way.tags[u"foo"], u"bar") def test_create_way_with_nodes_and_tags_and_tags_on_node(self): # Create way and nodes way = self.osmalchemy.way() way.nodes = [self.osmalchemy.node(51.0, 7.0), self.osmalchemy.node(51.1, 7.1), self.osmalchemy.node(51.2, 7.2), self.osmalchemy.node(51.3, 7.3), self.osmalchemy.node(51.4, 7.4)] way.tags[u"name"] = u"Testway" way.tags[u"foo"] = u"bar" way.nodes[2].tags[u"name"] = u"Testampel" way.nodes[2].tags[u"foo"] = u"bar" # Store everything self.session.add(way) self.session.commit() # Ensure removal from ORM self.session.remove() # Query for way and check way = self.session.query(self.osmalchemy.way).first() self.assertEqual(len(way.nodes), 5) self.assertEqual((way.nodes[0].latitude, way.nodes[0].longitude), (51.0, 7.0)) self.assertEqual((way.nodes[1].latitude, way.nodes[1].longitude), (51.1, 7.1)) self.assertEqual((way.nodes[2].latitude, way.nodes[2].longitude), (51.2, 7.2)) self.assertEqual((way.nodes[3].latitude, way.nodes[3].longitude), (51.3, 7.3)) self.assertEqual((way.nodes[4].latitude, way.nodes[4].longitude), (51.4, 7.4)) self.assertEqual(len(way.tags), 2) self.assertEqual(way.tags[u"name"], u"Testway") self.assertEqual(way.tags[u"foo"], u"bar") self.assertEqual(len(way.nodes[2].tags), 2) self.assertEqual(way.nodes[2].tags[u"name"], u"Testampel") self.assertEqual(way.nodes[2].tags[u"foo"], u"bar") def test_create_relation_with_nodes(self): # Create way and add nodes relation = self.osmalchemy.relation() relation.members = [(self.osmalchemy.node(51.0, 7.0), u""), (self.osmalchemy.node(51.1, 7.1), u""), (self.osmalchemy.node(51.2, 7.2), u""), (self.osmalchemy.node(51.3, 7.3), u""), (self.osmalchemy.node(51.4, 7.4), u"")] # Store everything self.session.add(relation) self.session.commit() # Ensure removal from ORM self.session.remove() # Query for way and check relation = self.session.query(self.osmalchemy.relation).first() self.assertEqual((relation.members[0][0].latitude, relation.members[0][0].longitude), (51.0, 7.0)) self.assertEqual((relation.members[1][0].latitude, relation.members[1][0].longitude), (51.1, 7.1)) self.assertEqual((relation.members[2][0].latitude, relation.members[2][0].longitude), (51.2, 7.2)) self.assertEqual((relation.members[3][0].latitude, relation.members[3][0].longitude), (51.3, 7.3)) self.assertEqual((relation.members[4][0].latitude, relation.members[4][0].longitude), (51.4, 7.4)) def test_create_relation_with_nodes_and_ways(self): # Create way and add nodes and ways relation = self.osmalchemy.relation() relation.members = [(self.osmalchemy.node(51.0, 7.0), u''), (self.osmalchemy.way(), u''), (self.osmalchemy.node(51.1, 7.1), u''), (self.osmalchemy.way(), u''), (self.osmalchemy.node(51.2, 7.2), u''), (self.osmalchemy.way(), u''), (self.osmalchemy.node(51.3, 7.3), u''), (self.osmalchemy.way(), u''), (self.osmalchemy.node(51.4, 7.4), u'')] relation.members[3][0].nodes.append(relation.members[8][0]) # Store everything self.session.add(relation) self.session.commit() # Ensure removal from ORM self.session.remove() # Query for way and check relation = self.session.query(self.osmalchemy.relation).first() self.assertEqual((relation.members[0][0].latitude, relation.members[0][0].longitude), (51.0, 7.0)) self.assertEqual((relation.members[2][0].latitude, relation.members[2][0].longitude), (51.1, 7.1)) self.assertEqual((relation.members[4][0].latitude, relation.members[4][0].longitude), (51.2, 7.2)) self.assertEqual((relation.members[6][0].latitude, relation.members[6][0].longitude), (51.3, 7.3)) self.assertEqual((relation.members[8][0].latitude, relation.members[8][0].longitude), (51.4, 7.4)) self.assertIs(relation.members[3][0].nodes[0], relation.members[8][0]) def test_create_relation_with_nodes_and_ways_and_tags_everywhere(self): # Create way and add nodes and ways relation = self.osmalchemy.relation() relation.members = [(self.osmalchemy.node(51.0, 7.0), u''), (self.osmalchemy.way(), u''), (self.osmalchemy.node(51.1, 7.1), u''), (self.osmalchemy.way(), u''), (self.osmalchemy.node(51.2, 7.2), u''), (self.osmalchemy.way(), u''), (self.osmalchemy.node(51.3, 7.3), u''), (self.osmalchemy.way(), u''), (self.osmalchemy.node(51.4, 7.4), u'')] relation.tags[u"name"] = u"weirdest roads in Paris" relation.members[3][0].nodes.append(relation.members[8][0]) relation.members[7][0].tags[u"foo"] = u"bar" relation.members[7][0].tags[u"bang"] = u"baz" relation.members[8][0].tags[u"name"] = u"Doppelknoten" # Store everything self.session.add(relation) self.session.commit() # Ensure removal from ORM self.session.remove() # Query for way and check relation = self.session.query(self.osmalchemy.relation).first() self.assertEqual((relation.members[0][0].latitude, relation.members[0][0].longitude), (51.0, 7.0)) self.assertEqual((relation.members[2][0].latitude, relation.members[2][0].longitude), (51.1, 7.1)) self.assertEqual((relation.members[4][0].latitude, relation.members[4][0].longitude), (51.2, 7.2)) self.assertEqual((relation.members[6][0].latitude, relation.members[6][0].longitude), (51.3, 7.3)) self.assertEqual((relation.members[8][0].latitude, relation.members[8][0].longitude), (51.4, 7.4)) self.assertIs(relation.members[3][0].nodes[0], relation.members[8][0]) self.assertEqual(relation.tags[u"name"], u"weirdest roads in Paris") self.assertEqual(relation.members[7][0].tags[u"foo"], u"bar") self.assertEqual(relation.members[7][0].tags[u"bang"], u"baz") self.assertEqual(relation.members[8][0].tags, relation.members[3][0].nodes[0].tags) class OSMAlchemyModelTestsSQLite(OSMAlchemyModelTests, unittest.TestCase): """ Tests run with SQLite """ def setUp(self): self.engine = create_engine("sqlite:///:memory:") self.base = declarative_base(bind=self.engine) self.session = scoped_session(sessionmaker(bind=self.engine)) self.osmalchemy = OSMAlchemy((self.engine, self.base, self.session)) self.base.metadata.create_all() OSMAlchemyModelTests.setUp(self) def tearDown(self): self.session.remove() self.engine.dispose() OSMAlchemyModelTests.tearDown(self) class OSMAlchemyModelTestsPostgres(OSMAlchemyModelTests, unittest.TestCase): """ Tests run with PostgreSQL """ def setUp(self): self.postgresql = Postgresql() self.engine = create_engine(self.postgresql.url(), client_encoding="utf-8") self.base = declarative_base(bind=self.engine) self.session = scoped_session(sessionmaker(bind=self.engine)) self.osmalchemy = OSMAlchemy((self.engine, self.base, self.session)) self.base.metadata.create_all() OSMAlchemyModelTests.setUp(self) def tearDown(self): self.session.remove() self.engine.dispose() self.postgresql.stop() OSMAlchemyModelTests.tearDown(self) class OSMAlchemyModelTestsMySQL(OSMAlchemyModelTests, unittest.TestCase): """ Tests run with MySQL """ def setUp(self): self.mysql = Mysqld() self.engine = create_engine(self.mysql.url() + "?charset=utf8mb4") self.base = declarative_base(bind=self.engine) self.session = scoped_session(sessionmaker(bind=self.engine)) self.osmalchemy = OSMAlchemy((self.engine, self.base, self.session)) self.base.metadata.create_all() OSMAlchemyModelTests.setUp(self) def tearDown(self): self.session.remove() self.engine.dispose() self.mysql.stop() OSMAlchemyModelTests.tearDown(self) class OSMAlchemyModelTestsFlaskSQLAlchemy(OSMAlchemyModelTests, unittest.TestCase): """ Tests run with SQLite """ def setUp(self): app = Flask("test") db = FlaskSQLAlchemy(app) self.session = db.session self.osmalchemy = OSMAlchemy(db) db.create_all() OSMAlchemyModelTests.setUp(self) def tearDown(self): self.session.remove() OSMAlchemyModelTests.tearDown(self) # Make runnable as standalone script if __name__ == "__main__": unittest.main() OSMAlchemy-0.1.3/test/test_util_db.py0000644000175000017500000002066013000162737017055 0ustar niknik00000000000000#!/usr/bin/env python3 # ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Tests concerning OSMAlchemy database utility code. """ # Standard unit testing framework import unittest # We want to profile test cases, and other imports import time import os # Helper libraries for different database engines from testing.mysqld import MysqldFactory from testing.postgresql import PostgresqlFactory # Module to be tested from osmalchemy import OSMAlchemy # SQLAlchemy for working with model and data from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker, scoped_session # Create database engine factories to enable caching Postgresql = PostgresqlFactory(cache_initialized_db=True) Mysqld = MysqldFactory(cache_initialized_db=True) # Dictionary to store profiling information about tests profile = {} def tearDownModule(): """ Global test suite tear down code """ # Purge caches of database engines Postgresql.clear_cache() Mysqld.clear_cache() # Print profiling info print("Utility code test times") print("=======================") print("") for suite in profile: print("%8.3f s\t%s" % (sum([profile[suite][test] for test in profile[suite]]), suite)) for test in profile[suite]: print("\t%8.3f s\t%s" % (profile[suite][test], test)) class OSMAlchemyUtilDbTests(object): """ Incomplete base class for common test routines. Subclassed in engine-dependent test classes. """ def setUp(self): if not self.__class__.__name__ in profile: profile[self.__class__.__name__] = {} profile[self.__class__.__name__][self.id().split(".")[-1]] = time.time() self.base = declarative_base(bind=self.engine) self.session = scoped_session(sessionmaker(bind=self.engine)) self.osmalchemy = OSMAlchemy((self.engine, self.base, self.session)) self.base.metadata.create_all() self.datadir = os.path.join(os.path.dirname(__file__), "data") def tearDown(self): self.session.remove() self.engine.dispose() profile[self.__class__.__name__][self.id().split(".")[-1]] -= time.time() profile[self.__class__.__name__][self.id().split(".")[-1]] *= -1 def test_import_osm_file(self): # Construct path to test data file path = os.path.join(self.datadir, "schwarzrheindorf.osm") # Import data into model self.osmalchemy.import_osm_file(path) # Ensure removal of everything from ORM self.session.remove() # Check number of elements nodes = self.session.query(self.osmalchemy.node).all() ways = self.session.query(self.osmalchemy.way).all() relations = self.session.query(self.osmalchemy.relation).all() self.assertGreaterEqual(len(nodes), 7518) self.assertGreaterEqual(len(ways), 1559) self.assertGreaterEqual(len(relations), 33) # Try to retrieve some node and make checks on it haltestelle = self.session.query(self.osmalchemy.node).filter_by(id=252714572).one() # Check metadata self.assertEqual(haltestelle.id, 252714572) self.assertEqual(haltestelle.latitude, 50.7509314) self.assertEqual(haltestelle.longitude, 7.1173853) # Check tags self.assertEqual(haltestelle.tags["VRS:gemeinde"], "BONN") self.assertEqual(haltestelle.tags["VRS:ref"], "65248") self.assertEqual(haltestelle.tags["name"], "Schwarzrheindorf Kirche") # Check node on a street wittestr = self.session.query(self.osmalchemy.way).filter_by(id=23338279).one() self.assertIn(haltestelle, wittestr.nodes) # Try to retrieve some way and do checks on it doppelkirche = self.session.query(self.osmalchemy.way).filter_by(id=83296962).one() # Verify metadata self.assertEqual(doppelkirche.id, 83296962) self.assertEqual(doppelkirche.visible, True) # Verify some tags self.assertEqual(doppelkirche.tags["name"], u"St. Maria und St. Clemens Doppelkirche") self.assertEqual(doppelkirche.tags["historic"], u"church") self.assertEqual(doppelkirche.tags["wheelchair"], u"limited") self.assertEqual(doppelkirche.tags["addr:street"], u"Dixstraße") # verify nodes on way nodes = (969195704, 969195706, 1751820961, 969195708, 969195710, 969195712, 969195714, 969195719, 969195720, 969195721, 969195722, 969218813, 969218815, 969218817, 969218819, 969218820, 969218821, 969218822, 969195740, 969195742, 969195745, 969195750, 969195751, 969195752, 969195753, 1751858766, 969195754, 969195759, 969218844, 969195704) for ref in nodes: nd = self.session.query(self.osmalchemy.node).filter_by(id=ref).one() # Verify existence self.assertIn(nd, doppelkirche.nodes) # Verify ordering self.assertIs(doppelkirche.nodes[nodes.index(ref)], nd) # Cross-check other nodes are not in way for ref in (26853096, 26853100, 247056873): nd = self.session.query(self.osmalchemy.node).filter_by(id=ref).one() self.assertNotIn(nd, doppelkirche.nodes) # Try to retrieve some relation and make checks on it buslinie = self.session.query(self.osmalchemy.relation).filter_by(id=1823975).one() # Check metadata self.assertEqual(buslinie.id, 1823975) self.assertEqual(buslinie.changeset, 40638463) # Check tags self.assertEqual(buslinie.tags["name"], u"VRS 640 Siegburg") self.assertEqual(buslinie.tags["ref"], u"640") self.assertEqual(buslinie.tags["type"], u"route") self.assertEqual(buslinie.tags["route"], u"bus") # Check members self.assertIn((haltestelle, "stop"), buslinie.members) self.assertEqual(list(buslinie.members).index((haltestelle, u"stop")), 16) self.assertIn((wittestr, ""), buslinie.members) self.assertEqual(list(buslinie.members).index((wittestr, "")), 109) class OSMAlchemyUtilDbTestsSQLite(OSMAlchemyUtilDbTests, unittest.TestCase): """ Tests run with SQLite """ def setUp(self): self.engine = create_engine("sqlite:///:memory:") OSMAlchemyUtilDbTests.setUp(self) def tearDown(self): OSMAlchemyUtilDbTests.tearDown(self) class OSMAlchemyUtilDbTestsPostgres(OSMAlchemyUtilDbTests, unittest.TestCase): """ Tests run with PostgreSQL """ def setUp(self): self.postgresql = Postgresql() self.engine = create_engine(self.postgresql.url(), client_encoding="utf-8") OSMAlchemyUtilDbTests.setUp(self) def tearDown(self): OSMAlchemyUtilDbTests.tearDown(self) self.postgresql.stop() class OSMAlchemyUtilDbTestsMySQL(OSMAlchemyUtilDbTests, unittest.TestCase): """ Tests run with MySQL """ def setUp(self): self.mysql = Mysqld() self.engine = create_engine(self.mysql.url() + "?charset=utf8mb4") OSMAlchemyUtilDbTests.setUp(self) def tearDown(self): OSMAlchemyUtilDbTests.tearDown(self) self.mysql.stop() # Make runnable as standalone script if __name__ == "__main__": unittest.main() OSMAlchemy-0.1.3/test/test_util_online.py0000644000175000017500000002515013042150177017754 0ustar niknik00000000000000#!/usr/bin/env python3 # ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016, 2017 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. """ Tests concerning OSMAlchemy online utility code. """ # Standard unit testing framework import unittest class OSMAlchemyUtilOnlineSQLToOverpassQLTests(unittest.TestCase): """ Tests for SQL to overpass conversion """ def test_trees_to_overpassql_bbox_node(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("&&", [(">", "latitude", 51.0), ("<", "latitude", 52.0), (">", "longitude", 7.0), ("<", "longitude", 8.0)])} # Expected result regex expected = r"^node\(51\.0000000,7\.0000000,52\.0000000,8\.0000000\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_bbox_node_le_ge(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("&&", [(">=", "latitude", 51.0), ("<=", "latitude", 52.0), (">=", "longitude", 7.0), ("<=", "longitude", 8.0)])} # Expected result regex expected = r"^node\(51\.0000000,7\.0000000,52\.0000000,8\.0000000\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_bbox_node_missing_east(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("&&", [(">", "latitude", 51.0), ("<", "latitude", 52.0), (">", "longitude", 7.0)])} # Expected result regex expected = r"^node\(51\.0000000,7\.0000000,52\.0000000,180\.0000000\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_bbox_node_missing_west(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("&&", [(">", "latitude", 51.0), ("<", "latitude", 52.0), ("<", "longitude", 8.0)])} # Expected result regex expected = r"^node\(51\.0000000,-180\.0000000,52\.0000000,8\.0000000\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_bbox_node_missing_north(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("&&", [(">", "latitude", 51.0), (">", "longitude", 7.0), ("<", "longitude", 8.0)])} # Expected result regex expected = r"^node\(51\.0000000,7\.0000000,90\.0000000,8\.0000000\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_bbox_node_missing_south(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("&&", [("<", "latitude", 52.0), (">", "longitude", 7.0), ("<", "longitude", 8.0)])} # Expected result regex expected = r"^node\(-90\.0000000,7\.0000000,52\.0000000,8\.0000000\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_id_node(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("==", "id", 1145698)} # Expected result regex expected = r"^node\(1145698\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_id_node_invalid(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": (">", "id", 1145698)} # Expect it to throw an exception with self.assertRaises(ValueError): # Run transformation res = _trees_to_overpassql(tree) def test_trees_to_overpassql_two_tags_node(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("&&", [("==", "amenity", "pub"), ("==", "name", "Bruchbude")])} # Expected result regex expected = r"^node\[\"amenity\"=\"pub\"\]\[\"name\"=\"Bruchbude\"\]->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_any_of_two_tags_node(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("||", [("==", "amenity", "pub"), ("==", "name", "Bruchbude")])} # Expected result regex expected = r"^node\[\"amenity\"=\"pub\"\]->\.s[0-9]+;node\[\"name\"=\"Bruchbude\"\]->\.s[0-9]+;\(\.s[0-9]+;\.s[0-9]+;\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_trees_to_overpassql_two_tags_in_bbox_node(self): # Import function to test from osmalchemy.util.online import _trees_to_overpassql # Test input tree = {"OSMNode": ("&&", [("==", "amenity", "pub"), ("==", "name", "Bruchbude"), (">", "latitude", 51.0), ("<", "latitude", 52.0), (">", "longitude", 7.0), ("<", "longitude", 8.0)])} # Expected result regex expected = r"^node\[\"amenity\"=\"pub\"\]\[\"name\"=\"Bruchbude\"\]\(51\.0000000,7\.0000000,52\.0000000,8\.0000000\)->\.s[0-9]+;\(\.s[0-9]+;\);$" # Run transformation res = _trees_to_overpassql(tree) # Check result self.assertRegexpMatches(res, expected) def test_normalise_overpassql(self): # Import function to test from osmalchemy.util.online import _normalise_overpassql # Test string oql = 'node(50.0,6.0,51.0,8.0)->.s1875312;node["name"="Schwarzreindorf Kirche"]->.s95682773;(.s1875312; .s95682773)->.s173859;.s173859 out meta;' # Expected result normalised_oql = 'node(50.0,6.0,51.0,8.0)->.s;node["name"="Schwarzreindorf Kirche"]->.s;(.s; .s)->.s;.s out meta;' # Run function res = _normalise_overpassql(oql) # Check result self.assertEqual(res, normalised_oql) def test_normalise_overpassql_with_catchy_string(self): # Import function to test from osmalchemy.util.online import _normalise_overpassql # Test string oql = 'node(50.0,6.0,51.0,8.0)->.s1875312;node["name"="Whatever.s192837465"]->.s95682773;(.s1875312; .s95682773)->.s173859;.s173859 out meta;' # Expected result normalised_oql = 'node(50.0,6.0,51.0,8.0)->.s;node["name"="Whatever.s192837465"]->.s;(.s; .s)->.s;.s out meta;' # Run function res = _normalise_overpassql(oql) # Check result self.assertEqual(res, normalised_oql) def test_escape_tag_nothing(self): # Import function to test from osmalchemy.util.online import _escape_tag # Test string tag = "foobar" # Expected result escaped_tag = "foobar" # Run function res = _escape_tag(tag) # Check result self.assertEqual(res, escaped_tag) def test_escape_tag_quote(self): # Import function to test from osmalchemy.util.online import _escape_tag # Test string tag = "foo\"bar" # Expected result escaped_tag = "foo\\\"bar" # Run function res = _escape_tag(tag) # Check result self.assertEqual(res, escaped_tag) def test_escape_tag_backslash(self): # Import function to test from osmalchemy.util.online import _escape_tag # Test string tag = "foo\\bar" # Expected result escaped_tag = "foo\\\\bar" # Run function res = _escape_tag(tag) # Check result self.assertEqual(res, escaped_tag) def test_escape_tag_quote_backslash(self): # Import function to test from osmalchemy.util.online import _escape_tag # Test string tag = "foo\\\"bar" # Expected result escaped_tag = "foo\\\\\\\"bar" # Run function res = _escape_tag(tag) # Check result self.assertEqual(res, escaped_tag) # Make runnable as standalone script if __name__ == "__main__": unittest.main() OSMAlchemy-0.1.3/AUTHORS0000644000175000017500000000017313042143155014107 0ustar niknik00000000000000Dominik George Eike „Eckehardt“ Tim Jesinghaus mirabilos OSMAlchemy-0.1.3/CHANGELOG0000644000175000017500000000145213042146630014253 0ustar niknik00000000000000Changelog for OSMAlchemy ======================== 0.1.3 ----- * Fix float formatting (avoid scientific notation) for bounding boxes. + Found by mirabilos during Veripeditus development. * Fix support for <= and >= operators in bounding box query. * Fix escaping of tag queries. 0.1.2 ----- * Ensure working with utf-8 everywhere + Fixes execution under non-utf-8 locales 0.1.1.post2 ----------- * Fix test discovery 0.1.1.post1 ----------- * Include test data * Include changelog in sdist * Fix encoding bug in setup.py * Fix major bug in OSM data importing * Fix test suite execution 0.1.post1 --------- * Changes for PyPI and other tools + Convert README to reStructuredText + Move tests/ to test/ * Add credits to README * Clean up some code 0.1 --- * Initial release OSMAlchemy-0.1.3/LICENSE0000644000175000017500000000347512777706374014102 0ustar niknik00000000000000OSMAlchemy - OpenStreetMap to SQLAlchemy bridge Copyright (c) 2016 Dominik George Copyright (c) 2016 Eike Tim Jesinghaus Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. Alternatively, you are free to use OSMAlchemy under Simplified BSD, The MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python itself. Exception for any OpenStreetMap data in this source repository, e.g. test data: © OpenStreetMap contributors, licensed under the Open Data Commons Open Database License (ODbL). You are free to copy, distribute, transmit and adapt our data, as long as you credit OpenStreetMap and its contributors. If you alter or build upon our data, you may distribute the result only under the same licence. The full legal code explains your rights and responsibilities. You can find the full legal code in the ODBL file. OSMAlchemy-0.1.3/MANIFEST.in0000644000175000017500000000017713002701307014574 0ustar niknik00000000000000include LICENSE include ODBL include CHANGELOG include AUTHORS include test/__init__.py include test/data/schwarzrheindorf.osm OSMAlchemy-0.1.3/ODBL0000644000175000017500000006134112777707027013530 0ustar niknik00000000000000## ODC Open Database License (ODbL) ### Preamble The Open Database License (ODbL) is a license agreement intended to allow users to freely share, modify, and use this Database while maintaining this same freedom for others. Many databases are covered by copyright, and therefore this document licenses these rights. Some jurisdictions, mainly in the European Union, have specific rights that cover databases, and so the ODbL addresses these rights, too. Finally, the ODbL is also an agreement in contract for users of this Database to act in certain ways in return for accessing this Database. Databases can contain a wide variety of types of content (images, audiovisual material, and sounds all in the same database, for example), and so the ODbL only governs the rights over the Database, and not the contents of the Database individually. Licensors should use the ODbL together with another license for the contents, if the contents have a single set of rights that uniformly covers all of the contents. If the contents have multiple sets of different rights, Licensors should describe what rights govern what contents together in the individual record or in some other way that clarifies what rights apply. Sometimes the contents of a database, or the database itself, can be covered by other rights not addressed here (such as private contracts, trade mark over the name, or privacy rights / data protection rights over information in the contents), and so you are advised that you may have to consult other documents or clear other rights before doing activities not covered by this License. ------ The Licensor (as defined below) and You (as defined below) agree as follows: ### 1.0 Definitions of Capitalised Words "Collective Database" – Means this Database in unmodified form as part of a collection of independent databases in themselves that together are assembled into a collective whole. A work that constitutes a Collective Database will not be considered a Derivative Database. "Convey" – As a verb, means Using the Database, a Derivative Database, or the Database as part of a Collective Database in any way that enables a Person to make or receive copies of the Database or a Derivative Database. Conveying does not include interaction with a user through a computer network, or creating and Using a Produced Work, where no transfer of a copy of the Database or a Derivative Database occurs. "Contents" – The contents of this Database, which includes the information, independent works, or other material collected into the Database. For example, the contents of the Database could be factual data or works such as images, audiovisual material, text, or sounds. "Database" – A collection of material (the Contents) arranged in a systematic or methodical way and individually accessible by electronic or other means offered under the terms of this License. "Database Directive" – Means Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended or succeeded. "Database Right" – Means rights resulting from the Chapter III ("sui generis") rights in the Database Directive (as amended and as transposed by member states), which includes the Extraction and Re-utilisation of the whole or a Substantial part of the Contents, as well as any similar rights available in the relevant jurisdiction under Section 10.4. "Derivative Database" – Means a database based upon the Database, and includes any translation, adaptation, arrangement, modification, or any other alteration of the Database or of a Substantial part of the Contents. This includes, but is not limited to, Extracting or Re-utilising the whole or a Substantial part of the Contents in a new Database. "Extraction" – Means the permanent or temporary transfer of all or a Substantial part of the Contents to another medium by any means or in any form. "License" – Means this license agreement and is both a license of rights such as copyright and Database Rights and an agreement in contract. "Licensor" – Means the Person that offers the Database under the terms of this License. "Person" – Means a natural or legal person or a body of persons corporate or incorporate. "Produced Work" – a work (such as an image, audiovisual material, text, or sounds) resulting from using the whole or a Substantial part of the Contents (via a search or other query) from this Database, a Derivative Database, or this Database as part of a Collective Database. "Publicly" – means to Persons other than You or under Your control by either more than 50% ownership or by the power to direct their activities (such as contracting with an independent consultant). "Re-utilisation" – means any form of making available to the public all or a Substantial part of the Contents by the distribution of copies, by renting, by online or other forms of transmission. "Substantial" – Means substantial in terms of quantity or quality or a combination of both. The repeated and systematic Extraction or Re-utilisation of insubstantial parts of the Contents may amount to the Extraction or Re-utilisation of a Substantial part of the Contents. "Use" – As a verb, means doing any act that is restricted by copyright or Database Rights whether in the original medium or any other; and includes without limitation distributing, copying, publicly performing, publicly displaying, and preparing derivative works of the Database, as well as modifying the Database as may be technically necessary to use it in a different mode or format. "You" – Means a Person exercising rights under this License who has not previously violated the terms of this License with respect to the Database, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation. Words in the singular include the plural and vice versa. ### 2.0 What this License covers 2.1. Legal effect of this document. This License is: a. A license of applicable copyright and neighbouring rights; b. A license of the Database Right; and c. An agreement in contract between You and the Licensor. 2.2 Legal rights covered. This License covers the legal rights in the Database, including: a. Copyright. Any copyright or neighbouring rights in the Database. The copyright licensed includes any individual elements of the Database, but does not cover the copyright over the Contents independent of this Database. See Section 2.4 for details. Copyright law varies between jurisdictions, but is likely to cover: the Database model or schema, which is the structure, arrangement, and organisation of the Database, and can also include the Database tables and table indexes; the data entry and output sheets; and the Field names of Contents stored in the Database; b. Database Rights. Database Rights only extend to the Extraction and Re-utilisation of the whole or a Substantial part of the Contents. Database Rights can apply even when there is no copyright over the Database. Database Rights can also apply when the Contents are removed from the Database and are selected and arranged in a way that would not infringe any applicable copyright; and c. Contract. This is an agreement between You and the Licensor for access to the Database. In return you agree to certain conditions of use on this access as outlined in this License. 2.3 Rights not covered. a. This License does not apply to computer programs used in the making or operation of the Database; b. This License does not cover any patents over the Contents or the Database; and c. This License does not cover any trademarks associated with the Database. 2.4 Relationship to Contents in the Database. The individual items of the Contents contained in this Database may be covered by other rights, including copyright, patent, data protection, privacy, or personality rights, and this License does not cover any rights (other than Database Rights or in contract) in individual Contents contained in the Database. For example, if used on a Database of images (the Contents), this License would not apply to copyright over individual images, which could have their own separate licenses, or one single license covering all of the rights over the images. ### 3.0 Rights granted 3.1 Subject to the terms and conditions of this License, the Licensor grants to You a worldwide, royalty-free, non-exclusive, terminable (but only under Section 9) license to Use the Database for the duration of any applicable copyright and Database Rights. These rights explicitly include commercial use, and do not exclude any field of endeavour. To the extent possible in the relevant jurisdiction, these rights may be exercised in all media and formats whether now known or created in the future. The rights granted cover, for example: a. Extraction and Re-utilisation of the whole or a Substantial part of the Contents; b. Creation of Derivative Databases; c. Creation of Collective Databases; d. Creation of temporary or permanent reproductions by any means and in any form, in whole or in part, including of any Derivative Databases or as a part of Collective Databases; and e. Distribution, communication, display, lending, making available, or performance to the public by any means and in any form, in whole or in part, including of any Derivative Database or as a part of Collective Databases. 3.2 Compulsory license schemes. For the avoidance of doubt: a. Non-waivable compulsory license schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme cannot be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; b. Waivable compulsory license schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme can be waived, the Licensor waives the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; and, c. Voluntary license schemes. The Licensor waives the right to collect royalties, whether individually or, in the event that the Licensor is a member of a collecting society that administers voluntary licensing schemes, via that society, from any exercise by You of the rights granted under this License. 3.3 The right to release the Database under different terms, or to stop distributing or making available the Database, is reserved. Note that this Database may be multiple-licensed, and so You may have the choice of using alternative licenses for this Database. Subject to Section 10.4, all other rights not expressly granted by Licensor are reserved. ### 4.0 Conditions of Use 4.1 The rights granted in Section 3 above are expressly made subject to Your complying with the following conditions of use. These are important conditions of this License, and if You fail to follow them, You will be in material breach of its terms. 4.2 Notices. If You Publicly Convey this Database, any Derivative Database, or the Database as part of a Collective Database, then You must: a. Do so only under the terms of this License or another license permitted under Section 4.4; b. Include a copy of this License (or, as applicable, a license permitted under Section 4.4) or its Uniform Resource Identifier (URI) with the Database or Derivative Database, including both in the Database or Derivative Database and in any relevant documentation; and c. Keep intact any copyright or Database Right notices and notices that refer to this License. d. If it is not possible to put the required notices in a particular file due to its structure, then You must include the notices in a location (such as a relevant directory) where users would be likely to look for it. 4.3 Notice for using output (Contents). Creating and Using a Produced Work does not require the notice in Section 4.2. However, if you Publicly Use a Produced Work, You must include a notice associated with the Produced Work reasonably calculated to make any Person that uses, views, accesses, interacts with, or is otherwise exposed to the Produced Work aware that Content was obtained from the Database, Derivative Database, or the Database as part of a Collective Database, and that it is available under this License. a. Example notice. The following text will satisfy notice under Section 4.3: Contains information from DATABASE NAME, which is made available here under the Open Database License (ODbL). DATABASE NAME should be replaced with the name of the Database and a hyperlink to the URI of the Database. "Open Database License" should contain a hyperlink to the URI of the text of this License. If hyperlinks are not possible, You should include the plain text of the required URI's with the above notice. 4.4 Share alike. a. Any Derivative Database that You Publicly Use must be only under the terms of: i. This License; ii. A later version of this License similar in spirit to this License; or iii. A compatible license. If You license the Derivative Database under one of the licenses mentioned in (iii), You must comply with the terms of that license. b. For the avoidance of doubt, Extraction or Re-utilisation of the whole or a Substantial part of the Contents into a new database is a Derivative Database and must comply with Section 4.4. c. Derivative Databases and Produced Works. A Derivative Database is Publicly Used and so must comply with Section 4.4. if a Produced Work created from the Derivative Database is Publicly Used. d. Share Alike and additional Contents. For the avoidance of doubt, You must not add Contents to Derivative Databases under Section 4.4 a that are incompatible with the rights granted under this License. e. Compatible licenses. Licensors may authorise a proxy to determine compatible licenses under Section 4.4 a iii. If they do so, the authorised proxy's public statement of acceptance of a compatible license grants You permission to use the compatible license. 4.5 Limits of Share Alike. The requirements of Section 4.4 do not apply in the following: a. For the avoidance of doubt, You are not required to license Collective Databases under this License if You incorporate this Database or a Derivative Database in the collection, but this License still applies to this Database or a Derivative Database as a part of the Collective Database; b. Using this Database, a Derivative Database, or this Database as part of a Collective Database to create a Produced Work does not create a Derivative Database for purposes of Section 4.4; and c. Use of a Derivative Database internally within an organisation is not to the public and therefore does not fall under the requirements of Section 4.4. 4.6 Access to Derivative Databases. If You Publicly Use a Derivative Database or a Produced Work from a Derivative Database, You must also offer to recipients of the Derivative Database or Produced Work a copy in a machine readable form of: a. The entire Derivative Database; or b. A file containing all of the alterations made to the Database or the method of making the alterations to the Database (such as an algorithm), including any additional Contents, that make up all the differences between the Database and the Derivative Database. The Derivative Database (under a.) or alteration file (under b.) must be available at no more than a reasonable production cost for physical distributions and free of charge if distributed over the internet. 4.7 Technological measures and additional terms a. This License does not allow You to impose (except subject to Section 4.7 b.) any terms or any technological measures on the Database, a Derivative Database, or the whole or a Substantial part of the Contents that alter or restrict the terms of this License, or any rights granted under it, or have the effect or intent of restricting the ability of any person to exercise those rights. b. Parallel distribution. You may impose terms or technological measures on the Database, a Derivative Database, or the whole or a Substantial part of the Contents (a "Restricted Database") in contravention of Section 4.74 a. only if You also make a copy of the Database or a Derivative Database available to the recipient of the Restricted Database: i. That is available without additional fee; ii. That is available in a medium that does not alter or restrict the terms of this License, or any rights granted under it, or have the effect or intent of restricting the ability of any person to exercise those rights (an "Unrestricted Database"); and iii. The Unrestricted Database is at least as accessible to the recipient as a practical matter as the Restricted Database. c. For the avoidance of doubt, You may place this Database or a Derivative Database in an authenticated environment, behind a password, or within a similar access control scheme provided that You do not alter or restrict the terms of this License or any rights granted under it or have the effect or intent of restricting the ability of any person to exercise those rights. 4.8 Licensing of others. You may not sublicense the Database. Each time You communicate the Database, the whole or Substantial part of the Contents, or any Derivative Database to anyone else in any way, the Licensor offers to the recipient a license to the Database on the same terms and conditions as this License. You are not responsible for enforcing compliance by third parties with this License, but You may enforce any rights that You have over a Derivative Database. You are solely responsible for any modifications of a Derivative Database made by You or another Person at Your direction. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. ### 5.0 Moral rights 5.1 Moral rights. This section covers moral rights, including any rights to be identified as the author of the Database or to object to treatment that would otherwise prejudice the author's honour and reputation, or any other derogatory treatment: a. For jurisdictions allowing waiver of moral rights, Licensor waives all moral rights that Licensor may have in the Database to the fullest extent possible by the law of the relevant jurisdiction under Section 10.4; b. If waiver of moral rights under Section 5.1 a in the relevant jurisdiction is not possible, Licensor agrees not to assert any moral rights over the Database and waives all claims in moral rights to the fullest extent possible by the law of the relevant jurisdiction under Section 10.4; and c. For jurisdictions not allowing waiver or an agreement not to assert moral rights under Section 5.1 a and b, the author may retain their moral rights over certain aspects of the Database. Please note that some jurisdictions do not allow for the waiver of moral rights, and so moral rights may still subsist over the Database in some jurisdictions. ### 6.0 Fair dealing, Database exceptions, and other rights not affected 6.1 This License does not affect any rights that You or anyone else may independently have under any applicable law to make any use of this Database, including without limitation: a. Exceptions to the Database Right including: Extraction of Contents from non-electronic Databases for private purposes, Extraction for purposes of illustration for teaching or scientific research, and Extraction or Re-utilisation for public security or an administrative or judicial procedure. b. Fair dealing, fair use, or any other legally recognised limitation or exception to infringement of copyright or other applicable laws. 6.2 This License does not affect any rights of lawful users to Extract and Re-utilise insubstantial parts of the Contents, evaluated quantitatively or qualitatively, for any purposes whatsoever, including creating a Derivative Database (subject to other rights over the Contents, see Section 2.4). The repeated and systematic Extraction or Re-utilisation of insubstantial parts of the Contents may however amount to the Extraction or Re-utilisation of a Substantial part of the Contents. ### 7.0 Warranties and Disclaimer 7.1 The Database is licensed by the Licensor "as is" and without any warranty of any kind, either express, implied, or arising by statute, custom, course of dealing, or trade usage. Licensor specifically disclaims any and all implied warranties or conditions of title, non-infringement, accuracy or completeness, the presence or absence of errors, fitness for a particular purpose, merchantability, or otherwise. Some jurisdictions do not allow the exclusion of implied warranties, so this exclusion may not apply to You. ### 8.0 Limitation of liability 8.1 Subject to any liability that may not be excluded or limited by law, the Licensor is not liable for, and expressly excludes, all liability for loss or damage however and whenever caused to anyone by any use under this License, whether by You or by anyone else, and whether caused by any fault on the part of the Licensor or not. This exclusion of liability includes, but is not limited to, any special, incidental, consequential, punitive, or exemplary damages such as loss of revenue, data, anticipated profits, and lost business. This exclusion applies even if the Licensor has been advised of the possibility of such damages. 8.2 If liability may not be excluded by law, it is limited to actual and direct financial loss to the extent it is caused by proved negligence on the part of the Licensor. ### 9.0 Termination of Your rights under this License 9.1 Any breach by You of the terms and conditions of this License automatically terminates this License with immediate effect and without notice to You. For the avoidance of doubt, Persons who have received the Database, the whole or a Substantial part of the Contents, Derivative Databases, or the Database as part of a Collective Database from You under this License will not have their licenses terminated provided their use is in full compliance with this License or a license granted under Section 4.8 of this License. Sections 1, 2, 7, 8, 9 and 10 will survive any termination of this License. 9.2 If You are not in breach of the terms of this License, the Licensor will not terminate Your rights under it. 9.3 Unless terminated under Section 9.1, this License is granted to You for the duration of applicable rights in the Database. 9.4 Reinstatement of rights. If you cease any breach of the terms and conditions of this License, then your full rights under this License will be reinstated: a. Provisionally and subject to permanent termination until the 60th day after cessation of breach; b. Permanently on the 60th day after cessation of breach unless otherwise reasonably notified by the Licensor; or c. Permanently if reasonably notified by the Licensor of the violation, this is the first time You have received notice of violation of this License from the Licensor, and You cure the violation prior to 30 days after your receipt of the notice. Persons subject to permanent termination of rights are not eligible to be a recipient and receive a license under Section 4.8. 9.5 Notwithstanding the above, Licensor reserves the right to release the Database under different license terms or to stop distributing or making available the Database. Releasing the Database under different license terms or stopping the distribution of the Database will not withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above. ### 10.0 General 10.1 If any provision of this License is held to be invalid or unenforceable, that must not affect the validity or enforceability of the remainder of the terms and conditions of this License and each remaining provision of this License shall be valid and enforced to the fullest extent permitted by law. 10.2 This License is the entire agreement between the parties with respect to the rights granted here over the Database. It replaces any earlier understandings, agreements or representations with respect to the Database. 10.3 If You are in breach of the terms of this License, You will not be entitled to rely on the terms of this License or to complain of any breach by the Licensor. 10.4 Choice of law. This License takes effect in and will be governed by the laws of the relevant jurisdiction in which the License terms are sought to be enforced. If the standard suite of rights granted under applicable copyright law and Database Rights in the relevant jurisdiction includes additional rights not granted under this License, these additional rights are granted in this License in order to meet the terms of this License. OSMAlchemy-0.1.3/README.rst0000644000175000017500000001470712766745520014556 0ustar niknik00000000000000OSMAlchemy ========== OSMAlchemy is a bridge between SQLAlchemy and the OpenStreetMap API. Goals ----- OSMAlchemy's goal is to provide completely transparent integration of the real-world OpenStreetMap data within projects using SQLAlchemy. It provides two things: 1. Model declaratives resembling the structure of the main OpenStreetMap database, with some limitations, usable wherever SQLAlchemy is used, and 2. Transparent proxying and data-fetching from OpenStreetMap data. The idea is that the model can be queried using SQLAlchemy, and OSMAlchemy will either satisfy the query from the database directly or fetch data from OpenStreetMap. That way, projects already using SQLAlchemy do not need another database framework to use OpenStreetMap data, and the necessity to keep a local copy of planet.osm is relaxed. If, for example, a node with a certain id is queried, OSMAlchemy will… - …try to get the node from the database/ORM directly, then… - …if it is available, check its caching age, and… - …if it is too old, refresh it from OSM, or… - …else, fetch it from OSM, and… - …finally create a real, ORM-mapped database object. That's the rough idea, and it counts for all kinds of OSM elements and queries. OSMAlchemy uses Overpass to satisfy complex queries. Non-goals ~~~~~~~~~ OSMAlchemy does not aim to replace large-scale OSM data frameworks like PostGIS, Osmosis or whatever. In fact, in terms of performance and otherwise, it cannot keep up with them. If you are running a huge project that handles massive amounts of map data, has millions of requests or users, then OSMAlchemy is not for you (YMMV). OSMAlchemy fills a niche for projects that have limited resources and cannot handle a full copy of planet.osm and an own API backend and expect to handle limited amounts of map data. It might, however, be cool to use OSMAlchemy as ORM proxy with an own API backend. Who knows? It might, as well, turn out that OSMAlchemy is an incredibly silly idea under all circumstances. Usage ----- Here are a few tiny examples of how to basically use OSMAlchemy: Installation ~~~~~~~~~~~~ OSMAlchemy can be installed just like any other standard Python package by one of… .. code-block:: console # pip3 install OSMAlchemy # python3 setup.py install …or what ever kind of distribution and install system you prefer. Using plain SQLAlchemy ~~~~~~~~~~~~~~~~~~~~~~ Make sure to get at least an engine from SQLAlchemy. Even better, get a declarative base and a scoped session: .. code-block:: python from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker, scoped_session engine = create_engine("sqlite:////tmp/foo.db") base = declarative_base(bind=engine) session = scoped_session(sessionmaker(bind=engine)) You can then initialise OSMAlchemy like so: .. code-block:: python osmalchemy = OSMAlchemy((engine, base, session), overpass=True) And probably install the databases: .. code-block:: python base.metadata.create_all() Using Flask-SQLAlchemy and Flask-Restless ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Imagine you have an SQLAlchemy object from Flask-SQLAlchemy bound to your Flask application. called db, and a Flask-Restless API manager as manager: .. code-block:: python from osmalchemy import OSMAlchemy osm = OSMAlchemy(db, overpass=True) db.create_all() osm.create_api(manager) You should now magically be able to query OSM via the REST API. Keep in mind that, with no filter provided, OSMAlchemy refuses to do automatic updates from Overpass. However, providing a query in the default JSON query way in Flask-Restless will give you live data and cache it in the database. Limitations ~~~~~~~~~~~ Only some basic SQL queries are supported by the online update code. This is because compiling SQLAlchemy's queries to OverpassQL is very complex. If you are very good at algorithms and building compilers, feel free to help us out! The following kinds of queries are fully supported: .. code-block:: python # A node with a specific id session.query(osmalchemy.node).filter_by(id=12345).one() # All nodes within a bounding box session.query(osmalchemy.node).filter( and_(latitude>51.0, latitude<51.1, longitude>7.0, longitude<7.1) ).all() # All nodes having a specific tag session.query(osmalchemy.node).filter( osmalchemy.node.tags.any(key="name", value="Schwarzrheindorf Kirche") ).all() You can go mad combining the two with and\_() and or\_(). You can also query for tags of ways and relations and for ways and relations by id. Not supported (yet) are queries for ways or relations by coordinates. You also cannot query for nodes related to a way or anything related to a relation - having a way or a relation, accessing it will, however, magically pull and update the nodes and members and add them to the database: .. code-block:: python # Get all nodes that are members of a (unique) named way session.query(osmalchemy.way).filter( osmalchemy.way.tags.any(key="name", value="My Unique Way") ).one().nodes This should, in reality, cover most use cases. If you encounter a use case that is not supported, please open an issue asking whether it can be supported (if you have an idea how it can be, please add it or even implement it and open a pull request). Projects using OSMAlchemy ~~~~~~~~~~~~~~~~~~~~~~~~~ OSMAlchemy was designed for use in the Veripeditus Augmented Reality framework. Development and standards ------------------------- Albeit taking the above into account, OSMAlchemy is developed with quality and good support in mind. That means code shall be well-tested and well-documented. OSMAlchemy is tested against the following SQLAlchemy backends: - SQLite - PostgreSQL - MySQL However, we recommend PostgreSQL. MySQL acts strangely with some data and is incredibly slow, and SQLite just doesn't scale too well (however, it is incredibly fast, in comparison). Authors and credits ------------------- :Authors: Dominik George, Eike Tim Jesinghaus :Credits: Special thanks to Mike Bayer from SQLAlchemy for his help with some SQLAlchemy bugs and pitfalls, and also some heads-up. :Contact: E-mail to osmalchemy@veripeditus.org License ------- OSMAlchemy is licensed under the MIT license. Alternatively, you are free to use OSMAlchemy under Simplified BSD, The MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python itself. OSMAlchemy-0.1.3/setup.py0000644000175000017500000000723312770753106014567 0ustar niknik00000000000000#!/usr/bin/env python3 # ~*~ coding: utf-8 ~*~ #- # OSMAlchemy - OpenStreetMap to SQLAlchemy bridge # Copyright (c) 2016 Dominik George # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Alternatively, you are free to use OSMAlchemy under Simplified BSD, The # MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python # itself. import importlib.util import os from setuptools import setup # Get some information for the setup MYDIR = os.path.dirname(__file__) # Find the version string from the __init__ file def find_version(package_name): filename = os.path.join(MYDIR, package_name, "__init__.py") with open(filename, "r") as file: for line in file.readlines(): if line.startswith("__version__"): version = line.split('"')[1] return version setup( # Basic information name = 'OSMAlchemy', version = find_version("osmalchemy"), keywords = ['osm', 'openstreetmap', 'proxy', 'caching', 'orm'], description = 'OpenStreetMap to SQLAlchemy bridge', long_description = open(os.path.join(MYDIR, "README.rst"), "r", encoding="utf-8").read(), url = 'https://github.com/Veripeditus/OSMAlchemy', # Author information author = 'Dominik George, Eike Tim Jesinghaus', author_email = 'osmalchemy@veripeditus.org', # Included code packages = ["osmalchemy", "osmalchemy.util"], # Distribution information zip_safe = True, install_requires = [ 'SQLAlchemy>=1.0.0', 'python-dateutil', 'overpass' ], tests_require = [ 'SQLAlchemy>=1.0.0', 'python-dateutil', 'overpass', 'psycopg2', 'Flask>=0.10', 'Flask-SQLAlchemy', 'testing.postgresql', 'testing.mysqld' ], extras_require = { 'Flask': [ 'Flask>=0.10', 'Flask-SQLAlchemy' ] }, test_suite = 'test', classifiers = [ 'Development Status :: 3 - Alpha', 'Environment :: Plugins', 'Intended Audience :: Developers', 'License :: OSI Approved :: MIT License', 'Programming Language :: Python :: 3 :: Only', 'Topic :: Database', 'Topic :: Scientific/Engineering :: GIS', 'Topic :: Software Development :: Libraries :: Python Modules' ] ) OSMAlchemy-0.1.3/PKG-INFO0000644000175000017500000002150213042151631014131 0ustar niknik00000000000000Metadata-Version: 1.1 Name: OSMAlchemy Version: 0.1.3 Summary: OpenStreetMap to SQLAlchemy bridge Home-page: https://github.com/Veripeditus/OSMAlchemy Author: Dominik George, Eike Tim Jesinghaus Author-email: osmalchemy@veripeditus.org License: UNKNOWN Description: OSMAlchemy ========== OSMAlchemy is a bridge between SQLAlchemy and the OpenStreetMap API. Goals ----- OSMAlchemy's goal is to provide completely transparent integration of the real-world OpenStreetMap data within projects using SQLAlchemy. It provides two things: 1. Model declaratives resembling the structure of the main OpenStreetMap database, with some limitations, usable wherever SQLAlchemy is used, and 2. Transparent proxying and data-fetching from OpenStreetMap data. The idea is that the model can be queried using SQLAlchemy, and OSMAlchemy will either satisfy the query from the database directly or fetch data from OpenStreetMap. That way, projects already using SQLAlchemy do not need another database framework to use OpenStreetMap data, and the necessity to keep a local copy of planet.osm is relaxed. If, for example, a node with a certain id is queried, OSMAlchemy will… - …try to get the node from the database/ORM directly, then… - …if it is available, check its caching age, and… - …if it is too old, refresh it from OSM, or… - …else, fetch it from OSM, and… - …finally create a real, ORM-mapped database object. That's the rough idea, and it counts for all kinds of OSM elements and queries. OSMAlchemy uses Overpass to satisfy complex queries. Non-goals ~~~~~~~~~ OSMAlchemy does not aim to replace large-scale OSM data frameworks like PostGIS, Osmosis or whatever. In fact, in terms of performance and otherwise, it cannot keep up with them. If you are running a huge project that handles massive amounts of map data, has millions of requests or users, then OSMAlchemy is not for you (YMMV). OSMAlchemy fills a niche for projects that have limited resources and cannot handle a full copy of planet.osm and an own API backend and expect to handle limited amounts of map data. It might, however, be cool to use OSMAlchemy as ORM proxy with an own API backend. Who knows? It might, as well, turn out that OSMAlchemy is an incredibly silly idea under all circumstances. Usage ----- Here are a few tiny examples of how to basically use OSMAlchemy: Installation ~~~~~~~~~~~~ OSMAlchemy can be installed just like any other standard Python package by one of… .. code-block:: console # pip3 install OSMAlchemy # python3 setup.py install …or what ever kind of distribution and install system you prefer. Using plain SQLAlchemy ~~~~~~~~~~~~~~~~~~~~~~ Make sure to get at least an engine from SQLAlchemy. Even better, get a declarative base and a scoped session: .. code-block:: python from sqlalchemy import create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker, scoped_session engine = create_engine("sqlite:////tmp/foo.db") base = declarative_base(bind=engine) session = scoped_session(sessionmaker(bind=engine)) You can then initialise OSMAlchemy like so: .. code-block:: python osmalchemy = OSMAlchemy((engine, base, session), overpass=True) And probably install the databases: .. code-block:: python base.metadata.create_all() Using Flask-SQLAlchemy and Flask-Restless ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Imagine you have an SQLAlchemy object from Flask-SQLAlchemy bound to your Flask application. called db, and a Flask-Restless API manager as manager: .. code-block:: python from osmalchemy import OSMAlchemy osm = OSMAlchemy(db, overpass=True) db.create_all() osm.create_api(manager) You should now magically be able to query OSM via the REST API. Keep in mind that, with no filter provided, OSMAlchemy refuses to do automatic updates from Overpass. However, providing a query in the default JSON query way in Flask-Restless will give you live data and cache it in the database. Limitations ~~~~~~~~~~~ Only some basic SQL queries are supported by the online update code. This is because compiling SQLAlchemy's queries to OverpassQL is very complex. If you are very good at algorithms and building compilers, feel free to help us out! The following kinds of queries are fully supported: .. code-block:: python # A node with a specific id session.query(osmalchemy.node).filter_by(id=12345).one() # All nodes within a bounding box session.query(osmalchemy.node).filter( and_(latitude>51.0, latitude<51.1, longitude>7.0, longitude<7.1) ).all() # All nodes having a specific tag session.query(osmalchemy.node).filter( osmalchemy.node.tags.any(key="name", value="Schwarzrheindorf Kirche") ).all() You can go mad combining the two with and\_() and or\_(). You can also query for tags of ways and relations and for ways and relations by id. Not supported (yet) are queries for ways or relations by coordinates. You also cannot query for nodes related to a way or anything related to a relation - having a way or a relation, accessing it will, however, magically pull and update the nodes and members and add them to the database: .. code-block:: python # Get all nodes that are members of a (unique) named way session.query(osmalchemy.way).filter( osmalchemy.way.tags.any(key="name", value="My Unique Way") ).one().nodes This should, in reality, cover most use cases. If you encounter a use case that is not supported, please open an issue asking whether it can be supported (if you have an idea how it can be, please add it or even implement it and open a pull request). Projects using OSMAlchemy ~~~~~~~~~~~~~~~~~~~~~~~~~ OSMAlchemy was designed for use in the Veripeditus Augmented Reality framework. Development and standards ------------------------- Albeit taking the above into account, OSMAlchemy is developed with quality and good support in mind. That means code shall be well-tested and well-documented. OSMAlchemy is tested against the following SQLAlchemy backends: - SQLite - PostgreSQL - MySQL However, we recommend PostgreSQL. MySQL acts strangely with some data and is incredibly slow, and SQLite just doesn't scale too well (however, it is incredibly fast, in comparison). Authors and credits ------------------- :Authors: Dominik George, Eike Tim Jesinghaus :Credits: Special thanks to Mike Bayer from SQLAlchemy for his help with some SQLAlchemy bugs and pitfalls, and also some heads-up. :Contact: E-mail to osmalchemy@veripeditus.org License ------- OSMAlchemy is licensed under the MIT license. Alternatively, you are free to use OSMAlchemy under Simplified BSD, The MirOS Licence, GPL-2+, LGPL-2.1+, AGPL-3+ or the same terms as Python itself. Keywords: osm,openstreetmap,proxy,caching,orm Platform: UNKNOWN Classifier: Development Status :: 3 - Alpha Classifier: Environment :: Plugins Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python :: 3 :: Only Classifier: Topic :: Database Classifier: Topic :: Scientific/Engineering :: GIS Classifier: Topic :: Software Development :: Libraries :: Python Modules OSMAlchemy-0.1.3/setup.cfg0000644000175000017500000000007313042151631014655 0ustar niknik00000000000000[egg_info] tag_build = tag_date = 0 tag_svn_revision = 0