Vector data#
Vector data are the second most common data type we work with in Geography - next to raster data, which we have covered already quite extensively during this course. The pages here have the goal to (a) give you an overview on the general structure of vector data, (b) make you familiar with some of the most common geospatial operations one does using vector data, and © provide code examples for these operations. As for the chapter on raster data, we avoid using external packages like shapely or geopandas. This is, because we want you to understand the data structure of vector data and develop ideas on how to use them in a very efficient way. Likewise, the examples provided here are by no means exaustive, but provide you with the key elements to work in the context of this course. At the very bottom of this page, we provide some links to additional sources. The book by Garrard is provided in moodle. Yet, be aware that this is a bit outdated and sometimes not fully “to the point”.
The OGR Simple Features Library#
In our geographic work, vector data are data in which geographic features are represented as discrete geometries:
Points: points represent the simplest type of gemetry. In geographical contexts they consist of two coordinates (e.g., latitude and longitude).Lines: Lines are a collection of at least two points in a pre-defined order, where the first and the last point are not identical.Polygons: Polygons are collections of at least three points in a pre-defined order, where the first and the last point are identical.
As you can see, all three types of vector data consist of individual points, stored as xy coordinates, so they are not fundamentally different in the way the carry spatial information. Compared to raster files, however, there are some distinct differences: (1) vector information only carry information at the locations of the points, and not between them (e.g., between two points that form a line. (2) For each geometry (or feature, as we will learn further below), we can store specific information (so-called attributes).
For our work here in the course, we will use the OGR Simple Features Library to work with the data - or simply: OGR. It is a library that is part of gdal, though, specifically focussing on working with vector formats. The most common formats you have probably worked with already, are .shp, GeoJSON, or kml. However, there are many more formats, some are more, some are less popular. Similar to gdal, OGR gives as a set of tools and libaries on-hand for reading, writing, and manipulating geospatial vector data. In addition, we will employ the OGR Spatial Reference System (OSR), which is also a part of gdal, but is focused on managing the spatial reference information of geospatial data. we will need this later on, when understanding and working with coordinate systems, projections, and datums used in voctor datasets.
Now let’s have a look at the general data structure of a vector file:
Fig. 4 Overview about the OGR vector structure. For more info and background visit the ogr documentation and see Garrard (2016) Geoprocessing in python chapters 3-5 (available in moodle).#
Similar to the raster data structure, we can see the hierarchical nature of a vector layer. Take some time and look at the graph to understand the structure of a vector file. The structure is always the same, independently of the type of vector data (i.e., point, line, or polygon)
Accessing vector data#
Now that we know the structure, we can start loading some data. First, we will need to import the ogr and osr libraries and define our working folder:
import os
from osgeo import ogr, osr
folder = "D:/OneDrive - Conservation Biogeography Lab/_TEACHING/__Classes-Modules_HUB/M8_Geoprocessing-with-python/data/"
Now, we can open the shapefile. The shapefile that we are using here is a subset of the gadm dataset, which contains countries in form of polygons. As for raster data, the first levels of the file access (i.e., the file and the layer) create pointers to the actual data on the disc:
# Open the shapefile
ds = ogr.Open(folder + "EU_vector/gadm36.shp")
ds
<osgeo.ogr.DataSource; proxy of <Swig Object of type 'OGRDataSourceShadow *' at 0x000001CD02BDB210> >
# Get the first layer in the file
lyr = ds.GetLayer()
lyr
<osgeo.ogr.Layer; proxy of <Swig Object of type 'OGRLayerShadow *' at 0x000001CD0419E700> >
Ok, this was easy. But now let’s see whether we can (a) get some general information about the shapefile, such as the projection, and (b) have a look at the different features and associated attributes within the layer:
# Spatial reference
sr = lyr.GetSpatialRef()
sr
<osgeo.osr.SpatialReference; proxy of <Swig Object of type 'OSRSpatialReferenceShadow *' at 0x000001CD04186010> >
As you can see, also the spatial reference is only a pointer to the actual information. If you want to store it as a variable and/or you want to read it, then you need to convert into something that the machine and store and work with. In the raster chapter we introduced the function .ExportToWkt():
sr = lyr.GetSpatialRef().ExportToWkt()
sr
'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]'
Now, that’s better. Like in the raster session we have the common lat/lon coordinate system (EPSG-code 4326). Below are some more code snippets, that allow you to get some additional information about the lyr:
# Get the name of the attributes
[field.name for field in lyr.schema]
['fid', 'iso_a2', 'NAME', 'FIPS_10_', 'ISO_A3', 'WB_A2', 'WB_A3']
# Get the total number of features in the lyr
lyr.GetFeatureCount()
9
Accessing features#
Knowing a layer’s properties is nice, but really what we want to work with is the information of the individual features. We can access features in two main ways: (1) through indexing, and (b) by iterating over all elements (i.e., features inside) of a layer. We combine this here with a function that allows us to access an attribute of a feature (.GetField())
First, let’s access a single feature, and get the name from the attribute 'NAME_0', which corresponds to the country name.
feat = lyr[5]
feat
<osgeo.ogr.Feature; proxy of <Swig Object of type 'OGRFeatureShadow *' at 0x000001CD04238AE0> >
ctr = feat.GetField('NAME')
ctr
'Switzerland'
Again, you can see that we access the feature through a pointer. However, getting the country name actually stores a value on the disc. Make sure to remember this when working with vector data.
Now, we rarely work with individual features, but instead we want to iterate over all features of a shapefile. We woould do this using a loop, and we got to know two different types of loops: the for-loop and the while-loop. Below we want to do the same as above, but now for all features of the shapefile. In case of the for-loop, this is straight forward:
for feat in lyr:
ctr = feat.GetField('NAME')
print(ctr)
Germany
Netherlands
Austria
Liechtenstein
Belgium
Switzerland
Czechia
Luxembourg
Poland
In case of the while-loop we need to employ another method associated with a layer: the .GetNextFeature() method. This methods works like a cursor inside a layer that always selects the next feature and works with it, until we want to access the next feature using the same method. Using .ResetReading() we can bring the cursor back to position 1:
feat = lyr.GetNextFeature()
while feat:
ctr = feat.GetField('NAME')
print(ctr)
# Get the next feature
feat = lyr.GetNextFeature()
# reset the reading
lyr.ResetReading()
Germany
Netherlands
Austria
Liechtenstein
Belgium
Switzerland
Czechia
Luxembourg
Poland
Access geometries#
The last part in terms of accessing the shapefile and its individual features is getting access to the geometry. As a recurring item, we have two options. (1) by creating a pointer to the geometry using the function .GetGeometryRef() and (2) by creating a copy of the geometry and copying it into memory. Both options have the application, as sometimes we don’t want to manipulate a geometry (e.g., calculate an intersection) but only want to use the geometry to do some selections or calculations (e.g., calculate the area of a geometry). Before you continue, think of at least two other applications, where your choice would be to creating a pointer to a geometry or creating a copy of a geometry in memory.
# Ge the feature
feat = lyr[5]
# Version I: fetches the pointer to a feature geometry. Returned object is an OGRFeature.
geom_ref = feat.GetGeometryRef()
geom_ref
<osgeo.ogr.Geometry; proxy of <Swig Object of type 'OGRGeometryShadow *' at 0x000001CD041CA5E0> >
# Version II: Create a copy of the geometry in memory
geom_geom = feat.geometry()
geom_geom
<osgeo.ogr.Geometry; proxy of <Swig Object of type 'OGRGeometryShadow *' at 0x000001CD05383840> >
We see that in both cases we only get to see the address in the memory. if we want to get to know the coordinates that the geometry is composed of, we need to take a bit a detour and convert the geometry into a json element:
import json
ring_dict = json.loads(geom_geom.ExportToJson())
ring_dict
{'type': 'Polygon',
'coordinates': [[[8.728973719134068, 46.10823576974391],
[8.677485391750196, 46.09579213029839],
[8.60183109429973, 46.122818895714566],
[8.51026045756495, 46.207878327807165],
[8.438120156269617, 46.23537014463381],
[8.42323734385719, 46.27583280294639],
[8.441634152733435, 46.43494451653792],
[8.385906922465184, 46.450205969091265],
[8.316267126318513, 46.43365257734048],
[8.28660486070453, 46.40535983631927],
[8.29146244339154, 46.378358873652715],
[8.192553751627255, 46.309164116610646],
[8.087340534124792, 46.27180204540798],
[8.073077841154136, 46.253611982247826],
[8.129508502830177, 46.19604438956186],
[8.132299032786491, 46.15935416276402],
[8.110594931346299, 46.1269530500795],
[8.025328817655055, 46.091141269326485],
[7.985848021160997, 45.999312218804654],
[7.898204793952982, 45.98194895302413],
[7.870201293240803, 45.940369641378744],
[7.84962011447879, 45.93971203023916],
[7.831232136091422, 45.914459514971746],
[7.706278521536756, 45.925724965871495],
[7.643026571884362, 45.966342690948494],
[7.541120647655901, 45.98411940094565],
[7.393842813332101, 45.91569975318675],
[7.286665894442562, 45.91342603746281],
[7.153547403919666, 45.87652903115642],
[7.090192095910922, 45.880508092789036],
[7.02208255966939, 45.9252599012787],
[6.98766605624652, 45.99311106977325],
[6.915112344729679, 46.048611546609365],
[6.8509302172676, 46.049645111240864],
[6.848553101246818, 46.08504346673087],
[6.869223673598278, 46.112328607212575],
[6.774345741705955, 46.13480780983984],
[6.765664103036487, 46.1516026818786],
[6.827675823358798, 46.2694766563498],
[6.750367876089285, 46.345518424039525],
[6.788106734235352, 46.40500800747762],
[6.762666872212138, 46.42926020495225],
[6.613683716883012, 46.45589937559545],
[6.482942340478747, 46.448587100029044],
[6.397676228969963, 46.40817616723784],
[6.301558067979397, 46.394481990609854],
[6.219495888549901, 46.32911118915278],
[6.216593978016747, 46.30950388730746],
[6.241820108631615, 46.2636888544014],
[6.276029905996948, 46.26312039701228],
[6.281197550718889, 46.2400727526164],
[6.255359335660102, 46.22110750117318],
[6.107874797112918, 46.13863192671509],
[6.028293090549282, 46.14793366985528],
[5.958839964707789, 46.13046698562103],
[5.982921182372477, 46.170826296903265],
[5.958529907317184, 46.211960762662784],
[6.089684691014861, 46.24637723268462],
[6.118607037201215, 46.33177110120149],
[6.135056598439502, 46.37040065475637],
[6.059013169593956, 46.417385884161895],
[6.060229124074902, 46.46502019262145],
[6.137978214810992, 46.557587748851354],
[6.118416788624285, 46.583462616098544],
[6.13185266104667, 46.59560658922562],
[6.429198853616439, 46.76081615737877],
[6.417106566637699, 46.8021572539805],
[6.44311766267925, 46.85145514411636],
[6.427751912774744, 46.90907582139897],
[6.442634722308632, 46.94416405733653],
[6.598697551869535, 46.986538843152665],
[6.665411823413165, 47.02129118498023],
[6.688252805126302, 47.04384791642425],
[6.676263872198966, 47.062399734280945],
[6.689699749441502, 47.078290362005575],
[6.838076252630053, 47.16813152337798],
[6.956246787471844, 47.24523107418896],
[6.9586239024318, 47.2905512824899],
[7.044303427031627, 47.34049654408822],
[7.003995809692348, 47.36814342856943],
[6.866639855345393, 47.35416501377758],
[6.939169952320597, 47.426844314608076],
[6.990973345587823, 47.452220913791436],
[6.973300005517475, 47.48909204713404],
[7.00079186947426, 47.49767037821678],
[7.142169386353665, 47.48765090985429],
[7.18083255724337, 47.48826521975897],
[7.168326865927081, 47.443565229167234],
[7.238090050421332, 47.416796847342354],
[7.336930049331483, 47.43185370852748],
[7.388218888470429, 47.433288891284],
[7.420563120089125, 47.45085703683307],
[7.414306678393336, 47.490177251832705],
[7.445995795635093, 47.4868841013985],
[7.467430051629757, 47.48190904919766],
[7.485776131704743, 47.49876746290044],
[7.476938519829145, 47.5148787023231],
[7.501123084833297, 47.51730738871431],
[7.505153853170223, 47.53301713618714],
[7.482726277613243, 47.542267142902354],
[7.520866597411644, 47.56341608328783],
[7.586028487006566, 47.58461852351222],
[7.65966638409214, 47.596579124709095],
[7.635895226598071, 47.56459144375676],
[7.609746950243298, 47.564746445142305],
[7.646901035347142, 47.551445243107686],
[7.727320446611278, 47.55042261873654],
[7.785550172044188, 47.563196126994114],
[7.819656622945952, 47.595338985899744],
[7.89820478639938, 47.58784561347949],
[7.912157432923323, 47.560560665838956],
[8.042278688971338, 47.560560665838956],
[8.087237184353985, 47.56738194255617],
[8.179014532995465, 47.61580284865459],
[8.23296472489886, 47.621952283986545],
[8.288568569089211, 47.61580284865459],
[8.316163768997113, 47.58784561347949],
[8.421100328957174, 47.58111192675463],
[8.448868860629954, 47.584280176722274],
[8.492380417223622, 47.61983359050431],
[8.522352743188625, 47.62190055579],
[8.551719146906036, 47.59686329824356],
[8.576305368398092, 47.595023102296146],
[8.580333703855052, 47.63900552035254],
[8.601624398203809, 47.63259770395755],
[8.607308802055703, 47.65629129066304],
[8.593588679698895, 47.65816850453589],
[8.458273971138976, 47.63988402686607],
[8.391301309692125, 47.66546375927365],
[8.392128132339353, 47.699518522729775],
[8.437913447987773, 47.72318626695773],
[8.46364830939544, 47.76390726702899],
[8.53651208430165, 47.77408761328202],
[8.551601609903688, 47.779255395466734],
[8.542299843129262, 47.79501652879686],
[8.583124222328443, 47.80023581646959],
[8.617437384288822, 47.757318737502814],
[8.644102423719568, 47.79101177876487],
[8.681929566362555, 47.75873975040655],
[8.713142127504103, 47.75742193244614],
[8.700429729879001, 47.72349643262581],
[8.717069536740082, 47.69455753871226],
[8.769882851240352, 47.69507432972595],
[8.770709670575663, 47.720860804689345],
[8.797581422028058, 47.72003411489709],
[8.856079146454043, 47.690681942396196],
[8.837682328455356, 47.687787904830955],
[8.851935112746071, 47.671281612797145],
[8.906205283127461, 47.65179538628344],
[9.016586140083895, 47.67889962315395],
[9.183397656213591, 47.670424571741606],
[9.196936885195052, 47.65613608846405],
[9.27321130474429, 47.65009003952592],
[9.547481676238394, 47.53454713873361],
[9.553058594201499, 47.51689139727299],
[9.584510128599902, 47.4807204653182],
[9.650345909588099, 47.452091819131425],
[9.649519082380959, 47.40971707660298],
[9.521154826668214, 47.262801032689836],
[9.497043626216094, 47.22893443529521],
[9.486433795177275, 47.19691776356692],
[9.495534336339764, 47.15595042804391],
[9.511853064409616, 47.12937239709614],
[9.502861367939467, 47.09469757098587],
[9.477023150713741, 47.06389838421279],
[9.533133797967842, 47.05552988529206],
[9.581202837002355, 47.05687037323997],
[9.669052777411785, 47.056198668670106],
[9.857981815846056, 47.01547766212633],
[9.87513838502492, 46.92742092806652],
[10.125187504325217, 46.846751170917955],
[10.201423383220881, 46.86683037781759],
[10.235116411132942, 46.92331262366993],
[10.295681188995452, 46.922692503892094],
[10.313664595151971, 46.964317981738574],
[10.373402543587575, 46.99625391458504],
[10.458461950445482, 46.936619295489706],
[10.453811079076281, 46.86442743606636],
[10.444922727306965, 46.82324123070971],
[10.417224166675194, 46.798850068917815],
[10.42869632652225, 46.75564844549454],
[10.399654179545218, 46.73554643716205],
[10.369165077743249, 46.67239773099673],
[10.395623417286851, 46.6388081513782],
[10.459082075574509, 46.6235635930363],
[10.451828551614776, 46.546704382537165],
[10.42590580770216, 46.53532610280307],
[10.295371138237567, 46.55108739174097],
[10.234703004730587, 46.57529777726472],
[10.23377282421701, 46.61798244019905],
[10.217856483813778, 46.6269741475451],
[10.087838582854868, 46.60439159565256],
[10.071198772129765, 46.56439404109331],
[10.032751510404436, 46.53297482287056],
[10.028100629875599, 46.48393386435326],
[10.04401696979603, 46.466983976917795],
[10.026343629175082, 46.44626173410232],
[10.133417197567432, 46.41401564603157],
[10.1332104882224, 46.38109768856322],
[10.091765989105816, 46.32895616753338],
[10.158945351988905, 46.2624486128159],
[10.145819540741014, 46.24332837691321],
[10.075746298873634, 46.22002230826545],
[10.04267338083769, 46.220487379574834],
[10.031717979077849, 46.26007154667689],
[9.977561074747879, 46.29810534661271],
[9.964021851103961, 46.35608632669129],
[9.918443233129894, 46.3711499407349],
[9.768064814944715, 46.33861962262979],
[9.720109087897153, 46.35089280790399],
[9.70843021885019, 46.311747976944865],
[9.674323770893931, 46.29180084021805],
[9.53645105355052, 46.29862219247705],
[9.444363645627547, 46.37528407827313],
[9.434648477307134, 46.498325692132994],
[9.384625689819062, 46.466415487648185],
[9.350829301131752, 46.49786054915217],
[9.282306351712931, 46.497369633774184],
[9.237967981087003, 46.43654660160613],
[9.260912310938206, 46.41665106694431],
[9.2751750086268, 46.33138500096686],
[9.224831435104791, 46.23118802437846],
[9.163788024117414, 46.17298925179278],
[9.09058679214395, 46.138166811580525],
[9.059167520459322, 46.061789033245724],
[9.002116741984013, 46.03930983103166],
[9.01555261231813, 45.99311106977325],
[8.980515991897734, 45.964378984371095],
[9.010798380079935, 45.92665517345312],
[9.063094929433445, 45.898956593742646],
[9.034362833067314, 45.848106978041976],
[9.002426796688678, 45.82071846206021],
[8.908884717822916, 45.828292603741595],
[8.912096392456736, 45.883401969582664],
[8.870961953717062, 45.94706735777295],
[8.767919146962475, 45.98308579795329],
[8.834375041491606, 46.06638825409176],
[8.808950234314858, 46.08974596176014],
[8.728973719134068, 46.10823576974391]]]}
Using these tools you are ready to access shapefiles, get some basic information about the layer, iterate over the individual features and access the attributes and geometries. In the next chapters, we will pay more attention to some more geographic applications. For some further reading, have a look at:
Garrard: Geoprocessing in python, chapters 3-5
Summary of some main methods using the ogr libraries. Some of the materials provided here are inspired by this website.