Working with geospatial data is really fun! We use this type of data all over the place in data science, and data warehouses have gotten really good at working with it (I'm looking at you, Snowflake!).
However, as you get started down the path of using geospatial data, you will likely run into a speed bump: .shp files
What is a .shp file?
A .shp file is a common way of encoding geospatial data in a binary format. This geospatial data will contain geospatial points, polygons (a collection of geospatial points that form an area), and multipolygons (more exotic polygons).
A .shp file will be bundled with other files with the same name but different file extensions (e.g., .shx, .dbf, .prj, etc.). In short, you will need all of them.
What is the problem with .shp files?
Unfortunately, most data warehouses don't read .shp files natively. If you are looking to load this dataset into your data warehouse, you are going to need to convert it to a compatible format like CSV.
Additionally, a .shp file might use a different coordinate system than the latitude / longitude system you are familiar with (WGS 84).
In this tutorial, we will convert a .shp file to CSV and transform a geospatial data to the latitude / longitude coordinate system.
The latitude / longitude coordinate system we are familiar with is called WGS 84 EPSG:4326.
Convert a .shp file to CSV
In this example, we'll use GeoPandas for the conversion.
import geopandas as gpd
# Import .shp file into a GeoPandas DataFrame
geopandas_df = gpd.read_file('Grid_100m.shp')
# Convert geospatial data to latitude/longitude coordinate system
converted_df = geopandas_df.to_crs('EPSG:4326')
# Write data to CSV
converted_df.to_csv('Grid_100m.csv', index=False)
If all goes well, you should now have a well-formatted CSV file that you can inspect with Excel (and load to your data warehouse).
Cheers! 🚀
Top comments (1)
thanks for the tutorial. For no-code users, shp converter csv offers quick conversion