Geoviews: Map Tile Overlaid With UK Choropleth Fails to Align Exactly
I have a ~107K row csv based off of the Office of National Statistics Postcode Lookup file which I loaded as DataFrame df.
(Please note that this link is for the original ONS file, not my altered one)
https://ons.maps.arcgis.com/home/item.html?id=4f71f3e9806d4ff895996f832eb7aacf
import pandas as pd
import numpy as np
import random
import copy
import plotly.express as px
import panel as pn
import holoviews as hv
import geoviews as gv
import geoviews.feature as gf
import cartopy
import cartopy.feature as cf
from geoviews import opts
from cartopy import crs as ccrs
pn.extension("plotly")
gv.extension("bokeh")
df = pd.read_csv("../df_example_dataset_221119.csv")
1st 25 Rows of df:
nspl_easting nspl_latitude nspl_localAuthorityDistrict nspl_longitude nspl_northing ... postcode_prefix postcode_order supergroup group subgroup
0 205580 52.016209 W06000009 -4.834655 239104 ... SA 111 Urbanites Ageing urban living Self-sufficient retirement
1 449021 51.740288 E07000180 -1.291435 204857 ... OX 91 Urbanites Ageing urban living Communal retirement
2 443448 53.508308 E08000018 -1.346353 401490 ... S 119 Hard-pressed living Industrious communities Industrious transitions
3 415327 54.619576 E06000047 -1.764161 524962 ... DL 60 Hard-pressed living Industrious communities Industrious hardship
4 461888 52.962238 E07000173 -1.080085 340937 ... NG 115 Urbanites Urban professionals and families Families in terraces and flats
5 441508 50.915695 E06000045 -1.410917 113080 ... SO 89 Cosmopolitans Students around campus Student communal living
6 393588 52.553703 E08000031 -2.096000 295100 ... WV 43 Suburbanites Semi-detached suburbia Multi-ethnic suburbia
7 440238 53.196899 E07000038 -1.399148 366815 ... S 119 Hard-pressed living Industrious communities Industrious hardship
8 197093 56.462948 S12000035 -5.295074 734958 ... PA 33 Rural residents Rural tenants Ageing rural flat tenants
9 652748 52.603550 E07000145 1.731206 307179 ... NR 108 Multicultural metropolitans Rented family living Private renting new arrivals
10 443254 53.615883 E08000036 -1.347626 413457 ... WF 70 Urbanites Ageing urban living Self-sufficient retirement
11 244584 52.002270 W06000010 -4.265472 236183 ... SA 111 Rural residents Ageing rural dwellers Renting rural retirement
12 342359 53.394840 E08000012 -2.868276 389019 ... L 93 Constrained city dwellers White communities Constrained young families
13 317949 56.591812 S12000048 -3.337703 745234 ... PH 11 Constrained city dwellers Challenged diversity Transitional Eastern European neighbourhood
14 522743 51.418400 E09000024 -0.236223 170296 ... SW 95 Suburbanites Suburban achievers Indian tech achievers
15 416903 50.993282 E06000054 -1.760528 121570 ... SP 32 Rural residents Rural tenants Ageing rural flat tenants
16 412438 53.683681 E08000033 -1.813157 420819 ... HX 13 Urbanites Ageing urban living Self-sufficient retirement
17 380633 53.561138 E08000002 -2.293852 407209 ... BL 45 Urbanites Ageing urban living Delayed retirement
18 465475 50.795639 E06000044 -1.072351 99975 ... PO 112 Multicultural metropolitans Rented family living Private renting new arrivals
19 525069 51.103195 E07000226 -0.215022 135281 ... RH 74 Multicultural metropolitans Rented family living Social renting young families
20 324536 55.648670 S12000026 -3.200686 640113 ... EH 110 Suburbanites Suburban achievers Ageing in suburbia
21 317378 54.844311 E07000026 -3.288153 550704 ... CA 58 Rural residents Rural tenants Rural life
22 491761 51.034000 E07000225 -0.692730 126884 ... GU 104 Rural residents Farming communities Agricultural communities
23 449460 54.650303 E06000001 -1.234992 528624 ... TS 81 Suburbanites Semi-detached suburbia Semi-detached ageing
24 457016 52.624683 E06000016 -1.159144 303321 ... LE 103 Multicultural metropolitans Rented family living Private renting new arrivals
Data Types:
In [35]: df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 107632 entries, 0 to 107631
Data columns (total 14 columns):
nspl_easting 107632 non-null int32
nspl_latitude 107632 non-null float32
nspl_localAuthorityDistrict 107632 non-null category
nspl_longitude 107632 non-null float32
nspl_northing 107632 non-null int32
nspl_outputAreaClassification 107632 non-null category
nspl_ward 107632 non-null category
nspl_westminsterParliamentaryConstituency 107632 non-null category
postcode 107632 non-null object
postcode_prefix 107632 non-null category
postcode_order 107632 non-null int32
supergroup 107632 non-null category
group 107632 non-null category
subgroup 107632 non-null category
dtypes: category(8), float32(2), int32(3), object(1)
memory usage: 4.4+ MB
The postcode_prefix column contains all 121 UK postcode areas.
In [50]: df["postcode_prefix"].value_counts()
Out[50]:
BT 2978
B 2625
S 2099
M 1988
NE 1975
...
WC 165
LD 140
KW 108
HS 53
ZE 30
Name: postcode_prefix, Length: 121, dtype: int64
While the postcode_order column contains numbers 1 - 121. 1 = lowest count of postcode_prefix in df (ie ZE), 121 = highest count (ie BT) in df.
In [42]: df["postcode_order"].value_counts()
Out[42]:
121 2978
120 2625
119 2099
118 1988
117 1975
...
5 165
4 140
3 108
2 53
1 30
Name: postcode_order, Length: 121, dtype: int64
UK Shapefile downloaded from:
https://geoportal.statistics.gov.uk/datasets/bbb0e58b0be64cc1a1460aa69e33678f_0/data
shapefile = "../Local_Authority_Districts_April_2019_Boundaries_UK_BUC/Local_Authority_Districts_April_2019_Boundaries_UK_BUC.shp"
gv.Shape.from_shapefile(shapefile, crs=ccrs.Mercator())
In [51]: shapes = cartopy.io.shapereader.Reader(shapefile)
...: list(shapes.records())[0]
Out[51]: <Record: <shapely.geometry.polygon.Polygon object at 0x1c35f30a50>, {'objectid': 1, 'lad19cd': 'E06000001', 'lad19nm': 'Hartlepool', 'lad19nmw': None, 'bng_e': 447157, 'bng_n': 531476, 'long': -1.27023005, 'lat': 54.67620087, 'st_areasha': 96512310.88728464, 'st_lengths': 50488.38708066442}, <fields>>
Set Latitude and Longitude and plot choropleth
heatmap = gv.Shape.from_records(shapes.records(), df, on={"lad19cd":"nspl_localAuthorityDistrict"}, value="postcode_order",
index=["postcode_prefix","nspl_outputAreaClassification","supergroup","group","subgroup"], crs=ccrs.Orthographic(central_longitude=-8.00, central_latitude=50.00)).opts(tools=["hover"], cmap = "Reds", colorbar=True, title="UK Postcode Heatmap", width=800, height=1350, alpha=0.4)
heatmap
Set Map Tile
wikimap =hv.Tiles('https://maps.wikimedia.org/osm-intl/{Z}/{X}/{Y}@2x.png', name="Wikipedia").opts(width=800, height=1350)
Final plot, including setting a point marker (latitude and longitude used here is for Big Ben)
wikimap * heatmap * gv.Points([(-0.116773,51.510357)]).opts(color='purple', size = 20)
Resultant Plot:
As you can see below, my choropleth very nearly overlays successfully over the map tile. However no matter how much I tweak the central_longitude and central_latitude values, I can't get them to line up exactly. It actually seems like I need to rotate my choropleth slightly some how.
I suspect that my issue is caused by either:
1. The shapefile that I am using
or
2. The projections that I have chosen
I tried to make both projections equal crs=ccrs.Mercator(), however this does not allow me to set the latitude and longitude for my choropleth thus preventing successfully overlaying this with the map tile.
Any pointers in the right direction would be hugely appreciated.
Thanks
Solution
Update on 23/11/19:
After posting this question I found: https://scitools.org.uk/cartopy/docs/v0.15/crs/projections.html
Here I found projection = OSGB. Switching to this projection resulted in my desired result:
- WSGIServerException: [Errno 8] nodename nor servname provided, or not known
- Python/Flask Login form throws 500 error on IIS
- Matplotlib plt.show() isn't showing graph
- How to avoid duplicate processing in python API server?
- Extract points/coordinates from a polygon in Shapely
- How do I detect collision in pygame?
- Typehint a method that returns new instance using a superclass classmethod
- How to process requests from multiiple users using ML model and FastAPI?
- How can I clear the interpreter console?
- What are the reasons for the differences in output results when using the groupby function in the Python pandas package?
- How to antialias picture to get rid of the jerkiness in the image
- Is there any difference between list(array) and array.tolist() in Python?
- How to move x-axis on top of the plot in plotnine?
- Pulling Zillow Rent Data from Zillow API
- Safest way to incrementally append to a file?
- Unable to install mysqlclient package on EC2 instance
- FastAPI - How to get app instance inside a router?
- SQLAlchemy: engine, connection and session difference
- Generating binary arrays with alternating values based on change indices in NumPy
- How can i solve this error: ValueError: The feature names should match those that were passed
- 'super' object has no attribute '__sklearn_tags__'
- Python: How to copy all attibutes from base class to derived one
- mitm proxy is closed, but the port is still occupied
- How to find pg_config path
- Why does "pip install" inside Python raise a SyntaxError?
- Plotting a matplotlib pie chart using 2 columns
- ImportError: No module named langchain.llms
- How do I disable the security certificate check in Python requests
- How can I import Psycopg2 into a Lambda Function on AWS?
- Django built in Logout view `Method Not Allowed (GET): /users/logout/`