45 minute visualisation!

I'm still commuting to London, and this morning while scrolling through Twitter, I wanted to know two things; one, how many points does it actually take to win the league (For Leicester [who will inevitably, probably regress over December], Arsenal and Manchester City) and two, what's the magic number that José Mourinho has to help his Chelsea team achieve to fight off relegation.

So, of course, I set about visualising it! Grabbing the data from statbunker.com, I expanded my view to look at 1st place, 4th place (to qualify for the Champions League) and 17th place (escaping relegation). It's a really simple visualisation, and instead of using the native box plot in 'show me', I created this with the help of the Analytics pane. I'll do a full write up on the how-to later.

Check it out below and let me know what you think!

Read More

An Ode to Brett Pitman*

(* and also Freddie Sears)

This post is an ode to Brett Pitman, a player who I have been a huge fan of and felt that he was *the* coup of the summer transfer window - My post on replacing Daryl Murphy highlighted the fact that Pitman actually was a great jack-of-all-trades as he had the ability to finish, score, win aerials and shoot often - Which based on the numbers I had on the scatterplot, placed him nicely between McGoldrick and Murphy. Not bad. Check out his radar below;

Compare that to Murphy in the same season, and you can see the number of goals scored is similar (albeit with less minutes), Pitman is better at holding onto the ball, and is more creative than Murphy (more key passes per 90)

This season, he hasn't scored at the same rate, but still has a similar conversion rate and a better shooting percentage (an Ipswich effect?) and still is a creative force - and has even bagged a couple of assists.

The thing I love about watching Pitman is his work rate, an indicative quality of many Ipswich players (Murphy, Tabb, Sears come to mind - and Anderson last year) and this work rate allows him to be a nuisance - and in the game against Rotherham, his diverse skill set allowed him to be the 'official' focal point of the system, though the strikers all ended up playing fairly narrow;

This nicely brings us to Freddie Sears - Sears is (according to #ITFC twitter, which is maybe one step up from TWTD in that there's still mad overreaction, but I feel I can get some level of level headed opinions from some) the best player Ipswich have, a quick guy with a neat finish who can hold onto the ball. So he's wasted on the wing. His radars last year and this aren't too shabby at all;

I actually disagree. Sears playing wide might mean that he has less opportunity to move inside, but it I feel his quality shines through, such that he'll engineer himself some shooting opportunities. But his discipline, ability to stick on the ball and relationship with Chambers seems to be better than Niles. The right-midfielder conundrum is a weird one for me; I have a feeling that a left sided bias for teams attacking in the Championship is a common theme - But I've not attempted to back it up with data. Tommy Oar isn't up to speed allegedly, Toure for me is no more than a squad player and in AMN & Bru are central players who can play wide (in a narrow fashion) 

Other notable comments;

As in some games around this time last year, Mick has shifted from 4-4-2 to a 4-3-3 (4-5-1 without the ball) - Of course, this is way forward, why didn't he play this before etc etc. I'm a bit meh about it all. I maintain two things; one, that Mick McCarthy & Terry Connor are fully in control and understand the dynamics & structure of the team better than many of us - and I think the summer signings are slowly bedding in, understanding their roles, responsibilities as well as that of their teammates around them. This , coupled with a couple of 'easier' games against Huddersfield, Bolton and Rotherham - who were blown away following a change of shape.

I was watching highlights of (almost) every Ipswich game back, and I noticed that a common theme in the patch around that Reading game was a) poor positioning and b) Skuse either having to drop back to save the full-backs or Smith/Berra having to cover, which meant an intelligent striker can simply nip in and score.

I still think scoring 5 was lucky and a part of the craziness of the Championship, than Ipswich being good. That said, Ipswich have gained some level of defensive stability in recent weeks - which for me stems from two things; first, the understanding between the midfielders and the defence, and second the movement of Ainsley Maitland Niles to the middle of the park. I mentioned in a previous post that given he drifts inside naturally, and seems to prefer to be central to the goal than playing wide. In addition, given his relatively high number of turnovers (though he also has a high number of regains) sometimes leaves Luke Chambers (who enjoys bombing forward) with a lot of ground to make up.

November ends with two games; Wolves tomorrow at home and Charlton away next week. I think this should be 4 points minimum; as I tweeted this morning, Ipswich have the highest percentage of shots taken against us in the 6 yard box, whereas Wolves have conceded the least - with almost 61% coming from outside the box. I feel Jackett's got his team organised well but they're lacking bite recently. I'll go for a sneaky Ipswich win or a draw - a clean sheet will be nice as always.

So, to Brett Pitman and Freddie Sears - You are the light which shines bright in a gloomy October.

Read More

Informative shapes, dynamic annotations & the beauty of wow

I've started my first placement after finishing at the Data School, and there are two key things I've experienced so far; firstly, I am so not used to being referred to as the 'Tableau Expert' - I still feel like there's way too much for me to learn, practice and get better at to even understand how it feels to be an 'expert' - in any field! Plus, from what I've gathered from what I've learnt and the amazing people in the community have shared; you never stop learning. Ever! And I think this is one of the best things in life in general.

But this leads me nicely into my second key thing (and not too soon, it was about to get real deep up here) which is the wow factor - Being on site and sitting near the rest of the team I've joined means they can throw questions at me as and when they need some support. And this is where I get to have some fun.

So one such query was regarding 'how do I best show...', and being able to cycle through some ideas, show some funky tricks and drawing out the 'wow!' was super rewarding. For instance, being able to use a shape in a dashboard to convey some information, or using an annotation instead of a label, or what highlight actions are. But that might be for a separate post - Actions can be a three step process.

The main one I wanted to talk about is more 'this is cool, Tableau is so smart' - And I referred to it as a 'dynamic annotation'

Why?

As you can see above; it's an annotation on the sum of sales, which is .. er.. dynamic. I didn't actually know that Tableau did this, but it was one of those 'of course that works' moments.

If you can't see from the .gif, what I've done is added a point annotation to my bar chart, and then the 'cool' part of this was that when you move the arrow around, the label (as it's a <Sum (Sales)> greyed out measure) actually changes depending on where the arrow was pointing.

So, it turns out that there are three different types of annotation;

• A mark annotation; which fixes itself to a mark on the view (which is greyed out in some cases, such as a line chart). This annotates the individual mark and only the text box itself is what is moveable (see below)

Note the 'yellow' indicates that this point is only moveable along it's own arrow, and the green means the highlighted red box is moveable. 

• A point annotation; where both the box and the point are moveable, but the beauty (as you can perhaps just about see above) is that due to the <SUM(Sales)> part, it's able to recalculate itself depending on where on the measure axis it is - imagine an invisible point at the end of each arrowhead. (see below)

Note; all green, all moveable; but also note the lack of dimensions within the annotation itself; as I said, where a mark annotated a specific point, a point indicates an area which is calculated via the measures. In this case, the two measure green pilled scatter plot has two measures.

• An area annotation; this is basically where you can visually draw the user to highlight an area on the chart - which also can act as a really nice caption. I tend to add a transparent colour here; and Tableau is so smart, it even only offers the lighter pastel colour palette. (see below)

Note; this is pretty straightforward, the yellow parts are moveable but fixed to an area and the green points are fully moveable and customisable.

The only time I've really used annotations is on my football blog where I used it to, well right click and annotate - I never really moved the arrow from the mark, as I was using a scatter plot. But I thought this point effect was very cool, and as I began to write this post, I found myself questioning why - which is when I noticed the coloured toggle shapes and the nature of the annotations.

I can't wait to discover more little nuances which I didn't know before in the coming months!

Read More

Reviewing that Zlatan viz

Zlatan Ibrahimovic was always described as an 'enigma' - People weren't sure how good he really was, he scored often but never really exploded, yet still commanded huge transfer fees - But I think since he moved to Paris St-Germain, he's really come into his own - A fact which was highlighted when he became the club's all-time highest goalscorer with 110 goals on the 4th of October 2015.

I made a visualisation to celebrate, using TransferMarkt.com as my data source, scraping the data using Import.io and then adding multiple sheets in Excel. Check out the viz below.

When I created it, I was quite excitable - it was current, I'd worked hard to get a lot of information on there, and I felt it had a infographic meets visualisation feel to it. Looking at it now, for the purposes of this blog, I think that there's scope for a makeover.

Let's talk through it a bit.
I kept a black background in order to utilise the gold text, which enabled me to have a 'champion' theme, and I added the image which the PSG Twitter account posted when Zlatan broke the record. Then I added a cumulative goals scored chart, coloured by season with a second overall running total sitting behind - I think this was a mistake, looking back. A simple running total would probably suffice, but I'm still a fan of the information that the tooltips provide.

The comparison to Pauleta was supposed to be a bit of context, but I think it's a bit distracting, and takes away from the achievement - Particularly as the website I used to get the data doesn't have the same granularity so therefore doesn't have the same effect.

Then the 'radio' chart. I'm super uncertain about this - I wanted to show the games in which Zlatan has scored or assisted a goal, to show his attacking influence and to show games where he had contributed to at least 3 goals. This needs to change. I think it's very hard to read, and there definitely is a better way to visualize this particular datapoint.

This was also the first time I had created a scrollable dashboard - one which allows the user to read left to right and scroll to see more - I liked this, but it definitely had it's issues when actually putting the dashboard together in Tableau.

All in all, on reflection, I think the aethetics, theme and layout of this visualisation was good - I just think that I need to spent a little bit more time thinking through whether something adds value, the underlying narrative of my story & essentially, what the most important points of the visualisation are.

I'll be doing a makeover of this over the coming weeks I think!

Read More

Let's go!

I've been meaning to create this blog for a while.

Since joining the Information Lab, three or four months after discovering Tableau, the world of Cairo, Tufte and Few, everything seems to have accelerated and I'm seeing, understanding and developing more and more new things every single day.

So the purpose of this blog is to attempt to post my thoughts on visualisations, books I've read, new and noteworthy articles, as well as the journey in creating some of the data hobbyism projects I undertake.

Currently, I have a bunch of blogs posted on the Data School website, one (so far!) on the Information Lab site & a completely separate football (kind of) meets data blog with a focus on the team I've grown up supporting, Ipswich Town.

I'm hoping that this blog serves as almost a data playground for me, allowing me to have a voice & mark my journey as I am exposed to and learn more about data! Plus, it allows me to use Blogger, as I think I can actually embed dashboards on here.

Read More

Analytics Thinkpiece;

I could I thought of a more imaginative title, but sometimes simplicity is key. I think this was a theme reflected recently when I came up with and delivered a 'Sports* Analytics 101 (*mainly football)" presentation to my colleagues at the Information Lab's Data School. I skimmed over some interesting things, and dwelled on some of the more laboured points - But I got some great feedback from my colleagues, and I hope they took on board something they didn't know before.

Similarly, analytics bloggers, writers and those who are just putting content out there get feedback and help understand things that others may not have known before. With the subject growing and key developments being made in the field, I've been wanting to share my thoughts for a few weeks now; namely since the anti-analytics pieces in national press and online as well as Marcotti's appearance on the excellent Analytics FC pod.

I feel inclined to mention Mark Thompson's (@ETNAR_uk) Sensible Stats twitter here - I think this is a great idea, and the crux of my two-part point of this piece. For me, these are fairly salient within the current state of analytics;

First, that the statistics used should be more accessible and easily understandable - explaining jargon, applying use cases for metrics and ideas and simply lowering the barriers to entry would perhaps make wider football fans more engaging towards such developments in this area. I feel that North Americans are more welcoming of such an approach (this is relatively speculative) but this is given the prevalence of statistics within sports such as baseball, NFL and NBA. This is a point which has been discussed before, and there are some amongst the content producers who are doing their best to make sure this is happening.

Second, is that as many people as possible should look to simply get data, use the data and just play with it.



I echo the tweets above; the best place to start is (for me anyway) where I started one July in 2013 - On Statsbomb. I feel like I was one of the first to follow the work that Colin, Ben & Ted were doing and it was simply excellent to watch it roll, and roll and roll into the hub it has become now. Even with TK at FCM/Brentford, James Yorke has done an excellent job in recruiting good talent who contribute regularly and with very good quality stuff - And it's still accessible & well explained.

Tom Worville has set up Fanalytics Blogs which auto-tweets selected blogs by people, which gives you the most up to date blogs out there, which is a great resource to keep up to date with readings and such.

But what if you want to get involved?



Ben Torvaney has written this post (which features my colleague Chris Love's work on BBC Text Commentary web scraping) on where to get data from when you don't have the opportunity or cash for Opta data here

The too-long didn't-read of it is, start with a simple copy and paste, clean it up a little and start playing. My personal holy grail, and a post I still refer people back to even today? Neil Charles' blog on doing football analysis in Tableau here - When this came out in February, it was the first time I downloaded and used Tableau. Today, 7 months later, it is the bread and butter of my job and I use it almost every day.



But if that ain't enough, and you just want a dataset to go and play with? Have a proper look at Ben's blog again - there are some really good links there and also (as the tweet above states) Paul Riley has a public expected goals Tableau dashboard and the data freely available to use, and Footy in the Clouds and Objective Footy and ChelseaStats are folks who have their data available to play with on their website.

In my eyes, there are those people who are doing the pioneering work within the space, the people who follow, read, critique and do their best to get involved and those who follow out of interest. For me, there's a real danger of those who are doing the cutting edge public stuff to run off into the distance while the rest of us are trying to gauge the ELOs from our ExpG's. Which brings me back, again to the crux of this article - that analytics should be made more accessible and bloggers should do their best to have some level of accessibility (though it's super hard not to immerse yourself in jargon sometimes) and that aspiring people who see what the key members are doing and like it? They should get involved.

Take your favourite team, ask a question and see if the data confirms it - then write about it. Then publish it. Even tag a few people in who you want to read it, and get the exposure you need. The rest will follow, and if you enjoy it, you'll inevitably want to get better and do more stuff.

With today's announcement about Analytics FC developments & the growing interest in the field, there's not been a better time to set up your Blogger or Wordpress account and get writing!

NB; there are some amazing people to follow on Twitter, and rather than listing them all and giving them the acknowledgement they deserve, I'm simply going to plug my Twitter list (I also recommend using Tweetdeck - so good)

https://twitter.com/Scribblr_42/lists/football-data-tweets

Read More

FIXED Level of Detail Intricacies; a Case Study

This blog was originally posted on The Information Lab: https://www.theinformationlab.co.uk/2015/11/04/fixed-level-detail-intricacies-case-study/

Preamble
Level of detail calculations are super duper tricky - Having used Tableau for the last four months at an intensive pace and level, I find myself steering away from level of detail where I can. So when an instance occurs when I need to use it, I either find myself thinking it through with minute detail, or referring back to a session with had the Queen of LOD's, Bethany Lyons - My notes actually come in handy in this instance!
As some of you may be aware, the Data School have already recruited for the next cohort of Data Padawans that The Information Lab are looking to mould into awesome Data Consultants. This intensive process included (as you can read here) a simple Tableau creation process, and followed by each interviewee getting given a similar data set. The beauty of this? Not only is it a chance to experience the world of Tableau and really get to grips with how it works, but it is also an opportunity to ask questions, learn and develop your skills as Andy and the rest of the team are always happy to answer questions, critique and help the interviewees during the process. This is an amazing opportunity for feedback, especially (in my experience anyway), a large amount of applications and notes made during the process disappear into the netherworld, never to be seen or heard from again. The transparency is amazing.

The Exercise
Back to the purpose of this post - As a warm-up a few weeks ago as the Data School training was winding down, Andy Kriebel set us a challenge to answer a question one of the candidates had asked; how to work out the minimum and maximum life expectancy for;

• Each Local Authority
• Each Gender
• For every date range
• Where there are no nulls, the metric used is average, and the age category is 'at birth'

The reason for doing analysis like this could be to see whether or not there is a change in the movement of the distance between the highest and lowest 'years left to live' value. Which is actually a good idea, as if this gap widens in some areas in the same year as it gets  smaller in others, then perhaps one ward is more successful than others and their practices should be implemented.

So creating something like this (here we have the basic calculations of MIN and MAX in the viz) should work right?


No - This doesn't seem to work as the visualisation is returning the same value for minimum and maximum for each one of these marks, as it's the lowest level of granularity. The difference is obvious when you take things at the local authority level. But the question, if you remember is this; if you rank the ward by any measure then you can't compare the difference in gender, for instance or see the minimum or maximum for the life expectancy (years left to live) measure.

The Action
What we want for this one is to not allow Tableau to aggregate at the default level, but instead to determine at which level of detail Tableau should be running it's calculations.
So the 'level of detail' that needs to be aggregated up & fixed to is at the Local Authority and Gender level. So my first attempt used a Fixed level of detail.
Fixed aggregates at the specific level of detail of the expression, regardless of what is present in the view. With this in mind, my first attempt was this.


Max LtL as { FIXED [Local Authority], [Gender] : MAX([Years Left to Live]) } for the maximum value

Min LtL as { FIXED [Local Authority], [Gender] : MIN([Years Left to Live]) } for the minimum value

LtL Diff as [Max Ltl] - [Min Ltl]

And the is what the result I was given was.

This seems a remarkable high range, even at birth - It doesn't make sense for the minimum life expectancy for a male or female to be 70 years apart from the maximum level of detail given. For instance in the example above, we can see that for the years 1999-2003, a female in the Eastbrook ward of Barking & Dagenham, the minimum years left to live is 14.9 years; a difference of almost 70 years.

All the values for the Local Authority are now at the same level of detail, but the result definitely can't be right! (I hope!)

However, this doesn't give the correct answer because in order for the fixed level of detail to work, all dimensions which are being addressed need to be in play within the calculation - so instead, our LOD calculation would be;


Max LtL as { FIXED [Local Authority], [Gender], [Metric],[Age Category],[Date Range] : MAX([Years Left to Live]) } for the maximum value


Min LtL as { FIXED [Local Authority], [Gender], [Metric],[Age Category],[Date Range] : MIN([Years Left to Live]) } for the minimum value


LtL Diff stays as [Max Ltl] - [Min Ltl]

Giving us this

Much better. Now in the years 1999-2003, a female in the Eastbrook ward of Barking & Dagenham has a minimum life expectancy of 77.4 years, with a small difference of 5.1 years. This seems to be correct.

The Lesson
So, the question was how to work out the minimum and maximum life expectancy for;

• Each Local Authority
• Each Gender
• For every date range
• Where there are no nulls, the metric used is average, and the age category is 'at birth'

And in order to do this, all the dimensions (blue pills) in the view must be included in the Level of Detail calculation. As you can see, before the colon you have every pill which is in the view. In order for the calculation to work, every dimension which is in the view must be taken in the calculation. This makes sure that all of these are held as fixed, meaning that the calculation is left at this high level of detail.

After conversing with Chris Love about this post, he mentioned that the first attempt can also be remedied using context filters. This is because the FIXED level of detail calculation doesn't take filters into account unless they are in context. So adding Metric and Age Category to context also works in this instance.

The reason for sharing this was to show an example where understanding how the " { FIXED " function can be used, but also to highlight once more that within Tableau, there is more than one way to reach a solution.

Have a look at the workbook I've uploaded Tableau Public; click on the image below to have it pop up.

It's got the two sheets - My first attempt, and second, with the data attached for you to explore further and understand the exercise.


If I've made any errors in how I've communicated or explained this, please let me know so I can edit the post accordingly, and more importantly, carry on learning.

Read More