Saturday, 6 June 2015

Analysing Managerial Performance - Is there a better way?

With the heavy turnover of managers across most football teams these days, it's often hard to judge what passes for success.

Recent studies showed the Championship to be one of the most volatile leagues with 20 managers being sacked over the course of the 2014/15 season - though Watford and Leeds accounted for 6 of those between them, so some context must be given. The recent BBC story (available here) is an OK piece but it doesn't really dig beneath the surface, with quotes from Richard Bevan about the sackings affecting 200 families a little odd - are chairman sacking managers and replacing them with single males?!? Or are 200 other families benefitting from the replacements. Anyway.....

The questions leads to why so many managers are being sacked, what are they being judged on and what makes a chairman pull the trigger.

The obvious answer is that a lack of time is often the route cause of a manager being sacked. A bad run at any point in the season can lead to the chairman being a little quick to get rid, take Steve McLaren as a case in point. After turning Derby from a bog standard mid table team to promotion challengers last season the pressure was on to deliver this year - that they were top on 1st March and finished 8th shows a drop in form but should he have been sacked? There were other circumstances yes, but it's odd that a manager can be sacked by a Championship club and go on to get a Premier League job, which looks the case with McLaren.

So, if a manager is given time how should they be judged?

1) Win Ratio

Many managers point to a win ratio as a badge of honour. This is the industry standard, the thing mainstream media use and is in all honesty a terrible way of looking at things. It is often talked about with no context, so how do we know what a good win ratio is? I'm sure Luis Enrique has a phenomenal win ratio, should all others be compared to him?

I'll use 5 Championship managers from the middle of the table (9th - 13th) and compare their records since they were appointed in all competitions

Gary Rowett (Birmingham City) -        44.12% (15 wins in 34 games)
Guy Luzon (Charlton Athletic) -          38.10% (8 in 21)
Russell Slade (Cardiff City) -                37.84% (14 in 37)
Gary Bowyer (Blackburn Rovers) -      37.27% (41 in 110)
Stuart Gray (Sheffield Wednesday) -    35.71% (30 in 84)

The above list shows that Gary Rowett has the best win ratio (sample size permitting) but there is not a massive difference between him and Gray. If Gray won his first 5 games of next season and Rowett lost his first 5 - unlikely but not unthinkable - Gray immediately has a better win ratio - this shows the volatility of it. So what is a better way?

2) Average points per game

Taking 3 points for a win and 1 point for a draw - if a team wins half and loses half their matches from a sample of 20 (50% win ratio - 30 points) has the manager done much better than one who has won 7 drawn 8 and lost 5? (35% win ratio - 29 points)
While they would have 1 point more, if they are purely being judged on Win Ratio they are much better off going for it and trying to win games than settling for a draw. I read recently about the philosophy of Paco Jemez the Rayo Vallecano manager in the Spanish La Liga, he had this to say:-

So are draws worthless? Taking the 5 managers above into account how do they compare?

Gary Rowett - 1.62 Points per game (55 points from 34 games)
Gary Bowyer - 1.46 Points per game (161 from 110)
Guy Luzon - 1.45 Points per game (29 from 20)
Russell Slade - 1.41 Points per game (52 from 37)
Stuart Gray - 1.37 Points per game (115 from 84)

So Gary Bowyer comes out much better in this ratio than just using Win Ratio. While it's still relatively close a different metric produces a different result. Does that mean Bowyer should be doing better than he is? At 1.46 points per game over a 46 game season he could expect 67 points - nowhere near enough for the Championship top 6 that Blackburn are aiming for.

3) Trophies/Promotion/Cup runs

While major trophies are almost unheard of outside the top flight teams these days (Sheffield Wednesday being the last team outside the top division to lift a major trophy - the League Cup in 1991) a team higher up the leagues will be judged on their achievements. Is it enough for Arsene Wenger to be happy with finishing 4th every season (alright, not this year!)? Should clubs at the higher end be judged if they do win a cup? If so, why do so many Premier League clubs pay little attention to the League Cup/Europa League in particular? The simple reason is the money. Stoke/Swansea/West Brom etc are much better staying the Premier League and reaping the financial rewards than winning a trophy and being relegated (look at the plight of Wigan - would they swap the FA Cup win for keeping their place in the Premier League)

So given this, should managers be judged on whether they achieve promotion from the Championship? Aitor Karanka was certainly unlucky and nobody would have expected Mick McCarthy to take Ipswich into a top 6 finish - but Steve McLaren has lost his job as Derby boss by coming 8th, Nigel Clough led Sheffield United to 5th and the play offs but still suffered the chop.

Again, looking at the managers above, while none have achieved promotion Gary Bowyer has been under pressure for not getting Blackburn near the play offs - though a successful FA Cup run somewhat took the heat off him. Russell Slade has barely been in the job a season but the Cardiff fans want him out. Stuart Gray and Gary Rowett on the other hand are not far off worshipped by the fans of their respective clubs. For this to be taken into consideration it must be based on perceived expectations. Gray may find himself under deeper pressure this season due to lifted expectations amongst the Sheffield Wednesday fans.

4) Points per £

Another method - and this may not work for all clubs, though with Financial Fair Play rules impacting across the whole of Europe it is worth considering - is which managers get the most points for their money. I did a similar style of summary for this on the MLS in 2012 as they publish the salaries of all players so you can work out a cost per point, at least from the playing staff.
More recently, Brentford seem to have taken this massively into account when appointing their new manager Marinus Dijkhuizen. I'm sure there were lots of other factors but his over performance with Excelsior in the Dutch Eredivisie compared to the budget was quite stark as this table by Martijn Hilhorst shows

Unfortunately most English clubs won't reveal their budget - most is simply guesswork - so a simpler way could be to look at overall operating costs, turnover etc. However, these take in several factors not necessarily linked to performance on the pitch, such as the Academy, Commercial operations, Community work etc, so may be a reach to just use a simple figure such as this.

However, a savvy chairman will at least be able to work out the return per budget for his manager, though may struggle to use this as a comparison figure.

The other danger with this is that most of the teams at the higher end of this table were battling relegation. To win/be promoted you need a bigger budget and the pay off from this is that the points expectation then goes up. If Marinus Dijkhuizen was given PSV's budget would he have achieved as well with Excelsior? Unlikely as he would have needed to gain 467 points!!

5) Attacking Style

Some teams naturally will play an exciting style, where others will play to the best of their abilities and use what they have. Sometimes these clash between expectations and reality.

A prime example of this would be a team like West Ham United and Sam Allardyce. This would always be a clash of styles between a manager who favours efficiency and making the most of what is available to a group of fans who want their team to play the "right way".

For me, in football there is no "right way" - the "right way" is generally whatever works - but I have to say from watching Sheffield Wednesday when Gary Meson was in charge the feeling changed quickly when a couple of defeats in a row happened - you don't mind watching poor football if the results match up but when you're watching poor football and losing the fans can quickly turn.

Anyway, back to Allardyce - West Ham generally play a more direct style and the fans want to see more attacking, possession football. This is fine, and probably goes a long way to what cost Allardyce his job at the end of a reasonable reason - they are in Europe, even if it is via the back door.

So what metrics should be looked at to see which teams play more attacking football?
While available statistics in the Championship are not as good as the Big 5 European leagues, the data is there behind the scenes from companies like Opta.
Even just using WhoScored we can look at things like Shot Share (shots for/shots against), Key Passes, Short Passes Per Game can all be used - I'll put a few of these in the table below for the managers I've named above, just to show an example of the different styles - this is simply a table and not analysis on any of the 5 and due to the limitations of the data this is taken over the full season (which usually combines more than 1 manager) - it's enough to paint a picture either way.

In a nutshell, the above shows:-

1) Charlton are regularly outshot by their opponents (Charlton have 0.55 shots for every 1 of the opposition)

2) Charlton have the best ratio of Chances created by Short Passing to Long Passing - They create 3.96 chances with a short pass for every long pass

3) Blackburn do more short passes than the other 4 teams, 3.86 short passes for every long pass (this is standard passes, regardless of location on the pitch)

4) Charlton have a much higher percentage of their shots from Open Play & Counter Attacks than via any other method (penalties, set pieces etc)

There are obviously caveats to the above data but it gives an idea that a chairman can use statistics to see how his team performs in an attacking sense.

6) Points over Expected

This utilises an idea from Simon Gleave, in that some games managers are EXPECTED to win.

Take Chelsea for example. They would be expected to win a home game against QPR, so when they do pick up the 3 points does it mean they are performing well? On the reverse of this, a team such as Swansea or Crystal Palace may be in mid table, but picking up more points than they are expected to get.

So we can see whether the 5 managers we are looking at have under or over performed in the Championship games they have been in charge of. To do this I will use the Bookmakers odds from Bet365 to calculate the probability that they would win the game.

To look at an example for Blackburn vs Ipswich on the final day of the 2014/15 season the odds were 3.3 for Blackburn, 3.5 for the draw and 2.3 for Ipswich - this means that the bookies thought Blackburn had a 29.6% chance of winning, there was a 27.9% chance of a draw and Ipswich had a 42.5% chance. We can use that to estimate the number of points they were EXPECTED to get from the game = Blackburn were expected to get 1.17 points, Ipswich were expected to get 1.55 points (Win % * 3 + Draw % * 1). The final result was a Blackburn win, so Blackburn over performed by 1.83 points and Ipswich under performed by 1.55 points. We can use this method in the table below to see how each manager has faired in league games under their tenure.

As you can see from the list above, all the managers are performing slightly better than expected - even the unpopular Slade has picked up more points than the bookies thought he would (this is due to their away form, they actually underperformed at home).

Again, this method isn't foolproof but it does give an idea of expectation and takes out some of the bias of having a better team.

7) Relationship with fans/chairman/players

Above all one thing that cannot be discounted is the relationship the Manager has with the fans, chairman and players. All the Win Ratios, Points per Pound and exciting football will do the manager no good if he doesn't have all 3 on board.
Take the example of Alan Pardew at Newcastle - the fans have been against him for a long time and though several tables showed he was over performing based on his expectations the relationship was at a point where his position was untenable. He has continued his relative success at Crystal Palace, where he shares a much better relationship.
Similar examples exist when managers have "lost the dressing room" - which supposedly happened to Beppe Sannino at Watford, despite them sitting top of the Championship after 6 games.
Losing the support of the chairman will always lead to the sack no matter how highly the fans regard you - it seems at Leeds it is better to have the support of the chairman than it is the fans or players which Neil Redfearn found out to his cost.

These are just some examples - I'm sure there are others but maybe it's time to look beyond the obvious and easy statistics and try something different.

Thanks for reading.

No comments:

Post a Comment