Monday, October 10, 2011

How I Evaluate Players

Disclaimer: Nothing in this post is going to be new, or groundbreaking. I'm not that advanced (yet). I've read a lot about this game, and I agree with some things more than others. I'm here to share with you what I agree with and what I believe.

1. Everything Is Relative

Too many times, I hear and read fans say things like, "Why did we sign Player X! He's terrible! Only 30 points last season and he's 34 years old!" or "Wow, Player Y is incredible. I'm so happy we traded for him yesterday! He's coming off four straight 60 point seasons!" Most fans can process simple stats like goals, assists, +/- (more on that later), etc to determine how good a player is. The problem is, they process these things in a vacuum.

It's not enough to tell me that Player X scored 30 points in his age 34 season, or that Player Y racked up 60 points four straight years. I need to know (at least) one other thing: cap hit. People know the cap exists, they might even have basic knowledge of how it works. Yet, too often, they can't seem to take it into account when evaluating talent.

This doesn't make any sense to me, because people use and process relative value in other aspects of life all the time, even at the very hockey games they watch and make these shallow observations. How many people out there don't like to buy beer or a cheeseburger at an NHL arena because it costs $9? And how many of those same people will gladly buy the same product before or after the game at the cheap dive bar two blocks from the arena?

I have read the entire NHL CBA and I have made it a point to know the salary cap and its rules like the back of my hand. I don't profess to know it better than any NHL GM, but I know it better than 99% of fans. I guarantee you this, and to be honest, I should know it that well if I take myself serious in this industry. It's possible that my perceptions are altered by the priority I put on the quirks of the cap that most people don't even know. But really, the extreme basics of the cap are easy to understand, and they should shape people's opinion more than they do.

Let's take a look at all of the NHL players with current cap hits above $7m, courtesy of capgeek.com, the greatest site in the universe:



Almost any fan would love to have anyone on this list on their team. Fans need to stop thinking that way. I look at this list and think ,"Wow, nearly every one of these guys is entirely overpaid." I consider only three of these players worth their price: Stamkos, Weber, and Doughty. If Crosby didn't have the concussion scare, he would be a fourth. I consider every other contract on this list to be a burden to the team, relatively speaking.

The NHL salary cap is a puzzle; it's a game within a game. It's like trying to pack belongings into a confined suitcase. Maybe you don't need your huge laptop. Maybe you are better off with just your cell phone, which admittedly can't do as much as your laptop, but it can get you by, and by packing the cell phone instead of the laptop, you have space for other useful additions. You're trying to create the most efficient load in the size of your suitcase. Efficiency and relative value is the key.

There are exceptions, I get that. Pittsburgh can afford to pay Evgeni Malkin what he wanted because they have managed to convince all of their other stars and role-players to take nice discounts. Florida can afford to take on Brian Campbell's huge deal because they needed to get to the salary cap floor. The Blue Jackets needed to make a major commitment to Rick Nash to get him to give up his UFA years and stay in Columbus. That's why I bolded and italicized "relatively speaking" above. But it's about time fans started to analyze trades and free agent signings thinking about the cap hits concurrently with the goals and assists.

Another relative aspect of statistics that I need to know: ice time. If a fringe defenseman puts up 30 points getting only 8 minutes a game, this is better than someone putting up 60 in 20 minutes a game. Not that difficult to understand, but often overlooked. Hint: look at Ben Lovejoy's stats from last season. He ranked 2nd in the entire NHL in even-strength points/60 minutes among defensemen with 40+ games. This made him the second most valuable Pittsburgh defenseman, behind only Kris Letang. Yes, Ben Lovejoy contributed more towards Pittsburgh's goal differential, given his ice time, than Paul Martin, Brooks Orpik, or Zbynek Michalek. Hard to believe, but true. (credit: Hockey Prospectus)

Important: Am I trying to say that I would rather have Lovejoy than Orpik? Or that I would like to give Lovejoy Orpik's minutes? Absolutely not. What I am saying is that Lovejoy fits his currently established role way better than probably anyone else in the league. And he's the type of player I would look to acquire for this role, on the cheap, if I were a GM.

This is so paramount to the way I think about hockey that I plan to blog and write about every major transaction, analyzing it from this standpoint. In the future, I'm also going to share with you my thoughts on best and worst contracts. Obviously, most of the "worst" entries are in the list above. My thoughts on "best" may surprise you. Lovejoy would be a prime example, for one. The reasoning behind the others mostly has to do with the rest of this blog entry.

2. Most Individual Stats Are Horrible, But Some Are More Horrible Than Others

+/- is the most overrated statistic ever. It's a lot like pitcher wins or batter RBI in baseball. Too much of it, and in fact the majority of it, depends on teammates. Whoever invented it had great intentions, but didn't think about the big picture. On the surface, it's a great idea: let's record goals scored while Player A is on the ice for both his team and the opponents, and whoever has the highest difference is probably a better player, right? The obvious flaw with this: a player can get a +/- change for plays that he potentially had nothing to do with. Example: David Backes is in front of the net on the Blues power play. Kevin Shattenkirk is on the point. Shattenkirk tries to make a cross-ice pass, it gets picked off for a breakaway, and Mike Richards pots the six billionth shorthanded goal of his career. Backes has no responsibility to defend that, he was doing everything right, and yet he takes a minus-1 for his efforts.

Most people can see this shortcoming. What isn't readily apparent is this: players sometimes ARE involved in the play, DO make consequential decisions, and can still get/not get pluses or minuses they do/don't deserve. That sounded confusing, so let me give you some extreme, made-up examples:

1) Alex Goligoski makes an incredible cross-ice pass to Jamie Benn. Benn tries to pick a corner, but hits the crossbar. This happens 45 times in the same game.

2) The Panthers are playing the Islanders, and it's a shooting gallery for Florida. Mark Streit is on the ice for 200 shots against his team. Of these 200, 40 are blocked, 40 miss the net, and 120 get through to Al Montoya, who saves every one. Streit is also on the ice for his team's only goal of the game, which happened to be on a dump-in that squirted by Jose Theodore. He finishes the night a +1.

3) Shane Doan is on the goal line and tries to make a cross-crease pass. He misses his target by 5 feet and the puck comes right to Erik Johnson, defending for Colorado. Johnson tries to clear, but Kyle Quincey's positioning is poor, and Johnson's shot bounces off Quincey's shin pad and into the net. An "own goal" that gives Doan a +1 and Johnson a -1.

In the first example, Goligoski deserves some credit for making a great pass, but he's at the mercy of his linemates. In the second, Streit deserves some disparagement for being part of a unit that allows so many shots to Florida, but his goalie stood on his head. (Or, even more in-depth..maybe Streit had nothing to do with it! Maybe his teammates were so, so bad and essentially responsible for 195 of the 200 shots. Maybe some of them actually went in. Either way, Streit's plus-minus wouldn't reflect his true value.) In the third example, neither player involved deserved what they got (except Quincey, who probably deserves a ticket to Lake Erie in this scenario).

Here's a real-life example: Toni Lydman was paired with Lubomir Visnovsky all last year for the Ducks. A fine player, yes. He finished T-2 in the NHL at final +/- at a +32. Visnovsky was a +18 on the year. Fellow Ducks D-man Cam Fowler was a -25. What gives?

Here's what happened (and I credit Hockey Prospectus for the stats). Lydman was incredibly lucky to achieve this mark. While he was on the ice, his teammates' shooting percentage was 11.7%. The NHL league average has been roughly 9% every year since the lockout (and trending downward). See a problem here? Lydman's teammates shot the lights out while he was on the ice. That 11.7% average is only going to go down. Furthermore, Lydman's goaltenders' save percentage while he was on the ice was .927, much better than the Ducks' overall team save percentage of .912. Once these statistics normalize, Lydman's +/- will too. There's no way around it.

Here's something that may surprise you: for every single goal that is scored, on average, 35% of it is luck! And of that 35%, 33% of it is related to shooting percentage. This is courtesy of Gabriel Desjardins via Hockey Prospectus, who ran a study that completely blew my mind.

Desjardins computed that there are three main components to scoring a goal: quality of shot location, taking the shot in the first place, and the quality of the shot itself. Having the ability to get the shot off requires no luck at all. Either you can or you can't. Shot location is slightly influenced by luck, but not much.  However, shooting percentage is completely fluctuating and is 33% luck. Good players will score more, yes. Good players can pick corners better than others, I'm not denying that. But it's largely like batting average in baseball. Good baseball players also can hit a lot of hard balls right at people. Those outs aren't indicative of their talent. Good hockey players can sometimes get robbed by better saves.

Here's what you, the skeptic, is probably thinking: "Maybe Lydman and his teammates are just that good! Maybe Lydman keeps making perfect passes where his teammates have easy shots!" Sorry, I don't buy it. Keeping up a team shot percentage of 11.7% and a team save percentage of .912 are just unsustainable.

Credit to Hockey Prospectus here: the Ducks' PDO (a stat that adds their team save percentage and shooting percentage) in 2010-11 was 101.6. Typically, league average hovers around 100. Think about this: teams face/create roughly 5000 shots in a season. A meager 1% swing in this mark (back towards the league average) equates to 50 goals. If Lydman were on the ice a third of the time, that's about 17 goals, or roughly half of his +/-! Lydman, and Visnovsky to an extent, were beneficiaries of the right linemates at the right time.

(For these reasons, Corey Perry's insane 17.2% shot percentage will also go down, by the way. I don't believe he's a 50-goal scorer ever again. One way this could turn out being wrong: Anaheim's total lack of forward depth. They need him/Ryan/Getzlaf to score all of their goals because no one else will, and he might get 50 due to sheer shot volume. But he won't ever get 50 on 290 shots again. I digress.)

It's admirable to measure team contributions to goal scoring and prevention. There is definitely some defensive liability in Cam Fowler's -25 and some testament to good play in Toni Lydman's +32. +/- just isn't the best way to measure these things. Luckily, Hockey Prospectus and others are working on a solution: Corsi. Corsi is a lot like +/-, except using shot attempts instead of goals. It completely removes the fluctation and luck in actual goal scoring. Remember, 35% of goal scoring is luck. 33% of this is actually finishing the shot. Getting shots off and being in position to do so is much less influenced by luck. That's what Corsi measures: the shot attempts for and against a player while they are on the ice. Corsi doesn't care if the shot was blocked, if it goes in, or if it misses the net. That's the luck-based aspect of goal-scoring. We only care about a player's ability to create or prevent shots, and in turn, possession.

What's the best way to win a hockey game? Allow absolutely no shots against your goaltender, and take as many as possible. Corsi measures the ability of a player to essentially do this, and it does so from both a defensive and offensive perspective. It does what +/- attempts to do in a much less fluky way.

Note: you can break this down further to a player's role, what zone they started in, etc. And I probably will in a future post. But for now, it's not important to the main point.

I imagine this went a lot like the people who realized the problems with batting average in baseball. "Hey, let's measure the percentage that a player gets a hit." Great idea! Fast forward a bit: "Wait a minute, a walk is just as good as a hit sometimes." So on-base percentage is invented. "Wait a minute, home runs are more valuable than walks sometimes." So slugging percentage is invented. And now we use all three, in context, to better evaluate a player. And we also use those advanced stats to create more advanced stats like VORP.

Hockey is headed the same route, and I can't wait for that transformation. We already have GVT (hockey's version of VORP), and I would imagine that we are headed down a route of win expectancy as well. It's a lot harder to quantify, because hockey is free-flowing, unlike the rigid, event-turn based nature of baseball. But it can and will be done.

If this section sounds like an entire quote of Hockey Prospectus, that's because it pretty much is. I can't and won't take credit for any of this. But HP has finally put into numbers and words what I've longed for forever. I can't wait to get to work with Corsi in the future as a better tool of analyzing players, and use it to develop more advance metrics. It's only going to become more prevalent.

3. What's The Difference Anyway?

Too many hockey fans measure team performance in wins and losses. Yes, winning and losing is the crux of the game, but it's too simplistic to think about it in this way. Just like Moneyball says about player acquisition: teams should be trying to buy runs, not wins. It's the same in hockey. What happens over an 82-game season is largely a product of goal differential. When front offices look to acquire players, they should be trying to acquire players that can put their teams into creating goal scoring opportunities, and shutting down opponent's abilities of doing the same.

Goal differential is the key. Improve your goal differential, and you improve your chances of winning games over the long term. Since goal scoring is fluctuating and very much influenced by luck, it's not enough to acquire the players with the highest goal totals. We need to look at the bigger picture, a larger sample size. We need to dig deeper. That's what the advanced stats I described in the last section do, and that's what GVT also does. It measures a player's contributions in goals over replacement level for the same amount of ice time. It's really a great summation of a player's contributions to a team.

Bottom line, even terrible teams can put up five goals in a game, or allow zero. Sometimes, they can even do it in the same game. But putting together a winning team is about some combination of the two over the long haul. I'm not interested in how we improve our goal differential. If we score 500 goals and allow 350, that's the same to me as if we score 200 and allow 50. Both teams are probably going to win the same number of games. For real life, think of it in context of a team scoring 200 and allowing 160, or a team scoring 230 and allowing 190. Two ways to skin a cat here; I'm going to pick whichever is more cost efficient given my team's constraints of the salary cap.

You can acquire skaters with a high defensive GVT, a high offensive GVT, or preferably, players that have both. Perhaps even more preferably, players that can do both in ways that don't show up on the back of his hockey card, and thus probably cost less. That's who I'm going to target. I don't know if there is a more important statement in this entire essay.

4. Scouting Is Still Important Too

People call baseball a team game, but it really isn't. Think about it. The only time any teamwork is needed is on turning double plays, cutoff throws, and the pitcher-catcher interaction. The rest of a baseball game (and the meat of it) is hundreds of disjointed events that create situations. These situations are then acted on entirely by individual effort. Alex Rodriguez has no control over what Robinson Cano does in front of him. If Cano gets on base with one out, maybe A-Rod tries to move him over. Maybe he tries to hit a homer. Regardless, there is no teamwork involved in A-Rod's success or failure here. This is why it is much easier to measure a baseball player's value than a hockey player's, especially a hitter's. That's also why I 100% subscribe to stats before scouting, as far as baseball is concerned. Give me a player that can get on base and not create outs. Very simplistic, I know, but this is a hockey blog. I think you get what I'm going for here.

In hockey, it's different. Hockey is perhaps the most team-intrinsic game there is. It's much harder to measure hockey in this way because of this team-based nature. Goal scoring is almost entirely a team effort. It flows together. Run scoring in baseball largely is not.

Here's a typical baseball sequence: Player A does something --> break in play --> player B comes up and does something else --> runs may or may not score.

Here's a hockey sequence: Player A passes to player B  --> teammates move simultaneously --> player B notices movement and passes to player C --> player C scores.

Yes, in baseball, one event leads to the next, but those two events are entirely independent of one another. The simultaneous, dynamic movement of hockey makes contribution more difficult to measure. This is why I cannot say (or no one should ever say) that I 100% prefer to ignore scouting in hockey.

What's the reason hockey is the hardest sport to play in existence? Because it involves an entirely new method of movement. Football, basketball, baseball, etc are hard, yes. I know. But what isn't hard is moving around in those sports. Anyone who plays them can move without thinking about it. LeBron James doesn't have to think about how he's going to run up the court. In hockey, that's not the case. The method of moving from point A to point B is an entirely new, not-naturally learned skill. And it's the entire foundation of the game. If you can't skate, you can't move, and if you can't move, you can't do anything. This is, simplistically, why I believe that scouts should 100% prefer skating ability to anything else, all things considered.

Positioning is also extremely important. When I watch a game, I'm watching to see where players are away from the puck. Are they putting themselves in position to create or deny future shot attempts (thus affecting their Corsi)? You can see how scouting and statistics work hand-in-hand here.

The key with these skills I am singling out is that they cannot be accurately represented in numbers. A player's shot strength translates to goal scoring, which can be measured in more advanced metrics. Of course I am interested in if a player has a laser, but I can "see" these things through statistics much more clearly than I can skating and positioning.

I had the fortune to attend about 30 home games of the Columbus Blue Jackets last season. Watching hockey live is an entirely different experience from on television. You can see these things much more clearly. You can see who is affecting the game more than others, and by attending several games in sequence, you can see who has the consistency. I learned more about hockey last year than the other 19 previous years of my life combined because I had the ability to attend games in person regularly. That cannot be replaced by a stat-sheet.

How These Things Work Together

I watch games taking all of these things into account. When I'm watching a team play, I'm thinking about the player's cap hit, his ability to create and deny shots, and his physical tools/hockey sense. I decide value based on these things. It's all about possession and goal differential for the cost it requires me to pay for it. Groundbreaking? No. But I'm not trying to break new ground...yet. I'm trying to assert myself in this industry.  I still have so, so much more to learn.

No comments:

Post a Comment