Updated May 10, 2012:
Updates:
Updated results through Spring'12 Classic
New charts showing team ranks by each classic season
New Overall Team Ratings
Individual Player Ratings (top 200) Appendix
Reformatted using forum code (no MS Word)
NHL'94 Rankings - A Statistical Look
A little while back I was reading this thread about an ongoing ladder league and I was intrigued by the idea of including the team rank in determining how much a win was “worth”. Meaning, a win using Edmonton vs. Detroit would mean more than a win using Chicago vs. Ottawa.
I wanted to come up with some empirical evidence to support the "what is the best team" discussions that come up from time to time. Chicago and Detroit are always mentioned as the best teams, but could I measure that, or support that notion quantitatively?
Below I go through my methods and findings in four sections:
Initial Team Rankings
Individual Player Rankings
Tying Players to Teams
Conclusions
(Spoiler shows the ranking of teams)
SECTION I – Initial Team Ranks
The first thing I did was take the ratings from the Building Lines with AJ series to come up with an initial ranking of teams. The ratings come from a respected A-level player (Angryjay93) and it is very thorough/detailed. I highly recommend reading it for all skill levels.
I weighted the forwards, defense, and goalies, 3-2-1 respectively, because that represents the number of players on the ice. I thought it was a good starting point in generating a team ranking. The results were as follows:
Well, that looked right, but was there anything I can do to support this ranking?
I decided to take the results from the classic league regular seasons (Spring'08 - Spring'12), for all divisions, and ranked the teams based on winning percentage. I figured individual skill would be factored out due to the volume of games and the general grouping of players (A,B,C, etc). I decided that the minimum was 100 games played (over 2 seasons). This eliminated anomalies such as IceStorm's season with the NYI (29-11, .745 win percentage, making them the #1 ranked team). The results:
The results were pretty amazing when compared to the weighted AJ rankings. Grouping the teams into 6 different tiers, the results matched closely with a few exceptions:
The colored tiers make comparisons easier. I decided that TIER I would be the top 5 teams, TIER II would be teams with an AJ rank above 7, TIER III above 6, TIER IV above 5, etc.
There were only 2 major discrepancies, and the rest of the teams fell nicely into place. The first discrepancy is CAL, who finished 10th in classic results, as compared to 5th in AJ's ranking and the other is LA, who finished 6th in classic compared to AJ's 12th. What is interesting here is that both teams are defined by very strong forwards, and poor goalies. CAL has a better ranked defense, giving them the higher AJ rank. Theoretically, CAL should do better than LA, however we can't completely eliminate things like user skill and team chemistry.
This also does not appear to be an anomoly. The chart below shows the ranks of the team win% for each of the 7 classic seasons included in this analysis. You'll notice that CAL doesn't finish better than 8th in ANY season, while LA finishes in the top 5 in 4 out of 7 seasons!
You may notice two other teams that didn't quite fall into place. VAN ranks 4th in classic, while AJ has them 6th. This is obviously very close and only shows up as a discrepancy because of the TIER system. The other is TOR finishing 13th, into TIER III vs AJ 10th (bottom of TIER II).
Despite those discrepancies, I felt comfortable with the general team rankings, both from a qualitative (AJ) and quantitative (Classic results) standpoint. SECTION II – Individual Players
My next goal was to take this down to the individual player level – what players made these teams so good? More specifically, what player attributes made them so good? I knew the player ratings in the original ROM didn't mean much as one major attribute – weight (the sole determinant of checking ability) -- was not even factored in, so I wanted to create a new player ranking based on attributes that mattered. I used two different methods to figure out player rankings as described below: METHOD 1 - New Player Rankings
Using Smozoma's Blitz Player Spreadsheet as a starting point, I modified the formulas to value lower weight (essentially keep the weight bug) and asked YOU, the community, how we should value the different player attributes.
The results of the survey can be seen here - GENS Player Attribute Survey
Based on those attributes, I created a new player ranking.
A quick glance at the top 5 forwards and defense made me feel I was on the right track: METHOD 2 - GDL Average Draft Positions
I took the draft results from seasons 5 through 8 of the GDL GENS Draft League and came up with an average draft position (ADP) for all the players. GDL allows owners to draft '94 players using a snake style draft and doesn't alter the original gameplay aside from team rosters. The ADP basically ranks all the players in '94 based indirectly on attributes that we ("we"= GDL team owners) value. Again, a quick glance at the top 5 forwards and defense showed I was on the right track:
The next step was to see if the new calculated rankings for the players I came up with correlated to the ADP. The results? Let's start with Defensemen. DEFENSEMEN
Weight reigns supreme with D! Out of the top 50 defensemen ranked by the GDL ADP compared to the calculated ratings, there were only 3 notable exceptions that stood out. I classified an exception as a difference of 15 or more spots between ADP and the calculated rank.
Patrice Brisebois (ADP 17, Calc 43) and Gord Hynes (ADP 19, Calc 60) were drafted considerably higher on average than their calculated indicated. The reason? Their weights are 5,4 respectively, and the rest of their attributes are pretty weak. Obviously weight is SUPER valued here. Of the top 16, the biggest difference was Petr Svoboda, whose ADP is 4, calc 10. Again, probably a weight bonus as Svoboda is a 5 weight.
The only other large variance was James Patrick, who was UNDERvalued at ADP 40, Calc 20. He is a 9 weight, but has other very respectable attributes such as 4 speed, 4 agility, 4 def awareness, 4 shot power, 4 stick handling, and 4 passing.
Other than those 3 mentioned, the top 50 defensemen were valued correctly from the survey attribute weights and GDL ADP. FORWARDS
After the initial survey results, the following players were drafted higher than their rating indicated:
Petr Klima, Russ Courtnall, Cliff Ronning, Tomas Sandstrom, Geoff Sanderson, and Brett Hull
Even though Speed and Shot Power were valued the highest in my attribute survey, they were apparently not valued high enough!
Meanwhile, other players such as Adam Oates, Gary Roberts, and Steve Larmer were undervalued. The comment element there is they all have strong awareness ratings, but were heavier.
Based on that initial result, I tinkered with the attributes values, increasing speed and shot power, and decreasing awareness, etc. and settled on this result:
These attribute ratings yielded just 1 variance in the top 50 forwards! That player? Tomas Sandstrom (ADP 25, Calc 41). Obviously Thomas' incredible shot (5/5) was valued over his 9 weight for GDL. (Side note: Increasing the shot power and accuracy attributes also yielded just one outlier, Cliff Ronning, as his speed and pass accuracy were valued over his weaker shot.)
My feeling is that we inherently value attributes in a more dynamic way than a static number. Certain combinations of attributes are valued greater (i.e. a player that has 5 shot and 5 accuracy is valued more than the calculation would indicate) and extreme ratings (6 speed, 6 shot power, 3 weight) also command greater value. I'll leave that for a future analysis. Anyway, I was happy that this weighting criterion for forwards provided a good estimate of player ratings as 49 of the top 50 calculated players correlated well with their ADP in GDL. GOALIES
Goalies? The GDL and the normal goalie rankings were nearly identical, suggesting that the goalie ratings in the original ROM are accurate, or we just don't know. Either way, this was straightforward. SECTION III – Players to Teams
Now that I had a good ranking of teams (SECTION I) and a good ranking of players (SECTION II). The question now was how do I marry the two? This becomes a little tricky as each team has a unique identity that defines how you can use them to win games. This is part of what makes '94 so great!
But of course I'll try. =) The goal was to see if there was a way to come up with team rankings based on the player ratings that matched the classic results/AJ rankings.
At first I ranked the teams based on the best 5 players (3F, 2D) and goalie, however this didn't make sense as the value of a team is more than the best 5 guys on the ice. I decided that it's important to weigh the #1 forward more than #2 and #3, and the #1 D more than #2. Chicago is the best example -- the reason the team is great on offense is because of Roenick, the #1F, helped by a decent support cast. Take Roenick out and the offense is vastly different. So, after some tinkering, I weighed the positions as follows: 1F (9), 2F (5), 3F (3), 1D (5), 2D (3), G (6) to come up with a total team score.
I ended up with these weights because it generated a team rank that came close to the classic/AJ rankings, with the exceptions below. The exceptions make sense when analyzed: Winnipeg - The biggest exception is that Winnipeg vaults into Tier 1, instead of the expected Tier 3. Housley actually ranks as the #1 Defenseman and Selanne the #6 Forward, and weighing those guys 6 and 3 times more is what drives up WPG's ranking. However, they don't have depth, and they are also known to have chemistry issues. From AJ’s analysis, "Although front loaded with superstars such as Phil Housley and Teemu Selanne, the Jets are not a premier team in the league. Their lack of depth and chemistry, chiefly at the forward position, allow opponents to key in on the speedy duo in an attempt to limit their damage. In the hands of a one on one specialist though, Housley and Selanne are as potent as any duo in the league." Dallas also drops, mainly due to their abysmal Defense and Goalie ratings. My opinion here is that people do well with Dallas using the "best defense is offense" strategy, mainly utilizing Modano and Courtnall's great speed together. Manual goalie in human play can also help alleviate some of that rating noise.
Lastly, Edmonton falls to Tier 4, given the lack of skill outside of Klima. "The main issue with the Oilers forward unit is that every player aside from Klima has a weakness that keeps them from being an upper echelon player". But the dynamic of Klima with Todd (low weight bruiser) and Simpson (big shot wing) do much better than the raw calculations suggest.
NYR & PHI also hover around the fringes of their tiers, each calculating to move up one tier -- the NYR being calculated as TIER II and PHI Tier III. Again, these are close enough and will probably continue to shift around as more classic results are added into this analysis. SECTION IV– Results
Based on all this data, this is how I break down the NHL'94 teams. The tiers are arbitrary, and open for debate (particularly Tiers II & III), but overall I think this is a solid representation of team ranks: TIER I (the strongest)
CHI
DET
BUF
MTL
CGY (Classic Results in Tier II) TIER II (strong)
VAN (Classic Results in Tier I)
BOS
DAL
WPG
TOR (Classic Results in Tier III) TIER III (good teams, competitive)
QUE
LA (Classic Results in Tier II)
EDM
NYR TIER IV (flawed, weak teams)
PHI
HFD
PIT
STL
WSH TIER V (poor teams)
NJ
NYI
SJ
TB TIER VI (The worst)
FLA
OTW
ANH
Below is the calculated team rankings (they differ from team strength) Appendix - Individual Player Ratings
