Modelling the Brownlow
This walkthrough will provide a brief, yet effective tutorial on how to model the Brownlow medal. We will use data from 2010 to 2018, which includes Supercoach points and other useful stats.
The output will be the number of votes predicted for each player in each match, and we will aggregate these to create aggregates for each team and for the whole competition. No doubt we'll have Mitchell right up the top, and if we don't, then we know we've done something wrong!
# Import modules libraries
import pandas as pd
import h2o
from h2o.automl import H2OAutoML
import numpy as np
from sklearn.preprocessing import StandardScaler
import os
import pickle
# Change notebook settings
pd.options.display.max_columns = None
pd.options.display.max_rows = 300
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
This walkthrough uses H2O's AutoML to automate machine learning. We have saved the models created by AutoML into the Github repo which can be loaded into the notebook to save you time training the model. However a drawback of H2O is that you can only load models if your currently installed verson of H2O is the same as the version used to create the model. You can check what version of H2O you have installed by running h2o.__version__:
If you have a different version of H2O installed you have two options. You can train the models yourself, all the code to do that is commented out in the notebook. Or you can pip uninstall H2O and then pip install H2O again, but specifically the 3.36.0.3 version. The code to do this is below and will only take 1 or 2 minutes:
# # Uncomment the code below if you need to uninstall the current version of H2O and reinstall version 3.36.0.3
# # The following command removes the H2O module for Python.
# pip uninstall h2o
# # Next, use pip to install this version of the H2O Python module.
# pip install http://h2o-release.s3.amazonaws.com/h2o/rel-zorn/3/Python/h2o-3.36.0.3-py2.py3-none-any.whl
EDA
Read in the data
I have collated this data using the fitzRoy R package and merging the afltables dataset with the footywire dataset, so that we can Supercoach and other advanced stats with Brownlow votes. Let's read in the data and have a sneak peak at what it looks like.
It looks like we've got about 76,000 rows of data and have stats like hitouts, clangers, effective disposals etc. Let's explore some certain scenarios. Using my domain knowledge of footy, I can hypothesise that if a player kicks 5 goals, he is pretty likely to poll votes. Similarly, if a player gets 30 possessions and 2+ goals, he is also probably likely to poll votes. Let's have a look at the mean votes for players for both of these situations.
Exploring votes if a bag is kicked (5+ goals)
Exploring votes if the player has 30+ possies & 2+ goals
As suspected, the average votes for these two situations is 1.87! That's huge. Let's get an idea of the average votes for each player. It should be around 6/44, as there are always 6 votes per match and around 44 players per match.
So the average vote is 0.12. Let's see how this changes is the player is a captain. I have collected data on if players are captains from wikipedia and collated it into a csv. Let's load this in and create a "Is the player captain" feature, then check the average votes for captains.
Create Is Player Captain Feature
captains = pd.read_csv('data/captains.csv').set_index('player')
def is_captain_for_that_season(captains_df, player, year):
if player in captains_df.index:
# Get years they were captain
seasons = captains_df.loc[player].season.split('-')
if len(seasons) == 1:
seasons_captain = list(map(int, seasons))
elif len(seasons) == 2:
if seasons[1] == '':
seasons_captain = list(range(int(seasons[0]), 2019))
else:
seasons_captain = list(range(int(seasons[0]), int(seasons[1]) + 1))
if year in seasons_captain:
return 1
return 0
brownlow_data['is_captain'] = brownlow_data.apply(lambda x: is_captain_for_that_season(captains, x.player, x.season), axis='columns')
This is significantly higher than if they aren't captain. What would be interesting is to look at the average difference in votes between when they were captain and when they weren't, to try and find if there is a 'captain bias' in brownlow votes. Go ahead and try. For now, we're going to move onto feature creation
Feature Creation
Let's make a range of features, including: * Ratios of each statistic per game * If the player is a captain * If they kicked a bag (4/5+) * If they kicked 2 and had 30+ possies
First we will make features of ratios. What is important is not how many of a certain stat a player has, but how much of that stat a player has relative to everyone else in the same match. It doesn't matter if Dusty Martin has 31 possessions if Tom Mitchell has had 50 - Mitchell is probably more likely to poll (assuming all else is equal). So rather than using the actual number of possessions for example, we can divide these possessions by the total amount of possessions in the game. To do this we'll use pandas groupby and transform methods.
Create Ratios As Features
%%time
# Get a list of stats of which to create ratios for
ratio_cols = ['CP', 'UP', 'ED', 'DE', 'CM', 'GA', 'MI5', 'one_perc', 'BO', 'TOG',
'K', 'HB', 'D', 'M', 'G', 'B', 'T', 'HO', 'I50', 'CL', 'CG', 'R50',
'FF', 'FA', 'AF', 'SC']
# Create a ratios df
ratios = (brownlow_data.copy()
.loc[:, ['match_id'] + ratio_cols]
.groupby('match_id')
.transform(lambda x: x / x.sum()))
feature_cols = ['date', 'season', 'round', 'venue', 'ID', 'match_id', 'player', 'jumper_no', 'team',
'opposition', 'status', 'team_score', 'opposition_score', 'margin', 'brownlow_votes']
# Create a features df - join the ratios to this df
features = (brownlow_data[feature_cols].copy()
.join(ratios))
Kicked A Bag Feature
Is Captain Feature
Won the Game Feature
30+ & 2+ Goals Feature
Previous Top 10 Finish Feature
I have a strong feeling that past performance may be a predictor of future performance in the brownlow. For example, last year Dusty Martin won the Brownlow. The umpires may have a bias towards Dusty this year because he is known to be on their radar as being a good player. Let's create a feature which is categorical and is 1 if the player has previously finished in the top 10. Let's create a function for this and then apply it to the afltables dataset, which has data back to 1897. We will then create a lookup table for the top 10 for each season and merge this table with our current features df.
afltables = pd.read_csv('data/afltables_stats.csv').query('Season >= 2000')
def replace_special_characters(name):
name = name.replace("'", "").replace("-", " ").lower()
name_split = name.split()
if len(name_split) > 2:
first_name = name_split[0]
last_name = name_split[-1]
name = first_name + ' ' + last_name
name_split_2 = name.split()
name = name_split_2[0][0] + ' ' + name_split_2[1]
return name.title()
afltables = (afltables.assign(player=lambda df: df['First.name'] + ' ' + df.Surname)
.assign(player=lambda df: df.player.apply(replace_special_characters))
.rename(columns={'Brownlow.Votes': 'brownlow_votes', 'Season': 'season', 'Playing.for': 'team'}))
### Create Top 10 rank look up table
brownlow_votes_yearly = (afltables.groupby(['season', 'player', 'team'], as_index=False)
.brownlow_votes
.sum())
brownlow_votes_yearly['yearly_rank'] = (brownlow_votes_yearly.groupby('season')
.brownlow_votes
.rank(method='max', ascending=False))
# Filter to only get a dataframe since 2000 and only the top 10 players from each season
brownlow_votes_top_10 = brownlow_votes_yearly.query('yearly_rank < 11 & season >= 2000')
brownlow_votes_top_10.head(3)
def how_many_times_top_10(top_10_df, player, year):
times = len(top_10_df[(top_10_df.player == player) & (top_10_df.season < year)])
return times
features['times_in_top_10'] = features.apply(lambda x: how_many_times_top_10(brownlow_votes_top_10, x.player, x.season), axis=1)
Average Brownlow Votes Per Game Last Season Feature
# Create a brownlow votes lookup table
brownlow_votes_lookup_table = (brownlow_data.groupby(['player', 'team', 'season'], as_index=False)
.brownlow_votes
.mean()
.assign(next_season=lambda df: df.season + 1)
.rename(columns={
'brownlow_votes': 'ave_votes_last_season'
}))
# Have a look at Cripps to check if it's working
brownlow_votes_lookup_table[brownlow_votes_lookup_table.player == 'P Cripps']
# Merge it to our features df
features_with_votes_last_season = (pd.merge(features, brownlow_votes_lookup_table.drop(columns='season'),
left_on=['player', 'team', 'season'],
right_on=['player', 'team', 'next_season'],
how='left')
.drop(columns=['next_season'])
.fillna(0))
Historic Performance Relative To Model Feature
It is well known that some players are good Brownlow performers. For whatever reason, they always poll much better than their stats may suggest. Lance Franklin and Bontempelli are probably in this category. Perhaps these players have an X-factor that Machine Learning models struggle to pick up on. To get around this, let's create a feature which looks at the player's performance relative to the model's prediction. To do this, we'll need to train and predict 7 different models - from 2011 to 2017.
To create a model for each season, we will use h2o's AutoML. If you're new to h2o, please read about it here. It can be used in both R and Python.
The metric we will use for loss in Mean Absolute Error (MAE).
As we are using regression, some values are negative. We will convert these negative values to 0 as it doesn't make sense to poll negative brownlow votes. Similarly, some matches won't predict exactly 6 votes, so we will scale these predictions so that we predict exactly 6 votes for each match.
So that you don't have to train these models yourself, I have saved the models and we will load them in. If you are keen to train the models yourself, simply uncomment out the code below and run the cell. To bulk uncomment, highlight the rows and press ctrl + '/'
h2o.init()
# Uncomment the code below if you want to train the models yourself - otherwise, we will load them in the load cell from disk
## Join to our features df
# aml_yearly_model_objects = {}
# yearly_predictions_dfs = {}
# feature_cols = ['margin', 'CP', 'UP', 'ED', 'DE',
# 'CM', 'GA', 'MI5', 'one_perc', 'BO', 'TOG', 'K', 'HB', 'D', 'M', 'G',
# 'B', 'T', 'HO', 'I50', 'CL', 'CG', 'R50', 'FF', 'FA', 'AF', 'SC']
# for year in range(2011, 2018):
# # Filter the data to only include past data
# train_historic = brownlow_data[brownlow_data.season < year].copy()
# # Convert to an h2o frame
# train_h2o_historic = h2o.H2OFrame(train_historic)
# # Create an AutoML object
# aml = H2OAutoML(max_runtime_secs=30,
# balance_classes=True,
# seed=42)
# # Train the model
# aml.train(y='brownlow_votes', x=feature_cols, training_frame=train_h2o_historic)
# # save the model
# model_path = h2o.save_model(model=aml.leader, path="models", force=True)
# # Get model id
# model_name = aml.leaderboard[0, 'model_id']
# # Rename the model on disk
# os.rename(f'models/{model_name}', f'models/yearly_model_{year}')
# # Append the best model to a list
# aml_yearly_model_objects[year] = aml.leader
# # Make predictions on test set for that year
# test_historic = brownlow_data[brownlow_data.season == year].copy()
# test_h2o_historic = h2o.H2OFrame(test_historic)
# preds = aml.predict(test_h2o_historic).as_data_frame()
# test_historic['predicted_votes'] = preds.values
# # Convert negative predictions to 0
# test_historic['predicted_votes_neg_to_0'] = test_historic.predicted_votes.apply(lambda x: 0 if x < 0 else x)
# # Create a total match votes column - which calculates the number of votes predicted in each game when the predictions
# # are unscaled
# test_historic['unscaled_match_votes'] = test_historic.groupby('match_id').predicted_votes_neg_to_0.transform('sum')
# # Scale predictions
# test_historic['predicted_votes_scaled'] = test_historic.predicted_votes_neg_to_0 / test_historic.unscaled_match_votes * 6
# # Aggregate the predictions
# test_grouped = (test_historic.groupby(['player', 'team'], as_index=False)
# .sum()
# .sort_values(by='brownlow_votes', ascending=False)
# .assign(mae=lambda df: abs(df.predicted_votes_scaled - df.brownlow_votes)))
# test_grouped['error'] = test_grouped.predicted_votes_scaled - test_grouped.brownlow_votes
# test_grouped['next_year'] = year + 1
# # Add this years predictions df to a dictionary to use later
# yearly_predictions_dfs[year] = test_grouped
# preds_errors = None
# for key, value in yearly_predictions_dfs.items():
# if preds_errors is None:
# preds_errors = value[['player', 'season', 'next_year', 'brownlow_votes', 'predicted_votes_scaled', 'error']]
# else:
# preds_errors = preds_errors.append(value[['player', 'season', 'next_year', 'brownlow_votes', 'predicted_votes_scaled', 'error']], sort=True)
# with open('data/prediction_errors_df.pickle', 'wb') as handle:
# pickle.dump(preds_errors, handle)
Look at that! A simple Machine Learning ensemble model, using AutoML predicted last year's winner! That's impressive. As we can see it also predicted Bontempelli would only score 11.26, when he actually scored 19 - a huge discrepency. Let's use this as a feature.
features_with_historic_perf_relative_to_model = \
(features_with_votes_last_season.pipe(pd.merge, preds_errors[['player', 'next_year', 'error']],
left_on=['player', 'season'],
right_on=['player', 'next_year'],
how='left')
.fillna(0)
.rename(columns={'error': 'error_last_season'})
.drop_duplicates(subset=['player', 'round', 'SC']))
Filtering the data to only include the top 20 SC for each match
Logically, it is extremely unlikely that a player will poll votes if their Supercoach score is not in the top 20 players. By eliminating the other 20+ players, we can reduce the noise in the data, as we are almost certain the players won't poll from the bottom half. Let's explore how many players poll if they're not in the top 20, and then filter our df if this number is not significant.
Since 2014, there have only been 24 players who have voted and not been in the top 20 SC.
features_with_sc_rank = features_with_historic_perf_relative_to_model.copy()
features_with_sc_rank['SC_rank_match'] = features_with_sc_rank.groupby('match_id').SC.rank(method='max', ascending=False)
# Filter out rows with a SC rank of below 20
features_with_sc_rank_filtered = features_with_sc_rank.query('SC_rank_match <= 20')
Modeling The 2017 Brownlow
Now that we have all of our features, we can simply create a training set (2012-2016), and a test set (2017), and make our predictions for last year! We will use AutoML for this process again. Again, rather than waiting for the model to train, I will save the model so you can simply load it in. We will also scale our features. We can then see how our model went in predicting last year's brownlow, creating a baseline for this years' predictions. We will then predict this year's vote count.
train_baseline = features_last_before_train.query("season < 2017")
holdout = features_last_before_train.query("season == 2017")
scale_cols = ['team_score', 'opposition_score', 'margin', 'CP', 'UP', 'ED', 'DE',
'CM', 'GA', 'MI5', 'one_perc', 'BO', 'K', 'HB', 'D', 'M', 'G',
'B', 'T', 'HO', 'I50', 'CL', 'CG', 'R50', 'FF', 'FA', 'AF', 'SC']
other_feature_cols = ['is_captain', 'kicked_a_bag', 'team_won', 'got_30_possies_2_goals', 'times_in_top_10',
'ave_votes_last_season', 'error_last_season', 'SC_rank_match']
all_feature_cols = scale_cols + other_feature_cols
# Scale features
scaler = StandardScaler()
train_baseline_scaled = train_baseline.copy()
train_baseline_scaled[scale_cols] = scaler.fit_transform(train_baseline[scale_cols])
holdout_scaled = holdout.copy()
holdout_scaled[scale_cols] = scaler.transform(holdout[scale_cols])
# Convert categorical columns to categoricals
train_baseline_h2o = h2o.H2OFrame(train_baseline_scaled)
holdout_h2o = h2o.H2OFrame(holdout_scaled)
for col in ['is_captain', 'kicked_a_bag', 'team_won', 'got_30_possies_2_goals']:
train_baseline_h2o[col] = train_baseline_h2o[col].asfactor()
holdout_h2o[col] = holdout_h2o[col].asfactor()
Below I have commented out training and saving the 2017 model. Rather than training it again, we will just load it in. Uncomment this part out if you want to train it yourself.
# # save the model
# model_path = h2o.save_model(model=aml_2017_model.leader, path="models", force=True)
# # Get model id
# model_name = aml_2017_model.leaderboard[0, 'model_id']
# # Rename the model on disk
# os.rename(f'models/{model_name}', f'models/brownlow_2017_model_v1')
# Load model in
aml_2017_model = h2o.load_model('models/brownlow_2017_model_v1')
# Predict the 2017 brownlow count
preds_final_2017_model = aml_2017_model.predict(holdout_h2o)
# Scale these predictions - change negatives to 0s and scale so each game predicts 6 votes total
holdout = (holdout.assign(predicted_votes=preds_final_2017_model.as_data_frame().values)
.assign(predicted_votes_neg_to_0=lambda df: df.predicted_votes.apply(lambda x: 0 if x <0 else x))
.assign(unscaled_match_votes=lambda df: df.groupby('match_id').predicted_votes_neg_to_0.transform('sum'))
.assign(predicted_votes_scaled=lambda df: df.predicted_votes_neg_to_0 / df.unscaled_match_votes * 6))
# Create an aggregate votes df and show the average SC points and goals scored
agg_predictions_2017 = (holdout.groupby(['player', 'team'], as_index=False)
.agg({
'brownlow_votes': sum,
'predicted_votes_scaled': sum,
'SC': 'mean',
'G': 'mean'})
.sort_values(by='brownlow_votes', ascending=False)
.assign(mae=lambda df: abs(df.brownlow_votes - df.predicted_votes_scaled))
.reset_index(drop=True))
So whilst our model predicted Dangerfield to win, it was pretty damn accurate! Let's find the MAE for the top 100, 50, 25, and 10, and then compare it to 2018's MAE in week, when the Brownlow has been counted.
Modelling This Year's Brownlow
Let's now predict this year's vote count. These predictions will be on the front page of the GitHub.
train = features_last_before_train.query("season < 2018")
test = features_last_before_train.query("season == 2018")
# Scale features
scaler = StandardScaler()
train_scaled = train.copy()
train_scaled[scale_cols] = scaler.fit_transform(train[scale_cols])
test_scaled = test.copy()
test_scaled[scale_cols] = scaler.transform(test[scale_cols])
# # save the model
# model_path = h2o.save_model(model=aml.leader, path="models", force=True)
# # Get model id
# model_name = aml.leaderboard[0, 'model_id']
# # Rename the model on disk
# os.rename(f'models/{model_name}', f'models/brownlow_2018_model_v1')
# Load model in
aml = h2o.load_model('models/brownlow_2018_model_v1')
# Predict the 2018 brownlow count
preds_final_2018_model = aml.predict(test_h2o)
# Scale these predictions - change negatives to 0s and scale so each game predicts 6 votes total
test = (test.assign(predicted_votes=preds_final_2018_model.as_data_frame().values)
.assign(predicted_votes_neg_to_0=lambda df: df.predicted_votes.apply(lambda x: 0 if x <0 else x))
.assign(unscaled_match_votes=lambda df: df.groupby('match_id').predicted_votes_neg_to_0.transform('sum'))
.assign(predicted_votes_scaled=lambda df: df.predicted_votes_neg_to_0 / df.unscaled_match_votes * 6))
# Create an aggregate votes df and show the average SC points and goals scored
agg_predictions_2018 = (test.groupby(['player', 'team'], as_index=False)
.agg({
'predicted_votes_scaled': sum,
'match_id': 'count'}) # shows how many games they played
.sort_values(by='predicted_votes_scaled', ascending=False)
.reset_index(drop=True))
Now that we have the top 25, let's also look at the top 3 from each team.
If you're looking for a round by round breakdown, just have a look at the test dataframe.
And there we have it! In a single notebook we have made a fairly good Brownlow predictive model. Enjoy.
Disclaimer
Note that whilst models and automated strategies are fun and rewarding to create, we can't promise that your model or betting strategy will be profitable, and we make no representations in relation to the code shared or information on this page. If you're using this code or implementing your own strategies, you do so entirely at your own risk and you are responsible for any winnings/losses incurred. Under no circumstances will Betfair be liable for any loss or damage you suffer.