Someone needs to teach Goldman Sachs about “markets”: World Cup edition

Bookmark and Share

I now have the full World Cup probabilities listed. This includes the likely outcome of every game and likely outcome of every team making it to every round. The predictions are as accurate as possible, based on historical correlations in both the World Cup specifically and my methods in general, and answer the question that most stakeholders ultimately care about: who is going to win. The prediction are updating every few minutes, allowing us to examine the impact of events during the game and early games on later games. There are many other predictions to follow, but I am confident in my accuracy and continuously updating make my predictions the most useful to interested stakeholders.

Two predictions that have been forwarded to me several time are by Bloomberg and Goldman Sachs. First, let me concede, while I try my best, these two strictly dominate me in style; these are really pretty reports! But, I do beat them as far as being a useful prediction.

These are pure fundamental models. Bloomberg does not provide much detail of this model. But, Goldman uses most of the same variables I use in my fundamental model: Elo rankings, goals for, goals against, dummy for type of match, home field, and home continent. There are a few small differences: I include friendly matches, I do not include home continent, etc. All of this is going to add up to noise for the predictions (i.e., not really make much of a difference). In short, both of our methods are completely sound.

The problem with pure fundamental models is that even the best fundamental models are lacking because the World Cup is an event held just once every four years without any regular season: there is a lot of idiosyncrasy in the event which is hard to capture in historical datasets. Goldman states, “To be clear, our model does not use any information on the quality of team or individual players that is not reflected in a team’s track record. For example, if a key player who was responsible for a team’s recent successes is injured, this will have no bearing on our predictions.” While this can be corrected for, to an extent, with individual-level data, they are referring to the idiosyncratic data that is included in the prediction market data. Thus, my prediction market-based predictions are going to be more accurate and updatable as the event progresses.

Goldman goes not to say that “There is no role for human judgment as the approach is purely statistical.” They should be applauded for that, but chided for not recognizing that the data exists; they do not need to add human judgment to note the effect of an injury.

This point is reflected well in the scatterplot of my forecasts for the 32 teams in reaching the round of sixteen and the round of eight. If the predictions were the same, they would run on the 45 degree diagonal; by definition the average prediction is 50% for reaching the round of 16 and 25% for reaching the round of 8. Notice that predictions from Bloomberg and Goldman are much flatter than mine: favorites are less favored and underdogs are more favored. This makes sense, this means we are all well calibrated in that the fundamental-based models accept that they have less information and more uncertainty.

There are two further peculiarities about the Goldman report: they compare their predictions to a broker not a market and they have Brazil as 48.5% likely to win. First, as one of the leading investment banks in the world, I am surprised they would compare their probabilities to the bid price of broker. Ladbrokes needs to make a profit and they do so by selling their predictions for more than they are worth. To guarantee a $1 return you would need to invest $1.18 to buy all of the teams to win. Second, the very, very under identified home field and continent advantage is what drives their prediction for Brazil to 48.5%. There are not that many World Cups to it is hard to identify the true advantage of hosting one. It is similar to home state in the presidency, which is also poorly identified. Brazil is extremely likely to win, I have them at 23%, but would Goldman advise their clients to buy Brazil long at 48.5%?!?

With that context, let me rewrite the entire column in a different way; if Goldman Sachs had a model for the price of an asset (e.g., MSFT stock in a month or Columbia to win the World Cup), but something just happened that shifts the underlying value of that asset far away from the model (e.g., a new CEO for MSFT or an injury to Columbia’s star player) would Goldman advise their clients to value the asset at the model’s price or the price on the open market? I would go with the market price …

Full coverage of the World Cup at: www.PredictWise.com/WorldCup