Why the College Football Playoff Selection Committee continues to reject AI

The evolution of college football rankings: The College Football Playoff Selection Committee has released its first rankings for the 2024 season, highlighting the continued reliance on human judgment in determining the sport’s top teams.

• Oregon, Ohio State, Georgia, Miami (FL), and Texas comprise the top 5 in the initial rankings.
• For the first time, the College Football Playoff will feature 12 teams, introducing new dynamics to the competition.
• The top four highest-ranked conference champions will receive first-round byes, while the next highest-ranked conference champion and top seven ranked teams will play in the first round.
• First-round games will be hosted by the higher-seeded teams at their home stadiums, adding a unique element of campus excitement.

The human element in team selection: The College Football Playoff Selection Committee, consisting of thirteen members, relies on intense debate and discussion to determine team rankings.

• Rankings are based on factors such as on-field performance, conference championships, strength of schedule, head-to-head results, and comparisons of results against common opponents.
• The human element in the selection process is not immune to bias, with factors like hype, brand recognition, and style points potentially influencing decisions.
• Last season’s controversial exclusion of undefeated ACC champion Florida State from the playoff in favor of Alabama highlighted the challenges of balancing fairness with judgment.

The legacy of the Bowl Championship Series (BCS): The current rejection of AI- or computer-based rankings in college football can be traced back to the unpopular legacy of the BCS system.

• From 1998 to 2013, the BCS determined the top two teams for the National Championship Game using a combination of polls and computer rankings.
• The system included six independent, statistically-based computer rankings that often produced results conflicting with human perception.
• The BCS occasionally exerted control over these computer polls, such as banning margin of victory from consideration in 2002, which fueled skepticism about the system’s fairness.

Computer rankings in the BCS era: The six computer polls used in the BCS system each had unique methodologies for evaluating team performance.

• Anderson & Hester focused on strength of schedule and heavily weighted wins against high-ranked teams.
• The Billingsley Report used a sequential approach, valuing consistent performance throughout the season.
• The Colley Matrix was a purely win-loss based system that emphasized teams’ records and opponents’ records.
• Massey Ratings integrated score margins and strength of schedule to assess overall team strength.
• Sagarin Ratings combined multiple ranking methods to account for team performance, schedule strength, and margin of victory.
• Wolfe Ratings used iterative calculations to evaluate teams’ rankings, focusing heavily on strength of schedule.

Criticisms of computer-based rankings: The BCS system faced significant criticism, leading to its eventual replacement by the College Football Playoff.

• Critics blamed “the computers” for producing rankings that felt disconnected from public expectations.
• The lack of transparency in the algorithms made it difficult for fans and analysts to understand how certain factors influenced rankings.
• These criticisms mirror recent debates about AI, where seemingly objective algorithms can introduce new, hidden biases.

The potential future of AI in college football rankings: While the College Football Playoff currently relies on human judgment, there may be opportunities to integrate AI in complementary roles.

• AI models could simulate game outcomes based on team and performance data, offering predictive insights into potential matchups.
• AI might be used to audit past College Football Playoff rankings to identify overlooked factors that inadvertently influenced committee decisions.
• The success of analytics and AI integration in other professional and collegiate sports suggests that college football may benefit from reconsidering its approach to computer-based decision-making.

Broader implications: The continued reliance on human judgment in college football rankings raises important questions about the balance between tradition and innovation in sports.

• While the College Football Playoff format has been widely welcomed as an improvement over the BCS, it reflects a rejection of analytics-driven decision-making that has proven valuable in other sports.
• As AI continues to advance and demonstrate its potential in various fields, college football may need to reconsider its stance on computer-based rankings to ensure fairness and objectivity in team selection.
• Finding a balance between human expertise and AI-driven insights could potentially lead to a more robust and transparent ranking system in the future.