It’s 2024 and there’s talk of a new poker coach in town. These days, we are asking AI everything under the sun, from business plans and wedding invitations to short stories and essays. Naturally, some are wondering if AI and poker could be a good pair.
"Can AI analyze poker hands and improve my poker skill?"
More and more beginner poker players are sure to be asking these questions in the future.
If you’re an experienced poker player, you’ve considered this already, and drawn your own conclusions. AI has a reputation for giving general information, missing subtleties and playing it safe, so to speak. But still, new large language models are coming out all the time. Perhaps GPT-4 Turbo, one of the latest models from OpenAI, will surprise you.
We were curious about how analysis from an AI and poker player compared.
The Hand AI and Phil Galfond Will Analyze Together
In one of his recent breakdowns, Phil Galfond took a look at a high-stakes cash game with some big names. As usual, he was concise and informative, as we have come to expect from one of poker’s unofficial ambassadors.
We’ll compare his insights to analysis from GPT-4, OpenAI’s multimodal large language model.
Context: The hand comes from the Super-High Roller cash game, which had an insane lineup on the day. Patrik Antonius, Doug Polk, Andrew Robl, Scott Seiver, Sam Trickett, and Dan Colman sit with relative amateurs Matt Kirk and Paul Newey.
The game is $400/$800 and everyone has fairly deep stacks, except for Andrew Robl. Matt Kirk starts with 293,000, but Robl has far more. His stack is Mariana Trench-deep at $1.3 million to start the hand (over 1,650 bb). The effective stack size in the hand will be about 360 bb.
Preflop:
- There's a $1,600 straddle from Antonius.
- Paul Newey raises to $4,500 from early position with .
- Matt Kirk 3-bets to $18,000 from the small blind with .
- Andrew Robl 4-bets to $50,000 from the big blind with .
- Matt Kirk 5-bets to $100,000.
- Robl calls.
Flop: The first three shared cards we see are .
- Matt Kirk continues to bet, sending $35,000 into the pot of $207,700 (about 16% of the pot).
- Robl calls.
Turn: The next card is the .
- Matt bets 18% of the pot, or $50,000 into $277,700.
- Robl thinks for a bit and then shoves all-in.
He covers Matt Kirk, who has about $120,000 left.
We see a snap fold from Kirk and Robl scoops the $327,700 pot and shows the bluff.
So, now you know how the action went, it's time to talk to our experts.
We’ll hear what Galfond had to say first, before turning to the metallic musings from GPT-4.
5 Questions Answered by Galfond and AI
Both coaches will answer 5 main questions about this Super-High Roller hand.
Each question will be answered by Phil Galfond first, followed by GPT-4 Turbo after it has a full summary of the hand. This includes all of the information that a human would have in an online poker hand replayer, but minimal details about player tendencies and strategies.
We’ll compare the human and AI responses too, using Phil Galfond as a benchmark.
Question: Why Did Andrew Robl 4-Bet Preflop?
Phil Galfond’s Analysis:
"Well, Paul Newey is a recreational player. Matt Kirk is hyper-aggressive and does like to (in general) attack recreational players in the game. So, I think Andrew felt that Matt was super wide here with this 3-bet which I think is totally true. Matt also has this tendency to... as the stakes get bigger fight harder.
I don't know how long the straddle was on for, but if it's a recent straddle, Matt is more likely to go for it.
Matt is also somebody who adjusts his game a lot based on how he's been doing winning or losing and he is very much prone to tilt. Not tilt in a way that he plays just terribly but he tilts in a way where he just wants to gamble more."
GPT-4 Turbo’s Analysis:
- Andrew Robl likely 4-bet preflop to apply pressure on his opponents, leveraging his deep stack.
- His choice of hand for a 4-bet bluff typically includes cards that block opponents' strong continuing ranges, such as an ace or a king, to reduce their likelihood of having premium hands.
- The 4-bet also serves to define the strength of his opponents' hands early in the hand, allowing him to make more informed decisions on later streets.
- By 4-betting, Robl aims to take control of the pot, setting himself up as the aggressor, which can lead to winning the pot without a showdown or building a larger pot when he has a strong hand.
Did AI Analyze the Spot Well?
Like Phil, GPT-4 talks about pressuring opponents, but the AI doesn’t talk about Robl’s reasoning or the straddle dynamic.
Overall, AI’s perspective on Robl’s 4-bet reasoning makes sense, but it makes a major mistake. How much does really block Kirk’s continuing range? Sure, it obviously blocks hands like , , and , but AI includes an or in the blocked range.
AI seems to understand the dynamics of stack sizes, hand ranges, and betting leads – but doesn’t fully understand how blockers work.
Question: Why Did Matt Kirk 5-Bet?
Phil Galfond’s Analysis:
"I think being out of position, he didn't want to call and check and face a c-bet and have to check-raise or do something else to win the pot, he wanted to take it down with the betting lead. So even if Andrew doesn't fold now Andrew could fold the flop to a small c-bet and Matt doesn't have to put in a ton of money to win the pot."
GPT-4 Turbo’s Analysis:
- Matt Kirk's 5-bet with likely aimed to capitalize on his aggressive image, pushing the narrative that he could have a strong hand, thus applying maximum pressure on Robl.
- Given his aggressive nature and willingness to gamble, Kirk might have believed Robl's 4-bet could be light, making a 5-bet a profitable move against Robl's potential bluffing range.
- The 5-bet also serves to isolate Robl, potentially taking down a significant pot preflop without having to see a flop.
- Kirk's decision to 5-bet rather than call or fold could be influenced by his stack size and the dynamics of high-stakes cash games, where aggressive plays are more common and can be highly effective.
Did AI Analyze the Spot Well?
On a basic level, AI shows us that it understands the table dynamics of bluffing and telling a story through actions.
However, it immediately makes a mistake. In the third bullet point, it says that the 5-bet isolates Robl. The two players are heads-up, so isolation plays don’t apply.
Also, GPT-4 mentions that Kirk may 5-bet instead of call due to his stack size. Kirk starts the hand with $293,000, but puts in $100,000 preflop, which definitely could appear strong to Robl. It’s possible that AI refers to the perceived strength (and increased folds) that putting in a third of a player’s stack offers. When we prompted AI to explain, it clearly didn’t understand or couldn’t compare Kirk’s small stack to Robl’s 4x larger stack.
Question: Why Did Matt Kirk Only Bet $35,000 on the Flop?
Phil Galfond’s Analysis:
"This is smaller than what I would normally expect from him.
While I do believe Andrew could have read into it, if I were in his shoes I wouldn't exactly know what to think, what I wouldn't think is that it's a hand like , probably not a hand like , it could be and it could be a hand like or that is just not really sure, but wants to get some value and protection. It could also be a flush draw, it could also be suited. It might make a lot of sense. He doesn't want to scare Andrew away, but Matt is somebody who doesn't have like one standard sizing. So his sizings are going to mean something but he's a very smart player and he's quite capable of leveling you. So you have to be careful."
GPT-4 Turbo’s Analysis:
- Pot Control: Kirk's relatively small bet of $35,000 into a pot of over $207,700 is likely an attempt to control the size of the pot with a marginal hand.
- Information Seeking: This bet size can also be seen as a probe to gauge Robl's strength. A call or raise from Robl could provide valuable information about his hand.
- Bluff Inducement: By betting small, Kirk might be trying to induce bluffs from weaker hands or floats from Robl, planning to take down a bigger pot on later streets.
- Value from Weaker Hands: Kirk may believe that only a small bet is needed to extract value from Robl's potential range of weaker hands that could still call a bet on this dry board.
Did AI Analyze the Spot Well?
Pot control and probe bets are two aspects of the c-bet that AI is correct about, following a similar line to Phil.
Then, it quickly gets into some suspicious strategy. Bluff inducing with King-high? If anything, that’s what Andrew Robl is seeking to accomplish. Kirk wants Robl to hit that fold button and not much else. Planning to take down floats from Robl later is a possibility, but not necessarily inducing bluffs. Matt isn’t in a position to call, as we see on the turn.
AI also seems to overvalue Kirk’s King-high. In the last bullet point, GPT-4 says that Matt could be value betting. This would take a savant-level read from him, although it’s not insane to put Robl on hands like , , and -type hands here.
Question: Why Did Matt Kirk Bet $50,000 on the Turn?
Phil Galfond’s Analysis:
"Matt fires out $50,000 and the reason this is interesting is because if Matt had a hand like Kings or Queens he would no longer be betting. I think he's no longer representing a lot of the hands that he was trying to represent on the flop. Now Aces, sure some kind of random suited. Yeah, for sure. off-suit, maybe. I think that's certainly possible. So, he can connect with this . However, he might check some of those hands, including he might check."
GPT-4 Turbo’s Analysis:
- The turn card ( ) likely did not improve Kirk's hand, as he was probably betting his king-high hand on the flop. By betting again on the turn, he's representing a strong hand, possibly an ace or a full house.
- Kirk may be trying to bluff Robl off a better hand, such as a pair of queens or a missed draw. By betting $50,000 into a $277,700 pot, Kirk is making a sizable bet that puts pressure on Robl to fold if he doesn't have a strong hand.
- Kirk's bet sizing leaves him with about $120,000 behind, which could be a strategic move to set up an all-in bluff on the river if Robl calls the turn bet.
- If Kirk genuinely has a strong hand like a full house or a flush draw that got there on the turn, betting $50,000 could be a value bet to extract more chips from Robl.
Did AI Analyze the Spot Well?
AI’s notes about representing full houses and targeting seem to show a decent understanding of hand ranges and blockers. It also agrees with Phil that Kirk could now be representing the .
However, AI doesn’t seem to understand bet sizing to the fullest degree. It calls the $50,000 bet into $277,700 “sizeable,” but less than one-fifth pot is anything but sizeable.
Also, GPT-4 doesn’t grasp the stack-to-pot ratio that would be in play on the river. If Robl had called the turn bet, Matt would have ~$120,000 behind against a pot of $377,000. That’s not the most attractive river sizing for a bluff with much fold equity, though that fact may make it appear strong.
Question: Why Did Andrew Robl Go All-In on the Turn?
Phil Galfond’s Analysis:
"Andrew has no pair, no draw, and what he's thinking here is... "Well, I don't think Matt has anything but the problem is if I call, Matt's very likely to just bluff shove the river and I don't want to call with Queen-high."
As we could see here, if Andrew were to call and Matt Kirk bluff shove the river and Andrew called again, he'd actually lose to the bluff. So, Andrew is thinking I want to win this pot. I don't believe him and I don't want to call.
Matt Kirk bets $50,000, he's got about $120k behind, so Andrew can risk $170k to pick up the $330k in the pot. It only has to work about a third of the time for it to be profitable and despite it having the potential to make him look really silly and for him to get it in completely dead, Andrew does make the hero shove here this is one of the sickest hands that I've seen in televised poker history."
GPT-4 Turbo’s Analysis:
- Capitalizing on Perceived Strength: The Ace on the turn could have strengthened Robl's perceived range, prompting him to push all-in to represent a strong hand or an actual strong holding.
- Applying Maximum Pressure: By going all-in, Robl puts maximum pressure on Kirk, forcing him to make a tough decision for his remaining chips, leveraging the significant pot size.
- Exploiting Kirk's Aggression: Knowing Kirk's aggressive and tilting tendencies, Robl's all-in could be a strategic move to exploit these traits, expecting Kirk to fold hands that missed the board.
- Protecting Equity: If holding a draw or a marginal hand, Robl's all-in could serve to protect his equity in the pot by denying Kirk the chance to see a free or cheap river card.
Did AI Analyze the Spot Well?
AI makes several safe predictions about Robl’s shove on the turn. It doesn’t give very deep insight but seems to be correct about the concepts. Repping the , putting Kirk to a decision, exploiting Kirk’s range, and denying free cards are all valid answers to this question.
Still, it misses the level of nuance that Phil touches on, by a long shot. While AI is repeating basic concepts, Galfond touches on Robl’s plans for the river and his pot odds.
Overall, How Did AI Poker Analysis Compare to Phil Galfond?
And so, how effective was the AI poker analysis?
In terms of general poker advice, GPT-4 scores well. It won’t unlock the secrets of the poker universe, but it can introduce and explain basic concepts. As we saw, it never reached the same level of insight that Phil Galfond did, but that was expected. These language models rarely step out on a limb and give advice that isn’t built from solid data. Therefore, it never produces the same hunches that a professional human poker player does.
In terms of accuracy, GPT-4 did well but made some errors along the way. It was wrong about certain bet sizings and significantly over-valued King-high in a 5-bet pot. This may be because it knows that Robl has Queen-high or the AI wasn’t able to find the right information on 5-bet ranges. It also said that Robl’s hand blocked hands that it didn’t block. As usual, you’ve got to check AI’s answers for accuracy.
In terms of ease of use, GPT-4 offered a good amount of information without much time spent. Inputting the hand context takes a few minutes, and if you’re accurate, you’ll increase the accuracy of the answers. We used Perplexity with its GPT-4 Turbo configuration, but using free options will probably produce very similar results.
Should You Use AI Poker Analysis?
New versions of GPT are sure to make AI and poker a better matchup, with more guidance and tailored answers. Want to learn about a certain poker move or summarize a concept? AI will probably give you an accurate answer quite quickly, even if a real coach's answer would be superior. You could even get it to look at some of your hands, which we'll talk about soon.
Even though a poker against AI competition was won by the machines a long time ago, the language models available to average users aren't impressive for analysis.
If you play on GGPoker, it's likely that your hand history will always be shielded from 3rd-party data mining since they've taken a stronger stance on it. Tracking software is what you'd use to feed AI information, but that will get you banned on GG. This could be a way to future-proof the game, but it also has downsides. Catching cheaters becomes almost impossible for private users without this kind of software.
Still, there’s a large chunk of data that AI won’t have access to, until far into the future.
What AI Poker Analysis Doesn’t Have Yet
The final frontier of AI poker analysis will probably be in live games.
Look at Andrew Robl as an example. He may be among the most well-known players and has appeared countless times in televised games, but also online as “good2cu.” There’s a ton of data out there on Robl, from live sessions and online hand histories.
But AI doesn’t have access to all of it and it won’t have that access for some time.
To build the kind of mental player profile that Phil Galfond is a long process and uses raw, unlabeled data. What that means is, video (or other data) that hasn’t been taught to the AI. Data labeling points out and classifies information for the AI, so we can use it later on. Body language and emotional tendencies for poker players haven’t been labeled by machine learning models yet, but in the future, it could be on the table.
While you could feed hand history to an AI language model, it won’t have anything new to tell you. It picks up your keywords, like "4-bet" or "3-bet", and feeds you information from top sources.
For now, the best service an AI can provide a poker player is general strategy and hand summaries. You can also ask about where the best poker bonus is found, but we've got that too.
Can You Analyze Hand Histories with GPT-4 or Other AI Models?
Of course, analyzing online hand histories is nothing new. Peter Eastgate was doing that to Isildur1 back in 2009, long before AI. Nowadays, apps like GTO Wizard are the standard for sifting through hand history.
When we asked to analyze raw hand data from two PokerStars hands, one of the models (Claude 3 Opus) made some huge mistakes. It forgot some actions and imagined players calling when they had folded.
GPT-4 was accurate and gave a small amount of strategic guidance, but nothing jaw-dropping. Here’s a link to that prompt and answer. When asked to speculate about a player’s unknown hole cards, GPT-4 took its usual stance and chose not to speculate. Future tweaks might improve the knowledge of ranges and make this more possible.
It's only a matter of time. AI and poker are likely to intertwine more as years go on, but large language models aren't useful enough yet. We'll follow their progress closely though.
While not exactly AI, we’ve got this story on poker bots that you might like. You’ll hear about how cheaters were able to automate digital players and skim millions of dollars off a poker site.
