‘Bot’ This

Most serious poker plays, especially those who play on-line are aware of the existence of “bots.” “Bot” is short for “robot” and the species of interest are those designed to play poker. A bot is, properly, an artificial intelligence (AI), a sophisticated piece of software that not only is programmed to make optimal decisions, but learns from its experiences.

There are many phony bots on the market, pieces of programming junk that you can buy or lease. None play poker better than you (at least I hope not). However, there is one poker bot that has achieved considerable fame. It’s a genuine AI, dubbed Polaris, and was developed by the members of the Computer Poker Research Group (CPRG) at the University of Alberta in Calgary. It has won several contests against other poker bots and made headlines recently when it outplayed a group of on-line pros.

You can visit the CPRG site (http://poker.cs.ualberta.ca/) and follow any of the links, including one that will let you play heads up against Poki, a “baby bot” whose game is good enough to be used in a poker training program. Other links will take you to tech reports and scientific publications.

Bots in general, poker bots in particular and the very notion of an AI are topics of endless fascination. Computer geeks love the sophisticated software. Mathematicians revel in the formal properties of the systems that underlie them. Applied scientists envision extensions into “partial information” domains like bidding auctions, commodities trading and currency exchanges. Poker players, of course, view them from a host of perspectives from envy to fear and loathing laced with heavy doses of paranoia.

The success of Polaris also seems to have fired the imagination of the media. Some called it the beginning of the end of poker. Others likened Polaris to Deep Blue, the chess AI that beat Gary Kasparov. Others warned ominously about mad scientists with clandestine bots lurking on the Internet, running roughshod over mere mortals —- tidbits that inflame the on-liners with paranoid tendencies. This is provocative stuff and we need to understand what’s really going on.

The game is: ​Limit Hold ‘Em (LH): Polaris won’t sit down (metaphorically speaking) in a game of Stud or Omaha. It only plays this one game. On the occasions where it was programmed for no limit (NLH), skilled human opponents consistently beat it. LH, of course, is more algorithmic than NLH. This is not to say that LH is not a complex, skillful game; merely an acknowledgement that it is easier to develop effective strategic generalizations in limit than no limit. It is also says nothing about the possibility of future bots playing world-class NLH, although this is a task of another order of magnitude. No one knows what the optimal strategy is for NLH and one may not exist.

​Heads-up: Polaris only plays against a single opponent. Heads-up play has a reduced number of variables compared with a game with multiple opponents. The computational burden on a bot that plays against more than one opponent is daunting and, worse, it isn’t clear what the maximally effective strategies are. Again, this isn’t an in-principle argument against developing such a bot, merely an acknowledgment of the difficulties.

​Duplicate Poker: The games were played in a version of poker modeled on duplicate bridge. The same cards are dealt to opponents at different times and each must play them from both sides. For example, one time you will play As,Kd against your opponent’s Ts,9s. Later, you will hold Ts,9s against your opponent’s As,Kd.

Duplicate play lowers variance by reducing the impact of luck. It doesn’t eliminate it, of course. For example, you may (correctly) fold a hand that someone calls with and a magical river presents your opponent with a pot you never got to see. However, compared with random dealing, duplicate is known to reduce variance by about two-thirds. This increases statistical power so that only one-ninth as many hands are needed to yield significant results, which is why the CPRG used it.

Some see duplicate poker as the way of the future. I don’t. Because it reduces the luck element weaker players will have fewer winning sessions and lose too regularly. The balance between luck and skill in poker as currently played fits my Goldilocks Rule —- it’s “just right.”

The Opponents: The “pros” in the Pros vs. Polaris competition were a group of young, experienced on-line players. After 3000 hands Polaris was up 195 small bets, a statistically significant result. In an earlier contest, Polaris took on two prominent pros, Phil Laak and Ali Eslami. It beat Eslami but Laak won enough so they eked out a small combined win. Our species (assuming that Laak is one of us) hailed this as a victory.

The stakes: An important but oft-unnoted feature is that Polaris only plays for “cybercash,” not real money. While there is little doubt that the pros are possessed of outsided egos (what top poker player is without one of these?), the fact that no actual harm could come to their bankrolls surely had an impact. The on-line hot shots lost a combined total of 195,000 cyber dollars. Would their play have been different if they were confronting the possibility of losing that much “real” money? Almost certainly. Would they have played better? Perhaps. Worse? Perhaps.

A more sober assessment: Given these factors, many of the concerns over Polaris’s triumphs seem unwarranted. You paranoids out there can retire to ruminating about hackers who can see your hole cards. But the success of the CPRG is significant and has implications for both science and poker. For one, it is at the cutting edge of AI programs that learn from feedback in a very complex game. And, importantly, it shows that a set of heuristics exists for optimal play of heads-up LH. This is enough to make any serious poker player think —- a lot.

There is more to discuss, but my editors get antsy when I go on and on. So, let’s stop here. Next time I’ll examine a number of related issues that have popped up in the blogs, web sites and other media about Polaris’s recent success.