"Libratus" Takes on the Top Poker Players

In 2015 an Artificial Intelligence (AI) dubbed “Claudico” played 80,000 hands of No Limit Hold ‘Em poker against four of the top online poker pros. The ‘bot (short for “robot”) was a program written by the AI experts at Carnegie Mellon University’s computer science department. While this might seem like a lot of hands to most folks, it turned out not to be enough data to settle the issue. The human players won but not enough to be statistically meaningful.

Starting today a newer more powerful poker playing computer (named Libratus) will take on the same four pros. This time they’ll be playing 120,000 hands each and hopefully settle the issue. If they win the four pros will get more than bragging rights, they will split a $200,000 prize. Most handicappers think Libratus still isn’t up the job and the international betting markets have the AI as a 4 - 1 and even 5 - 1 underdog.

There are some fascinating elements of this project that go way beyond poker. They involve the development of a remarkably effective procedure that is used to “teach” the computer how to play an effective game of poker. It uses an algorithm called the counterfactual regret-minimization routine that operates as follows. 

The AI starts out knowing nothing about poker other than the basics like hand rankings and the rules of the game. It’s dealt a hand and makes a decision about how to play it. It then sees whether it won or lost money on it and how much. The next step is the key (and requires a very fast, powerful computer): it runs through every other possible decision it could have made against all the things its opponent might have done. That is, it looks at all the counterfactuals and selects the one that would have minimized regret (won the most or lost the least) and moves it up in the hierarchy of possible ways to play that hand. The ones that would have won less or lost more are moved down.

Then they ran several trillion (that’s not a misprint) iterations — all played against a second instantiation of the program which was carrying out the same routines. Each time a better decision (i.e., one that “minimized” regret ) was found the hierarch of decisions was revised. The “regret” notion is used here in the sense of “Oh damn, I should have done that instead.” 

Over time the AI slowly homed in on the most effective strategies. Whether it has found the ones that can beat these top pros will be determined soon.

Note that this “brains v. bot” contest is being played “heads-up.” That is there are only two players in each hand, one of the humans and Libratus. Even though Libratus is playing at a world-class level of skill heads-up, poker is such a complex game that it cannot handle the computational load that having a third player at the table produces. The reason is that poker, unlike other complex games like chess and go where there are AI’s that can beat any human, is a game of partial, and sometimes unreliable information. Each player only knows some things but is missing the most important data, what one’s opponent’s cards are and what its bets mean. This simple fact makes the problem one of overarching complexity and arriving at an optimal set of decisions extremely difficult.

What’s even more fascinating is that the counterfactual regret-minimization algorithm is a general one. As the researchers at CMU are showing, it can be applied to any situation where the number of counterfactuals is large but within the computational capacity of the computer, where only some of the relevant information is available and where some of it may be misdirection and misleading. The applications they are exploring are in areas like medical diagnosis, cybersecurity, financial markets, economic decision-making, business and political negotiations, military situations — all circumstances where there is only incomplete and potentially misleading information but concrete decisions need to be made.

The Unkept Promises of the PEOTUS

I started compiling a list of things Trump said he would do — you know, as a check-sheet to go back to and see how’s he done in following up. This kind of thing is pretty routine. Here, for example, is a look back at how Obama fulfilled his promises.

But, since The Donald hasn’t taken the oath of office yet, I’m not interested in any specific legislative or governmental issues — those are either routine like cabinet nominations, appointments of advisors, tapping heads of agencies or they’re best regarded as premature like specific bills he might put forward for Congressional action once he’s in office.

What I’m interested in here is the dozen or so specific promises he’s made to the American people during the campaign and after his November 8th victory. Surprisingly — or not, depending on your political persuasion — the number which he followed through on is precisely zero. There are probably more of these faux pledges but this list shows pretty clearly what we’re dealing with in our new PEOTUS. At one time or another, Trump said he would:

•Release his personal and corporate tax returns

•Hold regular press conferences

•Provide documentation on Melania’s immigration status and whether she violated US immigration laws

•Specify what information about the Russian hacking that he claims he has that the CIA does not

•Explain how Mexico will pay for the wall now that he’s determined that US taxpayers will have to fund it

•Present a plan to prevent conflicts of interest between his duties as president and his corporation

•Clarify the operations of the Trump Foundation and why they did not violate New York State laws

•Provide an explanation why his international corporate links would not violate the Emolument Clause

•Explicate what his plans are for filling the vacuum that would be created by withdrawing from NAFTA

•Explicate what the impact of withdrawing or limiting the US’s role in NATO and the UN would be

•Present an analysis of the impact that “tearing up” the Iran agreement would have on the Middle East

•Put forward a detailed economic projection of the long-term impact of his proposed tax cuts

I’m not just inventing issues. These are all things Trump has said he will do, from promising to release his taxes if he’s elected, to ensuring us that he has full documentation that the future First Lady did not violate the guidelines of her visa when she first entered the country, to outlining what he knew that shows that the CIA was mistaken about the Russian hacking,[1] to promising to provide full economic analyses of a host of domestic and international proposals.

He has, of course, done none of these. It would be folly for anyone to expect that he will. In some cases transparency would be, well, revelatory like his tax returns, the full gory details of the Trump Foundation and Melania’s apparent visa violations.

In others they were just typical Trumpian bluster done to whip up his base but with no thought or plan behind them such as tearing up the Iran deal or withdrawing from NAFTA. It’s simply not possible for him to provide the analyses he promised since no such exist.

In still others the projections, if carried out honestly, would show them to be completely unworkable (dumping the Iran deal), economically disastrous (pulling out of NAFTA) or geopolitical suicide (reducing our role in NATO or the UN).

The bottom line here is we’re faced with something we’ve never confronted in the nearly 250 years of our Republic. We had a nominating process that was unlike anything we’ve seen before — more like an extended series of clips from The Twilight Zone in living color than a normal political contest. It was followed by an election unlike anything we’ve seen before — even Rod Serling wouldn’t have come up with what’s been played out before us. And now we’re really off the end of the pier, into the cold, dark waters of a socio-economic-political black hole.

I guess I better check my Twitter feed to see what shit’s coming down the pike now — you can bet it won’t be any effort by Trump to fulfill any of those promises he made. In fact, I’m quite certain that he wouldn’t remember making most of them….

---

[1] On this one, when pressed he said he’d let everyone know “on Tuesday or Wednesday” without specifying which Tuesday or Wednesday.

Lamb and Wild Mushroom Rigatoni

Back when we lived in New York, I had this dish (or one close to it) at an Italian restaurant on the Upper West Side. I like trying to recreate dishes. I’ve played with this one for a while. So it’s time to share. It’s dead easy, will take less than an hour and feeds two with leftovers for lunch tomorrow or, with a salad and bread can feed four.

It uses lamb broth which isn’t easy to find. Few markets in North America carry it, not even in cubes. Either make your own (this recipe from Ramsey is good) or find an online supplier. In a pinch substitute chicken or veggie stock.

Ingredients

olive oil

1/2 pound of lamb — shoulder works best, cut into bite-sized slices or cubes and dust with salt and pepper

1 onion, halved and sliced

1 garlic clove, smashed and chopped

2 sprigs of rosemary

1 pound or so of mushrooms — a mix of fresh white or crimini ‘shrooms and reconstituted dried wild ones such as chanterelles, pine, hedgehog and/or porcinis. Slice whole mushrooms and chop large pieces of wild. 

1/2 c white wine

1/2 c lamb broth

the mushroom soaking liquid, strained

2 T flour

a large bunch of a “green” — any. Chard, kale, spinach and collard all work. Each gives a slightly different flavor but all are good. Mustard greens are a bit strong but can be used.

chopped parsley for garnish

1/2 pound of rigatoni (or similarly shaped pasta) 

Directions

brown lamb in a T of olive oil and the rosemary — remove lamb (leave rosemary) and set aside

in same pan, sauté onions, garlic and mushrooms till they begin to caramelize (add olive oil if needed) — remove rosemary

add the flour and stir to mix

deglaze with wine and stir, scraping up the fond

add lamb broth and mushroom soaking liquid, stir, scraping up fond

return lamb

add greens

cover and simmer while cooking the pasta

combine everything and sprinkle with chopped parsley

Do We Have a Policing Crisis? By Larry E. Nevonen, JD, Guest Blogger:

The following is based on an article in the Wall Street Journal by an economics professor from Trinity College. The article supplied facts, sources, and some history of police-community relations that need broader exposure and exploration. Where do you stand on what we need to do?

The author raised a very difficult question: Factually, who has the better case? Black Lives Matter or Blue Lives Matter? In 2015, 41 officers were killed in the line of duty. There are about 900,000 officers. That means that officers bear a victimization rate of 4.6/100,000 officers. The average American faces a homicide rate of 4.5/100,000. The average American male faces a homicide rate of 6.6/100,000. 

On the other hand, police killed 1,207 Americans. This is a rate of 134/100,000 officers and is 30 times the overall base homicide rate. Justifiable homicides? Probably in almost all cases, but there are no statistics. The article asks the next tough question of are these killings necessary? Are there alternatives? In England and Germany police commit less than ½ of 1% of all homicides. How did we get here?

We know about the “War” on Drugs and the “War” on Crime. The big change in how those wars were fought and funded came from a 1994 crime bill signed by then President Bill Clinton. That bill gave out $20 billion to hire more police, build more prisons, and allow police departments to purchase surplus military equipment. A curious result? SWAT teams have existed for a long time. But the frequency of their use has grown to 50,000 SWAT raids per year nationally.

What happened to the relationship between the police and the community it served during this time period? The article quotes from the book “To Protect and Serve” by Norm Stamper, a man who started as a beat cop in San Diego and rose to become the Police Chief of Seattle for 6 years. When this ex-police chief states that he was trained to believe that his community was irrelevant, there is a problem. Rewards and promotions for officers were based on numbers of arrests, etc,. At the same time, Mr. Stamper quotes a fellow police chief as saying: “As someone who has spent 35 years wearing a police uniform, I’ve come to believe that hundreds of thousands of police officers commit perjury every year testifying.” In this environment is it any wonder that the police end up viewing citizens as numbers, or worse, revenue sources. At the same time, citizens often feel that the police cannot be trusted, when people know that when police get on the witness stand, they take an oath to tell the truth and proceed to lie. Other officers know and support this illegal and immoral conduct by keeping silent behind the blue line.      

The Justice Dept. report on Ferguson, Missouri painted a chilling picture. In 2013, in a town of 21,000 people, their courts issued 9,000 arrest warrants. Many of these warrants were based on failing to pay fines for parking tickets and even housing code violations including overgrown lawns. The city’s finance director had written to both the police chief and the city manager of the city’s financial needs that could be filled by increasing ticket writing.  Police promotions were based on “citation productivity” and the local judges and prosecutors assisted in this fund raising effort to fill the city’s coffers.  It was therefore predictable that any galvanizing event, such as the shooting of Michael Brown, would both trigger mass protest and get a fully militarized police response.  To quote the article:  “One important lesson from economics is that unaccountable government officials will not always act in the public’s behalf.”

And where has all this loss and expense gotten us?  Are we in the middle of a new crime wave?  For insight, the author turned to another book, “The Rise and Fall of Violent Crime in America”, by Barry Latzer.  In 1900 the American homicide rate was 6/100,000.  During Prohibition, it rose to 9/100,000 then falling to 4.5/100,000 in the 50’s.  A spike up to 11/100,000 occurred in the 70’s.  But the long term trend has been down from there to the current 4.5/100,000 which is among the lowest in the nation’s history.  Is this the result of the zero-tolerance and mass incarceration policies?  It turns out that the curve of homicide rates over time from our neighbor to the north, Canada, bears a remarkable resemblance to our results.  And Canada has not used either militarization of its police or mass incarceration to achieve these results.  Mr. Latzer concludes: “that the major determinants of a crime rate are likely cultural factors and economic opportunity.  The employed family man is going to be less interested in crime than the unemployed and unattached.” 

In this environment of violence and distrust what should a good citizen do?  All I can think of now is my disappointment with this state of affairs in this great nation that is mostly filled with well meaning hard working people including hard working peace officers.  I suggest that we start by opening discussions, using our blessed Freedom of Speech, to open dialogs at a variety of levels. 

 How do we get more openness and transparency in our Police Departments?  We desperately need better police relations with all our community.  We have been making some progress in police interactions with people with mental health problems by recharacterizing the contact as working with someone who needs help instead of working with someone who needs to be controlled.  There has also been progress in seeing child prostitutes as victims, not criminals.  Note how these are simple changes in an officer’s attitude about interacting with members of their community.  But it isn’t that simple.  Implementing these changes takes police management decisions and officer training.  We need to take any and all steps that lower tensions in police-community interactions and are focused on lowering the homicide rates in both blue and black groups.  That will take genuine dialog between both groups in a calm, dispassionate setting.  If we don’t conduct calm open “autopsies” of prior events, good and bad, there will be more such autopsies carried out in many venues in the future over bad events.  Without genuine dialog, not speeches, there are no winners, only losers, here. 

We need the police to lower their blue wall.  Admit it.  It exists.  The recent convictions of Sheriff Baca and Undersheriff Tanaka underscore an “above the law” mentality that must be rooted out at every level of police ethics.  When I hear an ex-police chief state on public television that the only way to get to the truth today in dealing with the police is to sue them, that kind of adversarial environment creates gridlock, not change or trust.  At the same time, front line police officers need a safe, both physically and career wise, avenue to report any violation of law by other officers.  It must be clear to all that their house is clean.  We need systems in place that keep the police accountable to the people.      

1. Our front line peace officers need more support than ever.  By militarizing them, the likelihood of good officers becoming victims of PTSD is very real.  And we need good, honest and healthy officers in the field.  Police career reward systems need to find the difficult way away from quotas, particularly any quota that drives the officer to treat his/her community as a revenue source.  Police management needs to do far better job in helping our front line cop do his/her job right and in a manner that ALWAYS EARNS the respect and admiration of the community being served.  Police management needs to do a much better job.

2. How do we reduce the number of police contacts where weapons are in play?  Do we really need 50,000 SWAT interventions a year in this country?  Can something be done to cut the supply of the weapons and ammunition that the cops fear to face the most as a way to increase their safety in the field?  What gun and ammunition control measures do the police suggest that would increase their safety in the field?  How can we genuinely make their dangerous job safer so that they can come home to their families?  How do we de-escalate on both sides?

3. Is it better to spend public funds on treatment instead of punishment?  In many areas, particularly drug use, it is clear that the punishment model has not worked.  How many police departments financially depend on drug bust forfeitures for a portion of their budget?  And for the many people who carry conviction records, their punishment never ends when barriers exist to those persons re-entering the job force at any level.  A job goes a long way to preventing recidivism.     

Think about these issues. Think about how you can help. Then think about who you vote for at any level. Who will push these problems in the right direction? Who might put gasoline on the fire? Your vote counts and matters. Use it. 

Each of us needs to be able to demand transparency from anyone in public service. And when the trust of the public is earned, only then should full support be given.

We can do better. We must do better. Too many lives have been lost and too much is at stake. Thank you for taking the time to consider this material. 

Political Rant: How We Got Here

FB political rant for the morning — as we contemplate the EC voting taking place as I write. It’s an effort to answer a question oft-asked these days: How the fuck did we get here?

The Republicans began taking power some thirty to forty years ago. The initial strategy was hatched by Lee Atwater and colleagues. Atwater was a brilliant if unscrupulous political operative. He started politically agnostic. He became a Republican simply because he grew up in the south and realized he couldn’t gain power as a Democrat. He was Karl Rove’s mentor. Look him up.

The strategy focused on systematically working at the lowest levels of government: school boards, fire commissioners, planning commissions, land-use and development committees. Very few folks run for these seats and with a bit of push from the party they got their people in. Those who showed a nose for government got tapped and groomed for the next levels: city councils, county councils. The more effective (or more popular) then ran for state legislatures.

They boosted their appeal to the electorate by reaching out to racists in the south, to evangelicals by staking out socially conservative positions on gender, marriage and abortion, to business-oriented voters by emphasizing lower taxes and campaigning vigorously against unions to the point where they succeeded in getting non-union workers voting to diminish the impact of the very organizations that could better their lot. Toss in a dollop of hatred for “elitists” and intellectual (especially scientists) and you have a heady brew.

It worked. Once they had control of a state they pushed their agenda, with ALEC writing prospective bills. And critically, they were in charge of redistricting after each census. With a little practice they became masters at gerrymandering.

During all this the Democratic party did essentially nothing to counter it, acting as though people would vote in their best interest. They knew that the country’s best interests were to be found in liberal policy and assumed they would win with sound economic and social positions — and counting on solid backing from minorities. But people do not vote in their best interest — obviously.

So now we are where we are. It’s a strange place — in a country whose citizenry, as virtually every poll taken in the past half-century shows, supports progressive positions on minimum wage, the environment, equal pay, parental leave, education, Social Security, health care, tax codes, business regulations, voting and gender rights, etc., etc. And what is this electorate doing? Watching helplessly as these policies get whittled away by a party dominated by bigots and oligarchs.

As Thom Hartmann put it: There are only three kinds of Republicans left. The wealthy, those with sufficient power and influence to have been bought by the wealthy and the duped. This last group is large and diverse and was created by policies put in place decades ago.

The election this year was special in several interesting ways. Most significantly “the duped” turned out to be a larger slice of the electorate than most politically savvy observers thought. Too large. They elected someone truly pathological. The center may not hold.

We are in very deep shit, deeper than most people realize. I don’t know how to turn it around. The primary hope I see on the horizon is a shift in demographics. Old white (duped) dudes are dying off. The millennial generation has more liberal and open values but they need to get active and they need to vote.

Or maybe the Trump administration will implode. Fasten seats belts.

When he knew he was dying, Atwater apologized for what he’d done and what he began . Too little, too late. As an old friend put it, just live an ethical life and you won’t have to apologize for anything.