Skip to content

Counterfactual Regret Minimization Game solution methods for Julia

License

Notifications You must be signed in to change notification settings

WhiffleFish/CounterfactualRegret.jl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CounterfactualRegret.jl

codecov

Installation

using Pkg; Pkg.add("CounterfactualRegret")

Implemented Solvers

Solver Name   Description        
CFRSolver   Vanilla CFR solver
CSCFRSolver Chance Sampling    
ESCFRSolver External Sampling  
OSCFRSolver Outcome Sampling  

Each solver takes optional kwarg method, which can be an instantiation of either Vanilla, Plus, or Discount types, which correspond to Vanilla CFR, CFR+, and discounted CFR respectively.

External Solver Packages

Name Status
DeepCFR CI codecov
ESCHER CI codecov
POMDPBestResponse CI codecov

Solving a Matrix Game

using CounterfactualRegret
const CFR = CounterfactualRegret
using CounterfactualRegret.Games
using Plots

game = MatrixGame([
    (1,1) (0,0) (0,0);
    (0,0) (0,2) (3,0);
    (0,0) (2,0) (0,3);
])
sol = CFRSolver(game; debug=true)
cb = CFR.ExploitabilityCallback(sol, 10) # optional callback to monitor training
train!(sol, 10_000; cb=cb)
plot(cb, lw=2)

Finding Kuhn Poker Nash Equilibrium with external sampling & discounting

Kuhn Poker Implementation & Game Definition Tutorial

game = Kuhn()
sol = ESCFRSolver(game; method=Discount=1.0, β=1.0, γ=1.0))
cb = CFR.ExploitabilityCallback(sol)
train!(sol, 100_001; cb=cb)

hist = cb.hist
exp_idxs = 10 .^(0:5) .+ 1
plot(
    hist.x[exp_idxs], hist.y[exp_idxs];
    xscale = :log10, lw=2, label="",
    xlabel = "Training Iterations",
    ylabel = "Exploitability"
)