Jerry Shan

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2023-127

May 12, 2023

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-127.pdf

In recent years, humans have made significant progress in building AIs for perfect and imperfect information games. However, trick-taking poker games are still considered a challenge due to their complexity. Tractor (a.k.a. ShengJi) is a 4-player trick-taking card game played with 2 decks of cards that involves competition, collaboration, and state and action spaces that are much larger than the vast majority of card games. Currently, there is no existing AI system that can play Tractor. In this work, we present ShengJi+, an effective AI system for the Tractor game powered by deep reinforcement learning and Deep Monte Carlo. ShengJi+ is trained using self-play for ∼1.2 million games and achieves 97.6% Leveling Rate over the random baseline agent. In addition to the main architecture, we also experiment with several training techniques for Tractor and discuss why they do or do not work based on the match statistics. Through case studies, we believe that ShengJi+ exhibits intelligent and rational playing strategies that resemble human Tractor players. We open-source our code to motivate future work on this topic and to introduce Tractor as a new benchmark for imperfect information multi-agent reinforcement learning, and to introduce ShengJi+ as a strong baseline for future research into this game.

Advisors: Joseph Gonzalez

---

BibTeX citation:

@mastersthesis{Shan:EECS-2023-127,
    Author= {Shan, Jerry},
    Title= {ShengJi+: Playing Tractor with Deep Reinforcement Learning},
    School= {EECS Department, University of California, Berkeley},
    Year= {2023},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-127.html},
    Number= {UCB/EECS-2023-127},
    Abstract= {In recent years, humans have made significant progress in building AIs for perfect and imperfect information games. However, trick-taking poker games are still considered a challenge due to their complexity. Tractor (a.k.a. ShengJi) is a 4-player trick-taking card game played with 2 decks of cards that involves competition, collaboration, and state and action spaces that are much larger than the vast majority of card games. Currently, there is no existing AI system that can play Tractor. In this work, we present ShengJi+, an effective AI system for the Tractor game powered by deep reinforcement learning and Deep Monte Carlo. ShengJi+ is trained using self-play for ∼1.2 million games and achieves 97.6% Leveling Rate over the random baseline agent. In addition to the main architecture, we also experiment with several training techniques for Tractor and discuss why they do or do not work based on the match statistics. Through case studies, we believe that ShengJi+ exhibits intelligent and rational playing strategies that resemble human Tractor players. We open-source our code to motivate future work on this topic and to introduce Tractor as a new benchmark for imperfect information multi-agent reinforcement learning, and to introduce ShengJi+ as a strong baseline for future research into this game.},
}

EndNote citation:

%0 Thesis
%A Shan, Jerry 
%T ShengJi+: Playing Tractor with Deep Reinforcement Learning
%I EECS Department, University of California, Berkeley
%D 2023
%8 May 12
%@ UCB/EECS-2023-127
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-127.html
%F Shan:EECS-2023-127