php poker generated random results do not match what is expected

Question

I have made a research on poker equities. What I did is use pokerstove program with some selected. And with that program calculated the equities using enumerate all method. So pokerstove results:

I also made a table where I store the random results:

CREATE TABLE poker_results
(
id int NOT NULL AUTO_INCREMENT,
matches_id int NOT NULL,  -- just for filtering in case I will want another starting hands.
result varchar(255) NOT NULL,
winner_hands varchar(255) NOT NULL,
PRIMARY KEY (id)
) 

The result column data looks like this: Jd,9d,Qh,5c,Kc - this represents the cards on board.

Winner_hands column data look like this: Ks-Ac,6c-Ad,3c-Ah (can be 1 hand won, can be all 8 hands won).

Here is the code which generates the results to database. Its on codeigniter framework. And also to not copy the whole poker.php, I just copy couple of functions which are used from it:

protected function get_probabilities($hands, $board_cards = array()) {
        $players = count($hands);

        $board = implode('', $board_cards);
        $board = trim($board);
        if (empty($board)) {
            $board = '-';
        }
        $hx = '';
        $hand_counter = 1;

        foreach ($hands as $hand) {
            $hx .= '&h' . $hand_counter++ . '=' . $hand[0] . $hand[1];
        }
        
        $url = 'http://' . $this->poker_server . '/?board=' . $board . $hx . '&auth=' . $this->auth;
        
        

        //Output exm. string '0.1342, 0.2042, 0.13525, 0.52635'
        //WAR!! check if alive
        $result = $this->parse_url_link($url);
        
        if (substr($result, 0, 3) == 'Err') {
            $this->utils->logging('ERROR', 'Poker server authorization failed!');
        }
        
        //echo $result;

        return array(
            'hands' => $hands,
            'prob' => explode(', ', $result),
            'board' => $board_cards
        );
    }


// protected because in child class needed
    protected function get_poker_winner($table_winners) {
        $to_return = array();
        foreach($table_winners['prob'] as $key => $val) {
            if ($val > 0) {
                $to_return[] = $table_winners['hands'][$key][0] . '-' . $table_winners['hands'][$key][1];
            }
        }
        
        return $to_return;
    }

poker_tests.php

<?php if ( ! defined('BASEPATH')) exit('No direct script access allowed');


include_once(APPPATH . 'controllers/poker.php');    
require_once APPPATH . "libraries/downloaded/Mersenne_twister.php";
use mersenne_twister\twister; 

class Poker_tests extends Poker {   
    
    public function __construct() {
        parent::__construct();
    }

    /**
     * Generates data in database with such structure:
     * CREATE TABLE matches
     * (
     * id int NOT NULL AUTO_INCREMENT,
     * player_cards varchar(255) NOT NULL,
     * 
     * PRIMARY KEY (ID)
     * )
     * 
     * CREATE TABLE poker_results
     * (
     * id int NOT NULL AUTO_INCREMENT,
     * result varchar(255) NOT NULL,
     * PRIMARY KEY (id)
     * ) 
     * 
     * 
     * Here 1 match will have many results, because we use same preflop cards.
     * 
     * Text results appended to pokerstove.txt
     * 
     * 376,992 games     0.013 secs  28,999,384 games/sec
     * 
     * Board: 
     * Dead:  
     * 
     *          equity      win     tie             pots won    pots tied   
     * Hand 0:  20.925%     20.53%  00.40%           77382       1504.50   { KhQs }
     * Hand 1:  06.215%     03.50%  02.72%           13190      10239.00   { 9c4d }
     * Hand 2:  06.396%     04.08%  02.32%           15379       8734.00   { 8d4c }
     * Hand 3:  18.906%     18.15%  00.76%           68426       2847.50   { AcKs }
     * Hand 4:  08.767%     06.91%  01.86%           26032       7019.50   { 9h2c }
     * Hand 5:  10.204%     09.83%  00.38%           37044       1424.00   { Ad6c }
     * Hand 6:  09.046%     08.67%  00.38%           32678       1424.00   { Ah3c }
     * Hand 7:  19.541%     18.08%  01.46%           68154       5514.50   { 8c7c }
     * 
     * 
     * ---
     * 
     * 
     */
    public function run() {

        $this->benchmark->mark('start');

        $this->output->enable_profiler(TRUE);

        //close the current connection, because its not needed
        $this->db->close();

        $db_poker = $this->load->database('poker_test', TRUE);

        $sql = "INSERT INTO poker_results (result, winner_hands, matches_id) VALUES (?, ?, ?)";

        // matches_id = 1. Insert new match
        $table8_hands = 'Kh,Qs,4d,9c,4c,8d,Ks,Ac,2c,9h,6c,Ad,3c,Ah,7c,8c';  // do test with this. Do those win the right amount of time?

        for ($i=0; $i < 400000; $i++) { // pradejus id 100194

            $flop = $this->poker_flop($table8_hands);
            $turn = $this->poker_turn($flop, $table8_hands);
            $river = $this->poker_stop($turn, $table8_hands);
            //echo json_encode($river) . '<br>';

            $db_poker->query($sql, array($river['river_board'],  implode(',', $river['winner_hands8']), 2));
        }
        
        $db_poker->close();

        $this->benchmark->mark('end');
        echo $this->benchmark->elapsed_time('start', 'end') . '<br>';
    }



    

    /**
     * 
     * Override - remove unneeded things for test from that function in poker.php
     * Generates 3 flop cards
     */
    public function poker_flop($table8_hands) {
            
    
        $table8_cards = explode(',', $table8_hands);
        
        $table8_results = $this->random_result($table8_cards, 3);
        
        return $table8_results;

    }

    /**
     * Generates 1 turn card
     * @param $table8_hands - same as match score in database. But here we have hardcoded in run function
     * 
     */
    public function poker_turn($table8_flop, $table8_hands) {
            
            $table8_cards = explode(',', $table8_hands);
            
            //Join players cards and opened board cards         
            $table8_reserved_cards = array_merge($table8_cards, $table8_flop);

            //Pass all opened cards and get one new board card          
            $table8_results = $this->random_result($table8_reserved_cards, 1);
            
            //Merge all opened board cards
            $table8_results = array_merge($table8_flop, $table8_results);

            // this is flop and turn both           
            return $table8_results;
            
    }

    /**
     * 
     * Generates 1 river card
     */
    public function poker_stop($table8_flop_turn, $table8_hands) {
            
        $table8_cards = explode(',', $table8_hands);
        
        $table8_reserved_cards = array_merge($table8_cards, $table8_flop_turn);
        
        $table8_results = $this->random_result($table8_reserved_cards, 1);
                    
        $table8_results = array_merge($table8_flop_turn, $table8_results);

        
        $table8_hands = $this->array_to_hands_array($table8_cards);
    
        $flop_turn_results = implode(',', $table8_results);

        // $this->benchmark->mark('code_start');
        //Get new probabilities - they will be needed to determine if hand has won or not. When prob. > 0 - then won
        $table8_prob = $this->get_probabilities($table8_hands, $table8_results);

        // $this->benchmark->mark('code_end');
        // echo $this->benchmark->elapsed_time('code_start', 'code_end');
        
        return array(
            'winner_hands8' => $this->get_poker_winner($table8_prob),
            'river_board' => $flop_turn_results
        );
            
    }


    /**
     * for second generation - new random function
     * @param  array  $reserved_cards 
     * @param  integer $cards_amount   
     * @return array
     */
    protected function random_result($reserved_cards, $cards_amount = 5) {

        $card_types = array('s', 'c', 'h', 'd');
        $card_values = array('A', 2, 3, 4, 5, 6, 7, 8, 9, 'T', 'J', 'Q', 'K');

        $deck = array();
        foreach ($card_values as $value) {
            foreach ($card_types as $type) {
                $deck[] = $value . $type;
            }
        }

        $remaining_deck = array_diff($deck, $reserved_cards);
        // make keys sequence:
        $remaining_deck = array_values($remaining_deck);

        $results = array();

        while (count($results) != $cards_amount) {          

            $rand_card_key = $this->random(0, (count($remaining_deck) - 1));
            
            $results[] = $remaining_deck[$rand_card_key];

            // remove from deck         
            unset($remaining_deck[$rand_card_key]);
            // make keys sequence:
            $remaining_deck = array_values($remaining_deck);
        }
        
        return $results;
    }


    /**
     * Picks random element from range
     * @param  integer $from        
     * @param  integer $to          
     * @return integer              
     */
    private function random($from, $to) {

        if (file_exists('/dev/urandom')) {
            $twister4 = new twister;
            $twister4->init_with_file("/dev/urandom", twister::N); 

            return $twister4->rangeint($from, $to);
        }
        else {
            return mt_rand($from, $to); 
        }
    }    
}

As we see in the random function - I use twister library with linux dev/urnadom when I do testing on linux and native mt_rand when I am on windows. Not noticing difference.

So to select results I use queries like this:

To get total count of results

select count(*) from poker_results where matches_id = 2 and id < 296351 

To get how much is total win (win + tie) of hand:

select count(*) from poker_results where matches_id = 2 and id < 296351 and winner_hands like '%Kh-Qs%' 

To get how much pots tied of hand:

select count(*) from poker_results where matches_id = 2 and id < 296351 and winner_hands like '%Kh-Qs%' and winner_hands != 'Kh-Qs'

The poker server is for getting the hand equity. By hand equity on river I determine if hand won or not - when hand equity is > 0 then it won.

This tool is used at another server - there is php code which runs the python program, but it does not matter here.

http://pokersleuth.com/programmable-poker-calculator.shtml

That tool is like pokerstove but it has command line version. By the way, don't buy this tool, we bought and they did not send the licence key, kind of like they don't care.

And now the results:

https://docs.google.com/spreadsheet/pub?key=0ArMZCQvNc-oQdEs0a1UyMkFGazVoN09KZmU1Q0FCU0E&output=html&richtext=true

Now if we compare, the hands win + tie more often than pokerstove or pokersleuth shows equity. And when I look ant tie % - its much bigger than poker stove shows. The sample is not so small. Pokerstove uses almost 400K games, there its bit less but the tendency remains the same. First I tried on much smaller sample like 10K games - same tendency. So when I will generate 100K more I am not surprised if result remains aproximately the same. Also this sample is generated on linux. But about same sample is generated also on windows and still the wins are more than pokerstove shows and sum of win % is 110-112 % also.

So we are not understanding - am I generating something bad or those programs show wrong? Programs showing wrong equities is not likely since they are used widely and probably should be tested a lot already.

Update:

I think I finally understood :) Pokersleuth computes the chances of a hand (two cards) winning knowing the 5 cards on the board. You are then comparing these chances to the real outcome (knowing the hands of all the other players). Right?

Right. I had to write here, cause it did not allow in comments to have extended discussions.

Update:

Generated more rows - currently 515989 on linux server, the % still remain approximately the same as before. So not moving

Answer 1

You have 296350 generated results, but 332911 wins & ties.

Does any part of this process reconcile the fact that when there's a tie, it's between two or more hands? It looks as though ties are being over counted.

Take your total win and tie %, 112.3371014%. Multiply (generated results divided by wins&ties). You will get exactly 100%. What does this suggest?

Also, look at Ad6c and Ah3c. Their tie count is exactly the same, and very likely the same hands (where the win was a pair of aces).

But for the hands where they tie, that hand is counted/weighted twice (once for Ad6c and once for Ah3c). That's why your tie % are at least double what they should be. You'll have to normalize tie counts by the number of hands that tie