From 5d8bbb2f3748e721d57dff4f3048f226b98b4fc7 Mon Sep 17 00:00:00 2001 From: Petr Baudis Date: Thu, 11 Mar 2010 21:37:45 +0100 Subject: [PATCH] tex: Strength Estimator cleanups --- tex/gostyle.tex | 44 +++++++++++++++++++++++++------------------- 1 file changed, 25 insertions(+), 19 deletions(-) diff --git a/tex/gostyle.tex b/tex/gostyle.tex index e6bd684..2632628 100644 --- a/tex/gostyle.tex +++ b/tex/gostyle.tex @@ -738,29 +738,34 @@ The neural network part of the project is written using the libfann C library\ci First, we have used our framework to analyse correlations of pattern vectors and playing strength. Like in other competitively played board games, Go players -receive real-world rating based on tournament games, and rank based on their -rating.\footnote{Elo-like rating system \cite{GoR} is usually used, +receive real-world {\em rating number} based on tournament games, +and {\em rank} based on their rating.% +\footnote{Elo-like rating system \cite{GoR} is usually used, corresponding to even win chances for game of two players with the same rank, and about 2:3 win chance for stronger in case of one rank difference.}% \footnote{Professional ranks and dan ranks in some Asia countries may -be assigned differently.} The amateur ranks range from 30kyu (beginner) to -1kyu (intermediate) and then follows 1dan to 7dan (9dan in some systems; -top-level player). Multiple independent real-world ranking scales exist -(geographically based) and online servers maintain their own user ranking; -the difference can be up to several stones. - -As the source game collection, we use Go Teaching Ladder -reviews\footnote{The reviews contain comments and variations --- we consider only the actual played game.} +be assigned differently.} +The amateur ranks range from 30-kyu (beginner) to 1-kyu (intermediate) +and then follows 1-dan to 7-dan\footnote{9-dan in some systems.} (top-level player). +Multiple independent real-world ranking scales exist +(geographically based), also online servers maintain their own user ranking; +the difference between scales can be up to several ranks and the rank +distributions also differ. \cite{RankComparison} + +As the source game collection, we use Go Teaching Ladder reviews archive% +\footnote{The reviews contain comments and variations --- we consider only the main +variation with the actual played game.} \cite{GTL} --- this collection contains 7700 games of players with strength ranging -from 30k to 4d; we consider only even games with clear rank information, and then -randomly separate 770 games as a testing set. Since the rank information is provided -by the users and may not be consistent, we are forced to take a simplified look -at the ranks, discarding the differences between various systems and thus increasing -error in our model.\footnote{Since -our results seem satisfying, we did not pursue to try another collection; +from 30-kyu to 4-dan; we consider only even games with clear rank information, +and then randomly separate 770 games as a testing set. +Since the rank information is provided by the users and may not be consistent, +we are forced to take a simplified look at the ranks, +discarding the differences between various systems and thus somewhat +increasing error in our model.\footnote{Since our results seem satisfying, +we did not pursue to try another collection; one could e.g. look at game archives of some Go server.} -First, we have created a single pattern vector for each rank, from 30k to 4d; +First, we have created a single pattern vector for each rank, from 30-kyu to 4-dan; we have performed PCA analysis on the pattern vectors, achieving near-perfect rank correspondence in the first PCA dimension% \footnote{The eigenvalue of the second dimension was four times smaller, @@ -768,10 +773,11 @@ with no discernable structure revealed within the lower-order eigenvectors.} (figure \ref{fig:strength_pca}). We measure the accuracy of strength approximation by the first dimension -using Pearson's $r$ (see \ref{pearson}), yielding satisfying value $r=0.979$. +using Pearson's $r$ (see \ref{pearson}), yielding quite satisfying value of $r=0.979$ +implying extremely strong correlation. Using the eigenvector position directly for classification of players within the test group yields MSE TODO, thus providing -reasonably satisfying accuracy. +reasonably satisfying accuracy by itself. To further enhance the strength estimator accuracy, we have tried to train a NN classifier on our train set, consisting -- 2.11.4.GIT