Computer Draughts Rating Lists

Discussion about development of draughts in the time of computer and Internet.
Post Reply
warpspeed
Posts: 11
Joined: Wed Oct 21, 2015 16:18
Real name: Erik van Zomeren

Computer Draughts Rating Lists

Post by warpspeed » Fri Apr 03, 2020 04:03

Are there any renowned computer draughts ELO rating lists of hardware/software combinations playing draughts according to the rules of FMJD like there are CCRL ELO rating lists for hardware/software combinations playing chess according to the rules of FIDE ?

If existing it also would be interesting how these ratings compare to ratings of humans in the FMJD ELO rating lists.

If not existing are there in this respect any estimates of the playing strength of the current best used hardware/software combinations compared to the playing strength of the worlds top human players?

About 4 and a half years ago I tried to get some picture of this in this topic :

The strength of the game program draughts. (which I experienced as a rather peculiar topic title)

There BertTuyt pointed to topic Feike Boomstra Match: a unique man-machine event in which was indicated top draughts programs didn't exceed top human level playing strength on the used hardware (or if so not by much). There was also a question how different hardware could influence the playing strength of a program. This can be tested of course by letting two different machines with the same draughts program play against each other. There will then be the difficulty of having different machines communicate in a proper way of course. For chess such tests, with the same chess engine playing on different machines against each other, have regularly been performed.

I'm interested in what the developments were in that respect since then.

For instance : how do the current best engine versions (of the strongest engines) compare to Scan 2.0 ?

warpspeed
Posts: 11
Joined: Wed Oct 21, 2015 16:18
Real name: Erik van Zomeren

Re: Computer Draughts Rating Lists

Post by warpspeed » Sat Apr 04, 2020 12:17

Well, maybe I can't expect a quick and easy answer. And besides : the world chess community is much larger than the worlds draughts community. On the rating list of the FIDE are about 360,000 players and on the rating list of the FMJD are about 2,200 players (men and women included). That's quite a difference.

Nevertheless I hope with even such a small community Computer Draughts will make a lot of progress soon. And this progress may even attract more people to play draughts.

Fabien Letouzey
Posts: 295
Joined: Tue Jul 07, 2015 07:48
Real name: Fabien Letouzey

Re: Computer Draughts Rating Lists

Post by Fabien Letouzey » Sat Apr 04, 2020 14:24

warpspeed wrote:
Fri Apr 03, 2020 04:03
For instance : how do the current best engine versions (of the strongest engines) compare to Scan 2.0 ?
I first thought your post was focused on human/computer comparisons (which I don't think exist), and missed that last line.

Scan 3 was a big jump in 2017, so programs are still making progress. I focused exclusively on variants, and therefore can't comment on international draughts (ID) level with precision, but 100 Elo seem likely (despite the tiny Elo scale). Even if top players could compete before that, I highly doubt they still can.

As a matter of fact, when was the last time a player claimed a win? And just in case it's on lidraughts.org, that version is limited in resources: one second on one core on a busy machine; it's bullet TC.

---

Regarding Elo estimation, I think that the problem is always the same: not enough player/engine games. We can get well-connected result graphs (in the sense of maths) for both categories, but next to nothing between them. This can only lead to separate Elo ratings. You can get more information from chess discussions about this.

I remember reading an old thread on this forum on computing Elo. I think Ed (?) mentioned using BayesElo (from chess) and ran it (or tried) on PDN files (which are virtually the same as PGN). However a difficulty is that ID has a defect that leads to an enormous "DrawElo", a BayesElo parameter, which was apparently (if memory serves) hardcoded in the source code.

It's certainly possible to compute Elo ratings from the tournaments run by Krzysztof. But don't forget that engine versions are different players. This will rule out Kingsrow maybe among others.

Fabien.

warpspeed
Posts: 11
Joined: Wed Oct 21, 2015 16:18
Real name: Erik van Zomeren

Re: Computer Draughts Rating Lists

Post by warpspeed » Sun Apr 05, 2020 07:48

Thanks Fabien for this clarification. A jump of about 100 elo from Scan 2.0 would be huge. This would put the current best ID programs playing on modest hardware very probably indeed beyond worldchampionship level.

As in chess matches or tournaments with machines playing with worldtop human players are rare. This makes elo comparison of humans versus machines indeed difficult. But separate elo rating lists of machines/engines only which do not necessarily have a relation with elo rating lists with humans is certainly possible.

Some elo rating comparison could indeed be derived from the tournaments of Krzysztof. But as you pointed out there is indeed the difficulty of the possibility of switching of versions in between during these tournaments.

warpspeed
Posts: 11
Joined: Wed Oct 21, 2015 16:18
Real name: Erik van Zomeren

Re: Computer Draughts Rating Lists

Post by warpspeed » Sat Apr 11, 2020 07:38

With the tournaments of Krzysztof Grzelak from 2017 onwards I noticed from year to year different hardware and different time controls. This too makes it difficult to make comparisons ratingwise.

I guess only with an initiative to create CIDRL (Computer International Draughts Rating Lists) with fixed time controls and fixed hardware like there is CCRL for chess one is able to get a better picture of the difference in playing strength of the engines ratingwise per time control and per hardware configuration.

Post Reply