Internet engine matches

Discussion about development of draughts in the time of computer and Internet.
Post Reply
mtaktikos
Posts: 6
Joined: Wed Aug 05, 2015 17:56
Real name: Michael Taktikos

Re: Internet engine matches

Post by mtaktikos » Wed Aug 05, 2015 22:07

Ed Gilbert wrote:http://edgilbert.org/InternationalDraug ... ources.zip

New files and changed files (for Windows compile). Also add bitcount.cpp to the project.
Thank you Ed, but it didn't work. My compiler doesn't support in pthread std:mutex, here the error message:
Image

Did you manage to get an exe file?

Ed Gilbert
Posts: 768
Joined: Sat Apr 28, 2007 14:53
Real name: Ed Gilbert
Location: Morristown, NJ USA
Contact:

Re: Internet engine matches

Post by Ed Gilbert » Wed Aug 05, 2015 22:37

Did you manage to get an exe file?
I added the Windows exe to the zip file. You may have to wait 15 minutes or so before getting it because the web server sometimes serves stale files from cache for a while. I also used static linking to libs so you don't need the VS2015 runtime dlls.

-- Ed

mtaktikos
Posts: 6
Joined: Wed Aug 05, 2015 17:56
Real name: Michael Taktikos

Re: Internet engine matches

Post by mtaktikos » Wed Aug 05, 2015 22:51

Ed Gilbert wrote:
Did you manage to get an exe file?
I added the Windows exe to the zip file. You may have to wait 15 minutes or so before getting it because the web server sometimes serves stale files from cache for a while. I also used static linking to libs so you don't need the VS2015 runtime dlls.

-- Ed
Thank you very much!
About every 30 minutes I make a download of the zip file, until now there is no exe packed herein,
but there is no haste, I will continue the downloads

Best regards,
Michael Taktikos

mtaktikos
Posts: 6
Joined: Wed Aug 05, 2015 17:56
Real name: Michael Taktikos

Re: Internet engine matches

Post by mtaktikos » Thu Aug 06, 2015 08:54

Wonderful! It's there and it works on my old computer. Thx again

Rein Halbersma
Posts: 1635
Joined: Wed Apr 14, 2004 16:04
Contact:

Re: Internet engine matches

Post by Rein Halbersma » Thu Aug 06, 2015 12:04

mtaktikos wrote:Wonderful! It's there and it works on my old computer. Thx again
Did you try to install Visual Studio 2015 Community Edition? It's free and includes everything you need (including profile guided optimizations).

mtaktikos
Posts: 6
Joined: Wed Aug 05, 2015 17:56
Real name: Michael Taktikos

Re: Internet engine matches

Post by mtaktikos » Thu Aug 06, 2015 15:00

Rein Halbersma wrote: Did you try to install Visual Studio 2015 Community Edition? It's free and includes everything you need (including profile guided optimizations).
No, I was not aware that Visual Studio is free, and thought, the MinGW compiler would be compatible with it. Until the prob with pthread_mutex occured, of course. Thanks for the suggestion, if there is in future again such a compatibility problem, then I will install Visual Studio to solve it

Ed Gilbert
Posts: 768
Joined: Sat Apr 28, 2007 14:53
Real name: Ed Gilbert
Location: Morristown, NJ USA
Contact:

Re: Internet engine matches

Post by Ed Gilbert » Thu Aug 06, 2015 16:12

I ran an engine match of 2-move start positions, kingsrow vs. scan, each using 6-piece dbs, 4 search threads, 120 moves in 3 minutes time controls. Results from kingsrow's perspective: +0, -33, =125, unknowns 0

Scan is really strong! Congrats to Fabien for creating a very powerful draughts program, and thanks for sharing the sources so the rest of us can try to understand what we need to do to move forward.

-- Ed

MichelG
Posts: 244
Joined: Sun Dec 28, 2003 20:24
Contact:

Re: Internet engine matches

Post by MichelG » Thu Aug 06, 2015 22:24

Ed Gilbert wrote:I ran an engine match of 2-move start positions, kingsrow vs. scan, each using 6-piece dbs, 4 search threads, 120 moves in 3 minutes time controls. Results from kingsrow's perspective: +0, -33, =125, unknowns 0

Scan is really strong! Congrats to Fabien for creating a very powerful draughts program, and thanks for sharing the sources so the rest of us can try to understand what we need to do to move forward.

-- Ed
That is quite an incredible result! I had not expected such a big difference.

I just started a match against scan with similar settings. I'll have some results for dragon tomorrow.

Michel

Rein Halbersma
Posts: 1635
Joined: Wed Apr 14, 2004 16:04
Contact:

Re: Internet engine matches

Post by Rein Halbersma » Fri Aug 07, 2015 12:33

Ed Gilbert wrote:The number of transposition table entries is 2^tt-size. That means 2 raised to the power of tt-size. If tt-size is 6, the number of entries is 2^6 = 64. If tt-size is 24, then number of entries is 2^24 = 16.7 million. Each tt entry is 16 bytes, so 16.7 million entries is 256mb. I don't know how scan behaves with different tt sizes, but clearly 6 is much too small. I would guess a value of 23 would be good for matches of 3 to 5 minutes. It's probably not critical (but 6 is very bad).

-- Ed
tt-size = 30 would correspond to 64 times as much, about 16Gb of TT entries so no wonder that didn't work (unless the machine had 24 or 32 Gb or RAM).

MichelG
Posts: 244
Joined: Sun Dec 28, 2003 20:24
Contact:

Re: Internet engine matches

Post by MichelG » Fri Aug 07, 2015 12:42

I have run a engine match between dragon v4.5 and scan with similar parameters as ed:
- 4 threads
- scan: 6 piece databases, 256 mb hash
- dragon: selected 8 piece databases, 6x6 eval patterns
- 3 minutes for 75 moves

I used 3 move ballots. The match is still running, but the first 158 games ended in
+0 -20 =138 (-43 elo)
from dragon's perspective. That confirms Ed's conclusion that scan is really strong :-) Congratz to Fabien indeed.

I think it is amazing that scan did not loose a single game against dragon or kingsrow in 316 games!

Does anyone else have results against scan? And has someone played a scan vs scan match?

Michel

edit: games included
Attachments
damexchange20150806.pdn
(968.89 KiB) Downloaded 79 times
Last edited by MichelG on Fri Aug 07, 2015 12:58, edited 1 time in total.

Rein Halbersma
Posts: 1635
Joined: Wed Apr 14, 2004 16:04
Contact:

Re: Internet engine matches

Post by Rein Halbersma » Fri Aug 07, 2015 12:45

MichelG wrote:I have run a engine match between dragon v4.5 and scan with similar parameters as ed:
- 4 threads
- scan: 6 piece databases, 256 mb hash
- dragon: selected 8 piece databases, 6x6 eval patterns
- 3 minutes for 75 moves

I used 3 move ballots. The match is still running, but the first 158 games ended in
+0 -20 =138 (-43 elo)
from dragon's perspective. That confirms Ed's conclusion that scan is really strong :-) Congratz to Fabien indeed.

I think it is amazing that scan did not loose a single game against dragon or kingsrow in 316 games!

Does anyone else have results against scan? And has someone played a scan vs scan match?

Michel
Ed & Michel: could you both post the games to this list?

Ed Gilbert
Posts: 768
Joined: Sat Apr 28, 2007 14:53
Real name: Ed Gilbert
Location: Morristown, NJ USA
Contact:

Re: Internet engine matches

Post by Ed Gilbert » Fri Aug 07, 2015 13:46

Match games kingsrow vs scan.
Attachments
dxpgames.pdn
(176.78 KiB) Downloaded 101 times

Fabien Letouzey
Posts: 285
Joined: Tue Jul 07, 2015 07:48
Real name: Fabien Letouzey

Re: Internet engine matches

Post by Fabien Letouzey » Fri Aug 07, 2015 18:59

MichelG wrote:I used 3 move ballots. The match is still running, but the first 158 games ended in
+0 -20 =138 (-43 elo)
from dragon's perspective. That confirms Ed's conclusion that scan is really strong :-) Congratz to Fabien indeed.
Thanks Ed and Michel!

While I'm not complaining by any means, those results lead to a puzzle for me. Dragon Draughts already has the most accurate evaluation (through years of effort) and it's also gaining a lot from a breakthrough table that Scan doesn't have. I don't believe that my (very naive) ProbCut implementation can compensate for that if at all, so where are the missing Elo points coming from?

You don't have to answer right now BTW :)

Fabien.

Rein Halbersma
Posts: 1635
Joined: Wed Apr 14, 2004 16:04
Contact:

Re: Internet engine matches

Post by Rein Halbersma » Fri Aug 07, 2015 19:28

Fabien Letouzey wrote: Thanks Ed and Michel!

While I'm not complaining by any means, those results lead to a puzzle for me. Dragon Draughts already has the most accurate evaluation (through years of effort) and it's also gaining a lot from a breakthrough table that Scan doesn't have. I don't believe that my (very naive) ProbCut implementation can compensate for that if at all, so where are the missing Elo points coming from?

You don't have to answer right now BTW :)

Fabien.
Wouldn't your bottom 4 patterns pick up potential breakthroughs? Btw, did you ever study particular pattern coefficients for plausibility?

Perhaps your qsearch is also sufficiently different from others to be a contributing factor?

BertTuyt
Posts: 1328
Joined: Wed Sep 01, 2004 19:42

Re: Internet engine matches

Post by BertTuyt » Fri Aug 07, 2015 19:54

I don't believe that my (very naive) ProbCut implementation can compensate for that if at all, so where are the missing Elo points coming from?
First of all sorry that i didnt post Damage - Scan Match results, as my computers are still at home (according Rein I should activate remote access). But most likely I will visit Holland in some weeks from now, so i will take stronger hardware with me.

During the tournament, Fabien informed me that at one point Scan was searching deeper in comparison with Damage. This was remarkable, as Damage used 8 cores, at 4 GHz, and normally is searching much deeper as opponents (and sometimes throwing babies away with the pruning bathwater). Damage uses a combination of MCP, FHR and LMR.

So I guess that maybe also search depth is a major factor, maybe Ed and/or Michel observed the same.
Looking at the code, and I might have a better look, at least the probcut implementation looks interesting (although as Fabien explained the original statistiscs part is missing). Next to that I need to better understand the move sorting and LMR implementation. As Rein mentioned also the Q-search could be an explanation.
But for now, my bet would be the combination probcut, move sort and LMR.

Last but not least, I really thank Fabien, you have given our community a big boost by this result and by sharing all the internals, as can be observed in this forum...

Bert

Post Reply