World Draughts Forum

It is currently Wed Sep 20, 2017 20:51

All times are UTC+01:00




Post new topic  Reply to topic  [ 348 posts ]  Go to page 1 2 3 4 524 Next
Author Message
 Post subject: Internet engine matches
PostPosted: Tue Jan 08, 2008 02:30 
Offline

Joined: Sat Apr 28, 2007 13:53
Posts: 725
Real name: Ed Gilbert
Location: Morristown, NJ USA
Hello draughts programmers,

This post is addressed to the authors of international draughts programs, and particularly those that have participated in discussions on this forum -- Bert, Gerard, Feike, Michel, etc. I would like to run some automated engine matches between our programs over the internet. Using the convenience of the internet and the DamExchange protocol, we should be able to run automated matches with large numbers of games, many more than are possible within the constraints of the manual, interactive tournaments like the Culemborg event. For me the purpose of such automated matches are to learn more about my program's strengths and weakness, to identify areas needing improvement, and to get a clearer idea of its playing strength relative to other strong programs. Probably you are all aware of Frank Mesander's DamExchange protocol and its convenient implementation in a library of function calls. This library makes is quite easy to add to a draughts program the capability of running automated matches against another program on a remote machine over LAN. I have been using this to run matches between two different configurations of Kingsrow, and also between Kingsrow and Dam2.2. I usually run these matches using 2-move ballots as the openings, in order to get good variation in games and prevent the programs from playing the same games over and over again. There are 81 of these 2-move ballots, but 2 of them (31-27 16-21 and 32-27 16-21) lose a man immediately so I exclude those. For each ballot, one game is played with Kingsrow as white and then a second game as black, so a total of 158 games are played. When played at time controls averaging 7 or 8 seconds per move these matches take a little longer than one day to complete. We could easily run one of these over a weekend, which would allow us to watch some of the games as they are played. If the match is interrupted, Kingsrow is set up to continue a partially completed match at the game where it left off, so we would not have to start all over at the first ballot.

I am most interested in running these matches with both engines configured with equal endgame databases, so for example if your engine uses a 6-piece db then I would run Kingsrow with a 6-piece db. I'm interested to see where Kingsrow gets itself into trouble rather than to see how the larger databases can get it out of trouble. If anyone has the interest to see what kind of a difference they make, we could also run additional matches with the larger databases.

Again, my purpose for proposing these matches is so that we can learn from them and improve our engines, and the results are only for my own information. I don't intend to make them public, but neither would I mind if you are pleased with your result and you want to publish it, discuss it on the forum, etc. My experience is that these things give lots of valueable information, and they're also fun! Why don't we give it a try?

-- Ed


Top
   
 Post subject:
PostPosted: Tue Jan 08, 2008 20:08 
Offline

Joined: Wed Sep 01, 2004 18:42
Posts: 1292
Ed, good initiative !
Lets start the preparation for this.

We can start from my perspective such a tournament soon (i have to modify an old Engine for this purpose) (although i as a result of a busy job , focus my draughts activities sin the weekends). If you want you can also use my private email h.tuyt@chello.nl

My purpose is to learn from this , and also i hope that we can share more ideas/thoughts regarding programming in this forum (like the chess community does).

Bert


Top
   
 Post subject:
PostPosted: Tue Jan 08, 2008 23:26 
Offline

Joined: Sat Apr 28, 2007 13:53
Posts: 725
Real name: Ed Gilbert
Location: Morristown, NJ USA
Hi Bert,

Great, thanks for agreeing to try this experiment. It should be some fun. We can run this whenever you are ready. I think any weekend for the next 3 or 4 weeks is ok for me. You will have to open up port 27531 on your router to allow TCP traffic originating on the WAN to pass through to your PC. I tried this here and then I was able to run a DamExchange match between two of my computers by giving the router's WAN IP address, 71.251.206.21, as the remote machine. If you're not familiar with this I can give you some help in setting it up. In DamExchange terms, I will be the initiator and you will be the follower. This will allow my machine to control the start position of each game so that we can run through all the 2-move ballots.

Do you have an "instant messaging" account on one of the big portals, like Yahoo or AOL? We could communicate that way when we are starting up the match.

-- Ed


Top
   
 Post subject:
PostPosted: Thu Jan 10, 2008 22:40 
Offline

Joined: Wed Sep 01, 2004 18:42
Posts: 1292
Ed, I examined the DamExchange and will start with a "simple" implementation in a "older" Engine (where i already use sockets , so i will try to built the DXP interface from the beginning).

I expect to have something working this weekend, so we could try to connect to each other late Saturday or somewhere Sunday, to play 1 game (so i know the basics work).
Can you write to me via my email if this is possible?

I don't know the time difference, and your time windows.
Next to that i can try to activate MSN, if you do the same (and again further details via Email), we can give this a try.

Im looking foward to it, and to be honest i don't care who will win, al least it will be fun , and we most definitely will learn.

Bert


Top
   
 Post subject:
PostPosted: Mon Jan 14, 2008 22:20 
Offline

Joined: Wed Sep 01, 2004 18:42
Posts: 1292
So this weekend , yesterday 13-January-2008, a historical moment (at least i guess so).

The first (or did some-one tried this before) cross-continental 10x10 draughts engine match (or at least the initial beginning), between Damage and KingsRow.

So far we played 10 games, till a DXP implementation-bug forced us to stop.

Will be continued.

Bert


Top
   
PostPosted: Mon Jan 21, 2008 16:09 
Offline

Joined: Sat Apr 28, 2007 13:53
Posts: 725
Real name: Ed Gilbert
Location: Morristown, NJ USA
This past weekend Bert and I ran a long engine match between kingsrow and damage. It was run over the internet using the DamExchange protocol. I think the match was a great success. The programs played 158 games with almost no operator involvement necessary. We could watch the action as it played out, and for part of the time we used Yahoo messenger to chat about the games and compare notes about what the engines were thinking. Some of the 2-move openings leave one side in a cramped position, and it was interesting to see how each program attacked and defended these. This is a great way to learn about the strengths and weaknesses of a draughts program, and I hope the other programmers will participate in similar matches.

Some details about the match mechanics: we had previously run some test games over the internet and discovered that Bert's internet connection through a wireless access point was not reliable, so for the full match he ran an ethernet cable from his router to the computer. This solved the problem and we had a solid LAN socket connection all weekend. Of course running this many games exercises the programs in ways they don't normally see, and sometimes exposes latent bugs. We discovered that Damage repeatedly crashed after 30 consecutive games, so Bert had to restart it about 5 times during the match. He thinks it is a string buffer overflowing in an output window and should be easy to fix. We started the match with time controls of 75 moves in 12 minutes. But after about 40 games we realized that kingsrow was running on a faster computer and this would make it difficult to interpret the results, so for the rest of the match I had kingsrow set to 75 moves in 5 minutes. Both programs were configured equally with 6-piece endgame databases, and pondering turned off. The match ran from Saturday morning to Monday morning. It would have finished some time on Sunday, but there were delays because we were not always present to restart when the 30 game crashes occured. Each game was concluded when Kingsrow saw a database win or loss, or if it saw database draws on 6 consecutive moves. Most of the games were 45 to 55 moves long. There is only 1 game that lasted for 75 moves without a conclusive result.

-- Ed


Top
   
 Post subject:
PostPosted: Mon Jan 21, 2008 19:25 
Offline

Joined: Wed Sep 01, 2004 18:42
Posts: 1292
I would like to add the game result, as I understand why Ed did not provide the info but i think i have the right to do so !!

Damage was really crushed by Kingsrow, so 40 Wins for Kingsrow, 2 wins for Damage, 115 Draws, and 1 game was so complex that after 75moves there was not yet a decisive results.

I would like to compliment Ed with this result, and it motivates me, to learn from this defeat, so Damage will return stronger as ever.

Keep in mind that Ed did not play with 9p databases, so he really raised the bar, so lets' take the challenge.

Bert


Top
   
 Post subject:
PostPosted: Mon Jan 21, 2008 20:00 
Offline

Joined: Tue Sep 30, 2003 21:11
Posts: 4227
BertTuyt wrote:
(...) and 1 game was so complex that after 75moves there was not yet a decisive results. (...)


I don't understand that. Why did you stop the 'complex game' after 75 moves? What was the position?


Top
   
 Post subject:
PostPosted: Mon Jan 21, 2008 20:36 
Offline

Joined: Wed Sep 01, 2004 18:42
Posts: 1292
We played totally automatic, without operator involvement.
So every game was automatically started with a different position, and stopped after 75moves max.

Bert


Top
   
 Post subject:
PostPosted: Mon Jan 21, 2008 20:37 
Offline
User avatar

Joined: Tue Aug 22, 2006 14:38
Posts: 1336
Real name: Joost de Heer
What does Kingsrow with the extra databases think of the final position of the aborted game?

_________________
Lasst die Maschinen verhungern, Ihr Narren...
Lasst sie verrecken!
Schlagt sie tot -- die Maschinen!


Top
   
 Post subject:
PostPosted: Mon Jan 21, 2008 21:12 
Offline

Joined: Tue Sep 30, 2003 21:11
Posts: 4227
BertTuyt wrote:
We played totally automatic, without operator involvement.
So every game was automatically started with a different position, and stopped after 75moves max.

Bert


Can you tell something more about the 'different positions'? Were those positions totally or nearly equal for both colors, or were any of them (very) bad for white or black?


Top
   
 Post subject:
PostPosted: Mon Jan 21, 2008 21:48 
Offline

Joined: Wed Sep 01, 2004 18:42
Posts: 1292
The positions were all the 2move ballots , except for the ones immediately loosing a man.
Both programs played every 2 move opening, so in balance.

Bert


Top
   
 Post subject:
PostPosted: Mon Jan 21, 2008 22:43 
Offline

Joined: Sat Apr 28, 2007 13:53
Posts: 725
Real name: Ed Gilbert
Location: Morristown, NJ USA
ildjarn wrote:
What does Kingsrow with the extra databases think of the final position of the aborted game?


Here is the game:

[Event "Game 25, opening 32-28 20-25"]
[White "Kingsrow"]
[Black "Damage 7.1"]
[Result "*"]
1. 32-28 20-25 2. 37-32 19-24 3. 34-30 25x34 4. 39x19 14x23 5. 28x19 13x24 6. 40-34 10-14 7. 41-37 14-19 8. 46-41 5-10 9. 45-40 10-14 10. 34-30 9-13 11. 44-39 18-23 12. 40-34 12-18 13. 30-25 7-12 14. 34-29 23x34 15. 39x30 1-7 16. 50-44 18-23 17. 32-28 23x32 18. 37x28 13-18 19. 44-39 8-13 20. 42-37 4-9 21. 39-34 18-23 22. 47-42 23x32 23. 37x28 12-18 24. 34-29 18-23 25. 29x18 13x22 26. 43-39 16-21 27. 31-26 21-27 28. 39-34 2-8 29. 41-37 8-12 30. 37-31 27-32 31. 38x18 12x32 32. 48-43 9-13 33. 42-38 13-18 34. 38x27 18-22 35. 27x18 17-21 36. 26x17 11x13 37. 31-27 13-18 38. 36-31 3-9 39. 34-29 7-11 40. 29x20 15x24 41. 31-26 11-17 42. 33-28 18-23 43. 27-21 23x32 44. 21x12 32-37 45. 12-7 37-41 46. 7-1 41-47 47. 43-39 24-29 48. 1x45 9-13 49. 26-21 6-11 50. 45-1 11-17 51. 21x12 13-18 52. 12x23 19x28 53. 1-7 47-38 54. 7-12 38-20 55. 49-43 20-15 56. 12-7 15-20 57. 7-18 20-47 58. 18-45 47-20 59. 45-7 20-42 60. 7-18 42-47 61. 18-34 47-20 62. 34-40 20-47 63. 40-18 47-42 64. 18-12 42-15 65. 12-1 15-42 66. 1-34 42-15 67. 34-12 15-47 68. 12-18 47-42 69. 18-36 42-47 70. 36-31 28-32 71. 31-18 14-19 72. 18-4 47-15 73. 4-36 15-47 74. 36-22 47-42 75. 22-4 42-15 76. 4-31 *

With the 9-pc db, at move 45 kingsrow sees a white database win. But after move 47. 43-39, it no longer sees a win, and it cannot tell for certain if the position is won. If white plays 1-45 then the win is preserved, but kingsrow could not know this during the match.

-- Ed


Top
   
 Post subject: Another engine match
PostPosted: Thu Feb 07, 2008 03:48 
Offline

Joined: Sat Apr 28, 2007 13:53
Posts: 725
Real name: Ed Gilbert
Location: Morristown, NJ USA
Last weekend Bert and I ran another engine match between Kingsrow and Damage. It turns out that Bert had not loaded most of the 6-piece endgame db slices into Damage for the first match, and this time he ran the 64-bit version on his core2-duo. As before, we ran with equal time controls, single-threaded engines, pondering off, and 6-piece databases for both engines. Game times were 75 moves in 11 minutes. Bert asked me to post the results, which were less lopsided than the last time. Kingsrow vs Damage: 30 wins, 6 losses, 120 draws, 2 unknowns. We both found some areas that can be improved from this match, so we'll work on improvements for a while and then try it again in a month or two.

-- Ed


Top
   
 Post subject:
PostPosted: Thu Feb 07, 2008 20:26 
Offline

Joined: Wed Sep 01, 2004 18:42
Posts: 1292
Ed, thanks for your post.
Small correction I was running Damage 10.0 on a Quadcore Q6600.
Only using 1 core [img]http://laatste.info/bb3/images/smilies/icon_rolleyes.gif[/img]

Bert


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 348 posts ]  Go to page 1 2 3 4 524 Next

All times are UTC+01:00


Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Limited