First of all a big thanks for the new site. Its an excellent source of advice which I have recommended to several freinds and colleagues and on chess forums.

I've been into engine chess for about 2 years but have stilllearned valuable tips from your advice on book making and particularly book tuning. I read your advice on book making just now and it hadn't occurred to me to select a small sample as a "learn from " base. I have always done the learn from Database against the whole game base, whic may explain some anomalies I have been seeing in my engine play !

I have been using the Tour Book 14.0v with some success ove rthe past couple of weeks and have just seen that you have released version 14.Oz.

Two questios: What source have you used for the book. Is it all Rybka games or a mixture of engines ?

Is it all blitz games or some longer ones mixed in ? What you are changing between versions ? I can see that the size of the book in terms of number of games is still N=18,497 but the ctg file size is increasing each time ?

Many thanks,

Mark

_______________________________________________________________________________________________

Mark:

Thank you very much for your kind words.

Yes, I have found that in general the learn base should be considerably smaller than the main book base. The main base must contain many obscure and mostly defensive lines against a variety of threats. While the learn file should consist of the strongest current opening lines for both black and white. Tour Book 14.0z is primarily a blitz book tuned for 3-0 time controls. However it does quit well at 1-0 and 5-0 times. At 16-0 it will begin to show some weakness. It is made from games that are mostly 3-0 and 5-0 times.

It is my thinking that engine opening books should be made time and engine specific. I brake the time controls for books down like this; one book for 3-0 one for 16-0 and one for 30+ times. Also as I mention different engines favor different positional themes you should use a book that is tuned to the engine that you are playing.

The reason the number of game in the book is staying the same and the configuration file is increasing in size is that the book is in the hand tuning phase. All new lines and moves are being added by hand after deep positional analyses.

Best Regards:

Kevin:

Hi Kevin,

Hope you don't mind but could I just run one last thing by you.

As an experiment yesterday I followed your book making advice to make a seed book and started with all server games from January 2007 to January 2008. I then removed all draws and games with less than 35 moves and from what was left I extracted all games where elo of both players was at least 2700. This gave a game base of 31,605.

I then created a learn base using games of Big Rabbit, BigMc, Zapchessbook and Golden Boy (these seem to be consistently high elo players) giving me a learn base of 3,667. I then compiled the book using Absolute 100 (I meant to ask - do you use Absolute or ECO Relative to compile book ?). I then ran a learn from DB using my learn base ticking black and white wins.

I had short run on the server using the GUI book settings you suggested and when I looked back at some of the losses I found that my book played quite long into what was not a very good line. When I checked against Tour Book 14.Ov and 14.Oz I found that your books had stopped well short of that position eg. stopped at move 14 whereas my book line played on to move 23. Is this because as part of tuning you have cut short certain lines and removed them , or is maybe because you compiled using ECO Relative rather than Absolute ?

Regards,

Mark

------------------------------------------------------------------------------------------

Mark:

It appears that you have done everything right. As for absolute or ECO, I always use Absolute as it will give the book better depth.

As for the new book following unproductive lines to great depths, yes this will happen at first. As you play the book more and add games that you have won as well as your opponents wins against your book; the new lines will begin to supersede the losing lines. At this point you have not set any red or green moves to be followed, so with the recommended book settings given; the fledgling book is using only the percentage of games won with any given move as it criteria for choosing.

As games are added to the learn file you will almost always notice the depth of the book reducing. (always add the games that you wish your book to learn, to both the main game base and the learn file) The slow reduction of the books depth in this stage of development is a good thing. After you have added 2000 or so games (adding primarily games played with the book you are adding them into) the book will stabilize. Once this happens you can go to the hand tuning phase and it is here that you can extend the winning lines to their full depth.

Good luck and Best Regards:

Kevin:

I wanted to ask you a question regarding book building...

What are the pro and con of building a book out of 2 or more books?

Thanks!

Anonymous:

-----------------------------------------------------------------------------------------

Anonymous:

This is some thing that I do not have a lot of experience with. You are I believe referring to the Fritz GUI function in Books…Import Book.

“Import book” merges the contents of a different book into the current one.

“Import old book” converts a book in the older (FBK) format and imports the contents.

The program will also convert and import books from the programs Genius, Rebel and Chessmaster. If a move is already in the openings book it will be marked as a preferred tournament move.

I put this function in much the same category as the GUI Book Learn. It works on an elementary level but probably will not produce the results you are looking for. The problem is that when you use this function to attempt to merge two all ready highly advanced books. (books with extensive move weights and hand enabled red-green tuning marks) The weights and move preference marks become convoluted creating conflicts for the GUI when the book is in play. It would be nice if the best lines of two books could be combined so easily. It also would nice if book learning actually worked.

However having said this there are techniques that some book makers use to utilizes the book learn function. Playing a newly created book without weights or marks with the book learn turned on to identify faulty lines; that can then be later addressed and altered manually.

I am sure that some book development technique may successfully employ the Import Book function but it not something that I have ever used.

Kevin:

Hi Kevin,

I will be attentive about your updated books. I have an old and poor pc (AMD 2.4 Ghz Athlon xp) with 768 ram. Nevertheless I win some dual core 2 with 4 threads and draw with some 8 threads.

I am very happy to have found a good book. only a question I need to know :

I played with 72 megas of hash tables and 4 megas of table bases. Is this configuration correct? what will be the best configuration with your book?

Jean:

------------------------------------------------------------------------------------------

Jean:

The amount of hash table file size that you use is dependent upon your system and the engine being used. The conventional wisdom is to use one half of your available system RAM. I think I agree with this in long time control games. However in blitz play 3-0 5-0 times, you do not need that much. A smaller hash size (with most engines) will give you a bit more nodes per second but will reduces the depth in Ply especially in the end game.

So it is a compromise and you must find the setting that gives you equipment the best performances. Also most, if not all engines will only use hash settings in multiples of 8 (16-32-64-128 256-512) so your 72Mb is being rounded down to 64Mb.

When you are trying to decide the optimum setting look closely at the depth in the end game play. If your engine does not seem to be getting good depth here you may need to go to the next hash size.

As you have described your system I would say that you have it about right… 64 or 128 in blitz play.

As for the EGTB Cache this depends on how many egtbs that you are using. The speed of your hard drive can come into play with egtb as well.

This Cache is actually only the references file for the bases stored on the hard drive. The RAM stored file is interrogated by the engine as the number of pieces on the board become few. By the way it may take several games for this file to load up fully into the cache.

This is what I have found to be about right:

For all 3-4-5 man about 5 Giga bites…use 8Mb of cache

For all 3-4-5 and a few 6 man (CB Turbo 3) 48Gb…use 16Mb of cache

For blitz play I use 140Gb of EGTBs all on a Raptor 10,000rpm hard drive, with a cache size of 32Mb.

Kevin:

Just came across your web page which I found very interesting. Regarding your settings for the book, can you clarify please.

Book settings:(Frayer Tour Book)
Use Book: on
Tournament Book: on
Varity of Play: max - (all the way to left)
Influence of learn Value: max + (all the way to right)
Learning Strength: max - (all the way to left)
Minimum games: 1
Up to move: 100

On my Fritz 11 the max settings are all the way to the right and the min settings all the way to the left for all three settings not just learn value..

Regards:

Mike:

-----------------------------------------------------------------------------------------

Mike:

You are right it is not as clear as it could be. Varity of Play is Max - (Minus) or the minimum variety setting. You must look close but two are max - and one is max+.

My intent is for the book to use the minimum amount of variety pf play and learn strength and the maximum amount of influence of learn value. (all the way; left, right, left)

I want the book to play a narrow repertoire of the sharpest lines. (especially in blitz times) I do not want the book to use the GUI learn function as I feel it is detrimental in such a narrowly focused book. The learn function does not learn any new lines or add any moves to you book at all. What it does is to mark moves as being successful or unproductive. In order for the book to actually learn, it needs a little assistance from us using the book development techniques and hand tuning process I have talked about in my blog.

Hope this has been of some help, please do not hesitate to write me any time with questions.

Kevin:

Hi Kevin

Thanks for the reply. The effect of these settings for Tour 14.1b is that if you play each move that is recommended, you end up with the following :

From this on move 9 for black there are no more moves recommended, all I get is 0-0, h5 and Nbd7 showing 0 in the end column of the opening book pane [%] the engine takes over with no more moves to play. However if the engine does play 0-0 then white follows with the recommended move 0-0-0 and suddenly black now has a recommended move 10. Nbd7 but no move for 11 then we are ok till move 18 then back to book moves for 19 through to move 37.

You then end up the following game :

120'/40+60'/20+30' n/a

256MB, Tour Book 14.1b.ctg, N-3XTA4YATYOZ7W 1.e4 c5 2.Nf3 d6 3.d4 cxd4 4.Nxd4 Nf6 5.Nc3 a6 6.Be3 e5 7.Nb3 Be6 8.f3 Be7 9.Qd2 0–0 10.0–0–0 Nbd7 11.g4 b5 12.g5 Nh5 13.Nd5 Bxd5 14.exd5 f6 15.gxf6 Rxf6 16.Na5 Qf8 17.Nc6 Nf4 18.Kb1 Bd8 19.Nxd8 Qxd8 20.c4 bxc4 21.Bxc4 Nb6 22.Bb3 a5 23.a3 Qd7 24.Rc1 a4 25.Ba2 Nc8 26.Rc4 Ng6 27.Ka1 Rxf3 28.h4 Nge7 29.Rhc1 Qe8 30.Qe2 Rf8 31.Bb1 Nf5 32.Bf2 Nce7 33.Be4 Rb8 34.Rc7 Rb3 35.Ra7 Qb5 36.Qxb5 Rxb5 37.Rc2 g6 Line

Rybka scores this as more or less equal :

New game - Rybka 2.3.2a mp, 120'/40+60'/20+30'
5rk1/R3n2p/3p2p1/1r1Ppn2/p3B2P/P7/1PR2B2/K7 w - - 0 1

Analysis by Rybka 2.3.2a mp :

38.Rd2 Rb3 39.Bd3 Kh8 40.Rxa4 Ng8 41.h5 gxh5 42.Ra7 Nge7 43.Ka2 Rbb8 44.Be1 h6
= (0.02) Depth: 17 00:00:00 1kN
38.Rd2 Rb3 39.Bd3 Kh8 40.Rxa4 Ng8 41.h5 gxh5 42.Ra7 Nge7 43.Ka2 Rbb8 44.Be1 h6
= (0.02) Depth: 18 00:00:00 1kN
38.Rd2 Rb3 39.Bd3 Kh8 40.Rxa4 Ng8 41.h5 gxh5 42.Ra7 Nge7 43.Ka2 Rbb8 44.Be1 h6
= (0.03) Depth: 19 00:01:20 17235kN

(Hurd, n/a 24.02.2008)

Wouldn,t it make more sense to allow the book to play the moves at 9,11 and 18 to save any time for the engine

Regards

Mike

------------------------------------------------------------------------------------------

Mike:

Yes, absolutely; this is a good example of a place to do a bit of hand tuning. What you have found here is what I call a transposition hole. It is somewhat common in books that have been heavily hand tuned. Because there is often more than one way to arrive at a position, it is some times necessary to fill in these holes when they are discovered. This in fact streamlines the book and makes it stronger.

Also do not hesitate to do deep positional analyses of lines that end early, particularly for black. 14.1b as with many books has better attacking lines as white and black defenses can always be improved.

You are absolutely on the right track; continue filling in gaps and extending winning lines, while at the same time looking for better alternatives when you feel the book lines are unproductive. The book will become stronger and your Elo will increase.

Kevin:

Hi Kevin,

I am rather new to book making, and surfing the web I just stopped into your great guide for this purpose. Here you have my sincerely congratulation to you!

I would like to make you a question. In the website, you write: First never add games directly to an existing book. Add the games to the database that the book was made from and then make a fresh book from the updated game bases. Importing games into an existing book is probably the most conmen mistake that is made in book making.

I have the following questions:

1) Why is this an error?

2) Why always is better to add games played by the same book you are testing?

Thank you very much for helping,

Best regards!

José :

Kevin: