Saturday, September 15, 2007

ChessDB Review

Why spend hundreds of bucks on a chess database when you can download ChessDB, a great free chess database tool? ChessDB is open source (yeah!), and once you've installed it just click 'Tools' and 'Download Games' to download a free 3.5 million game database. Huzzahh!

ChessDB is a descendent of SCID (no longer maintained, and which charged for the actual database). It is developed by Dr. David Kirkby. So far I'm lovin' it. It has the standard database features: search for games in the database based on position and all sorts of other criteria. It has a game tree display for each move that includes frequencies and performance statistics. One unique feature is that you can right-click on any two moves in the game tree, and it does a statistical analysis to determine whether one move is actually better than the other (for the stat-heads out there, it even gives a p-value!). Also, ChessDB can directly download games from ICC, FICS, and TWIC.

They are still working on shoring up the documentation, such as the online tutorial, and the help guide that comes with the program is not searchable, which is sometimes frustrating. But this is an Open Source project, so you can take part in its development. This is in contrast to commercial software, in which your only recourse is to complain and then pray. The ChessDB developer wants help translating the documentation into languages other than English.

Because much of the help and documentation features are still in development, it is sometimes frustrating to figure out how to do some very basic things. For instance, how do I filter the games by position and then save the matching games into a little mini-database? I couldn't figure it out on my own. Luckily, Dr Kirkby is very helpful, and explained it to me within the day when I posted a message to the ChessDB user group (it's easy: create a new database, and then drag the filtered games from the huge database using the Windows-->Database Switcher widget).

All in all, the benefits absolutely dwarf any difficulties I have had with the program. It is easy and fun to use, and the developer is responsive and helpful. I have been wanting a database for a few months now, but didn't want to fork out the three hundred bucks for Chessbase. Thank goodness I found ChessDB, and now have 3.5 million games to explore (including over 8000 with 1. e4 e5 2. d4).

30 Comments:

Blogger transformation said...

this is just fantastic.

someone gifted CB9 to me, and mega game collection a year ago, and i JUST laboriously unzipped the last TWIC database games, from TWIC 468 to 670--a LOT of work--another 305,000 games, added to above, but if i didnt have it, would surely use this resource.

thank you.

i sincerely know how happy you feel. i was esctatic when i got mine. congratulations!

its still nice, however, to have our own seriously pruned version that you live with day to day, game by game of course, but that is something you can generate from this source data. im so happy for you BDK.

9/15/2007 09:41:00 PM  
Blogger takchess said...

off topic but wanted to send you this info. These types of things is why Alekhine played his version of the gambit not the Danish Gambit proper. I suggest you do play around with the Danish anyway it is nice to have 2 strong bishops on the diagonals. an early d5 by black is known as the Schlecter defense . it does suck some of the life out of the Danish.



http://www.chessgames.com/perl/chesslike.pl?gid=1182249

also this is an interesting way to give back the pawn

http://www.chessgames.com/perl/chessgame?gid=1038022

9/15/2007 09:57:00 PM  
Blogger Pale Morning Dun - Errant Knight de la Maza said...

" it is sometimes frustrating to figure out how to do some very basic things."

Oh yeah? Try Chessbase, and you'll know the meaning of frustration. I like my chessbase and my fritz, but they have never been great on documentation. So much so they try and substitute documentation with articles by Steve Lopez on how to actually do things in Fritz and Chessbase.

Now exactly what other products have you purchased in the past that decided to hand off their documentation and tutorials to the occassional online article that is at the whim of the author? I expect more from Germans for goodness sake.

Rant off, I'll have to check that dbase out.

9/15/2007 10:38:00 PM  
Blogger transformation said...

and let us not forget, besides fondness for Civil War trivia and a very good writer at that, Mr. Lopez is a self proclaimed consumer of not insignificant quantities of beer ale, in large cc mugs.

9/15/2007 11:13:00 PM  
Blogger Aleph said...

There's another actively developed version of SCID here. That's the version I personally use and prefer, as it has many more features than the original SCID or ChessDB.

There's a discussion of some of the differences between these versions here

9/15/2007 11:52:00 PM  
Anonymous Anonymous said...

SCID-PG is better IMO.
http://prolinux.free.fr/scid/

9/16/2007 12:21:00 AM  
Blogger Blue Devil Knight said...

Thanks for the tip on the other continuation of SCID.

Does SCID come with a huge free database, or do you have to pay?

I guess there is some animosity toward ChessDB's developer out there. For those who want to see a taste of the controversy between Skid and ChessDB, see this post. I was not aware of this, but got ChessDB solely because it seemed for SCID you have to pay for games, while in ChessDB they are free. Please correct me if I'm wrong, with links to the SCID free game info! (In general open source projects tend to have awful, unwieldy, unorganized web sites, so it is hard to find even the simplest information like this at their sites).

9/16/2007 12:44:00 AM  
Blogger Blue Devil Knight said...

Tak: I like the Alekhine variation!

9/16/2007 12:50:00 AM  
Blogger Blue Devil Knight said...

For the pro ChessDB side of the controversy, see this.

I know I like ChessDB , and it has a free database associated with it, unlike SCID. What's so great about a free database reader if you have to pay for a database? That's just silly. See this page to purchase a database for SCID...a million games....ChessDB, on the other hand, over 3.5 million games for free: that's the decider for me.

My database program isn't for doing tactical problems, or opening repertoire, or playing games (all the assets of the other offshoot of SCID): it's for exploring a database--its statistics and the like. If my "open source" database program doesn't come with a database, then I don't really need it! I've got plenty of tactical puzzles, opening repertoire resources. I just needed a database. Focus of purpose tends to lead to better software, especially in the Open Source model.

So, after looking into it a little bit, I have to say: given my goals, ChessDB is winning 1-0.

9/16/2007 09:56:00 AM  
Anonymous Anonymous said...

Thanks for pointing out the different forks of SCID. I have been using SCID since a long time now and always found it a great chess database program even though its development is halted.

On the database issue: I have a close to 500000 games database I put together mainly from TWIC plus a few others. You can download it for free from my site (see chess game downloads here) in pgn and SCID DB format. I plan to update it once or twice a year.

9/16/2007 12:13:00 PM  
Blogger takchess said...

So are you trying to tell me that no means no ?

8)

9/16/2007 02:23:00 PM  
Anonymous Anonymous said...

Shane Hudson the SCID author tried to make some small pittance for his hundreds of hours of coding SCID. So he tried to sell CD-ROMs with a database. I believe Shane became seriously ill & hospitalized and halted his work on SCID. link
For all SCID-pg's extra features, it's worth noting that ChessDB somehow uses significantly more RAM. SCID-pg has bitmapped (not TrueText) pieces and is compatible with UCI engines such as Rybka and Toga (200+ points stronger than Crafty). Where did you get the idea that CHessDB "comes with" a free database? Any database that ChessDB can open, so can Scid-pg. I am not affiliated with either program, so ultimately i dont care what you use. In fact i was a user and promoter of ChessDB in the beginning (last December thru March), but Scid-PG is simply more robust, easier on the eyes, and talks with Rybka unlike ChessDB.

9/16/2007 02:46:00 PM  
Blogger Unknown said...

If you wanted to use chessbase products (which are nice, even though the documentation may be lacking) you can buy chessbase light 2007. I believe it's relatively cheap, around $20-30 or so. You'd then have to buy mega database 2007 I suppose for the 3.5 million game database. I'm not sure what the price of that is.

Chessbase light is pretty nice, visually the pieces look much better due to the new anti-aliasing feature (although why this hasn't been put into chessbase 9 as an update is scandalous). I'd say for pure visuals on a chess server, chessbase light 2007 or fritz 10 on the playchess server is the best I've seen so far. Chessbase also has some features such as colored squares and arrows (which I use in my instructional videos) which are nice, too.

Now that chessbase light is out there really needs to be some detailed review that points out all the differences between chessbase light 2007 premium and chessbase 9. I think a normal player could probably get away with chessbase light as their only database program from the looks of it though.


One advantage you have on playchess (when using the free chessbase light 2007 or fritz 10) is that you can use multimedia features, such as talking to your opponents with voice, also there is a nifty analysis board you can whip out when watching or reviewing a game, it allows you to move pieces around yet keep your eye on the main game as well.

9/16/2007 03:18:00 PM  
Blogger Blue Devil Knight said...

Thanks for all the interesting info.

Anon asks: "Where did you get the idea that CHessDB "comes with" a free database? "

In ChessDB, I click on a button and download, for free, 3.5 million games. As opposed to SCID, which charges for games on a CD (which takes time to get). I am still not sure if the Skid-pg fork of scid has a helpful and easy download a zillion games button.

9/16/2007 03:23:00 PM  
Anonymous Anonymous said...

If you want to use SCID with the 3.5 mil db, then just copy the db file to your scid directory.... :p

9/16/2007 11:48:00 PM  
Blogger Blue Devil Knight said...

Anon: yes, I could, or I could just click 'open' in ChessDB.

Clearly I stepped unwittingly into the middle of a spat between two programmers :)

9/17/2007 01:00:00 AM  
Blogger Blue Devil Knight said...

My apologies, I meant to say a petty spat.

9/17/2007 01:07:00 AM  
Blogger Unknown said...

If skid-pg really does have more features, better graphics, etc., why not use it? Sure, it's nice to have a database all-in-one program, but if you can take 5-10 minutes and copy the database to skid-pg, and get more features + better graphics, wouldn't that be a better option?

Also there are many free databases on the internet, which you can find on google which I assume you can use with skid-pg (pgn databases should work with any database program).

Regardless of what I just said - I'm not really part of this argument because I don't use skid-pg OR chessdb! I've never used either of them. :)

I will probably check out the features of these database programs (and the quality of the databases themselves, which is very important) and maybe compare them to chessbase 9 (or chessbase light).

Chessbase 9 is pretty nice, though I can't really mention any specific features because I don't know if the other 2 programs contain those features. There was one thing that was neat though: A while back, I was interested in playing an opening that I had seen once in a polgar game. I didn't remember the specifics, but I remembered it was a sicilian in which polgar played a6-b5 and then Bc5 + Qb6. In chessbase I could actually search for these moves themselves (instead of just a single position) and I found the game she played immediately. I think it's called a 'maneuver search' or something similar. Very interesting feature for researching ideas.

9/17/2007 05:08:00 PM  
Blogger Blue Devil Knight said...

Zen: good points. If it is true that scid-pg uses less ram, that would be important, as it takes a long time to search through 3.5 million games.

9/17/2007 05:16:00 PM  
Anonymous Anonymous said...

hi, it's Anon again. use "Fragext" (google it) to defrag your database files. you will see a huge speed increase.

Scid-PG and ChessDB are not friends.. but they are brothers, both being sons of SCID. For me, Rybka compatibility settles it once and for all. However, i'd rather see people use ChessDB than chessbase, because more ChessDB and/or Scid-PG users means that we can soon download stuff in Scid format rather than blasted Chessbase.

9/17/2007 09:01:00 PM  
Blogger Unknown said...

oh man...wow...3.5 million freakin games....i'm downloading it...

9/18/2007 12:13:00 AM  
Blogger crow with no mouth said...

The current beta version of ChessDB is compatible with UCI, so it can talk to Rybka or other such engines (it includes Toga II, same as the current Scid version).

As for this question of the databases: any database that ChessDB downloads for you is free. Just look at the README which explains where it's downloading it from (sourceforge or chesstutor) and go there and get it yourself. The menu option is only a convenience and doesn't mean you can't use that database with some other software.

There are also free databases at icofy.net (one with about 3 million games). I'm not sure how much they overlap; I'm guessing the 3.5 million game database that ChessDB points you to is the icofy one plus TWIC or something like that, but I haven't checked.

9/18/2007 01:19:00 PM  
Blogger Blue Devil Knight said...

Crow with no mouth: thanks for the information. It seems a lot of people want to badmouth ChessDB, and it seems more personal than objective.

9/18/2007 01:27:00 PM  
Blogger katar said...

ChessDB obviously ripped off Scid-PG's UCI engine support. ChessDB even borrowed SCID-pg's GUI including GUI for features that Scid-pg has and ChessDB doesn't. Maybe Kirkby will figure out how to rip off those corresponding features as well, or at least he will figure out how to cover his tracks by removing the remnants of Scid-pg's GUI.

Very ironic considering Kirkby's claims that Scid-pg ripped off HIS code. But if you check Kirkby's allegations the allegedly lifted "code" only involves irrelevant documentation-- definitely not any programming for new features. The documentation originally sourced from Shane Hudson anywya, before Kirkby's global search&replace operation changing even Shane's engine "Scidlet" to "ChessDBlet".

By the way, the linked directory above contains an archive of all my old blog posts. But do not advertise or publicly link to it, please. Please share the ChessDB-ripoff picture though. Peace out.

9/22/2007 03:25:00 AM  
Blogger Blue Devil Knight said...

Katar:

They are both coming off as petty and childish. At any rate, this is Copylefted, open source, where it is encouraged to copy and share the work with others, to modify the work, distribute modified bits of the code, etc (I stole some of that from Wiki copyleft entry).

However, because something is copylefted doesn't mean it is ok to take it and not acknowledge where you got it! It's just dishonest and probably violates the terms of most copylefts [great 2 c the cryptoblog, incidentally].

I'll ask Dr Kirby about this.

9/22/2007 12:32:00 PM  
Blogger Blue Devil Knight said...

Katar:

Kirby openly acknowledges the UCI code is from scid-pg. He acknowledges scid-pg in the code itself, in the web updates, etc.. Since it is copylefted, that is his right and should be encouraged.

See here for some of the ChessDB code acknowledging scid-pg.

It sounds like you have bought into this silly game perpetuated in part by Pascal whatever who maintains scid-pg.

Incidentally, what if I defrag my whole hard drive, should that also work to make it faster?

9/22/2007 08:31:00 PM  
Blogger crow with no mouth said...

I can answer the last part-- yes, defragging the whole hard drive works fine. It sped up opening my 3+ million game database by 3x or more.

If you have some other custom defrag tools you can have them defrag only the database files. But that is not necessary, and doing the whole drive is fine.

9/23/2007 02:26:00 AM  
Anonymous Anonymous said...

Kirby openly acknowledges the UCI code is from scid-pg. He acknowledges scid-pg in the code itself, in the web updates, etc.. Since it is copylefted, that is his right and should be encouraged.

See here for some of the ChessDB code acknowledging scid-pg.


I just looked at the "here" link. Is that an acknowledgment? It sounds like a child victory: "i can take your work, so i took it!". No thank you ...

I also followed some of discussions on the mailing lists between the 2. For me at least it seems obvious that Kirkby is the big mouth here. He also manages to stay on google top searches with the same denigratory post, sticked wherever he can.

12/10/2007 04:28:00 PM  
Blogger Blue Devil Knight said...

Laser, yes it is very childish. It isn't just him though.

12/10/2007 08:57:00 PM  
Blogger Aftermath said...

Is ChessDB dead? No updates since 2007.

11/15/2012 06:21:00 PM  

Post a Comment

<< Home