RubyForge - 30 million records
Another growth milestone for RubyForge - the database now contains over 30 million records. This is up from 20 million records just 5 months ago, which means that the database is growing at around 60K records per day. The biggest table is gem_downloads, which contains over 12M records. Fortunately we don't do much querying of this table, just lots and lots of inserts. The next largest table is frs_dlstats_file which gets mostly inserts, too, except for the nightly stats runs.
As always, the excellent PostgreSQL database engine continues to handle the load without problems. Go PostgreSQL!
Cool. How big is that, physically? How many queries per second are you handling, and on what hardware? Any funky graphs? :)
Here at Newzbin we have something like 930 million records, using MySQL and some moderately beefy hardware (half a dozen 16G dual Opterons and 4-24 SCSI disks). The database comes to roughly 240GB, with some archived read-only tables compressed, and we see an average of about 500 queries/second per server.
Graphs: http://support.newzbin.com/rrd/database
Posted by: Thomas Hurst | June 30, 2007 at 11:26 AM
Thomas - whew, that's quite a setup! Good to hear that MySQL scales that far... we're using it on another job and the database is growing quickly, but it's nowhere near that big yet. RubyForge-wise, the DB is 4 GB on disk runnning on a single dual Xeon box with RAID 5 SCSI. It's frequently IO bound, so we may be getting some new hardware soon... we'll see.
Posted by: Tom Copeland | June 30, 2007 at 03:12 PM