« PMD exam on javablackbelt | Main | Java to Python converter using JJTree »
Just noticed that RubyForge hit some nice round numbers this weekend:
As the Oompa-Loompa say, the numbers "keep on growing and there's no sign of slowing"!
On the downside, I had to disable ViewVC as it was getting hammered by bots and was driving the load average up over 10. Yikes. Anybody have a machine that could host RubyForge's ViewVC and perhaps accept an hourly rsync?
TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451d3c069e200e5505cd38f8833
Listed below are links to weblogs that reference RubyForge round numbers:
» RubyForge round numbers from PsychicProgrammer.com
Just noticed that RubyForge hit some nice round numbers this weekend: As the Oompa-Loompa say, the numbers keep on growing and there's no sign of slowing! On the downside, I had to disable ViewVC as it was getting hammered by...Link:... [Read More]
This is only a preview. Your comment has not yet been posted.
As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.
Having trouble reading this image? View an alternate.
Bots? I wonder what they're scraping. And why. Maybe it's for stuff like Google's code search.
I (almost) never use it myself. If I want to look at the code for a project I just, you know, download it. :)
Posted by: Daniel Berger | March 12, 2007 at 11:13 AM
Yeah, mostly bots... just crawling every link they find... ah well.
Posted by: tomcopeland | March 12, 2007 at 12:10 PM
Could liberal use of nofollow on those sections help?
Posted by: spicycode | March 12, 2007 at 10:14 PM
spicycode - yeah, although, we want the bots to crawl those pages... just not at such a furiuos rate. Blah....
Posted by: tomcopeland | March 13, 2007 at 05:37 AM
Not that it matters much to me now that I've figured it out, but do you think it would be appropriate to put something more informative than "PAGE NOT FOUND" on the ViewVC pages? I must have tried like thirty ViewVC pages for various projects; all the while hoping that ViewVC was just temporarily down and would be back up any minute.
Posted by: Meekish | March 14, 2007 at 02:50 AM
Could you use robots.txt to deny crawlers rather than disable access entirely?
Posted by: Jeremy Kemper | March 14, 2007 at 02:54 AM
Meekish - good point, I've removed the links, thanks for the nudge.
Jeremy - yeah, from what I've seen, they don't seem to respect robots.txt. Actually, we want the code to get crawled, we just don't want it to get crawled at such a tremendous rate. But we may get an offsite solution here shortly... stay tuned....
Posted by: tomcopeland | March 14, 2007 at 05:42 AM
There is a setting in the Google Webmaster Tools that allows you to decrease the crawl rate. I haven't actually tried it, but it may be what you're looking for.
Posted by: PJ Hyett | March 17, 2007 at 03:48 PM
PJ - Yup, but then there's msnbot, and etc., etc. But anyhow, Matt Bauer has kindly put together a server for me to run ViewVC on, so I just need to set that up, set up an rsync job, and redirect things...
Posted by: tomcopeland | March 17, 2007 at 04:00 PM