mod_security woes

A customer had an unfortunate experience with mod_security recently. They were getting occasional HTTP 500 responses from their Rails app on certain large pages.  ExceptionNotifier wasn't reporting any stacktraces, and a check of the actual log file didn't show any problems either.  Even connecting to the production machines and running the same request using script/console's app.get worked fine!

Finally they took an entire slice out of their production architecture and made the request while watching the Apache logs.  And lo and behold - mod_security was seeing a large response and returning a 500 code.
This was a surprise since mod_security had been (we thought) configured in "logging-only" mode. 

Lessons learned are 1) load up the staging environment with lots of data to shake out any such issues, 2) study the mod_security settings to ensure it'll do what's expected, and 3) if ExceptionNotifier and app.get
tells you no exception is happening in the Rails app, widen your search.