- use realistic data volumes -- use production sized data sets in QA. What happens when a million rows are returned?
- don't rely on data producers -- make sure you limit the results and don't rely on the producers of the data to always keep a reasonably sized data set. Things will change and you will break.
- put limits into other application-level protocols - web service, RMI, XML-RPC calls are all vunerable to returning huge collection of objects which will consume tons of memory.
Sunday, April 11, 2010
Release It! - Chapter 4.11 Unbounded Result Sets
Unless your explicitly limit the size of your result sets, you will run out of memory at some point. QA tends to have much smaller result sets than production which makes finding issues around large result sets difficult. A good API will allow for limiting of result sizes. Objects that contain collections of other objects are a typical place where large result sets can overwhelm the system. There are less than perfect ways to get a database to limit results but they are vendor specific. Unbounded Result Sets are a common cause of slow responses.