MMS • Raul Salas
Sort Memory errors in Mongodb and how to resolve them!
So you installed Mongodb and everything seems to be humming along for months and then all of the sudden your application comes to a grinding halt because of the following strange error your receiving from Mongodb:
OperationFailed: Sort operation used more than the maximum 33,554,432 bytes of RAM. Add an index, or specify a smaller limit.’ on server
This error is caused by Mongodb engine exceeding memory capacity limits on sorting when Mongodb cannot use an index. The default memory limit for sorting data is 32 MB. A good first step would be to optimize the query with the explain query parameter, However, to get your application online quickly there is a quick fix for this issue. The default parameter for an internal system parameter called “InternalQueryExecMaxBlockSortBytes” will need to be modified. Let’s verify what the parameters are now on this particular Mongodb instance
db.adminCommand({getParameter: ‘*’})
This will return a large document with the internalQueryExecMaxBlockingSortBytes the value of 33,554,432 bytes (33 MB). So now let’s increase this parameter to 134,217,728 bytes (134 MB) by executing the following command:
db.adminCommand({“setParameter”:1,”internalQueryExecMaxBlockingSortBytes”:134,217,728})
Now execute your query to see if the error goes away! You may need to increase the parameter if your query continues to fail. It would be nice if Mongodb included a metric to monitor this parameter’s usage in a future release.
Update for Mongodb 4.0:
$sort
and Memory Restrictions
The $sort
stage has a limit of 100 megabytes of RAM. By default, if the stage exceeds this limit, $sort
will produce an error. To allow for the handling of large datasets, set the allowDiskUse
option to true
to enable $sort
operations to write to temporary files. See the allowDiskUse
option in db.collection.aggregate()
method and the aggregate
command for details.
Raul Salas
Raul@mobilemonitoringsolutions.com