MongoDB 3.2.1 | wiredTiger
New documents are inserted.
There are no conditions or difficult sampling.
Collection statistics:
"count" : 149380522 "size" : 5582314593.0 "avgObjSize" : 37 "storageSize" : 12036964352.0 "capped" : false "totalIndexSize" : 1537122304.0
/etc/security/limits.d/99-mongodb.conf
mongod soft fsize unlimited mongod hard fsize unlimited mongod soft cpu unlimited mongod hard cpu unlimited mongod soft as unlimited mongod hard as unlimited mongod soft nofile 65536 mongod hard nofile 65536 mongod soft nproc 65536 mongod hard nproc 65536
There is nothing conspicuous in the output of dmesg .
mongostat:
insert query update delete getmore command % dirty % used flushes vsize res qr|qw ar|aw netIn netOut conn time *0 *0 217 *0 0 139|0 2.0 29.4 0 2.6G 2.3G 0|0 0|1 136k 59k 6 2016-02-15T14:45:49+03:00 *0 *0 175 *0 0 109|0 0.7 29.4 1 2.6G 2.3G 0|0 0|1 108k 50k 6 2016-02-15T14:45:50+03:00 *0 *0 43 *0 0 32|0 0.5 29.4 0 2.6G 2.3G 0|0 0|1 29k 27k 6 2016-02-15T14:45:51+03:00 *0 *0 10 *0 0 7|0 0.6 29.4 0 2.6G 2.3G 0|0 0|1 6k 20k 6 2016-02-15T14:45:52+03:00 *0 *0 28 *0 0 15|0 0.6 29.4 0 2.6G 2.3G 0|0 0|1 14k 23k 6 2016-02-15T14:45:53+03:00 *0 *0 2 *0 0 1|0 0.6 29.4 0 2.6G 2.3G 0|0 0|1 425b 18k 6 2016-02-15T14:45:54+03:00 *0 *0 1 *0 0 2|0 0.6 29.4 0 2.6G 2.3G 0|0 0|1 368b 18k 6 2016-02-15T14:45:55+03:00 *0 *0 17 *0 0 14|0 0.6 29.5 0 2.6G 2.3G 0|0 0|1 11k 21k 6 2016-02-15T14:45:56+03:00 *0 *0 11 *0 0 10|0 0.6 29.5 0 2.6G 2.3G 0|0 0|1 8k 20k 6 2016-02-15T14:45:57+03:00 *0 *0 4 *0 0 2|0 0.6 29.5 0 2.6G 2.3G 0|0 0|1 2k 18k 6 2016-02-15T14:45:58+03:00
top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2222 mongod 20 0 4200976 3,692g 6664 S 3,3 23,9 0:50.26 mongod
Strong insertion delays started from about 140 million documents.
If you create a new collection and write to it, then everything works as it should.
It records about 300 documents per second.
Example of inserting a document (golang + labix.org/mgo):
if _, err := c.UpsertId(bson.ObjectIdHex("xxxxxxx"), bson.M{ "$set": bson.M{"field": "string"}, }); err != nil { panic(err) }
Actually, how to fix this problem?