Memory Tunning Method
Download
Mode fokus
Ukuran font
Memory tuning aims to achieve an optimal balance between machine resources and performance by providing a sufficient and stable memory usage to ensure normal system performance rather than simply reducing the memory usage. This document describes why and how TencentDB for MongoDB memory is occupied. It also provides troubleshooting methods and solutions for you to tune the database performance in time.
Storage Engine Memory
The cache size of WiredTiger (TencentDB for MongoDB storage engine) cannot exceed 60% of the actual requested instance memory specification. If the storage engine cache reaches 95% of the limit, the instance load will be high. If the dirty data cache percentage reaches 20% in the storage engine, user threads will be blocked.
Troubleshooting
You can view the engine memory usage in MongoDB Shell through
db.serverStatus().wiredTiger.cache. In the returned information, the value after bytes currently in the cache indicates the cache data size in bytes. See the following figure.{......"bytes belonging to page images in the cache":6511653424,"bytes belonging to the cache overflow table in the cache":65289,"bytes belonging to page images in the cache":8563140208,"bytes dirty in the cache cumulative":NumberLong("369249096605399"),......}
You can view the dirty data Cache percentage of the storage engine in real time on the MongoStatus feature of DBbrain performance trend. For details, see mongostat.
Usage suggestions
If Cache Dirty of the storage engine stays above 20%, you can troubleshoot as follows:
A. Control the volume of data written per unit time.
c. Increase the number of threads for cleaning dirty data. A higher value consumes more instance resources, so adjust the value with caution. The default value is
threads_max=4,threads_min=1. You can set it as follows:db.runCommand({"setParameter":1, "wiredTigerEngineRuntimeConfig":"eviction=(threads_max=8,threads_min=4)"})
Note:
Adjusting the number of threads for cleaning dirty data via db.runCommand is only suitable for replica set architecture version 4.0 and later.
If Cache Used of the storage engine stays above 95%, you can troubleshoot as follows:
a. Analyze the slow log in the database by using Slow SQL Analysis.
c. Increase the number of threads for cleaning dirty data.
Memory of Connection and Request
Each connection needs to be handled by a corresponding request thread. Too many request threads cause frequent context switching and increase memory overhead. Each thread can use a maximum of 1 MB of thread stack, which typically ranges from tens of KB to a few KB.
When a request command is received, a request context is created. During this process, many temporary buffers may be allocated, such as request packets, response packets, and temporary buffers for sorting. These buffers are released when the request ends. However, this release only returns them to the memory allocator
tcmalloc. tcmalloc preferentially returns these buffers to its own cache and then gradually releases them to the operating system. In many cases, high memory utilization occurs because tcmalloc does not return memory to the operating system in a timely manner, which can cause memory usage to reach tens of GB.Troubleshooting
The size of memory that
tcmalloc has not returned to the operating system can be viewed using the command dbserverStatus().tcmalloc.tcmalloc.formattedString or
db.serverStatus().tcmalloc. As shown in the following figure, you can use db.serverStatus().tcmalloc.tcmalloc.formattedString to query the information. The memory corresponding to Bytes in use by application refers to the memory actually consumed by the Mongod node, and Bytes in page heap freelist indicates the memory that has not been returned to the operating system.
Log in to the MongoDB console. On the System Monitoring page, view the trend chart for the instance monitoring metric Connection Percentage. Connection percentage refers to the ratio of the current number of connections in the cluster to the maximum number of connections.
Usage suggestions
1. Adjust the memory reclamation rate.
The
tcmallocReleaseRate parameter is used to control memory release and specifies the amount of memory to release when memory usage reaches a certain ratio. A higher tcmallocReleaseRate value allows MongoDB to release memory faster, but it also impacts performance. Adjust this parameter as required when configuring tcmallocReleaseRate to reach a balance between memory usage and performance requirements. Run the following command to set tcmallocReleaseRate, which ranges from 1 to 10.When
tcmallocAggressiveMemoryDecommit is set to 1, memory can be reclaimed directly and quickly. The execution method is as follows:db.adminCommand( { setParameter: 1, tcmallocReleaseRate: 5.0 } )db.runCommand( { getParameter: 1, tcmallocAggressiveMemoryDecommit:1} )
Note:
tcmallocReleaseRate is applicable to TencentDB for MongoDB version 4.2 and later.2. Control the number of concurrent connections.
We recommend that you create a maximum of 100 long connections to the database. By default, the MongoDB Driver can establish a connection pool of 100 with the backend. When there are many clients, reduce the connection pool size of each client. Generally, we recommend that you keep the total number of long connections to the database under 1000. If the connection percentage is high, see Analysis and Solutions for High Connection Utilization to optimize it.
3. Reduce the memory overhead per request.
For example, you can reduce collection scans and memory sorting by creating indexes. To create an index, see MongoDB Indexes.
4. Upgrade the memory specification.
If the memory utilization continues to rise when the number of connections is appropriate, we recommend that you upgrade the memory specifications to avoid drastic decline in system performance due to memory overflow and large cache purge.
Upgrade CPU and memory configuration of Mongod. For more information, see Adjusting Mongod Node Specification.
You can add shards for the sharded cluster. For detailed directions, see Adjusting Shard Quantity.
Metadata Memory
TencentDB for MongoDB maintains some metadata for each collection, such as index information, number of documents, and storage engine options. The management of this information will occupy some system resources. If there is a large amount of memory metadata in your collection or index, a lot of memory will be occupied. Especially in versions earlier than TencentDB for MongoDB 4.0, a large number of file handles will be opened during full logical backup. If they are not returned to the operating system in time, the memory usage will increase rapidly. Furthermore, in lower versions of TencentDB for MongoDB, file handles may not be deleted after a large number of collections are deleted, which may also cause memory leaks.
Troubleshooting
To count the number of collections and indexes in MongoDB, use the following commands separately:
use <database>;db.getCollectionNames().filter(function(c) { return !c.startsWith("system."); }).lengthdb.getCollectionNames().forEach(function(col) { print("Indexes for " + col + ": " + db.getCollection(col).getIndexes().length); })
Query the number of collections as prompted below:
mongos> db.getCollectionNames().filter(function(c) { return !c.startsWith("system."); }).length4
Query the index number of each collection as follows:
mongos> db.getCollectionNames().forEach(function(col) { print("Indexes for " + col + ": " + db.getCollection(col).getIndexes().length); })Indexes for nba: 3Indexes for t1: 1Indexes for test: 1Indexes for user: 1
Usage suggestions
We recommend that you keep the number of collections within a few thousand for MongoDB. Otherwise, MongoDB will experience a significant performance drop when managing these collections.
Memory Usage in Index Creation
MongoDB may consume a large amount of memory when creating indexes. In normal business data writes, the secondary node maintains a
buffer of about 256 MB for data playback. For MongoDB versions earlier than 4.2, when the primary node creates indexes in a non-background manner, the backend replays the recorded index creation operations and creates indexes serially, which may consume up to 500 MB of memory. For MongoDB 4.2 and later versions, the background option is deprecated by default. Indexes are created in the foreground, but secondary nodes are allowed to create indexes by replaying the index creation operations recorded by the primary node in parallel. This speeds up index creation, but it also incurs some memory overhead on secondary nodes. Therefore, creating multiple indexes at the same time may cause instance memory overflow. In addition, secondary nodes may consume more memory during data playback, especially after the primary node completes index creation.Troubleshooting
On the Active Sessions tab of the Real-time Sessions page in DBbrain, check whether there are multiple processes adding indexes. As shown in the following figure, you can find the CreateIndex key field in the Command column.
Usage suggestions
Don't create multiple large indexes simultaneously during peak hours.
To avoid excessive memory usage, create indexes in the background whenever possible. For details on how to create indexes, see the MongoDB official website.
If the memory is insufficient, consider increasing the system memory. For details, see Adjusting Mongod Node Configuration Specifications.
To reduce the memory usage, we recommend that you use more effective index types, such as text search index and geospatial index. Use DBbrain's index recommendation feature to select the optimal index.
For larger collections, use sharding technology to disperse the storage of data and indexes. By doing so, the memory pressure of a single instance will be reduced.
Memory Produced in Logical Backup and Primary-Secondary Switch
Logical backup generally will produce large number of data scans, causing high memory usage. If you select secondary node or hidden node for logical backup, the memory used by secondary node will be evidently higher than that of the primary node.
In primary-secondary switch, the original primary node will be switched as the secondary node. Thus, the memory usage of the current secondary node is higher than that of the current primary node.
Troubleshooting
1. Log in to the MongoDB console, and check whether there is an running backup task on the Task Management page. The task being executed will be displayed on the top of the task list. Click Task Details in the Operation column, and check whether Logical Backup is selected. For detailed direction, see Task Management.
2. In Task Type on the Task Management tab, search for Promote to Primary Node to confirm whether the primary-secondary switch is performed.
3. Click the System Monitoring tab, and then view the memory utilization of the primary node and secondary node respectively from the Cluster Overview drop-down menu on the left.
Usage suggestions
As physical backup is performed by copying the physical file, memory usage is not so high. If your business has high requirements on the node memory, select physical backup mode. For more information, see Data Backup.
If reading a secondary node is not set after the primary-secondary switch, restart the corresponding secondary node in the console to free up the memory. If the secondary node has a request, perform the operation during off-peak hours.
Bantuan dan Dukungan
Apakah halaman ini membantu?
Anda juga dapat Menghubungi Penjualan atau Mengirimkan Tiket untuk meminta bantuan.
masukan