Why does my java project use more and more physical memory after running for a period of time?

  java, question
One of the company's java ee projects running on tomcat mainly functions as an interface server.  The physical memory on the server is 8G. When tomcat starts up, the initial Heap and Max heap memory of jvm are both set to 1400M m. After the project runs, the physical memory is almost 2G.  However, after running slowly for a few days, using the top command to see the memory usage, the java process takes up more and more memory, and finally uses up the physical memory. However, it will not increase after that (swap will not be used). At the same time, tomcat does not feel any impact on its performance, nor will it throw OOM or something. When the load is low, the Central Processor usage rate will not exceed 10%.
 Later, I did the stress test of the interface for this project. The internal processing logic of the interface is to accept the request message and save it to the database. After 30 minutes of stress test, the physical memory of the machine will be used more and more until it is fully used. However, the response speed of the interface has no effect.  During the whole pressure measurement process, I used visualVM to observe the jvm, and found that the Heap basically only used more than 500 m, and did not reach the maximum value (the initial value and the maximum value assigned to the virtual machine when the project was started were 1400M).  After the pressure test, there are more than 50 threads in the live state (the maximum number of threads configured by tomcat is 300). It seems that the next ThreadDump is all threads related to http requests.
 I would like to ask, is this normal?  Since the allocated Heap has not been used up, where is the slowly growing physical memory used (is it the thread Stack?  ), why is this physical memory not released as the load decreases?
 The JVM startup parameters of the project and some parameters set by tomcat server.xml are posted here:
Export java _ opts = "-server-xm1400m-xm1400m-xss1512k-xx: add AggressiveOpts -XX: add usebiasedlocking-xx: permsize = 128m-xx: maxpermsize = 256m-xx: add DisableExp  LICIT GC-XX: MAXTENURING THRESHOLD = 31-XX: Add UseConcMarkSweepGC -XX: Add USEPARNEWPGC-XX: Add CMSParallelRemarkEnabled -XX: Add USECMSCOMPACTATFLECTION-XX: LARG  EPageSizeInBytes=128m -XX: plus UseFastAccessorMethods -XX: plus usecmsinitiatingoccupancyonly-djava.awt.headless = true "
<Connector port="9010" protocol="HTTP/1.1"
 IEncoding="UTF-8"  minSpareThreads="25" maxSpareThreads="75"
 acceptCount="300"  maxThreads="300" maxProcessors="1000" minProcessors="5"
 URIEncoding="UTF-8" />
The younger brother's experience in JVM optimization is basically zero. If you can solve your doubts greatly, I would appreciate it. By the way, do you have any books on JVM optimization in Kneel to beg?  T_T

Use perftools to see if there is an out-of-heap memory leak.