when I try to run a model, my ram usage is significantly higher than the amount of memory I specified with the memory keyword. Lets say I am using nastran64.exe mem=5Gb. In the task manager max memory for analysis.exe is 5Gb, .f04 and .log file also 5 Gb. Before I started the job, my memory usage was 7 Gb but during runtime it's increasing up to 20-40Gb. I haven't started any other program and I tested this on two different machines with both NX Nastran v7.1 and NX Nastran v9.0. Neither the taskmanager nor the resource manager shows which process is using all this memory. Could it be that there is a memory leak or something?
OS: Windows 7
This has being explained in detail a few posts ago, take a look to this:
Not a memory leak at all, this is Windows OS that use unallocated RAM memory to speed I/O task and then to improve disk performance, for this reason this is very important to have as much RAM as possible, as well as fast hard disc to allocate the nastran SCRATCH files, the best option is a RAID-0 based in a collection of SSD.
Are you sure? I am talking about unavailable ram, which is not cached! For I/O speedup Nastran uses the buffer cache, so if I'm right I would expect high cached data in the taskmanager, which isn't the case. I compared the performance of nx nastran to another fe solver and the other one is almost 2 hours faster, since he isn't paging at all. Both solver were configured for smp, both parallel sparse solver and during runtime of other fe solver I indeed see a lot of cached data for I/O speedup!
I attached a screenshot of the taskmanager, when I solve my model with a non nastran solver... It isn't paging at all. That's how it should look like, in my opinion.
You can't really compare the mechanics of Nastran to any other solver. Nastran was born in the days of paper punch cards, 9 track tape and low memory mainframes. Historically, it's architecture assumes that most jobs will not fit into core memory and therefore, almost everything is written to disk. Newer subsystems do more work in core memory and ongoing work is being done to remove unnecessary IO operations.
Due to the above, Nastran does much more IO that more contemporary solvers, even for small jobs that would easily fit into available memory. Historically, this was all file IO. As Blas noted, modern operating systems (Linux and Windows Vista and later (not XP)) all use unallocated physical memory as an IO cache. If a solver is doing everything in the core memory of its process, there will be no IO and you will not see this caching. Nastran always does a fair amount of IO, so you will always see this caching with a Nastran job.
As Jim pointed out, NX Nastran does a fair amount of I/O. The heart of nastran is the DMAP architecture (open architecure) which allows users to modify the standard nastran solution sequences. Data between two successive DMAP statements cannot be assumed to be persisent in memory but rather preserved in the database files.
Nastran is also designed to be more efficient for (very) large problem sizes. For these classes of problems, one cannot solve everything incore.
Until Windows Vista64/Windows 7, there was no concept of memory cache on windows. For these older OS, nastran was using filemapping to do the I/O. Filemapping uses virtual memory to map disk I/O to memory. With windows 7 and later, Microsoft introduced a memory manager concept which reserves a portion of unallocated memory for I/O cache. Linux OS does the same thing as well. However, Linux allocates 100% of the unallocated memory for I/O cache.
In order to run nastran, we recommend the use of multiple disks in a RAID0 configuration (software RAID) for nastran scratch file system. Use of SSDs in this RAID configuration will give you the best performance. You can also use the higher RPM SAS/SATA drives as well. The throughput time for nastran run will be drastically reduced when you have a high performance I/O system.
Sorry but I still don't get it!
I have two Parallel Sparse Solver (NX Nastran) and a non nastran one.
I have a model with 2.5 Mio DOF, so it does not fit into memory:
NX Nastran Parallel Sparse Solver needs 1h,43m to solve the model and heavily starts writing
to the windows paging file. While running the job, it eats up all my ram (24 GB).
How can I prevent Nastran from doing this? I specified memory=5Gb and I used the nastran64.exe not the nastran64L.exe...
So why does nastran allocate more than 5 Gb? All for I/O speedup? I thought Nastran would use the memory cache to do this, which you can see in the taskmanager!
There you can see: Total, Cached, Available and Free.
And cached physical memory is still available to use, e.g. if you start another process.
Like in the attached file the non nastran still has a lot of available ram and a lot of chached physical memory and this solver solves the model within 14 minutes. I guess because it does not use the windows paging file and only uses his own scratch file. It also does a lot of I/O!. I don't see, why Nastran should be over 1 hour slower than the non nastran solver!
To make it more clear, I attached a screenshot of the taskmanger during nx nastran solving. Simply compare it to the one I posted before!
When I say there is no ram, I am talking about unavailable ram!
Cached memory (at least the standby list) is still available ram.
If you are using NX Nastran 7.1, you probably will be paging since as I explained earlier, that version uses filemapping to get around the poor windows I/O. For that version of nastran, you'll need to put the following line in your rcf file (sysfield=mapio=no) to turn off the file mapping. If you are using NX Nastran 9.0, this is automatically set by nastran.
When performing I/O requests, nastran makes native calls to windows I/O subsystem. It is the subsystem that tends to use additional memory on the system (not used for nastran) to do the I/O cache. This is exactly what you see in the performance monitor. If you want to turn off memory cache for disk I/O, you'll need to contact Microsoft support. There is a registry setting to turn this off.
It is the sparseness of the matrices and not the total amount of DOFs that determine whether a solve can be performed in-core. For example, having MPCs, RBE elements, Glue connections, etc can make the matrices more dense.
Nastran relies more on the disk and less on direct memory. The other software you are pointing out probably relies less on disk and more on memory. This is just an inherent software design to cater to different sizes and classes of problems.
It is NOT recommended to run nastran on any hardware and expect significant peformance speed ups. However, as explained in the earlier post, you'll need to run nastran on a system that has multiple disks set up in a RAID0 and have the swap file system on a separate disk (separate from the RAIDed disks)
You are posting images showing the overall memory usage for all processes on the machine. Please go to the Processes tab in TaskManager and look at the Working Set (Memory) column for the analysis.exe process. This will not exceed the value requested by the MEM keyword. The IO caching is done by the operating system outside of the Nastran processes.
To be clear: Nastran is not "eating up" all of you ram. The Operating System is utilising a resource (unallocated ram) that would otherwise be sitting idle to help improve performance.