Cancel
Showing results forΒ 
Search instead forΒ 
Did you mean:Β 

Teamcenter 11.2.3.1 extreme Performance issue

Creator
Creator

Hello Board

We have a extreme unusual performance issue in our TC. The TC speed is very fluctuating, for example checkout needs sometimes 10 seconds and the a few seconds later the same procedure needs about 2 minutes. The structure expand needs the double time than on a reference system. And we've got the feeling that the system extrimly slow down when 2 or 3 people loads big structures to NX. We see in our server logs that normaly all TC Servers and the Database Server have a maximum CPU usage of 20%, but somtimes the DB Server had a peak up to 50%. In the TCLogs we see that especialy when we open structures there are a lot of sql queries. The reason for that seems to be an occurrence "In Context" overwriting thats automaticly is in our DB because we activates the NX arrangememt sync. GTAC believes that this is no system failure but they are creating a performance patch for TC11.5 to fasten this queries up. I did not believe that this is our only problem here, but there is nothing concrete I can say. I thing these fluctuating is some DB Server Issue or something in the 4tier communication. 

 

Maybe someone else had the same problems and found a solution or can gave me a hint what we can check.

 

Here are oure system data: 

 

TC11.2.3.1

NX10.0.3 MP19

2 Enterprise Server, each for 64 User

MSSQL 2014 based Database

1 FMS

1 Webtier Server 

1 Dispatcher Server 

1 T4EA Server

 

I know this issue is not very concrete, but sometimes it helps to talk about to find solutions πŸ˜‰

 

Greetings Jens

15 REPLIES

Re: Teamcenter 11.2.3.1 extreme Performance issue

Solution Partner Phenom Solution Partner Phenom
Solution Partner Phenom
If you're not measuring the performance then nothing is wrong.

Randy Ellsworth, Teamcenter Architect, Applied CAx, LLC
NX 11 | SW 2016 | Creo 4 | TcUA 11.4
Evaluating: AW 3.4

Re: Teamcenter 11.2.3.1 extreme Performance issue

Creator
Creator
Hello Randy

I can live with that when we have no fluctuation. When TC is all the time slow than its slow, but when its somtimes fast and in the next minute very slow, than I'got a bad feeling with that.

Greetings Jens

Re: Teamcenter 11.2.3.1 extreme Performance issue

Solution Partner Phenom Solution Partner Phenom
Solution Partner Phenom
Prove it. You can't fix it if you don't know what's broken. That may sound harsh or insensitive and for that I apologize. Use-cases are important and so are metrics. Without those its all just a guess or a feeling which doesn't lead anywhere useful.

Randy Ellsworth, Teamcenter Architect, Applied CAx, LLC
NX 11 | SW 2016 | Creo 4 | TcUA 11.4
Evaluating: AW 3.4

Re: Teamcenter 11.2.3.1 extreme Performance issue

Creator
Creator
Hi
This is my problem, I know there is something broken but I can't find the problem and I'am glad for every hint. Maybe someone has a similar problem and share it or someone ask the right questions so I'm move in the right direction.

Greetings Jens

Re: Teamcenter 11.2.3.1 extreme Performance issue

Solution Partner Phenom Solution Partner Phenom
Solution Partner Phenom

Let's start at the database instance...

MS SQL 2014 > Apply CU1 from Microsoft. https://support.microsoft.com/en-us/help/2887899/
Also, see SFB-Teamcenter-6464

 

Now indexes...

Run this SQL to check indexes (fragmentation >50%, page_count >15)

SELECT SI.NAME AS INDEX_NAME, OBJECT_NAME(DMV.object_id) AS TABLE_NAME , 
avg_fragmentation_in_percent 
FROM sys.dm_db_index_physical_stats( DB_ID(), NULL, NULL, NULL, 'Detailed')
as DMV LEFT OUTER JOIN sys.indexes AS SI ON DMV.OBJECT_ID = SI.OBJECT_ID 
AND DMV.INDEX_ID = SI.INDEX_ID WHERE avg_fragmentation_in_percent > 50.0 
AND page_count > 15 AND SI.NAME IS NOT NULL 
AND index_type_desc IN('CLUSTERED INDEX', 'NONCLUSTERED INDEX') 
ORDER BY avg_fragmentation_in_percent DESC;

Run in a TC command window

 

index_verifier.exe -u=infodba -p=infodba -g=dba -o=DO_IT

Rerun the SQL and compare stats. Same as first run.

 

Collect all the SQL statements that start with 'ALTER' from the text file (DO_IT).

Execute the 'ALTER' commands in SQL server manager.

Rerun the SQL and compare stats. No rows returned. Indexes are successfully defragmented.


Randy Ellsworth, Teamcenter Architect, Applied CAx, LLC
NX 11 | SW 2016 | Creo 4 | TcUA 11.4
Evaluating: AW 3.4

Re: Teamcenter 11.2.3.1 extreme Performance issue

Creator
Creator
Hello Randy

Thanks for you reply. We check your idea and see that we allready done this points. Any other ideas?

Greetings Jens

Re: Teamcenter 11.2.3.1 extreme Performance issue

Solution Partner Phenom Solution Partner Phenom
Solution Partner Phenom
Next is to check if there are any other databases running on the database server. Then you'll have to tell me more about your database installation. Is the DB software is installed in the default location? Does your database instance files live in the same location as the default? Processor, RAM, Disk space? Not RAID 5?

This will complete the database layer and we can move up to the application layer.

Randy Ellsworth, Teamcenter Architect, Applied CAx, LLC
NX 11 | SW 2016 | Creo 4 | TcUA 11.4
Evaluating: AW 3.4

Re: Teamcenter 11.2.3.1 extreme Performance issue

Creator
Creator
Hello Randy

Sorry for the late reply, but I had to wait for the informations from out IT:

1. Is the DB software installed in the default location? οƒ  Yes
2. Does your database instance files live in the same location as default? οƒ  No, D:\Databases, .ldf-files and mdf-files in same location
3. Processor οƒ  CPU Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz οƒ  Virtual processors: 8
4. RAM οƒ  96 GB
5. Storage HP Fusion
6. Disk Space οƒ  350
a. Teamcenter Data οƒ  126 GB
b. Teamcenter Log οƒ  100 GB
7. No RAID
8. SQL Server 2014 Enterprise AlwaysOn

Greetings Jens

Re: Teamcenter 11.2.3.1 extreme Performance issue

Solution Partner Phenom Solution Partner Phenom
Solution Partner Phenom
Your database server is beefy. Especially if it is only running a single instance of Teamcenter (Siemens recommendation). If there are other DB instances on the same server then this will reduce the performance. Some last things you can do for MSSQL are:

Separate the DB files (MDF, LDF and TEMPDB) on to high-performance, fault tolerant, distinct physical disk volumes (not an appliance). For instance, E:\SQLData\tc.mdf (and tc.ndf if present), F:\SQLData\tc.ldf, G:\SQLData\tempdb.mdf, H:\SQLData\templog.ldf. Use RAID 10 or 5 for MDF/NDF w/10k RPM or faster. Use RAID 1 or 10 for LDF w/10k RPM or faster (do not use RAID 5 for LDF).

Minimize Auto-Growth. Frequent auto-growth hurts performance so validate the current size of the files (data + indexes) and add a 20% buffer. Leave auto-growth enabled. This assumes that you have already sized the DB files by estimating the long-term (1-3 years) growth of data+indexes and added a buffer (med size DB add 35%, large DB add 25%) for the data file and sized the log file to be 20% of the data file.

Set the min/max memory to avoid memory pressure. For 96GB RAM use min 73,728MB (72GB) and max 88,064MB (86GB).

Monitor disk and performance to establish a baseline so you know when something isn't quite right. We focused on the DB server because small changes can make big gains in performance due to the number of calls being made. As we go through the rest of the tiers, the chance for gaining a large performance jump is less and less.

Randy Ellsworth, Teamcenter Architect, Applied CAx, LLC
NX 11 | SW 2016 | Creo 4 | TcUA 11.4
Evaluating: AW 3.4