Recently we have had an old problem with the server free space running low.
After purging the Log files, (as done before) we are now getting a lot of "File not found" errors. Note, there is some debate in the office wether this problem was existant before the purge or not, either way it is very recent.
If I search for the problem files individually, I can find them. I can even navigate to them in the Sharepoint site. But I cannot open them.
I checked the crawl log an it seems there are exactly 5,550 errors, (Which seems like a suspciiously neat number!). Which are "file not found" errors.
We have ~16,000 files in the database.
Does anyone have any experience with this, or any ideas?
We are running ST7, MP8.
Sharepoint foundataion 2010
SQL Server 2008 R2
Solved! Go to Solution.
You have to be careful deleting transaction logs as they not only log all past transactions but can also be used to log transactions that have not yet been commited to the DB as a performance cache. I don't know what may have happened when you ran out of space, but it is possible you had transactions that had not yet been commited and you deleted them by deleting the logs. Having 5550 errors seems a bit excessive for it to be about the logs though...
Also, a proper regularly (daily) ran maintenance plan can keep the log file sizes down.
Ahhh...... Stephen and Ken, I can always count on you!
I am glad you said that. This was my suspicion. However i am in an unfortuante position where I am not familiar at all with the workings of Sharpoint, so i wasn't sure wether or not i shoud re-index the database. (Also i don't have an IT Guy at the moment).
At the risk of asking dumb questions;
Wwhat does re-indexing do, and is there any risk of data loss? (Should I wait until i get the IT Guy to run a fresh backup first?)
Do all the clients need to be off line, (With respect ot the Insight Sharepoint)?
We didn't just delete logs, (We have been ther before!). I will have to check what the Purge procedure actually is. I thought we were using dbshrink, but again I am not certain.
A bit more digging and i discovered that all the files that we are unable to open are from a specific date. Overnight I had the log files restored, (Thanks to my IT Guy working in his holiday!) and everything seems to be OK now. With the exception of the availiable space on the server again.
A proper maintenance procedure is something I cannot get my head around. I am still grappling with the concept that ~13gb of cad data is filling a 250gb drive!. Even after the purge we only get back to ~200gb.
What is a normal ratio of data to log size we should expect?
Can I ask is there some guidelines/tutorials/advice anywhere that would help us implement a procedure to keep this under control?
Thanks again guys for you input.
Log files keep a copy of every transaction including any new or modified binary files (the whole file, not just it's name), so if there are many saves of files the transaction logs can get big ina hurry. What kind of recovery mode you are running on your Content DB (Simple or Full) will dictate how the log files truncate. In Simple recovery mode, they are supposed to truncate after a checkpoint (when the data has been written to the DB from memory). In Full recovery mode, the log files truncate after they are backed up via the SQL DB Backup command up to the point where a checkpoint had occured. When logs are truncated, they do not actually shrink on disk and if they are kept up with timely backups, it's not a problem and shrinking them actually can degrade performance and is futile effort as they will only grow back.
So if your Content DB is running in Simple recovery mode, they shouldn't be a problem. Given that yours are a problem, your DB is most likely set to Full recovery and would require a backup to truncate them. Either way, you should be backing up your DB every night.
Firstly - My Apologies Stefan for calling you Stephen! (Brain fade!)
Thanks Ken, I'll have a sit down with the IT Guy to get a handle on exactly what is backup and recovery mode we are employing.
Also; The Central Administration "Problems and Solutions" is suggesting there is a lot of unused space and recomemnds running the SQL Management Studio database shrink application. Does this work?
By default, the same place that the database file (.MDF) resides. They have a .LDF extension. But it is possible and I believe it to be a best practice for performance to put the logs on another disk than where the DB file is stored, but that is a manual setup thing a DBA would have had to do.
Unfortunately MS sells SharePoint as this self contained product and makes it seem like it just takes care of the SQL maintenance, but that is not really the case, espcially when you are running a heavy transaction based app like Insight on top of it.
Difference between Full and Simple recovery is this:
We have always ran our content DB in Simple recovery mode, but there is a risk of substantial data loss if anything fails and corrupts the DB.
Your maintenance plans can automate the backup process, shrink DBs/logs and/or their files, rebuild indexes, update statistics, and many other maintenance tasks. SQL is a complex beast so if your Insight server is a production server, it would be worth your time and money to hire a SQL DBA consultant who is well versed on handling SharePoint DBs to take a look at it and set it up properly. They know the tricks and exactly what is needed. We are lucky enough to have 2 or 3 SQL DBA's on staff but still didn't have the expertise to solve a transaction problem when our DB's got big and had to resort to some outside help.
One word of advice, if doing maintenance for the first time (or after an extended period) and the disk space is low, try to do it after hours and if possible through a weekend -- unless in an emergency situation.
I had a customer one time that had never performed any SQL maintenance and was low on disk space. We implemented SQL maintenance plans to get the disk space back -- it took over 24 hours for the first run to complete and customer was down for a day waiting on the maintenance to complete. Subsequent maintenance was quick though. Not trying to scare you and YMMV but just wanted to forewarn.
As for the size of the database for only having 13 Gb of CAD files, do you have SharePoint versioning turned on? This will greatly increase the number of files created and stored in the database.