Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Make SQL Server faster at manipulating data - turn off transaction logging?

I use SQL Server 2005 as a data store for a lot of data I do analytic work on. This is not a transactional database as I am not hitting it with updates or capturing real time data. I get a few gigs of data from my clients, load them into SQL Server and do a series of manipulations. I then grab bits of this data and pull them into R where I do most of my analysis. I then push a little data into tables in SQL Server and maybe do a join or two.

I have a heck of a time with the logs in SQL Server getting big and I assume it takes some degree of overhead to create these. How can I configure SQL Server so that it runs with little or no logging? If things get corrupt I am happy to just start from the beginning. Any ideas how to make this all faster?

BTW, no need to tell me how to shrink logs, I'm already doing that. But I wish I didn't have to make the logs in the first place. I'm only using the DB to house the data because its too big to fit into memory in R.

Should I be using a more simple DB than Sql Server? Feel free to tell me I am killing an ant with a sledgehammer. But please recommend a more appropriate sized hammer. :)

like image 256
JD Long Avatar asked Feb 21 '09 00:02

JD Long


People also ask

How do I stop SQL Server transaction log full?

If the log has never been backed up, you must create two log backups to permit the Database Engine to truncate the log to the point of the last backup. Truncating the log frees logical space for new log records. To keep the log from filling up again, take log backups regularly and more frequently.

How do I reduce transaction log?

To reduce the physical size of a physical log file, you must shrink the log file. This is useful when you know that a transaction log file contains unused space. You can shrink a log file only while the database is online, and at least one virtual log file (VLF) is free.

Should I shrink my transaction log?

A shrink operation will be useful after performing an operation that creates a large number of Transaction Logs. Shrinking the Transaction Log file to a size smaller than the size of the Virtual Log File is not possible, even if this space is not used.

What causes SQL transaction log growth?

Large database transactions, such as importing large amounts of data, can lead to a large transaction log file. Transaction log backups not happening fast enough causes the SQL log file to become huge. SQL log files also enlarge due to incomplete replication or availability group synchronization.


1 Answers

How can I configure SQL Server so that it runs with little or no logging? I

I don't believe you can.

However if you configure the database (each database on a server can be different) for simple backups the log file won't grow until you back it up. This is done by setting the recovery mode to "simple".

With simple backups the log is only used to hold the state of transactions until they are fully written into the main database.

like image 84
Richard Avatar answered Nov 02 '22 19:11

Richard