I'm trying to determine worst case disk speed, so I wrote the following function.
static public decimal MBytesPerSec(string volume)
{
string filename = volume + "\\writetest.tmp";
if (System.IO.File.Exists(filename))
System.IO.File.Delete(filename);
System.IO.StreamWriter file = new System.IO.StreamWriter(filename);
char[] data = new char[64000];
Stopwatch watch = new Stopwatch();
watch.Start();
int i = 0;
for (; i < 1000; i++)
{
file.Write(data);
if (watch.ElapsedMilliseconds > 2000)
{
break;
}
}
watch.Stop();
file.Close();
System.IO.File.Delete(volume + "\\test.txt");
decimal mbytessec = (i * 64 / watch.ElapsedMilliseconds);
return mbytessec;
}
The function works OK, but the writes are getting cached, so the speed is not worst case.
In WIN32 C++, I would simply create the file with FILE_FLAG_NO_BUFFERING
, FILE_FLAG_WRITE_THROUGH
options, and then make sure to follow the non-cached writing rules (write to the file at sector size offsets, with minimum of 4k writes)
I found one article that discusses the .NET technique.
So I wrote a new function (ignore the math errors).
static public decimal MBytesPerSecNonCached(string volume)
{
const FileOptions FILE_FLAG_NO_BUFFERING = (FileOptions)0x20000000;
string filename = volume + "\\writetest.tmp";
using (FileStream fs = new FileStream(filename, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None, 1024, FileOptions.WriteThrough | FILE_FLAG_NO_BUFFERING))
{
byte[] data = new byte[65535];
int i = 0;
Stopwatch watch = new Stopwatch();
watch.Start();
for (; i < 1000; i++)
{
fs.Write(data, 0, 65535);
if (watch.ElapsedMilliseconds > 2000)
{
break;
}
}
watch.Stop();
fs.Close();
System.IO.File.Delete(filename);
decimal mbytessec = (i * 64 / watch.ElapsedMilliseconds);
return mbytessec;
}
}
This function works for 4k, 16K and 32K write sizes, but once I try 64K write sizes, I get an exception:
IO operation will not work. Most likely the file will become too long or the handle was not opened to support synchronous IO operations.
So, how can I fix this so I can test with larger than 32KB write sizes (64KB to 4096KB)?
Bypassing your cache is simple, safe, and causes no permanent changes. In most Windows and Linux browsers: Hold down Ctrl and press F5 .
cBioPortal provides the option of caching information on the backend to improve performance. Without caching, every time a request is received by the backend, a query is sent to the database system for information, and the returned data is processed to construct a response.
Benefits of CachingBecause memory is orders of magnitude faster than disk (magnetic or SSD), reading data from in-memory cache is extremely fast (sub-millisecond). This significantly faster data access improves the overall performance of the application.
Cached data are files, scripts, images, and other multimedia stored on your device after opening an app or visiting a website for the first time. This data is then used to quickly gather information about the app or website every time revisited, reducing load time.
Try some unmanaged code:
[DllImport("kernel32", SetLastError = true)]
static extern unsafe SafeFileHandle CreateFile(
string FileName, // file name
uint DesiredAccess, // access mode
uint ShareMode, // share mode
IntPtr SecurityAttributes, // Security Attr
uint CreationDisposition, // how to create
uint FlagsAndAttributes, // file attributes
IntPtr hTemplate // template file
);
const uint FILE_FLAG_NO_BUFFERING = 0x20000000;
SafeFileHandle handle = CreateFile("filename",
(uint)FileAccess.Write,
(uint)FileShare.None,
IntPtr.Zero,
(uint)FileMode.Open,
FILE_FLAG_NO_BUFFERING,
IntPtr.Zero);
var unBufferedStream = new FileStream(handle,FileAccess.Read,blockSize,false);
now you should have access to an unbuffered stream which you can read and write however you please with no constraints
For the record....you can also disable caching like this:
[DllImport("KERNEL32", SetLastError = true)]
public extern static int DeviceIoControl(IntPtr hDevice, uint IoControlCode,
IntPtr lpInBuffer, uint InBufferSize,
IntPtr lpOutBuffer, uint nOutBufferSize,
ref uint lpBytesReturned,
IntPtr lpOverlapped);
[DllImport("KERNEL32", SetLastError = true)]
public extern static int CloseHandle(
IntPtr hObject);
[StructLayout(LayoutKind.Sequential)]
public struct DISK_CACHE_INFORMATION
{
public byte ParametersSavable;
public byte ReadCacheEnabled;
public byte WriteCacheEnabled;
public int ReadRetentionPriority;//DISK_CACHE_RETENTION_PRIORITY = enum = int
public int WriteRetentionPriority;//DISK_CACHE_RETENTION_PRIORITY = enum = int
public Int16 DisablePrefetchTransferLength;//WORD
public byte PrefetchScalar;
}
public void SetDiskCache(byte val)
{
IntPtr h = CreateFile("\\\\.\\PHYSICALDRIVE0", (uint)FileAccess.Read | (uint)FileAccess.Write, (uint)FileShare.Write, IntPtr.Zero, (uint)FileMode.Open, 0, IntPtr.Zero);
DISK_CACHE_INFORMATION sInfo = new DISK_CACHE_INFORMATION();
IntPtr ptrout = Marshal.AllocHGlobal(Marshal.SizeOf(sInfo));
Marshal.StructureToPtr(sInfo, ptrout, true);
uint dwWritten = 0;
int ret = DeviceIoControl(h,IOCTL_DISK_GET_CACHE_INFORMATION,IntPtr.Zero,0,ptrout,(uint)Marshal.SizeOf(sInfo),ref dwWritten,IntPtr.Zero);
sInfo = (DISK_CACHE_INFORMATION)Marshal.PtrToStructure(ptrout,typeof(DISK_CACHE_INFORMATION));
sInfo.ReadCacheEnabled = val;
// acuma trimite structura modificata
IntPtr ptrin = Marshal.AllocHGlobal(Marshal.SizeOf(sInfo));
Marshal.StructureToPtr(sInfo, ptrin, true);
ret = DeviceIoControl(h, IOCTL_DISK_SET_CACHE_INFORMATION, ptrin, (uint)Marshal.SizeOf(sInfo), IntPtr.Zero, 0, ref dwWritten, IntPtr.Zero);
CloseHandle(h);
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With