c# slow socket speed

Question

I'm trying to write a client/server file transfer system. Currently it works, and I've profiled it, and I can't seem to send data any faster than maybe 2-4 megabytes per second. I've tuned my code so that I can read data from the disk in the hundreds of megabytes per second, and the performance wizard shows nothing above a 1-3 between my disk read and my socket write, so my code is setup(it would seem) to push out data as fast as the nic/cpu/motherboard whatever could handle it.

I suppose the question is, why isn't that the case?

Here is some code, so that you can get an idea of what I've got set up here.

Socket code(as streamlined as I can make it)

namespace Skylabs.Net.Sockets
{
    public abstract class SwiftSocket
    {
        public TcpClient Sock { get; set; }

        public NetworkStream Stream { get; set; }

        public const int BufferSize = 1024;

        public byte[] Buffer = new byte[BufferSize];

        public bool Connected { get; private set; }

        private Thread _thread;

        private bool _kill = false;

        protected SwiftSocket()
        {
            Connected = false;
            Sock = null;
            _thread = new Thread(Run);
        }
        protected SwiftSocket(TcpClient client)
        {
            _Connect(client);
        }
        public bool Connect(string host, int port)
        {
            if (!Connected)
            {
                TcpClient c = new TcpClient();
                try
                {
                    c.Connect(host, port);
                    _Connect(c);
                    return true;
                }
                catch (SocketException e)
                {
                    return false;
                }
            }
            return false;
        }
        public void Close()
        {
            _kill = true;
        }
        private void _Connect(TcpClient c)
        {
            Connected = true;
            Sock = c;
            Stream = Sock.GetStream();
            _thread = new Thread(Run);
            _thread.Name = "SwiftSocketReader: " + c.Client.RemoteEndPoint.ToString();
            _thread.Start();
        }
        private void Run()
        {
            int Header = -1;
            int PCount = -1;
            List<byte[]> Parts = null;
            byte[] sizeBuff = new byte[8];
            while (!_kill)
            {
                try
                {
                    Header = Stream.ReadByte();
                    PCount = Stream.ReadByte();
                    if (PCount > 0)
                        Parts = new List<byte[]>(PCount);
                    for (int i = 0; i < PCount; i++)
                    {
                        int count = Stream.Read(sizeBuff, 0, 8);
                        while (count < 8)
                        {
                            sizeBuff[count - 1] = (byte)Stream.ReadByte();
                            count++;
                        }
                        long pieceSize = BitConverter.ToInt64(sizeBuff, 0);
                        byte[] part = new byte[pieceSize];
                        count = Stream.Read(part, 0, (int)pieceSize);
                        while (count < pieceSize)
                        {
                            part[count - 1] = (byte)Stream.ReadByte();
                        }
                        Parts.Add(part);
                    }
                    HandleMessage(Header, Parts);
                    Thread.Sleep(10);
                }
                catch (IOException)
                {
                    Connected = false;
                    if(System.Diagnostics.Debugger.IsAttached)System.Diagnostics.Debugger.Break();
                    break;                    
                }
                catch (SocketException)
                {
                    Connected = false;
                    if (System.Diagnostics.Debugger.IsAttached) System.Diagnostics.Debugger.Break();
                    break;
                }
            }
            HandleDisconnect();
        }
        public void WriteMessage(int header, List<byte[]> parts)
        {
            try
            {
                byte[] sizeBuffer = new byte[8];
                //Write header byte
                Stream.WriteByte((byte)header);
                if (parts == null)
                    Stream.WriteByte((byte)0);
                else
                {
                    Stream.WriteByte((byte)parts.Count);

                    foreach (byte[] p in parts)
                    {
                        sizeBuffer = BitConverter.GetBytes(p.LongLength);
                        //Write the length of the part being sent
                        Stream.Write(sizeBuffer, 0, 8);
                        Stream.Write(p, 0, p.Length);
                        //Sock.Client.Send(p, 0, p.Length, SocketFlags.None);
                    }
                }
            }
            catch (IOException)
            {
                if (System.Diagnostics.Debugger.IsAttached) System.Diagnostics.Debugger.Break();
                _kill = true;
            }
            catch (SocketException)
            {
                if (System.Diagnostics.Debugger.IsAttached) System.Diagnostics.Debugger.Break();
                _kill = true;
            }
        }
        protected void WriteMessage(int header)
        {
            WriteMessage(header,null);
        }
        public abstract void HandleMessage(int header, List<byte[]> parts);
        public abstract void HandleDisconnect();

    }
}

File Transferer code(the class that sets up the socket, loads the file etc)

namespace Skylabs.Breeze
{
    public class FileTransferer
    {
        public String Host { get; set; }
        public string FileName { get; set; }
        public string FilePath { get; set; }
        public string Hash { get; set; }
        public FileStream File { get; set; }
        public List<TransferClient> Clients { get; set; }
        public const int BufferSize = 1024;
        public int TotalPacketsSent = 0;
        public long FileSize{get; private set; }
        public long TotalBytesSent{get; set; }
        private int clientNum = 0;
        public int Progress
        {
            get
            {
                return (int)(((double)TotalBytesSent / (double)FileSize) * 100d);
            }
        }
        public event EventHandler OnComplete;
        public FileTransferer()
        {

        }
        public FileTransferer(string fileName, string host)
        {
            FilePath = fileName;
            FileInfo f = new FileInfo(fileName);
            FileName = f.Name;
            Host = host;
            TotalBytesSent = 0;
            try
            {
                File = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.Read, BufferSize, FileOptions.SequentialScan);
                File.Lock(0,File.Length);
            }
            catch (Exception e)
            {
                ErrorWindow er = new ErrorWindow(e);
                er.ShowDialog();
            }

        }
        public bool Grab_Next_Data_Chunk(ref byte[] buffer, out int size, out long pos)
        {
            lock (File)
            {
                pos = File.Position;
                size = 0;
                if (pos >= FileSize - 1)
                    return false;
                int count = File.Read(buffer, 0, (FileSize - pos) >= FileTransferer.BufferSize ? FileTransferer.BufferSize : (int)(FileSize - pos));
                //TotalBytesSent += count;
                size = count;
                TotalPacketsSent++;
                return true;
            }
        }
        public bool Start(int ConnectionCount)
        {
            Program.ServerTrace.TraceInformation("Creating Connections.");
            if (Create_Connections(ConnectionCount) == false)
            {
                return false;
            }
            File.Seek(0, SeekOrigin.Begin);
            FileSize = File.Length;
            Clients[0].Start(this,0);

            List<byte[]> parts = new List<byte[]>(1);
            parts.Add(BitConverter.GetBytes(FileSize));
            Clients[0].WriteMessage((int)Program.Message.CFileStart, parts);

            Program.ServerTrace.TraceInformation("Sent start packet");

            for (clientNum = 1; clientNum < ConnectionCount; clientNum++)
            {
                Clients[clientNum].Start(this, clientNum);
            }
            return true;

        }
        private bool Create_Connections(int count)
        {
            Clients = new List<TransferClient>();
            for (int i = 0; i < count; i++)
            {
                TransferClient tc = new TransferClient();
                if (tc.Connect(Host, 7678) == false)
                    return false;
                Clients.Add(tc);
            }
            return true;
        }
        public void AddClient()
        {
            TransferClient tc = new TransferClient();
            tc.Connect(Host, 7678);
            tc.Start(this, clientNum);
            clientNum++;
            Clients.Add(tc);
        }
        public void RemoveClient()
        {
            Clients.Last().Kill();
        }
        public void AdjustClientCount(int newCount)
        {
            int dif = newCount - Clients.Count;
            if (dif > 0)
            {
                for(int i=0;i<dif;i++)
                    AddClient();
            }
            else
            {
                for(int i=0;i<Math.Abs(dif);i++)
                    RemoveClient();
            }
        }
        public void ClientDone(TransferClient tc)
        {
            List<byte[]> parts = new List<byte[]>(1);
            parts.Add(ASCIIEncoding.ASCII.GetBytes(FileName));
            tc.WriteMessage((int)Program.Message.CPartDone,parts);

            tc.Close();
            Clients.Remove(tc);
            if (Clients.Count == 0)
            {
                Program.ServerTrace.TraceInformation("File '{0}' Transfered.\nTotal Packets Sent: {1}", FilePath,
                                                     TotalPacketsSent);
                File.Unlock(0,File.Length);
                File.Close();
                File.Dispose();
                if(OnComplete != null)
                    OnComplete.Invoke(this,null);
            }

        }

    }
    public class TransferClient : Skylabs.Net.Sockets.SwiftSocket,IEquatable<TransferClient>
    {
        public FileTransferer Parent;
        public int ID;
        private bool KeepRunning = true;
        public Thread Runner;
        public void Start(FileTransferer parent, int id)
        {
            this.Sock.Client.
            Parent = parent;
            ID = id;
            List<byte[]> p = new List<byte[]>(1);
            p.Add(Encoding.ASCII.GetBytes(Parent.FileName));
            WriteMessage((int)Program.Message.CHello, p);
        }
        public void Kill()
        {
            KeepRunning = false;
        }
        private void run()
        {
            while (KeepRunning)
            {
                List<Byte[]> p = new List<byte[]>(3);
                byte[] data = new byte[FileTransferer.BufferSize];
                int size = 0;
                long pos = 0;
                if (Parent.Grab_Next_Data_Chunk(ref data,out size,out pos))
                {
                    p.Add(data);
                    p.Add(BitConverter.GetBytes(size));
                    p.Add(BitConverter.GetBytes(pos));
                    WriteMessage((int)Program.Message.CData, p);
                    Parent.TotalBytesSent += size;
                }
                else
                {
                    break;
                }
                Thread.Sleep(10);
            }
            Parent.ClientDone(this);
        }
        public bool Equals(TransferClient other)
        {
            return this.ID == other.ID;
        }

        public override void HandleMessage(int header, List<byte[]> parts)
        {
            switch (header)
            {
                case (int)Program.Message.SStart:
                    {
                        Runner = new Thread(run);
                        Runner.Start();
                        break;
                    }
            }
        }

        public override void HandleDisconnect()
        {
            //throw new NotImplementedException();
        }
    }
}

I would like to stress that in FileTransferer.Get_Next_Data_Chunk there is almost no delay, it reads really fast, in the 100's of megabytes per second. Also, the WriteMessage for the socket is streamlined and fast as I believe humanly possible.

Maybe there is a setting or something? or a different protocol?

Any idea's would be more than welcome.

I forgot to mention, this program is being built SPECIFICALLY for LAN environments, which have a max speed of 1000Mbps(or bytes, I'm not sure, that would also be nice if someone would clarify that as well.)

Answer 1

Firstly, your network speed will be bits per second, it's almost never bytes per second.

Second you're likely limited by having to constantly open and close the file with the IO readers. Unless you are running on a SSD this is going to cause significant increased overhead due to drive seek times.

To solve this, try to increase your buffer size to something larger as 1024 is quite small. I usually use around 262144 (256K) for my buffer sizes.

In addition to that, you will want to pipeline the file IO like this:

ReadBlock1
loop while block length > 0
    TransmitBlock1 in separate thread
    ReadBlock2
    Join transmit thread
end loop

Using the above pipeline you can usually get up to double your transmission speed.

When you implement pipelined file IO, you no longer need to worry about the problem of making your buffer size too large, unless your files are always going to be size of < 2 * BufferSize, since you say that you're dealing with in excess of 100mb files you need not worry about this case.

Other things you could do to improve this would be to use .

Also remember that in .NET file IO is very often synchronous despite use of threading.

For further reading see: http://msdn.microsoft.com/en-us/library/kztecsys.aspx

Edit: just to add if you think the problem is the network and not the file IO, then just comment out the network part to make it appear instant, what speed do you then get? What about the reverse, what if you make your file reading just always return an empty new byte[BufferSize] how does that affect your copy speed?