"Throttle" a function

Question

I'm developing an MMORPG game server, while this approach is not needed for 80% of the functions within the game server, the 20% of the functions that would use it account for 99% of resource usages.

I'm trying to accomplish a throttled function. Say you call a massive function, or you're searching a massive data structure for a specific criteria. That one call being fired could result in massive CPU usage.

For example, a massive function, when called, utilizes a steady 30% of the CPU over 4 seconds to complete, say we're searching millions of entries in an auction system by criteria. Imagine we have 4 people doing this at once. I want to make that same method to take only 1% CPU or less, taking perhaps more than 15-20 seconds to return the same result without starving hardware.

As of now I'm using this class.

public class AsyncLazy<T> : Lazy<Task<T>> 
{ 
    public AsyncLazy(Func<T> valueFactory) : 
        base(() => Task.Factory.StartNew(valueFactory)) { }
    public AsyncLazy(Func<Task<T>> taskFactory) : 
        base(() => Task.Factory.StartNew(() => taskFactory()).Unwrap()) { } 
}

This doesn't block in initialization of the value, and doesn't block when consuming the lazy value. However, during massive operations the executing the operation starves the hardware to complete the task, resulting in 30% CPU usage over 4 seconds. My goal is to throttle calls over a virtually unlimited amount of time so they can complete their task without starving hardware.

Edit: Thanks to Scott Chamberlin for correcting me with proper "lingo". I'm not looking to call a method lazily, but I'm looking to "Throttle" specific method calls.

Answer 1

How can you throttle a function using the your programming language?

As mentioned by willaien, Thread Priority is one of the only CPU throttling mechanism you have inside the process, and even that is indirect. This is because it is not an app's responsibility to directly control CPU utilization; it is the OS and processor's responsibility to manage system resources such as threading. This allows the OS to prevent a single app from crashing everything.

How can you throttle your function outside of your programming language?

You can break up the intense calculation into a completely separate service. Then place this service on another machine that you can directly control the resources of from a system or OS level (for example adding more memory or CPU power by hardware or using VMs). With a VM, you can "throttle" the function by setting the VMs CPUs. You can "throttle" a pure hardware solution or VMs using middleware like a load balancer or using a queue to limit the number of in-process requests. This would satisfy your requirement of each calculation taking as long as it needs. And that other service could simply be a distributed cache, (as Scott Chamberlain mentioned) a database stored proc, or query where the database is on a different machine.

If you take this route, you may want to investigate .NET's async pattern. This would also ensure that your original service is not thread bound by threads that are blocking when using your distributed service.