Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to make two SQL queries really asynchronous

Tags:

My problem is based on a real project problem, but I have never used the System.Threading.Tasks library or performing any serious programming involving threads so my question may be a mix of lacking knowledge about the specific library and more general misunderstanding of what asynchronous really means in terms of programming.

So my real world case is this - I need to fetch data about an user. In my current scenario it's financial data so let say I need all Accounts, all Deposits and all Consignations for a certain user. In my case this means to query million of records for each property and each query is relatively slow itself, however to fetch the Accounts is several times slower than fetching the Deposits. So I have defined three classes for the three bank products I'm going to use and when I want to fetch the data for all the bank products of certain user I do something like this :

List<Account> accounts = GetAccountsForClient(int clientId);
List<Deposit> deposits = GetDepositsForClient(int clientId);
List<Consignation> consignations = GetConsignationsForClient(int clientId);

So the problem starts here I need to get all those three list at the same time, cause I'm going to pass them to the view where I display all users data. But as it is right now the execution is synchronous (If I'm using the term correctly here) so the total time for collecting the data for all three products is:

Total_Time = Time_To_Get_Accounts + Time_To_Get_Deposits + Time_To_Get_Consignations

This is not good because the each query is relatively slow so the total time is pretty big, but also, the accounts query takes much more time than the other two queries so the idea that get into my head today was - "What if I could execute this queries simultaneously". Maybe here comes my biggest misunderstanding on the topic but for me the closest to this idea is to make them asynchronous so maybe then Total_Time won't be the time of the slowest query but yet will be much faster than the sum of all three queries.

Since my code is complicated I created a simple use case which I think, reflect what I'm trying to do pretty well. I have two methods :

public static async Task<int> GetAccounts()
{
    int total1 = 0;
    using (SqlConnection connection = new SqlConnection(connString))
    {
        string query1 = "SELECT COUNT(*) FROM [MyDb].[dbo].[Accounts]";
        SqlCommand command = new SqlCommand(query1, connection);
        connection.Open();
        for (int i = 0; i < 19000000; i++)
        {
            string s = i.ToString();
        }
        total1 = (int) await command.ExecuteScalarAsync();
        Console.WriteLine(total1.ToString());
    }
    return total1;
}

and the second method :

public static async Task<int> GetDeposits()
{
    int total2 = 0;
    using (SqlConnection connection = new SqlConnection(connString))
    {
        string query2 = "SELECT COUNT(*) FROM [MyDb].[dbo].[Deposits]";
        SqlCommand command = new SqlCommand(query2, connection);
        connection.Open();
        total2 = (int) await command.ExecuteScalarAsync();
        Console.WriteLine(total2.ToString());
    }
    return total2;
}

which I call like this:

static void Main(string[] args)
{
    Console.WriteLine(GetAccounts().Result.ToString());

    Console.WriteLine(GetDeposits().Result.ToString());
}

As you can see I call GetAccounts() first and I slow the execution down on purpose so I give a chance the execution to continue to the next method. However I'm not getting any result for a certain period of time and then I get all printed on the console at the same time.

So the problem - how to make so that I don't wait for the first method to finish, in order to go to the next method. In general the code structure is not that important, what I really want to figure out is if there's any way to make both queries to execute at the same time. The sample here is the result of my research which maybe could be extended to the point where I'll get the desired result.

P.S I'm using ExecuteScalarAsync(); just because I started with a method which was using it. In reality I'm gonna use Scalar and Reader.

like image 773
Leron_says_get_back_Monica Avatar asked Oct 10 '14 16:10

Leron_says_get_back_Monica


People also ask

Are SQL queries asynchronous?

SQL Server allows applications to perform asynchronous database operations. Asynchronous processing enables methods to return immediately without blocking on the calling thread.

Can I run 2 SQL queries at once?

You can include multiple SQL statements on the SQL query panel. The exceptions are CALL and CREATE PROCEDURE statements. These statements must be used alone in a query.

How do I combine multiple SQL queries in one?

To combine two or more SELECT statements to form a single result table, use the set operators: UNION, EXCEPT or INTERSECT.

Are SQL queries synchronous?

The sync-sql module is designed to make synchronous queries to the database. Since normal SQL queries are asynchronous in node. js but if you want to run it synchronously, then it is possible using this module. In synchronized SQL, the result set is returned when the query is executed.


2 Answers

When you use the Result property on a task that hasn't completed yet the calling thread will block until the operation completes. That means in your case that the GetAccounts operation need to complete before the call to GetDeposits starts.

If you want to make sure these method are parallel (including the synchronous CPU-intensive parts) you need to offload that work to another thread. The simplest way to do so would be to use Task.Run:

static async Task Main()
{
    var accountTask = Task.Run(async () => Console.WriteLine(await GetAccounts()));
    var depositsTask = Task.Run(async () => Console.WriteLine(await GetDeposits()));

    await Task.WhenAll(accountTask, depositsTask);
}
like image 199
i3arnon Avatar answered Oct 13 '22 08:10

i3arnon


Here's a way to to perform two tasks asynchronously and in parallel:

Task<int> accountTask = GetAccounts();
Task<int> depositsTask = GetDeposits();

int[] results = await Task.WhenAll(accountTask, depositsTask);

int accounts = results[0];
int deposits = results[1];
like image 42
abatishchev Avatar answered Oct 13 '22 08:10

abatishchev