Querying a view immediately after writing to underlying tables in SQL Server 2014

Question

I am having an issue where if I write to a table (using Linq-to-SQL) which is a dependency of a view, and then immediately turn around and query that view to check the impact of the write (using a new connection to the DB, and hence a new data context), the impact of the write doesn't show up immediately but takes up to a few seconds to appear. This only happens occasionally (perhaps 10-20 times per 10,000 or so writes).

This is the definition of the view:

CREATE VIEW [Position].[Transactions]
WITH SCHEMABINDING
AS
(
  SELECT
    Account,
    Book,
    TimeAPIClient AS DateTimeUtc,
    BaseCcy AS Currency,
    ISNULL(QuantityBase, 0) AS Quantity,
    ValueDate AS SettleDate,
    ISNULL(CAST(0 AS tinyint), 0) AS TransactionType
  FROM Trades.FxSpotMF
  WHERE IsCancelled = 0

  UNION ALL

  SELECT
    Account,
    Book,
    TimeAPIClient AS DateTimeUtc,
    QuoteCcy AS Currency,
    ISNULL(-QuantityBase * Rate, 0) AS Quantity,
    ValueDate AS SettleDate,
    ISNULL(CAST(0 AS tinyint), 0) AS TransactionType
  FROM Trades.FxSpotMF
  WHERE IsCancelled = 0

  UNION ALL

  SELECT
    Account,
    Book,
    ExecutionTimeUtc AS DateTimeUtc,
    BaseCcy AS Currency,
    ISNULL(QuantityBase, 0) AS Quantity,
    ValueDate AS SettleDate,
    ISNULL(CAST(1 AS tinyint), 1) AS TransactionType
  FROM Trades.FxSpotManual
  WHERE IsCancelled = 0

  UNION ALL

  SELECT
    Account,
    Book,
    ExecutionTimeUtc AS DateTimeUtc,
    QuoteCcy AS Currency,
    ISNULL(-QuantityBase * Rate, 0) AS Quantity,
    ValueDate AS SettleDate,
    ISNULL(CAST(1 AS tinyint), 1) AS TransactionType
  FROM Trades.FxSpotManual
  WHERE IsCancelled = 0

  UNION ALL

  SELECT
    Account,
    Book,
    ExecutionTimeUtc AS DateTimeUtc,
    BaseCcy AS Currency,
    ISNULL(SpotQuantityBase, 0) AS Quantity,
    SpotValueDate AS SettleDate,
    ISNULL(CAST(2 AS tinyint), 2) AS TransactionType
  FROM Trades.FxSwap

  UNION ALL

  SELECT
    Account,
    Book,
    ExecutionTimeUtc AS DateTimeUtc,
    QuoteCcy AS Currency,
    ISNULL(-SpotQuantityBase * SpotRate, 0) AS Quantity,
    SpotValueDate AS SettleDate,
    ISNULL(CAST(2 AS tinyint), 2) AS TransactionType
  FROM Trades.FxSwap

  UNION ALL

  SELECT
    Account,
    Book,
    ExecutionTimeUtc AS DateTimeUtc,
    BaseCcy AS Currency,
    ISNULL(ForwardQuantityBase, 0) AS Quantity,
    ForwardValueDate AS SettleDate,
    ISNULL(CAST(2 AS tinyint), 2) AS TransactionType
  FROM Trades.FxSwap

  UNION ALL

  SELECT
    Account,
    Book,
    ExecutionTimeUtc AS DateTimeUtc,
    QuoteCcy AS Currency,
    ISNULL(-ForwardQuantityBase * ForwardRate, 0) AS Quantity,
    ForwardValueDate AS SettleDate,
    ISNULL(CAST(2 AS tinyint), 2) AS TransactionType
  FROM Trades.FxSwap

  UNION ALL

  SELECT
    Account,
    c.Book,
    TimeUtc AS DateTimeUtc,
    Currency,
    ISNULL(Amount, 0) AS Quantity,
    SettleDate,
    ISNULL(CAST(3 AS tinyint), 3) AS TransactionType
  FROM Trades.Commission c
  JOIN Trades.Payment p
    ON c.UniquePaymentId = p.UniquePaymentId
    AND c.Book = p.Book
)

while this is the query generated by Linq-to-SQL to write to one of the underlying tables:

INSERT INTO [Trades].[FxSpotMF] ([UniqueTradeId], [BaseCcy], [QuoteCcy], [ValueDate], [Rate], [QuantityBase], [Account], [Book], [CounterpartyId], [Counterparty], [ExTradeId], [TimeAPIClient], [TimeAPIServer], [TimeExchange], [TimeHandler], [UniqueOrderId], [IsCancelled], [ClientId], [SequenceId], [ExOrdId], [TradeDate], [OrderCycleId], [CycleIndex])
  VALUES (@p0, @p1, @p2, @p3, @p4, @p5, @p6, @p7, @p8, @p9, @p10, @p11, @p12, @p13, @p14, @p15, @p16, @p17, @p18, @p19, @p20, @p21, @p22)

and this is the query generated by Linq-to-SQL to check the effect of the write:

SELECT
  SUM([t0].[Quantity]) AS [Item2],
  [t0].[Currency] AS [Item1]
FROM [Position].[Transactions] AS [t0]
WHERE ([t0].[Book] = @p0)
AND ([t0].[DateTimeUtc] < @p1)
GROUP BY [t0].[Currency]

Also, this is the Linq-to-SQL code that generates the write (using F# type providers):

type Schema = Microsoft.FSharp.Data.TypeProviders.DbmlFile<"TradeDb.dbml", ContextTypeName="TradeDb"> 

use db = new Schema.TradeDb(connectionString)
let trade = new Schema.Trades_FxSpotMF()
(* omitted: set object properties corresponding to column values here... *)
db.Trades_FxSpotMF.InsertOnSubmit(trade)
db.SubmitChanges()

while this is the corresponding Linq-to-SQL that generates the read:

use db = new Schema.TradeDb(connectionString)
query { for t in db.Position_Transactions do
        where ( t.Book = book &&
                t.DateTimeUtc < df.MaxExecutionTimeExcl
              )
        groupBy t.Currency into group
        let total = query { for x in group do sumBy x.Quantity }
        select (group.Key, total)
      }
|> Map.ofSeq

I would have thought System.Data.Linq.DataContext.SubmitChanges() would only return once the write transaction was complete, and that any subsequent query of the view must contain the effect of the write... what am I missing/doing wrong?

Answer 1

I finally got to the bottom of this: the DB writes are done in their own threads, with the main thread waiting for all the write threads to complete before checking the results. However, there was a bug in the code which checked whether all the threads were complete, causing the main thread to do the check too early.