Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

SQL Server custom counter stored procedure creating dupes

I created a stored procedure to implement rate limiting on my API, this is called about 5-10k times a second and each day I'm noticing dupes in the counter table.

enter image description here

It looks up the API key being passed in and then checks the counter table with the ID and date combination using an "UPSERT" and if it finds a result it does an UPDATE [count]+1 and if not it will INSERT a new row.

There is no primary key in the counter table.

Here is the stored procedure:

USE [omdb]
GO
/****** Object:  StoredProcedure [dbo].[CheckKey]    Script Date: 6/17/2017 10:39:37 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[CheckKey] (
@apikey AS VARCHAR(10)
)
AS
BEGIN

SET NOCOUNT ON;

DECLARE @userID as int
DECLARE @limit as int
DECLARE @curCount as int
DECLARE @curDate as Date = GETDATE()

SELECT @userID = id, @limit = limit FROM [users] WHERE apiKey = @apikey

IF @userID IS NULL
    BEGIN
        --Key not found
        SELECT 'False' as [Response], 'Invalid API key!' as [Reason]
    END
ELSE
    BEGIN
        --Key found
        BEGIN TRANSACTION Upsert
        MERGE [counter] AS t
        USING (SELECT @userID AS ID) AS s
        ON t.[ID] = s.[ID] AND t.[date] = @curDate
        WHEN MATCHED THEN UPDATE SET t.[count] = t.[count]+1
        WHEN NOT MATCHED THEN INSERT ([ID], [date], [count]) VALUES (@userID, @curDate, 1);
        COMMIT TRANSACTION Upsert

        SELECT @curCount = [count] FROM [counter] WHERE ID = @userID AND [date] = @curDate

        IF @limit IS NOT NULL AND @curCount > @limit
            BEGIN
                SELECT 'False' as [Response], 'Request limit reached!' as [Reason]
            END
        ELSE
            BEGIN
                SELECT 'True' as [Response], NULL as [Reason]
            END
    END
END

I also think some locks are happening after introducing this SP.

The dupes aren't breaking anything, but I'm curious if it's something fundamentally wrong with my code or if I should setup a constraint in the table to prevent this. Thanks

Update 6/23/17: I dropped the MERGE statement and tried using @@ROWCOUNT but it also caused dupes

BEGIN TRANSACTION Upsert
UPDATE [counter] SET [count] = [count]+1 WHERE [ID] = @userID AND [date] = @curDate
IF @@ROWCOUNT = 0 AND @@ERROR = 0
INSERT INTO [counter] ([ID], [date], [count]) VALUES (@userID, @curDate, 1)
COMMIT TRANSACTION Upsert
like image 360
bfritz Avatar asked Jun 18 '17 04:06

bfritz


1 Answers

A HOLDLOCK hint on the update statement will avoid the race condition. To prevent deadlocks, I suggest a clustered composite primary key (or unique index) on ID and date.

The example below incorporates these changes and uses the SET <variable> = <column> = <expression> form of the SET clause to avoid the need for the subsequent SELECT of the final counter value and thereby improve performance.

ALTER PROCEDURE [dbo].[CheckKey]
    @apikey AS VARCHAR(10)
AS

SET NOCOUNT ON;
--SET XACT_ABORT ON is a best practice for procs with explcit transactions
SET XACT_ABORT ON; 

DECLARE
      @userID as int
    , @limit as int
    , @curCount as int
    , @curDate as Date = GETDATE();

BEGIN TRY;

    SELECT
          @userID = id
        , @limit = limit 
    FROM [users] 
    WHERE apiKey = @apikey;

    IF @userID IS NULL
    BEGIN
        --Key not found
        SELECT 'False' as [Response], 'Invalid API key!' as [Reason];
    END
    ELSE
    BEGIN
        --Key found
        BEGIN TRANSACTION Upsert;

        UPDATE [counter] WITH(HOLDLOCK) 
        SET @curCount = [count] = [count] + 1 
        WHERE
            [ID] = @userID 
            AND [date] = @curDate;

            IF @@ROWCOUNT = 0
            BEGIN    
                INSERT INTO [counter] ([ID], [date], [count]) 
                    VALUES (@userID, @curDate, 1);
            END;

        IF @limit IS NOT NULL AND @curCount > @limit
        BEGIN
            SELECT 'False' as [Response], 'Request limit reached!' as [Reason]
        END
        ELSE
        BEGIN
            SELECT 'True' as [Response], NULL as [Reason]
        END;

        COMMIT TRANSACTION Upsert;

    END;

END TRY
BEGIN CATCH
    IF @@TRANCOUNT > 0 ROLLBACK;
    THROW;
END CATCH;
GO
like image 76
Dan Guzman Avatar answered Nov 12 '22 08:11

Dan Guzman