Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can scalar functions be applied before filtering when executing a SQL Statement?

I suppose I have always naively assumed that scalar functions in the select part of a SQL query will only get applied to the rows that meet all the criteria of the where clause.

Today I was debugging some code from a vendor and had that assumption challenged. The only reason I can think of for this code failing is that the Substring() function is getting called on data that should have been filtered out by the WHERE clause. But it appears that the substring call is being applied before the filtering happens, the query is failing. Here is an example of what I mean. Let's say we have two tables, each with 2 columns and having 2 rows and 1 row respectively. The first column in each is just an id. NAME is just a string, and NAME_LENGTH tells us how many characters in the name with the same ID. Note that only names with more than one character have a corresponding row in the LONG_NAMES table.

NAMES: ID, NAME
    1, "Peter"
    2, "X"
LONG_NAMES: ID, NAME_LENGTH
    1, 5

If I want a query to print each name with the last 3 letters cut off, I might first try something like this (assuming SQL Server syntax for now):

SELECT substring(NAME,1,len(NAME)-3)
    FROM NAMES;

I would soon find out that this would give me an error, because when it reaches "X" it will try using a negative number for in the substring call, and it will fail. The way my vendor decided to solve this was by filtering out rows where the strings were too short for the len - 3 query to work. He did it by joining to another table:

SELECT substring(NAMES.NAME,1,len(NAMES.NAME)-3) 
    FROM NAMES 
        INNER JOIN LONG_NAMES 
            ON NAMES.ID = LONG_NAMES.ID;

At first glance, this query looks like it might work. The join condition will eliminate any rows that have NAME fields short enough for the substring call to fail.

However, from what I can observe, SQL Server will sometimes try to calculate the the substring expression for everything in the table, and then apply the join to filter out rows. Is this supposed to happen this way? Is there a documented order of operations where I can find out when certain things will happen? Is it specific to a particular Database engine or part of the SQL standard? If I decided to include some predicate on my NAMES table to filter out short names, (like len(NAME) > 3), could SQL Server also choose to apply that after trying to apply the substring? If so then it seems the only safe way to do a substring would be to wrap it in a "case when" construct in the select?

like image 557
Peter Recore Avatar asked Oct 25 '22 17:10

Peter Recore


1 Answers

Martin gave this link that pretty much explains what is going on - the query optimizer has free rein to reorder things however it likes. I am including this as an answer so I can accept something. Martin, if you create an answer with your link in it i will gladly accept that instead of this one.

I do want to leave my question here because I think it is a tricky one to search for, and my particular phrasing of the issue may be easier for someone else to find in the future.

TSQL divide by zero encountered despite no columns containing 0

EDIT: As more responses have come in, I am again confused. It does not seem clear yet when exactly the optimizer is allowed to evaluate things in the select clause. I guess I'll have to go find the SQL standard myself and see if i can make sense of it.

like image 111
Peter Recore Avatar answered Oct 27 '22 11:10

Peter Recore