Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

in SQL, why is this JOIN returning the key column twice?

I'm sorry if this is a stupid question, but I can't seem to get my head around it. I'm fairly new to SQL and this behavior would be strange in R or Pandas or other things that I'm used to using.

Basically, I have two tables in two different databases, with a common key user_id. I want to join all the columns with

SELECT * FROM db1.first_table t1 
JOIN db2.second_table t2 
ON t1.user_id = t2.user_id

Great, it works. Except there are two (identical) columns called user_id. This wouldn't really matter, except that I am doing this in pyspark and when I try to export the joined table to a flat file I get an error that two of the columns have the same name. There are work-arounds for this, but I'm just wondering if someone can explain why the join returns both user_id columns. It seems like it is an inner join so by definition the columns are identical. Why would it return both?

As a side question, is there an easy way to avoid this behavior?

Thanks in advance!

like image 341
seth127 Avatar asked Jul 25 '17 19:07

seth127


People also ask

How do you avoid duplicates in join?

How do I prevent duplicate rows from joining multiple tables? Solution. Select column values in a specific order within rows to make rows with duplicate sets of values identical. Then you can use SELECT DISTINCT to remove duplicates.

Why inner join returns duplicate rows?

Using an Incomplete ON Condition Unwanted rows in the result set may come from incomplete ON conditions. In some cases, you need to join tables by multiple columns. In these situations, if you use only one pair of columns, it results in duplicate rows.

How do I ignore duplicate columns in SQL?

There is no option in MySQL to ignore duplicate columns in a CREATE TABLE statement. You should avoid using SELECT * . In this case, it gives duplicate column names because you join tables that contain columns of the same name. You will have to write out the list of columns you want to keep, instead of using SELECT * .

Does inner join cause duplicates?

SQL inner join creates duplicates.


1 Answers

SELECT * returns all columns from all tables of the query. That includes both user_id columns - one from table A, one from table B.

The best practice is to list the column names you want returned specifically, though another option to shorten the list would be:

SELECT TableA.*, 
       TableB.col1, 
       TableB.col2, 
       ...rest of B columns except user_id
like image 129
Aaron Dietz Avatar answered Oct 22 '22 00:10

Aaron Dietz