Query optimization with in (select ...) clause

Question

I'm using Firebird WI-V3.0.4.33054 on Windows.

I'm having trouble optimizing this query, that uses an in clause with a select:

update CADPC p set p.STA = 'L'
where p.COD in (select distinct CODPC from CADPCI_Rec where IDNfr = 27)
and not exists (select * from CADPCI where CODPC = p.COD)

The plan for this query is (and the obvious problem is the P NATURAL part):

PLAN SORT (CADPCI_REC INDEX (PK_CADPCI_REC))
PLAN (CADPCI INDEX (FK_CADPCI_CODPC))
PLAN (P NATURAL)

Select Expression
    -> Filter
        -> Unique Sort (record length: 36, key length: 8)
            -> Filter
                -> Table "CADPCI_REC" Access By ID
                    -> Bitmap
                        -> Index "PK_CADPCI_REC" Range Scan (partial match: 1/3)
Select Expression
    -> Filter
        -> Table "CADPCI" Access By ID
            -> Bitmap
                -> Index "FK_CADPCI_CODPC" Range Scan (full match)
Select Expression
    -> Filter
        -> Table "CADPC" as "P" Full Scan

If, on the other hand, I manually run the select distinct, copy the results and paste in the query, like this:

update CADPC p set p.STA = 'L'
where p.COD in (5699, 5877, 5985)
and not exists (select * from CADPCI where CODPC = p.COD)

Now the optimizer chooses a sensible plan for the P table and the query runs very quickly:

PLAN (CADPCI INDEX (FK_CADPCI_CODPC))
PLAN (P INDEX (PK_CADPC, PK_CADPC, PK_CADPC))

Select Expression
    -> Filter
        -> Table "CADPCI" Access By ID
            -> Bitmap
                -> Index "FK_CADPCI_CODPC" Range Scan (full match)
Select Expression
    -> Filter
        -> Table "CADPC" as "P" Access By ID
            -> Bitmap Or
                -> Bitmap Or
                    -> Bitmap
                        -> Index "PK_CADPC" Unique Scan
                    -> Bitmap
                        -> Index "PK_CADPC" Unique Scan
                -> Bitmap
                    -> Index "PK_CADPC" Unique Scan

I have also tried exists in both conditions, but the result is the same: the subquery is re-evaluated for each row.

update CADPC p set p.STA = 'L'
where exists (select * from CADPCI_Rec where IDNfr = 27 and CODPC = p.COD)
and not exists (select * from CADPCI where CODPC = p.COD)

Plan:

PLAN (CADPCI_REC INDEX (PK_CADPCI_REC))
PLAN (CADPCI INDEX (FK_CADPCI_CODPC))
PLAN (P NATURAL)

Select Expression
    -> Filter
        -> Table "CADPCI_REC" Access By ID
            -> Bitmap
                -> Index "PK_CADPCI_REC" Range Scan (partial match: 1/3)
Select Expression
    -> Filter
        -> Table "CADPCI" Access By ID
            -> Bitmap
                -> Index "FK_CADPCI_CODPC" Range Scan (full match)
Select Expression
    -> Filter
        -> Table "CADPC" as "P" Full Scan

So, the question is: can I somehow make the engine choose the indexed plan when the in clause contains a select (typically only a couple of records)?

Answer 1

You may try EXECUTE BLOCK and FOR SELECT to "invert control".

Essentially, executing an anonymous ad-hoc procedure

EXECUTE BLOCK AS
  declare id INTEGER;
BEGIN
  for select distinct t1.CODPC from CADPCI_Rec t1 
    left join CADPCI t2 on where t2.CODPC = t1.CODPC
  where t2.CODPC is NULL and t1.IDNfr = 27
  into :id
  do 
    update CADPC p set p.STA = 'L' where p.COD = :ID and p.STA <> 'L';
END

---

You might also use Global Temporary Tables (GTTs) and then create list of IDs before actually deleting.

Database preparation (creating a body-less table):

CREATE GLOBAL TEMPORARY TABLE CADPC_mark_IDs
   ( ID integer )
ON COMMIT DELETE ROWS

And then the commands would be like

insert into CADPC_mark_IDs(ID)
select distinct t1.CODPC from CADPCI_Rec t1 
   left join CADPCI t2 on where t2.CODPC = t1.CODPC
where t2.CODPC is NULL and t1.IDNfr = 27

then

update CADPC p set p.STA = 'L'
where p.COD in (select * from CADPC_mark_IDs) and p.STA <> 'L'

then

commit; -- clear the in-memory table for next uses

---

one more option, like Mark suggested, would be using MERGE, after you converted "where not exist" into "left join" (already done above, hopefully correct).

Something along that

merge into CADPC p
  using (
    select distinct t1.CODPC as id from CADPCI_Rec t1 
      left join CADPCI t2 on where t2.CODPC = t1.CODPC
    where t2.CODPC is NULL and t1.IDNfr = 27
  ) t
on (t.id = p.COD) and (p.STA <> 'L')
when matched then update set p.STA = 'L'

Answer 2

Try using exists for both:

update CADPC p
    set p.STA = 'L'
    where exists (select 1 from CADPCI_Rec where r.IDNfr = 27 and p.COD = r.CODPC) and
         not exists (select 1 from CADPCI c2 where c2.CODPC = p.COD);

In particular, you want an index on CADPCI_Rec(CODPC, IDNfr).