Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Forward (or Backward filling) in postgres

The problem is to fill missing values in a table. In pandas, one can use forward (or backward) filling to do so as shown below:

$> import pandas as pd
$> df = pd.DataFrame({'x': [None, 1, None, None, 2, None, 3, None]})
$> df['y'] = df['x'].fillna(method='ffill')
$> df
    x   y
0 NaN NaN
1   1   1
2 NaN   1
3 NaN   1
4   2   2
5 NaN   2
6   3   3
7 NaN   3

Is there a way to do that in SQL and more precisely in PostGres? I guess window functions could help but i couldn't figure out how.

In PostGres, it would be like:

sandbox=# SELECT x, ??
FROM
  (SELECT NULL AS x
   UNION ALL SELECT 1 AS x
   UNION ALL SELECT NULL AS x
   UNION ALL SELECT NULL AS x
   UNION ALL SELECT 2 AS x
   UNION ALL SELECT NULL AS x
   UNION ALL SELECT 3 AS x
   UNION ALL SELECT NULL AS x) a;
 x 
---

 1


 2

 3

(8 rows)
like image 575
Guillaume Thomas Avatar asked Jun 18 '16 12:06

Guillaume Thomas


People also ask

What is forward and backward fill?

Forward filling and backward filling are two approaches to fill missing values. Forward filling means fill missing values with previous data. Backward filling means fill missing values with next data point.

How many indexes can be created on a table in Postgres?

Indexes can have up to 32 columns, including INCLUDE columns. (This limit can be altered when building PostgreSQL.) Only B-tree currently supports unique indexes. An operator class with optional parameters can be specified for each column of an index.

Can we create index on view in PostgreSQL?

We can create an index only on a materialized view not a regular view in PostgreSQL. Materialized View is a database object that stores the result of a pre-calculated query of the database.


1 Answers

window functions here

so many aliases since your query is very sensitive to order. I added more empty x lines to prove it is prune to several empty lines...

select x,y from (
select r,x, case when y is not null then y else min(y) over (partition by x order by r) end y from (
SELECT row_number() over() r,x, case when x is not null then x else lag(x) over () end y
FROM
  (SELECT NULL AS x
   UNION ALL SELECT 1 AS x
   UNION ALL SELECT NULL AS x
   UNION ALL SELECT NULL AS x
   UNION ALL SELECT NULL AS x
   UNION ALL SELECT NULL AS x
   UNION ALL SELECT 2 AS x
   UNION ALL SELECT NULL AS x
   UNION ALL SELECT 3 AS x
   UNION ALL SELECT NULL AS x
   ) a
   ) b
order by r
   ) c
   ;

enter image description here

like image 92
Vao Tsun Avatar answered Sep 30 '22 20:09

Vao Tsun