Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

M Code(Power Query) to Remove Empty columns that runs fast

I use the below code in Power Query to remove empty columns from a table with a lot of columns. It runs very slow and I am looking for a way to speed it up. Basically if all the entries in a given column is null, the column should be removed

//Remove Empty Columns
ColumnstoKeep = List.Select(
        Table.ColumnNames(#"Expanded"),each List.NonNullCount(Table.Column(#"Expanded",_)) <>0 ),

RemoveEmptyColumns = Table.SelectColumns(#"Expanded",ColumnstoKeep),
like image 919
Rafadan Avatar asked Oct 29 '25 17:10

Rafadan


1 Answers

Here is another answer which will stop once the first non null column is reached.

enter image description here

enter image description here

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUTIyAhIoKFYHKoNNAipujE0CUzwWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t, Column3 = _t, Column4 = _t, Column5 = _t, Column6 = _t, Column7 = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", Int64.Type}, {"Column2", Int64.Type}, {"Column3", Int64.Type}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}}),
    sampleTable = Table.ReplaceValue(#"Changed Type","",null,Replacer.ReplaceValue,{"Column4", "Column5", "Column6", "Column7"}),
    columnNames = Table.ColumnNames( sampleTable),
    Custom1 = List.Generate(
        ()=> [name = columnNames{index}, index = 0] ,
        each  [index] < List.Count(columnNames) and List.NonNullCount(Table.Column(sampleTable,[name])) <>0,
        each [name = columnNames{index}, index = [index]+ 1] ,
        each [name]
        )
in
    Custom1
like image 129
davidebacci Avatar answered Oct 31 '25 12:10

davidebacci



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!