Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Reducing a Matlab table with

What is the fastest way to reduce a MATLAB table with multiple rows per timestamp to a table with only one row per timestamp and at the same time merging the other values into this one row ? (See: Example of before and after)

As I have to handle large amounts of data is there a way of doing this in parallel (parfor, ...) ?

BEFORE:

Timestamp  Value01   Value02   Value03
_________  _______   _______   _______

1001       01        02        []          
1001       []        []        []          
1001       []        []        03          
1002       []        []        07          
1002       []        09        []          
1003       04        01        []
1003       []        []        []         
1004       05        06        08 

AFTER:

Timestamp  Value01   Value02   Value03
_________  _______   _______   _______

1001       01        02        03          
1002       []        09        07          
1003       04        01        []         
1004       05        06        08 
like image 905
user1833863 Avatar asked Oct 31 '22 13:10

user1833863


1 Answers

I found your question interesting and tried to find solution. Now want to show you my approach.

First of all I tried to use union function but it works a bit confusing for me with tables so only way I found - is to convert table to cell and numeric data, work with it and then create new table.

Here the code:

tab = table2cell(MyTable(:,2:end))                  % convert to cell
tab( cellfun('isempty',tab) ) = {[0]}               % replace [] with [0]
tab = cell2mat(tab)                                 % convert to numeric
t = MyTable(:,{'Timestamp'})                        % lets take a time
t = table2array(t)                                  % to numeric too
t = t - 1000
fun = @(x) sum( tab( find (t == x),:),1)            % find the sum of rows with the same time
arr = [1:t(end)]'   %'                              % how many sums we need
carr = arrayfun(fun, arr, 'uniformoutput',false)    
result = cell2mat(carr)

And the result is:

 result =

 1     2     3
 0     9     7
 4     1     0
 5     6     8

The last step - create new table. It's just the example:

Value01 = result(:,1)
Value01 = num2cell(Value01)
Value01(find([Value01{:}]==0)) = {[]}
%... the same for Value02 and Value03
NewTable = table(1000+arr,Value01, Value02, Value03, 'VariableNames',{'Timestamp', 'v1','v2','v3'})


NewTable = 

Timestamp    v1     v2     v3 
_________    ___    ___    ___

1001         [1]    [2]    [3]
1002         []     [9]    [7]
1003         [4]    [1]    [] 
1004         [5]    [6]    [8]

Already I finished this answer I thought about your data... Looks like it's strings - because you have 01 not 1... But I still think my solution is right just need to rewrite a bit if you really have a strings :)

like image 198
Mikhail_Sam Avatar answered Nov 15 '22 08:11

Mikhail_Sam