What is the fastest way to reduce a MATLAB table with multiple rows per timestamp to a table with only one row per timestamp and at the same time merging the other values into this one row ? (See: Example of before and after)
As I have to handle large amounts of data is there a way of doing this in parallel (parfor, ...) ?
BEFORE:
Timestamp Value01 Value02 Value03
_________ _______ _______ _______
1001 01 02 []
1001 [] [] []
1001 [] [] 03
1002 [] [] 07
1002 [] 09 []
1003 04 01 []
1003 [] [] []
1004 05 06 08
AFTER:
Timestamp Value01 Value02 Value03
_________ _______ _______ _______
1001 01 02 03
1002 [] 09 07
1003 04 01 []
1004 05 06 08
I found your question interesting and tried to find solution. Now want to show you my approach.
First of all I tried to use union
function but it works a bit confusing for me with tables
so only way I found - is to convert table to cell
and numeric
data, work with it and then create new table.
Here the code:
tab = table2cell(MyTable(:,2:end)) % convert to cell
tab( cellfun('isempty',tab) ) = {[0]} % replace [] with [0]
tab = cell2mat(tab) % convert to numeric
t = MyTable(:,{'Timestamp'}) % lets take a time
t = table2array(t) % to numeric too
t = t - 1000
fun = @(x) sum( tab( find (t == x),:),1) % find the sum of rows with the same time
arr = [1:t(end)]' %' % how many sums we need
carr = arrayfun(fun, arr, 'uniformoutput',false)
result = cell2mat(carr)
And the result is:
result =
1 2 3
0 9 7
4 1 0
5 6 8
The last step - create new table. It's just the example:
Value01 = result(:,1)
Value01 = num2cell(Value01)
Value01(find([Value01{:}]==0)) = {[]}
%... the same for Value02 and Value03
NewTable = table(1000+arr,Value01, Value02, Value03, 'VariableNames',{'Timestamp', 'v1','v2','v3'})
NewTable =
Timestamp v1 v2 v3
_________ ___ ___ ___
1001 [1] [2] [3]
1002 [] [9] [7]
1003 [4] [1] []
1004 [5] [6] [8]
Already I finished this answer I thought about your data... Looks like it's strings
- because you have 01
not 1
... But I still think my solution is right just need to rewrite a bit if you really have a strings
:)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With