How to use await in a parallel foreach?

Question

So I sepnt the better part of the night trying to figure this out.

I was fortunate to get introduced to the parallel.foreach yesterday and it works like I want it to do except from one detail.

I have the following:

        Parallel.ForEach(data, (d) =>
        {
            try
            {
                MyMethod(d, measurements);
            }
            catch (Exception e)
            {
              // logg
            }

        });

Within the method "MyMethod" I have alot of logic that gets done and most of it is fine but I make api calls where I fetch data and I use async task for this to be able to use "await" in order for the code to wait untill that specific part gets executed and then move on:

    private async void MyMethod(PimData pimData, IEnumerable<ProductMeasurements> measurements)
    {
        try
        {
           // alot of logic but most relevant part 

            await Task.WhenAll(ExecuteMeasurmentAndChartLogic(pimData.ProductNumber, entity));
            await Task.WhenAll(resourceImportManager.HandleEntityImageFiles(pimData.ProductType + pimData.ProductSize,SwepImageType.Png, ResourceFileTypes.ThreeD, entity, LinkTypeId.ProductResource));

            await Task.WhenAll(resourceImportManager.HandleEntityImageFiles(pimData.ProductSketch, SwepImageType.Png, ResourceFileTypes.Sketch, entity, LinkTypeId.ProductResource));

        }
        catch (Exception e)
        {
            // logg
        }
    }

Problems:

1 For starters the loop finishes before all code is finished

2 Second problem is that I get "Task was canceled" in alot of api calls

3 And third as mentioned above, the code does not wait for each method to full execute.

I cant get it to execute everything in ExecuteMeasurmentAndChartLogic() method before moving forward to the next step.

This gives me the following issues (more issues):

In this method I create an item and add it to the db, and this item needs more info that I get from an api call that is done inside of ExecuteMeasurmentAndChartLogic() but the problem is that several items get craeated and have to wait for the rest of the data which is not what I desire.

SIDE-NOTE: I am aware of that crating an item and adding to the db before all data is there is not best practice but I am integrating toward PIM and the process for that is delicate

I want several threads running but at the same time I want the fuill logic to execute for each item before moving on to the next method.

Clarify:

Several items running

Each item handels ALL the logic it needs to handel before moving on to the next part of the code, noramly do this with await.

In the code above resourceImportManager() method gets executed before ExecuteMeasurmentAndChartLogic() is finished. which is what I dont want.

Instead of a Parallel.ForEach I used :

    Task task1 = Task.Factory.StartNew(() => MyMethod(data, measurements));
    Task.WaitAll(task1);

but that wasnt much of a help

Fairly new to this and havent been able to understand where I am doing it wrong.

EDIT: Updated the problems with this

EDIT: this is how ExecuteMeasurmentAndChartLogic() looks like:

    public async Task ExecuteMeasurmentAndChartLogic(string productNumber, Entity entity)
    {
        try
        {
            GrafGeneratorManager grafManager = new GrafGeneratorManager();
            var graphMeasurmentList = await MeasurmentHandler.GetMeasurments(productNumber);

            if (graphMeasurmentList.Count == 0) return;

            var chart = await grafManager.GenerateChart(500, 950, SystemColors.Window, ChartColorPalette.EarthTones,
                "legend", graphMeasurmentList);

            await AddChartsAndAddToXpc(chart, entity, productNumber);
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
        }

    }

EDIT: Background to this: I make a call to a api to get alot of data. For each item in this data I need to make an api call and get data that I apply to the item.

After reading comments which alos got me thinking in a diffirent way. I can perhaps loop through all my items and do minor logic for them and add a url in a task list and make a seperate task that executes this one by one.

WIll keep this updated

Answer 1

Don't use Parralel.ForEach at all. Make your method to return Task instead of void, collect all the task and wait them like:

Task.WaitAll(data.Select(d => MyMethod(d, someParam)).ToArray());