I use the following method in odder to retrieve data by passing pageIndex
(1) and pageSize
(500) for each HTTP call.
this.demoService.geList(1, 500).subscribe(data => {
this.data = data.items;
});
The response has a property called isMore
and I want to modify my method in odder to continue HTTP calls if isMore
is true. I also need to merge the returned values and finally return the accumulated values.
For example, assuming that there are 5000 records and until 10th HTTP call, the service returns true for isMore
value. After 10th HTTP call, it returns false and then this method sets this.data
value with the merged 5000 records. For this problem, should I use mergeMap
or expand
or another RxJs
operator? What is the proper way to solve this problem?
Update: I use the following approach, but it does not merge the returned values and not increase the pageIndex. For this reason it does not work (I tried to make some changes, but could not make it work).
let pageIndex = 0;
this.demoService.geList(pageIndex+1, 500).pipe(
expand((data) => {
if(data.isComplete) {
return of(EMPTY);
} else {
return this.demoService.geList(pageIndex+1, 500);
}
})
).subscribe((data) => {
//your logic here
});
Update II:
of({
isMore : true,
pageIndex: 0,
items: []
}).pipe(
expand(data => demoService.geList(data.pageIndex+1, 100)
.pipe(
map(newData => ({...newData, pageIndex: data.pageIndex+1}))
)),
// takeWhile(data => data.isMore), //when using this, it does not work if the total record is less than 100
takeWhile(data => (data.isMore || data.pageIndex === 1)), // when using this, it causing +1 extra HTTP call unnecessarily
map(data => data.items),
reduce((acc, items) => ([...acc, ...items]))
)
.subscribe(data => {
this.data = data;
});
Update III:
Finally I made it work by modifying Elisseo's approach as shown below. Howeveri **I need to make it void and set this.data
parameter in this getData()
method. How can I do this?
getData(pageIndex, pageSize) {
return this.demoService.geList(pageIndex, pageSize).pipe(
switchMap((data: any) => {
if (data.isMore) {
return this.getData(pageIndex+1, pageSize).pipe(
map((res: any) => ({ items: [...data.items, ...res.items] }))
);
}
return of(data);
})
);
}
I want to merge the following subscribe part to this approach but I cannot due to some errors e.g. "Property 'pipe' does not exist on type 'void'."
.subscribe((res: any) => {
this.data = res;
});
getData(pageIndex, pageSize) {
return this.demoService.getList(pageIndex, pageSize).pipe(
switchMap((data: any) => {
if (!data.isCompleted) {
return this.getData(pageIndex+1, pageSize).pipe(
map((res: any) => ({ data: [...data.data, ...res.data] }))
);
}
return of(data);
})
);
}
stackblitz
NOTE: I updated pasing as argument pageIndex+1 as @mbojko suggest -before I wrote pageIndex++
UPDATE 2
Using expand operator we need take account that we need feed the "recursive function" with an object with pageIndex -it's necesarry in our call- for this, when we make this.demoService.getList(data.pageIndex+1,10)
we need "transform the result" adding a new property "pageIndex". for this we use "map"
getData() {
//see that initial we create "on fly" an object with properties: pageIndex,data and isCompleted
return of({
pageIndex:1,
data:[],
isCompleted:false
}).pipe(
expand((data: any) => {
return this.demoService.getList(data.pageIndex,10).pipe(
//here we use map to create "on fly" and object
map((x:any)=>({
pageIndex:data.pageIndex+1, //<--pageIndex the pageIndex +1
data:[...data.data,...x.data], //<--we concatenate the data using spread operator
isCompleted:x.isCompleted})) //<--isCompleted the value
)
}),
takeWhile((data: any) => !data.isCompleted,true), //<--a take while
//IMPORTANT, use "true" to take account the last call also
map(res=>res.data) //finally is we only want the "data"
//we use map to return only this property
)
}
Well we can do a function like this:
getData() {
of({pageIndex:1,data:[],isCompleted:false}).pipe(
expand((data: any) => {
return this.demoService.getList(data.pageIndex,10).pipe(
tap(x=>{console.log(x)}),
map((x:any)=>({
pageIndex:data.pageIndex+1,
data:[...data.data,...x.data],
isComplete:x.isComplete}))
)
}),
takeWhile((data: any) => !data.isComplete,true), //<--don't forget the ",true"
).subscribe(res=>{
this.data=res.data
})
}
See that in this case we don't return else simple subscribe to the function and equal a variable this.data to res.data -it's the reason we don't need the last map
Update 3 by Mrk Sef
Finally, if you don't want your stream to emit intermittent values and you just want the final concatenated data, you can remove the data concatenation from expand
, and use reduce
afterward instead.
getData() {
of({
pageIndex: 1,
data: [],
isCompleted: false
})
.pipe(
expand((prevResponse: any) => this.demoService.getList(prevResponse.pageIndex, 10).pipe(
map((nextResponse: any) => ({
...nextResponse,
pageIndex: prevResponse.pageIndex + 1
}))
)
),
takeWhile((response: any) => !response.isCompleted, true),
// Keep concatenting each new array (data.items) until the stream
// completes, then emit them all at once
reduce((acc: any, data: any) => {
return [...acc, ...data.data];
}, [])
)
.subscribe(items => {
this.data=items;
});
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With