My Angular (v1.2.15) application needs to process a large (60k entries) index in Elasticsearch (v2.4). The index is too large to use traditional paging using from
and size
parameters in the Elasticsearch query. Instead I'm using the Scroll Api, as recommended.
I can get this working as expected using an HTTP client (Postman). The call sequence is as follows:
POST https://example.com/myindex/mytype/_search?scroll=1m
{
"query":{
"term":{
"someFlag":false
}
},
"sort":["_doc"]
}
This returns:
{
"_scroll_id": "$$28$$VLgIOB3qj91KEA5az5jSwA==$$QXix74BkZ7CRjjBL54dy2gnVav8=cXVlcnlBbmRGZXRjaDsxOzc2MDI5OlJyQzFUX0oxUkV1aW9XclU4M2RPOFE7MDs=",
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 10002,
"max_score": null,
"hits": [
...
]
}
}
Using the _scroll_id value in the next call:
POST https://example.com/myindex/mytype/_search?scroll=1m
{
"scroll":"5m",
"scroll_id":"$$28$$VLgIOB3qj91KEA5az5jSwA==$$xfMhpKpO0ZUjhlY7gI3Yt__DIeY=cXVlcnlBbmRGZXRjaDsxOzc2MDMyOlJyQzFUX0oxUkV1aW9XclU4M2RPOFE7MDs="
}
Returns:
{
"_scroll_id": "$$28$$VLgIOB3qj91KEA5az5jSwA==$$xfMhpKpO0ZUjhlY7gI3Yt__DIeY=cXVlcnlBbmRGZXRjaDsxOzc2MDMyOlJyQzFUX0oxUkV1aW9XclU4M2RPOFE7MDs=",
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 10002,
"max_score": null,
"hits": [
...
]
}
}
as expected.
However, when I attempt to make the equivalent calls using Angular's $http
service, the first call succeeds, but no HTTP call appears to be made on subsequent calls.
var getSavedSearchIds = function () {
var url = configService.getElasticUri() + "/_search?scroll=1m";
var config = {
headers: {
"Authorization": "Basic " + configService.getElasticAuthorisationData(),
"Content-Type": "application/json"
}
};
var query = {
query: {
term: {
someFlag: false
}
},
sort: ["_doc"]
};
return $http.post(url, query, config).then(function (data) {
return data;
});
}
var getNextSavedSearchIds = function (scrollId) {
var url = configService.taskElasticUri + "_search/scroll";
var config = {
headers: {
"Authorization": "Basic " + configService.getElasticAuthorisationData(),
"Content-Type": "application/json"
}
};
var query = {
scroll: "1m",
scroll_id: scrollId
};
return $http.post(url, query, config).then(function (data) {
return data;
});
}
According to the network monitor in chrome developer tools, no HTTP request is made for the second and subsequent calls.
This looks like an Angular issue, but I haven't been able to find a solution to it, or information about it.
How can I get requests for data pages to succeed after the initial call?
NB. Angular 1.2.15 is a constraint on this solution. It can't be changed because of potential impacts in the broader application scenario.
UPDATE
The code that calls these two methods looks like this:
savedSearchService.getSavedSearchIds()
.then(function (batch) {
var scrollId = batch.scrollId;
var ids = batch.Ids
while (ids.length > 0) {
...
savedSearchService.getNextSavedSearchIds(scrollId)
.then(function (batch) {
scrollId = batch.scrollId;
ids = batch.Ids
...
});
}
return;
})
then()
callback is never hit.