Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 2 down vote favorite

I'm writing a simple Node.JS app that works with the GitHub API to pull user statistics. I'm sending a GET request to /repos/:user/:repo_name/stats/contributors which should return to me a JSON string. Here's my function:

function getRepoCommits(token,repos,user,callback) {
    for (var i=0; i<repos.length; i++) {
        var returnChunk = '';
        var full_name = repos[i].full_name;
        //console.log(full_name);
        var options = {
            host: 'api.github.com',
            path: '/repos/'+full_name+'/stats/contributors',
            method: 'GET',
            headers: {
                'Accept': 'application/json',
                'Authorization': 'token '+token,
                'Content-Type': 'application/x-www-form-urlencoded',
            }
        };

        var request = https.request(options, function(res) {
            //res.setEncoding('utf8');
            res.on('data', function(chunk) {
                returnChunk += chunk;
            });
            res.on('end', function(chunk) {
                console.log(returnChunk);
                var stats = JSON.parse(returnChunk);
                console.log(stats.length);
                for (var j=0;j<stats.length;j++) {
                }
                if (i == repos.length-1) {
                    //callback();
                }
            })
        });

        request.on('error',function(err) {
            callback(err);
            console.log(err);
        });
        request.end();
    }
}

The area of interest is right here:

            res.on('end', function(chunk) {
                console.log(returnChunk);
                var stats = JSON.parse(returnChunk);
                console.log(stats.length);
                for (var j=0;j<stats.length;j++) {
                }
                if (i == repos.length-1) {
                    //callback();
                }
            })

When I check the value of returnChunk, it is a valid JSON string in the format :

[{json}]

However, when the function reaches JSON.parse, it throws an error:

SyntaxError: Unexpected token [

Basically, it's appending an extra [] to the end of the string before parsing it. It becomes

[{json}][]

I've tried for hours on end to figure out how to deal with this problem, but I can't seem to figure out why it is doing this. Any thoughts?

share|improve this question
 
What do you mean it's appending the [] to the end? Where do you see that in your logging? –  Blue Skies 21 hours ago
2  
I see what's happening. You're making several asynchronous requests in a loop, and having them all append to the same returnChunk variable, so all your data is getting mixed together. Remember, JavaScript doesn't have block scope, only function scope, so it's as if you put var returnChunk = "" at the top of the function. –  Blue Skies 21 hours ago
1  
@BlueSkies - you should post that as an answer... –  Alexei Levenkov 21 hours ago

1 Answer

up vote 2 down vote accepted

You're making several asynchronous requests in a loop, and having them all append to the same returnChunk variable, so all your data is getting mixed together.

Remember, JavaScript doesn't have block scope, only function scope, so it's as if you put var returnChunk = "" at the top of the function.

A solution would be to use .forEach() instead of a for statement so that the callback will give you a new scope for each iteration.

repos.forEach(function(repo, i) {
    var returnChunk = '';
    var full_name = repo.full_name;
    //console.log(full_name);
    var options = {
        host: 'api.github.com',
        path: '/repos/'+full_name+'/stats/contributors',
        method: 'GET',
        headers: {
            'Accept': 'application/json',
            'Authorization': 'token '+token,
            'Content-Type': 'application/x-www-form-urlencoded',
        }
    };

    var request = https.request(options, function(res) {
        //res.setEncoding('utf8');
        res.on('data', function(chunk) {
            returnChunk += chunk;
        });
        res.on('end', function(chunk) {
            console.log(returnChunk);
            var stats = JSON.parse(returnChunk);
            console.log(stats.length);
            for (var j=0;j<stats.length;j++) {
            }

// !!! This will be unreliable because you don't know which one will finish last.
//     You could maintain a separate counter that is incremented as each "end" fires
//        to make sure the `callback()` happens on the last one.
            if (i == repos.length-1) {
                //callback();
            }
        })
    });

    request.on('error',function(err) {
        callback(err);
        console.log(err);
    });
    request.end();
});
share|improve this answer
 
Thanks a bunch! It worked perfectly. It was one of those d'oh moments that just goes to show how much I have to learn! I would upvote, but I don't have enough rep points yet! –  upinthecloude 21 hours ago
 
Glad it helped. –  Blue Skies 20 hours ago

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.