current community

your communities

more stack exchange communities

Stack Exchange
tour help
careers 2.0
Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 0 down vote favorite
1
 
uniqueElementArray= Array.filter(function(itm,i,Array){
                return i==Array.indexOf(itm);
                });

Array contains set of 1000 or even more than 1000 elements. We want to get unique elements from this array. We are using function which is mentioned above finally we are getting unique Elements array but it's creating performance issue. How can i increase the performance. It's taking too much time in IE8 and IE9.w

share|improve this question
    
In IE8? are you using a polyfill? Try jsperf.com –  elclanrs May 9 '13 at 6:26
    
If JQuery can be an option, you can use $.unique(yourArray). Here is the documentation Jquery-Unique –  Ahmet DAL May 9 '13 at 6:28
    
What elements does the array contain and is it sorted? You can use setTimeout to process it without locking the main thread If it's sorted and contains (for example) only strings then you can optimise it a little more –  HMR May 9 '13 at 6:29
    
@AhmetDAL jQuery.unique(array) works on arrays of DOM elements only, not strings or numbers. –  Ankit Jaiswal May 9 '13 at 6:33
    
No it is working on them too. Did you try it ? –  Ahmet DAL May 9 '13 at 6:39

4 Answers 4

up vote 2 down vote

I recommend using underscore.js:

var otherArray = _.uniq(array);

You can see how it's implemented here.

share|improve this answer
    
For future visitors; doesn't work on array containing objects: console.log(_.unique([{a:1},{a:1}]).length) will give you 2 –  HMR May 9 '13 at 7:14
    
So would the original slow solution in the question. JS doesn't have a sense of deep equality by default. _uniq() does provide an optional iterator argument that can help with this. –  user24359 May 9 '13 at 7:19
    
Yes, I see the OP uses indexOf and [{a:0}].indexOf({a:0}) returns -1 So the OP is using Array of simple type. The warning was not for the OP but for future stumblers upon that want to use it for an array containing objects. –  HMR May 9 '13 at 8:32
    
The problem I had with IE8 is that it doesn't matter if the array has even a couple of hundred elements. Had to do it asynchronous, not sure if _ works with larger arrays, I'll give it a shot. –  HMR May 9 '13 at 8:34
    
I'll admit I haven't actually tried it on IE8-- pretty sad if it can't handle it. Looking at underscore's implementation, offhand I'd only expect a big speed up if there are lots of duplicate items, so it might not help. One thing you might try is sorting the array first and then using _uniq(array, true) –  user24359 May 9 '13 at 8:37
up vote 0 down vote

Use it this way :

var array = Array.prototype.indexOf ?
function(arr, val) {
    return arr.indexOf(val) > -1;
} :

function(arr, val) {
    var i = arr.length;
    while (i--) {
        if (arr[i] === val) {
            return true;
        }
    }
    return false;
}

function removeRedundant(arr, equals) {
  var val, origArr = arr.slice(0);
  arr.length = 0;

  for (var i = 0, len = origArr.length; i < len; ++i) {
    val = origArr[i];
    if (!array(arr, val)) {
        arr.push(val);
    }
  }

  return arr;
}

var arr = [1, 2, 2, 1, 3];
removeRedundant(arr);
console.log(arr); // [1, 2, 3]
share|improve this answer
up vote 0 down vote

If you can sort the array (change order of elements), this function seems handy:

unique = function(in_array) {
    var last, result=[];
    in_array.sort();
    for (var i in in_array) if (in_array[i] != last) result.push(last=in_array[i]);
    return result;
}

EDIT:

See also this answer, I think it's great!.

share|improve this answer
up vote 0 down vote

In IE8 I've experienced absurdly slow performance when comparing a bunch of floats. In that case asynchronous handling is the only option you have to not get the "script takes too long" message.

IE8 does not have Array.filter function so your code would not work in IE8 anyway.

//create an array with 10.000 floats
var arr=[];
for(i=0;i<10000;i++){
  arr.push(Math.random());
}
//adding one double float to make sure it will be removed
arr.push(arr[5]);
console.log("array created:",arr.length);
var now=new Date();
function getUniqueDone(arr){
    console.log("done asynch",(new Date().getTime())-now);
    console.log("length of array:"+arr.length);//usually is 10.000
}
// works only on arrays containing numbers or strings
function getUnique(arr,callback){
  var i,j,len,counter,
  chuncksOf=1000;//In IE8 I get script warnings at 150 already,
  //  FF and Chrome could handle 1000
  //  depending on what your array contains you can test a bit and
  //  set chuncksOf accordingly. current value does nnot work in IE8
  //  you have to set it to 100 and wait 20 seconds for output
  if(arr.sort){
      // on very large arrays this still might have a problem
      console.log("sorting");
      arr.sort();
      console.log("sort done");
      i=0;
  }else{
      i=arr.i;
      arr=arr.arr;
  } 
  len=arr.length;
  counter=0;
  while(i<len&&counter<chuncksOf){
    for(j=i+1;j<arr.length;j++){
      if(arr[i]===arr[j]){
        arr.splice(j,1);
        j--;
      }
    }
    i++;
    counter++;
  }
  if(i>=len){
      callback(arr);
  }else{
      var pass={
        arr:arr,
        i:i
      }
console.log("i is now:",i);
      setTimeout(function(){
          getUnique(pass,callback);
      },0);
  }
}
console.log("getting unique");
getUnique(arr,getUniqueDone);
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.