current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
SharePoint Stack Exchange is a question and answer site for SharePoint enthusiasts. It's 100% free, no registration required.
up vote 1 down vote favorite

I am working on creating a POC for a huge SharePoint site. The site is actually a seperate site collection that holds 2-3 document libraries. These document libraries are expected to grow rapidly in size and reach a size of 100000 documents in next 2 years (documents are all images of size 3-5MB). I have following questions. If somebody got some experience please share some suggestions.

  1. As the content grows, will there be a performance impact with the custom search/filter webparts that uses CAML and SPQuery?

  2. Is there any better alternative to CAML such as Search API to return items from libraries?

Our end users access these libraries only through custom webparts, not through native SP views.

Thanks in advance -Arun

share|improve this question
3  
You should look into SharePoint list and library thresholds. CAML is the fastest besides SQL to fetch data from lists and libraries in SharePoint, so stick to CAML is my advice. –  Daniel Ziga Nov 7 '13 at 12:24
    
could you specify what SharePoint version you are using? –  TempaC Nov 7 '13 at 13:09
    
It's SharePoint 2010 Enterprise –  Arun Nov 7 '13 at 13:26

2 Answers 2

up vote 4 down vote

SharePoint has some boundaries like

  • Boundaries: Static limits that cannot be exceeded by design
  • Thresholds: Configurable limits that can be exceeded to accommodate specific requirements
  • Supported limits: Configurable limits that have been set by default to a tested value

For more information see the best practise for designing large lists

share|improve this answer
up vote 2 down vote

Before making any recommendations you should review SharePoint 2010 best practices for Search erformance. Articles like this one (even though they were written in the beta timeframe) hold valuable insight into large list/library handling. http://blogs.msdn.com/b/enterprisesearch/archive/2010/06/09/sharepoint-2010-search-dogfood-part-3-query-performance-optimization.aspx

Using Query is always going to be faster that any other approach. The only issue is "freshness" of the index. This means ensuring that you are configuring search to crawl the corpus to keep the index fresh. Whether you chose query (and which query syntax you choose) over another approach will also be determined by how you are creating your search experience for your end users.

Note that in SharePoint 2013 the Search.asmx and SQL Query Language approaches are depricated in favor of REST and Keyword Query Language. So if you are planning on implementing a SharePoint 2013 Search farm anytime soon, that may drive your decision. Note that you can implement SharePoint 2013 Search against 2010 farms and gain the advantages of Continuous Crawling and significantly improved search experience without having to upgrade your 2010 site collections.

BTW, 100,000 is not that "huge" particularly for images, which tend to be slim on metadata. You generally plan additional index partitions at 10M documents. Are you planning to employ a custom iFilter to handle the image metadata or just go with the out of the box null filter?

share|improve this answer
    
Thank you Mathew for the suggestions. As you correctly mentioned, the image libraries that we are planning don't have much meta data associated. Also we have to show the documents available in the search results as soon as they get uploaded. With around 50,000 images uploaded and an index on "Title" column, I am able to run CAML queries without any performance issues. With your recomandations I will continue to work on my POC to identify the best possible solution for us. –  Arun Nov 17 '13 at 7:09
    
You should also look at 2013 Continuous Crawling. It's built for keeping indexes significantly fresher than 2010. –  Matthew McDermott Nov 17 '13 at 19:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.