Patching into JSON APIs (Twitter)

Patching into JSON APIs (Twitter)


It's always fun to watch the trending topics on Twitter. It, like so many other popular online destinations, has a JSON API. Let's have some fun. Here's what we're going to build. You can see the listview on the left-side and the search view on the right-side.

At this point, I'm going to dispense with the academically correct practice of separating the CSS and JS from the HTML. Aside from the libraries, all page-specific code (HTML, CSS, and JS) will be located within the single page. The following code is our starting base page. It is twitter.html in the code bundle for the chapter:

<!DOCTYPE html>  
<html>  
  <head>   
    <meta charset="utf-8">   
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.0, user-scalable=no">   
    <title>Chapter 5 - News</title>       
    <link rel="stylesheet" href="http://code.jquery.com/mobile/1.3.0/jquery.mobile-1.3.0.min.css" />   
    <script src="http://code.jquery.com/jquery-1.8.2.min.js"></script>  
    <script src="js/jsrender.min.js" type="text/javascript"></script>
  <script src="http://code.jquery.com/mobile/1.3.0/jquery.mobile-1.3.0.min.js"></script>

This next bit of styling will help our Twitter results look more Twitter-ish:

    <style type="text/css">     
      .twitterItem .ui-li-has-thumb .ui-btn-inner a.ui-link-inherit, #results .ui-li-static.ui-li-has-thumb{       
         min-height: 50px;       
         padding-left: 70px;     
      } 
      .twitterItem .ui-li-thumb, #results .ui-listview .ui-li-icon, #results .ui-li-content{       
         margin-top: 10px;
         margin-left: 10px;     
      }     
      .twitterItem .ui-li-desc{       
         white-space:normal;       
         margin-left:-25px;       
      }     
      .twitterItem .handle{       
        font-size:80%;       
        font-weight:normal;         
        color:#aaa;     
      }     
      .twitterItem .ui-li-heading{       
        margin: 0 0 .6em -25px;     
      }   
    </style> 
  </head>   
  <body>  

This page is pretty much just a placeholder that will be filled in once we get back results from hitting the Twitter API:

  <div id="home_page" data-role="page"> 	
    <div data-role="header"><h1>Now Trending</h1></div>   
    <div data-role="content">
      <ul id="results" data-role="listview" data-dividertheme="b">
      </ul>
    </div>
  </div>  

The following script is the processing core of the page.

  <script type="text/javascript"> 
    $(document).on("pagebeforeshow", "#home_page",  function(){ 	

     //before we show the page, go get the trending topics
     //from twitter
    $.ajax({       
      url:"https://api.twitter.com/1/trends/daily.json",
        dataType:"jsonp",       
        success: function(data) {       
          var keys = Object.keys(data.trends);       

          //Invoke jsRender on the template and pass in
          //the data to be used in the rendering.
          var content = $("#twitterTendingTemplate")
           .render(data.trends[keys[0]]);

          //Inject the rendered content into the results area 
          //and refresh the listview
          $("#results").html( content ).listview("refresh"); 
        }	
      })
      .error(function(jqXHR, textStatus, errorThrown){                  
        alert(textStatus+" - "+errorThrown);     
      });
    });    

    $(document).on('click', 'a.twitterSearch', function(){     
      var searchTerm = $(this).attr("data-search");     

      //take the search term from the clicked element and 
      //do a search with the Twitter API
      $.ajax({        
        url:"http://search.twitter.com/search.json?q="+escape(searchTerm),        
        dataType:"jsonp",       
        success: function(data){

          //create a unique page ID based on the search term
          data.pageId = searchTerm.replace(/[# ]*/g,"");             
          //add the search term to the data object
          data.searchTerm = searchTerm; 

          //render the template with JsRender and the data    
          var content = $("#twitterSearchPageTemplate").render(data);  

          //The rendered content is a full jQuery Mobile 
          //page with a unique ID.  Append it directly to the 
          //body element
          $(document.body).append(content); 	
          
          //switch to the newly injected page
          $.mobile.changePage("#"+data.pageId);       
        }     
      })
      .error(function(jqXHR, textStatus, errorThrown){                  
        alert(textStatus+" - "+errorThrown);     
      });   
    });     
  </script>  

Following are the two JsRender templates:

  <script id="twitterTendingTemplate" type="text/x-jsrender"> 
    <li class="trendingItem">     
      <a href="javascript://" class="twitterSearch" data-search="{{>name}}">       
        <h3>{{>name}}</h3>     
      </a>   
    </li> 
  </script>  

  <script id="twitterSearchPageTemplate" type="text/x-jsrender">   
    <div id="{{>pageId}}" data-role="page" data-add-back-btn="true">     
      <div data-role="header">
        <h1>{{>searchTerm}}</h1>
      </div>     
      <div data-role="content">
        <ul id="results" data-role="listview" data-dividertheme="b">
          {{for results}}           
            <li class="twitterItem">             
            <a href="http://twitter.com/{{>from_user}}">   
              <img src="{{>profile_image_url}}" alt="{{>from_user_name}}" class="ui-shadow ui-corner-all" /> 
              <h3>{{>from_user_name}} 
                <span class="handle">
                  (@{{>from_user}})<br/>
                  {{>location}} 
                  {{if geo}}
                    {{>geo}}
                  {{/if}}
                </span>
              </h3>               
              <p>{{>text}}</p>             
            </a>           
          </li>         
        {{/for}} 	      
      </ul>     
    </div>   
  </div> 
</script>  
</body> 
</html>

OK, that was a lot of code to throw at you all at once, but most of it should look pretty familiar at this point. Let's start with explaining some of the newest stuff.

Normally, to pull data into a web page, you are subject to the same-domain policy even when you're pulling JSON. However, if it's coming from another domain, you'll need to bypass the same-domain policy. To bypass the some-domain policy, you could use some sort of server-side proxy such as PHP's cURL (http://php.net/manual/en/book.curl.php) or the Apache HTTP Core Components (http://hc.apache.org/) in the Java world.

Let's just keep things simple and use JSONP (also known as JSON with Padding). JSONP does not use a normal Ajax request to pull information. Despite the fact that the configuration options are for the $.ajax command, behind the scenes, it will execute the call for the data as a standalone script tag as follows:

 <script type="text/javascript" src=" https://api.twitter.com/1/trends/daily.json?callback=jQuery172003156238095834851_1345608708562&_=1345608708657"></script>

It is worth noting that JSONP is called using a GET request. That means that you can't use it to pass sensitive data, because it would be instantly viewable through network traffic scanning or simply looking at a browser's request history. So, no logging in over JSONP or passing anything sensitive. Got it?!

Before the actual request is made, jQuery will create a semi-random function name that will be executed once the response is received from the server. By appending that function name as the callback within the URL, we are telling Twitter to wrap their response to us with this function call. So, instead of receiving JSON script like {"trends": …}, we have a script written to our page that starts as follows:

jQuery172003156238095834851_1345608708562({"trends": …}). 

The reason this works is because the same-domain policy does not exist for scripts. Handy, yes? After the script is loaded and the callback has processed, we will have the data in JSON format. In the end, the execution under the sheets is vastly different, but the results are the same as you would get with regular getJSON requests from your own domain.

Here is a slice of the response back from Twitter:

jQuery1720026425381423905492_1345774796764({
  "as_of": 1345774741,
  "trends": {
    "2012-08-23 05:20": [
       {
         "events": null,
         "name": "#ThingsISayTooMuch",
         "query": "#ThingsISayTooMuch",
         "promoted_content": null
       },
       {
         "events": null,
         "name": "#QuieroUnBesoDe",
         "query": "#QuieroUnBesoDe",
         "promoted_content": null
       },
       {
          "events": null,
          "name": "#ASongIKnowAllTheLyricsTo",
          "query": "#ASongIKnowAllTheLyricsTo",
          "promoted_content": null
       },

Next, we whittle down the response to only the part we want (the very latest set of trending topics) and pass that array into JsRender for … well… rendering. It may seem more simple to just loop through the JSON and use string concatenation to build your output but take a look at the following template and tell me that's not going to be a lot cleaner to read and maintain:

<script id="twitterTendingTemplate" type="text/x-jsrender"> 
  <li class="trendingItem">     
    <a href="javascript://" class="twitterSearch" data-search="{{>name}}">       
      <h3>{{>name}}</h3>     
    </a>   
  </li> 
</script>  

The text/x-jsrender type on the script will ensure that the page does not try to parse the inner contents as JavaScript. Since we passed in an array to JsRender, the template will be written for every object in the array. Now that is simple! Granted, we're only pulling the name out of the data object, but you get the idea of how this works.

Let's look at the next significant block of JavaScript:

$(document).on('click', "a.twitterSearch", function(){     
  //grab the search term off the link     
  var searchTerm = $(this).attr("data-search");          
 
  //do a Twitter search based on that term     
  $.ajax({       url:"http://search.twitter.com/search.json?q="+escape(searchTerm),        
   dataType:"jsonp",       
   success: function(data){         
     //create the pageID by stripping 
     //all non-alphanumeric data         
     var pageId = searchTerm.replace(/[^a-zA-Z0-9]+/g,"");                  
     //throw the pageId and original search term 
     //into the data that we'll be sending to JSRenderdata.pageId = pageId;
     data.searchTerm = searchTerm;          	      

     //render the page and append it to the document body         $(document.body).append($("#twitterSearchPageTemplate")
       .render(data));                  

     //set the page to remove itself once left          
     $("#"+pageId).attr( "data-" + $.mobile.ns 
       + "external-page", true )
       .one( 'pagecreate', $.mobile._bindPageRemove );                  
     //switch to the new page          
     $.mobile.changePage("#"+data.pageId);   
    }
  })
  .error(function(jqXHR, textStatus, errorThrown){
    //If anything goes wrong, at least we'll know.           
    alert(textStatus+" - "+errorThrown);     
  });    
});

First, we pull the search term from the attribute on the link itself. The search term itself is somewhat inappropriate as an id attribute for dynamically rendered pages; so, we'll strip out any spaces and non-alphanumeric content. We then append the pageId and searchTerm attribute to the JSON object we received back from Twitter. Following is a sample of returned data from this call:

jQuery1720026425381423905492_1345774796765({
    "completed_in": 0.02,
    "max_id": 238829616129777665,
    "max_id_str": "238829616129777665",
    "next_page": "?page=2&max_id=238829616129777665&q=%23ThingsISayTooMuch",
    "page": 1,
    "query": "%23ThingsISayToMuch",
    "refresh_url": "?since_id=238829616129777665&q=%23ThingsISay
TooMuch",
    "results": [
        {
            "created_at": "Fri, 24 Aug 2012 02:46:24 +0000",
            "from_user": "MichelleEspra",
            "from_user_id": 183194730,
            "from_user_id_str": "183194730",
            "from_user_name": "Michelle Espranita",
            "geo": null,
            "id": 238829583808483328,
            "id_str": "238829583808483328",
            "iso_language_code": "en",
            "metadata": {
                "result_type": "recent"
            },
            "profile_image_url": "http:\/\/a0.twimg.com\/profile_images\/2315127236\/Photo_20on_202012-03-03_20at_2001.39_20_232_normal.jpg",
            "profile_image_url_https": "https:\/\/si0.twimg.com\/profile_images\/2315127236\/Photo_20on_202012-03-03_20at_2001.39_20_232_normal.jpg",
            "source": "&lt;a href=&quot;http:\/\/twitter.com\/&quot;&gt;web&lt;\/a&gt;",
            "text": "RT @MuchOfficial: @MichelleEspra I'd be the aforementioned Much! #ThingsISayTooMuch",
            "to_user": null,
            "to_user_id": 0,
            "to_user_id_str": "0",
            "to_user_name": null,
            "in_reply_to_status_id": 238518389595840512,
            "in_reply_to_status_id_str": "238518389595840512"
        }
        
}

So, we'll take this response and pass it into the renderer to be transformed against twitterSearchPageTemplate:

<script id="twitterSearchPageTemplate" type="text/x-jsrender"> 
    <div id="{{>pageId}}" data-role="page" data-add-back-btn="true">     
      <div data-role="header">
        <h1>{{>searchTerm}}</h1>
      </div>     
      <div data-role="content">
        <ul id="results" data-role="listview" data-dividertheme="b">
          {{for results}}           
            <li class="twitterItem">             
            <a href="http://twitter.com/{{>from_user}}">   
              <img src="{{>profile_image_url}}" alt="{{>from_user_name}}" class="ui-shadow ui-corner-all" /> 
              <h3>{{>from_user_name}} 
                <span class="handle">
                  (@{{>from_user}})<br/>
                  {{>location}} 
                    {{if geo}}
                      {{>geo}}
                    {{/if}}
                </span>
              </h3>               
              <p>{{>text}}</p>             
            </a>           
          </li>         
        {{/for}}       
      </ul>     
    </div>   
  </div> 
</script> 

These are simple implementations. The examples on GitHub show many more options that are worth exploring. Check out http://borismoore.github.com/jsrender/demos/ for details on creating more complex templates. This is a rapidly changing library (most client-side templating libraries are). So don't be surprised if, by the time you read this, there are a lot more options and slightly changed syntax.

Once we have the results of the transformation, we'll be ready to append the new page's source to the document's body and then programmatically change to this new page.