Archive for the ‘HowTo’ Category

Against the Browser’s Will: Make Mashups Talk Across Domains

Friday, March 31st, 2006

I used some of my previous blog posts and turned it into an article for DevX that explains different ways on how to make cross domain requests from within a browser. Here is the link: Against the Browser’s Will: Make Mashups Talk Across Domains.

Web Services + JSON = Dump Your Proxy

Thursday, December 15th, 2005

In my post on "How to build a Maps Mash-up" I mentioned that there are different ways to overcome the browser security restrictions to retrieve data from another domain (cross-domain restriction). The previous sample used the software proxy method to make the Web service requests and this post talks about a way to make a request without a proxy. It’s the dynamic script tag method.

Today Yahoo! added a new output option for part of their Web services called JSON. This makes it possible to make the JavaScript WS request without using the XMLHTTPRequest object. It is a great way to pull data from another domain because you can dump your proxy and all the data will not route through your server anymore. I will talk about the pros and cons of both these approaches later, but first I want to give an overview of what JSON is, how it works and show some sample code.

What is JSON?

On Doug Crockford’s page it reads like that: "JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate." And that’s how it look like:

{"ResultSet":{"Result":[{"precision":"city","Latitude":"34.05217","Longitude":"-118.243469","Address":"","City":"LOS ANGELES","State":"CA","Zip":"","Country":"US"}]}}}

The string above is returned by Y! Geocoder for the query “LA”. JSON is a serialized JavaScript object, which JavaScript can turn back into an object. For Yahoo! WS the structure of the JSON string is similar to the XML result but the difference between and attribute and element can’t be made. The following is a comparison of the XML result for the same call.

<ResultSet ... >
  <Result precision="city">
    <Latitude>34.05217</Latitude>
    <Longitude>-118.243469</Longitude>
    <Address></Address>
    <City>LOS ANGELES</City>
    <State>CA</State>
    <Zip></Zip>
    <Country>US</Country>
  </Result>
</ResultSet>

One way to get from JSON to a JavaScript object is to call eval(), with the string as argument. The following sample uses the Geocoder result to display LA’s Latitude and Longitude in an alert box. This is just static.

eval.html
<HTML>
  <BODY>
    <script language"javascript">
      var location = eval({"ResultSet":{"Result":[{"precision":"city","Latitude":"34.05217","Longitude":"-118.243469","Address":"","City":"LOS ANGELES","State":"CA","Zip":"","Country":"US"}]}});
      alert("Lat:" + location.ResultSet.Result[0].Latitude + " Lon: " + location.ResultSet.Result[0].Longitude );
    </script>
  </Body>
</HTML>

This is nice but doesn’t do too much in the real world. The problem was to get the data from a Web service that is located on another domain imported without using a proxy.

The secret sauce

Adding the <Script> tag dynamically in the DOM tree of the browser is the answer and the JSON response helps to get the data in a format that is easy to digest for JavaScript. When a Script tags gets dynamically added to the DOM tree the code (script URL) gets executed on the fly. The trick is that instead pointing to a JavaScript library, we include a Web service request in the tag that returns data in the above mentioned format. The Yahoo! Web services that offer the JSON output option also supports a parameter called ‘callback’ and all it does is wrap the return data in a function with the name of the callback value. http://api.local.yahoo.com/MapsService/V1/geocode?appid=dantheurer&location=la&output=json&callback=getLocation would result in something like this getLocation({"ResultSet":{"Result":[{"precision":"city",….) which tries to call the getLocation function (callback) that needs to be implemented to deal with the data.

Below is a sample that takes a location as an input parameter, then calls the Y! Geocoder WS and displays Long / Lat in the page.

geocodeJson.html
<script type="text/javascript" src="jsr_class.js"></script>
<script type="text/javascript">
  //<![CDATA[
  var appid = "dantheurer";

  //That is the callback function that is specified in the request url and gets executed after the data is returned
  function getLocation(jData) {
    if (jData == null) {
      alert("There was a problem parsing search results.");
      return;
    }
    //get the values out of the object
    var lat = jData.ResultSet.Result[0].Latitude;
    var lon = jData.ResultSet.Result[0].Longitude;
    //build some html
    var smart = "Long: " + lon + "<br />" + "Lat: " + lat;
    //add it in the page DOM
    document.getElementById('result').innerHTML = smart;
  }

  function addGeocode() {
    var location = document.getElementById("geoquery").value;
    // Build the Yahoo! web services call
    request = 'http://api.local.yahoo.com/MapsService/V1/geocode?appid=' + appid + '&location=' + location + '&output=json&callback=getLocation';
    // Create a new script object
    aObj = new JSONscriptRequest(request);
    // Build the script tag
    aObj.buildScriptTag();
    // Execute (add) the script tag
    aObj.addScriptTag();
  }
  //]]>
</script>

Jason, my cube neighbor, wrote a really nice class that deals with the dirty work of adding, removing and making sure the tags are unique. If you open up the file, it even has a security warning from the inventor of JSON in there. Below is the code snippet:

jsr_class.js

function JSONscriptRequest(fullUrl) {
  // REST request path
  this.fullUrl = fullUrl;
  // Keep IE from caching requests
  this.noCacheIE = '&noCacheIE=' + (new Date()).getTime();
  // Get the DOM location to put the script tag
  this.headLoc = document.getElementsByTagName("head").item(0);
  // Generate a unique script tag id
  this.scriptId = 'YJscriptId' + JSONscriptRequest.scriptCounter++;
}

// Static script ID counter
JSONscriptRequest.scriptCounter = 1;

// buildScriptTag method
JSONscriptRequest.prototype.buildScriptTag = function () {
  // Create the script tag
  this.scriptObj = document.createElement("script");

  // Add script object attributes
  this.scriptObj.setAttribute("type", "text/javascript");
  this.scriptObj.setAttribute("src", this.fullUrl + this.noCacheIE);
  this.scriptObj.setAttribute("id", this.scriptId);
}

// removeScriptTag method
JSONscriptRequest.prototype.removeScriptTag = function () {
  // Destroy the script tag
  this.headLoc.removeChild(this.scriptObj);
}

// addScriptTag method
JSONscriptRequest.prototype.addScriptTag = function () {
  // Create the script tag
  this.headLoc.appendChild(this.scriptObj);
}

Here is what the script does in some bullet points

  • Build request URL with input parameter and callback function
  • Build the script tag that contains the request URL
  • Add the new tag to the DOM tree
  • As soon as the tag gets addes, the WS request gets executed and what gets returned is the JSON response wrapped in a function call. The name of the function got specified in the callback parameter.
  • The response, which is a function call now calls the matching function with the JSON data as parameter. This is where the data can get extracted.
  • The script tag gets removed again

The sample I wrote for the Maps launch depends on a PHP proxy. I took that sample and wrote a version that uses JSON instead. Maybe it’s because it’s new, but for some reason I like the new version better.

jsonSample

All this is not just a hack that might not work again tomorrow because of a browser upgrade. Most of the dynamic ads use the dynamic script tag to display themselfes depending on the context. There are of course ups and downs for both technologies and not everyone agrees, but here are some points to think about.

Proxy method

  • More robust, error handling is easier
  • More control on the server side
  • It has some security implications on the server side as the proxy could be abused.
  • The server side can have additional functionality implemented that is hidden to the caller e.g. login, exchange secrets…

Dynamic script tag

  • No XML parsing necessary
  • Performance win
  • No traffic gets routed (and counted) on your end.
  • JSON converters don’t know that they should define an array if they is only one nested element in a tag, even if the Schema allows 1..n
  • More cross-browser issues
  • Positive impact on rate limiting if it’s done per IP
  • No need to set up a proxy

For more information about JSON, JavaScript and Web services have a look at our brand new Yahoo! Developer Network JavaScript Developer Center.

How to build a Maps Mash-up

Thursday, November 3rd, 2005

Maps are cool! Not only do they provide better aerial images, what makes a map even more valuable is to add data to it. This HowTo describes how this can be done and how to overcome the XmlHttpRequest cross-domain restriction.

This post will discuss the following parts needed for the Mash-up:

  • Map API that lets you embed a map into your website
  • Data source
  • Ways to get around cross-domain restriction

There are a couple of choices for embeddable Maps out there. The “oldest” one is Google Maps, but there is also Virtual Earth, and today the new Yahoo! Maps launched! This is really good stuff. Yahoo! offers different ways of working with their Maps.

  • The simple API takes a geoRSS formated XML file and overlays the data on yesterday’s YMaps technology
  • There are three ways to interact with the brand new Flash Maps. One of them is JavaScript, making it really easy to write super slick applications
  • The last one is an AJAX API that works pretty similar to what’s already out there.

The sample is located at http://theurer.cc/maps/y/ and you can have a look at the JavaScript code by viewing the page source. It uses the AJAX API and demonstrates the following:

  • Geocode an address and display on the map
  • Overlay Traffic information like accidents and construction zones
  • Overlay local search information

The data sources used are three REST Web services that Yahoo also recently opened up. They are easy to program against as it is just a URL with parameters that returns XML.

Browsers like Firefox and IE enforce a cross-domain restriction, which does not allow HTTP calls from one domain to another. Since REST works over HTTP and the services are not hosted on the same server as the Maps application, we need to find a way to get around that. There are different ways to do that.

  • Software proxy
  • Apache mod_rewrite
  • Dynamically loading .js files via script tag and pass data instead of JS code.

The easiest approach is to write a proxy that lives on the same server as the application and just pass the Request and Response to and from the web server. This has a slight security risk, but the example shows how you could limit it. For our scenario it works perfectly.

Those were the basics. Let’s look at some code! The full source:

<html>
<head>
  <script type="text/javascript"
src="http://api.maps.yahoo.com/ajaxymap?v=2.0
&appid= dantheurer"></script>
  <style type="text/css">
    #mapbox {
      height: 600px;
      width: 600px;
  } </style>
</head>
<body>
  <div id="mapbox"></div>
  <script type="text/javascript">
    //init map
    var ymap =
    new YMap(document.getElementById('mapbox'));
    //define a point to start at
    var startPoint = new YGeoPoint(37.34,-122.02);
    //center and draw map at given point
    ymap.drawZoomAndCenter(startPoint, 7);
  </script>
</body>
</html>

The appId in the script is included only for tracking purposes. Please replace it with your own. You can get an appid here. Once you have the code working, we can go ahead and add controls to the map.
//init map
var ymap = new YMap(document.getElementById('mapbox'));
//add controls
ymap.addPanControl();
ymap.addZoomLong();

All that and more can be found in the GettingStartedGuide for AJAX. It gets a little more interesting when data sources come into play. For this sample, I demonstrate the easiest way to overcome the cross-domain restriction. A PHP file will serve as a proxy. Yes, this has some security implications but the proxy is limited to a request with a specific (Yahoo!) domain. For the use case of a free Web service, this is “good enough”. This is all that’s needed:

yproxy.php
<?php
  header("Content-type: text/xml\n\n");
  $url = $_SERVER['QUERY_STRING'];
  //strpos allows only calls to specified endpoint
  if ( strpos($url,"http://api.local.yahoo.com/") === 0)
  {
    $ch = curl_init($url);
    curl_exec($ch);
  }
?>

To run the proxy, the PHP Curl extension needs to be enabled.

Now that the proxy is in place we are able to make a REST call to api.local.yahoo.com. I want to follow what happens if the ‘AddLocation’ Button is pressed. The sample has an input box where the user can input an address.

<div class="inputelement">
<button onClick="addGeocode()">
Add location</button> <input id="geoquery"
value="5th Avenue,New York" size="50">
</input> - enter an address</div>

Once the button is pressed, the function addGeocode() gets called. Yes, the name could be better… and what it does is assemble the REST query and call the function that actually calls the proxy.


var query = document.getElementById("geoquery").value;
var uri =
"http://api.local.yahoo.com/MapsService/V1/geocode?
appid=dantheurer&location=" + query;
callWS(uri, "geocode");

The callWS() does an XmlHttpRequest to the proxy with the REST URL as parameter and displays ‘Loading’ while it waits for the proxy to complete the request. The PHP proxy uses CURL to make the actual request, and the callback method parseResult() gets executed once the call has succeeded.

var url = 'yproxy.php?' + encodeURI(target);
xmlhttp.open('GET', url, true);
xmlhttp.onreadystatechange = function() {
  if(xmlhttp.readyState == 4 && xmlhttp.status == 200) {
    document.getElementById('result').innerHTML = '';
    parseResult(xmlhttp.responseText, service);
  } else {
  document.getElementById('result').innerHTML =
  "Loading...";
  }
};
xmlhttp.send(null);
}

From here, all that’s left is parsing the XML and adding the markers. In this sample I used the xmlw3cdom.js parser from SourceForge. It’s more than I need, but it makes things really easy. This DOM parser requires the following SAX parser xmlsax.js to deal with the XML object. I will not discuss how to parse the XML, because it’s easier to just look it up in the source file.

The results are used to create a Point and a String for the Smartwindow, which get passed to the createYMarker() method.


var point =
new YGeoPoint(parseFloat(lat),parseFloat(lon));
var smart = "Long: " + lon + "
" + "Lat: " + lat;
var myMarker = createYMarker(point, i, smart);

It creates the Markers on the map.


var marker = new YMarker(point,'id'+number);
marker.addLabel(number);
YEvent.Capture(marker,EventsList.MouseClick, function() { marker.openSmartWindow(msg) });
return marker;

This is a pretty simple example but this can be the foundation for a much more complex mash-ups. Once this part is done, it only gets better.

Keep mashing!

Maps API – 1,2,3 for non developers

Tuesday, September 13th, 2005

I got the following question more than once: “I am not a developer, but how can I make use of the Yahoo! Maps API…”.

I will make an attempt to explain all the steps that need to be done to display multiple POI (points of interest) using the API. The result should look similar to this link
http://api.maps.yahoo.com/Maps/V1/annotatedMaps?
appid=YahooDemo&
xmlsrc=http://developer.yahoo.net/maps/sample.xml
.

It contains the address of the Maps API server and two parameters, appid and xmlsrc.

Written another way:
http://api.maps.yahoo.com/Maps/V1/annotatedMaps?
appid=YOURAPPID&xmlsrc=YOURXMLFILE

The first part is static
http://api.maps.yahoo.com/Maps/V1/annotatedMaps?
and the values after ‘appid=’ and ‘xmlsrc=’ need to be replaced.

YOURAPPID To get an appid click here.
YOURXMLFILE would point to the file that contains all the locations you want to display.

The easiest way to get started with your own map is to copy our sample file and modify it. Create your own file on a server that is accessible for the outside. If you don’t have your own web server / domain, you can sign up for free web space e.g. at Geocities and host your file there. Geocities offers you a web interface to upload files. Go to our sample XML file at http://developer.yahoo.net/maps/sample.xml, do a right click and choose show ‘Page Source’. Mark the complete page and copy it to the Geocities web interface to or a text file if you have ftp access to a server and save it e.g. as sample.xml.

Now you can complete the link above with you xmlsrc. For Geocities it would look something like that http://www.geocities.com/yourname/sample.xml. If you enter the full link from above in your browser with the modified parameters, you should see your own map.

The final step would be to change the locations to display your own. Have fun!