Dan Theurer » Yahoo

Launching the Un-Launch-Able

Dan — Fri, 29 Sep 2006 16:12:26 +0000

Opening up Yahoo! I’ve been looking forward to this day and writing this exact post for quite some time now! So this is it! Wow, this feels great! We just pushed Browser Based Authentication (BBAuth) out the door.

Let’s start this post with what BBAuth is and what it can be used for. It was designed to allow third-party applications to interact with user-specific data with the users’ consent. On top of doing the obvious, it supports Single Sign-On out of the box.

That means you can build applications that instead of creating your own sign-up flow, which requires users to pick yet another username and password, you can let them sign in with their existing Yahoo! account. The best thing about it is that it’s safe, the YahooId does not get shared with the applications. Your application needs to redirect the user to the Yahoo! BBAuth login and after the user successfully logs in, your app will receive an encrypted and unique userid for each user that logs in. This sample application makes use of SSO.

Good or bad? That is up-to-you do decide. My opinion is that this can make navigating the web so much easier for users, and I am one of them! I don’t want to have to remember ‘x’ usernames and ‘y ‘passwords and keep adding to the list everyday. There are also other ways of dealing with that problem, but here is a solution that is really straightforward. Feel free to leave a comment and let me know what you think! I want to add that this is not driven by a huge initiative to get everyone on Yahoo!, but an attempt to put out another tool that developers can decide to adopt if they like it.

But that’s not all! Yahoo! Photos opened up an API that takes advantage of BBAuth as well. I wrote a sample application that is using it, which allows user to view and update titles and descriptions for photos stored on Yahoo!. The ajaxy parts are using the YUI libraries. On top of all that, we are doing a private beta for developers who attend our public Hackday! The new Yahoo! Mail is opening up their backend!! Appid sign-up will be limited for now but stay tuned for future updates.

For me, this has been quite a ride from the first time we talked about making BBAuth happen until today, the day we finally launched. In a big company like Yahoo!, you need to get input / approval from quite some folks if you want to do something out of the box and open up the company. All that makes sense and is justifiable but sometimes I wish it would have been faster. On the other side, I learned a lot about the company I work for, how big companies work in general, egos, friends and allies and most importantly how you get stuff done that is obviously not on everyone’s “need this today” list.

Like most platform projects that have to support a lot of different use-cases, the list of people that have made this happen is very long and I don’t even want to try to list them all. Instead I want to send a big “general” thank you out to all the thinkers and do-ers, the try-to-stop-it-ers and the must-have-today-ers! Thanks to all of you for making it what it is today!

Further reading: On the Developer Network page we have the official announcement. Jeremy posted something on his blog as well. Without his help to clear last minute “congestions” I am not sure if would have gone out today :)

Hackday – Final Countdown

Dan — Wed, 27 Sep 2006 08:01:05 +0000

It’s getting close! A little over two days are left before we kick off the Yahoo Hack Day! A lot has been written, the guest list / registration will close very soon and we are about to buckle up and go on the ride! There will be classes on Friday during the day ranging from social networks to performance, and in the evening we will have a huge performance that goes way beyond all the usual corporate music gigs, you name it. People are already speculating and we will see who gets it right at the end.

Besides the entertainment and classes we will have networking, coding, camping, presentations, wifi, food (not free – sorry, we spent the $ somewhere else), parking and and and. It will be a lot of fun! If you didn’t sign up already, this is your last call! You can do this on Upcoming.org.

Sidenote: The timer on the hackday page gets powered by the RESTful getTime web service that I wrote for fun. How much more basic then getTime can a web service get? That’s not the point :) The nice thing about it is that it supports JSON and serialized PHP on top of the XML output.

Jobs, Jobs, Jobs

Dan — Thu, 07 Sep 2006 15:26:27 +0000

Yahoo! is hiring and both, my wife and I have very interesting job openings within our teams:

- Sr. Engineer, Exceptional Performance
- Product Manager, Developer Network

If you or someone you know is interested (and qualified), send me a resume (dan at yahoo-inc . com). Both jobs are on-site in Sunnyvale, California. Also, I think I’m suppose to include this lawyer stuff: In the interest of full disclosure, I get a referral bonus and you get a great job at a great company! Yahoo! Inc. is an equal opportunity employer. For more information or to search all of our openings, please visit http://careers.yahoo.com.

Technical Yahoo, Sr, Yahoo! Exceptional Performance

Come join the Exceptional Performance team! We are responsible for measuring and improving the performance of all Yahoo! products worldwide. We’re looking for someone who doesn’t accept the status quo and instead is driven to make our products the best they can be for our users. You will be responsible for:
- Gathering and evangelizing performance best practices across the company.
- Working as a performance consultant across all product teams within Yahoo!.
- Designing experiments for measuring performance and analyzing the results.
- Helping build tools to quantify product performance.

Basic Qualifications:
- BS in Computer Science or related field
- 3+ years experience with web applications
Preferred Qualifications:
- Excellent communication and collaboration skills for working with engineering teams and product management
- Proficiency in at least three of these areas: Perl, PHP, JavaScript, CGI, MySQL, HTTP, Windows SDK
- Attention to detail

If you love to push systems to their optimum we’re waiting to hear from you.

Product Manager, Yahoo! Developer Network

The Yahoo! Developer Network ( http://developer.yahoo.com/) is continuing to open up Yahoo! products and services across all of Yahoo’s services to provide an easy-to-use yet powerful platform for third-party developers to build new and innovative products and businesses. Inspired ideas and innovations are bubbling up from third parties every day, and your job is to help the Yahoo! Developer Network team enable this creativity, and make it flourish.

The perfect candidate is a highly-collaborative and strategic thinker who is extremely analytical, detail oriented, and not afraid to deal with tactical issues to achieve strategic goals as we build the next-generation web. Good prioritization skills, excellent written and oral communication skills, quantitative smarts, and a high degree of comfort managing multiple projects, leading teams, and working with senior executives are required. You should enjoy working in a fast-paced environment with dynamic responsibilities and should have very strong communication, presentation and relationship-building skills.

Some responsibilities include:
- Developing requirements, prioritizing enhancements, translating into MRDs/PRDs and driving projects to completion.
- Defining, tracking and improving key product metrics and developing tests against those metrics
- Conducting competitive analysis and identifying areas for differentiation.
- Aggressively identifying opportunities for improvement, and incorporating new product offerings into the Yahoo! Developer Network platform on an ongoing basis.
- Helping refine existing processes to support routing, tracking, and fulfillment requests from third-party developers and partners
- Developing and maintaining a working knowledge of systems throughout Yahoo! to inform resolution of requests from third-party developers and partners
- Working with cross-functional teams and identifying and working with the appropriate technical and business owners to facilitate decision-making and cross-functional communication

BA/BS or equivalent required, MBA preferred (or significant equivalent experience). Prior experience in the online media industry and internet-focused companies is a plus, especially experience with third-party developers and partners.

Flickr and Upcoming.org ‘rev’ their consumer product

Dan — Mon, 28 Aug 2006 22:11:45 +0000

Upcoming.org now integrates with Flickr and Yahoo! Local; Flickr added geotagging functionality to the organizr – That rocks!

More on Upcoming.org: “Yeah, we’ve been busy. What’s new? Undiscovered events, Flickr photos for events, buddy icons, new event pages, and more than we can remember.”

More on Flickr: “Flickr’s great for exploring photos by photographer, tag, time, text and group, and now it’s also great for exploring photos by place.” – APIs are in the make

Fifa World Cup Germany 2006 – Mashup

Dan — Wed, 12 Apr 2006 02:30:55 +0000

Now Yahoo has it all – international maps, hybrid and satellite view! This was reason enough to write a simple mashup that uses the new features! Check out the Fifa World Cup Germany 2006 Mashup. The documentation is located here.

JSON re-loaded – Travel API lauched

Dan — Wed, 21 Dec 2005 00:44:13 +0000

On Thursday Yahoo! launched JSON support for Search and the Maps API’s and today Travel went live with two new calls that support JSON as well. The new services return detailed information about a trip created with the Yahoo! Travel Trip Planner.

I put together a little page that uses the two new calls with the JSON output to display trip details.

spiffYSearch

Dan — Tue, 20 Dec 2005 21:51:42 +0000

Kent Brewster wrote a really cool app that uses Y! Search JSON and can be included in any web site to display search results dynamically in that page.

Wordpress on Yahoo

Dan — Tue, 20 Dec 2005 20:15:12 +0000

After Yahoo! annouced that they will offer Moveable Type as the default blogging tool, WordPress just made the followig announcement: “Well today we’re very excited to announce we’re adding a new host to the page with a familiar name – Yahoo! ” Read the full post.

Web Services + JSON = Dump Your Proxy

Dan — Thu, 15 Dec 2005 20:45:46 +0000

In my post on "How to build a Maps Mash-up" I mentioned that there are different ways to overcome the browser security restrictions to retrieve data from another domain (cross-domain restriction). The previous sample used the software proxy method to make the Web service requests and this post talks about a way to make a request without a proxy. It’s the dynamic script tag method.

Today Yahoo! added a new output option for part of their Web services called JSON. This makes it possible to make the JavaScript WS request without using the XMLHTTPRequest object. It is a great way to pull data from another domain because you can dump your proxy and all the data will not route through your server anymore. I will talk about the pros and cons of both these approaches later, but first I want to give an overview of what JSON is, how it works and show some sample code.

What is JSON?

On Doug Crockford’s page it reads like that: "JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate." And that’s how it look like:

{"ResultSet":{"Result":[{"precision":"city","Latitude":"34.05217","Longitude":"-118.243469","Address":"","City":"LOS ANGELES","State":"CA","Zip":"","Country":"US"}]}}}

The string above is returned by Y! Geocoder for the query “LA”. JSON is a serialized JavaScript object, which JavaScript can turn back into an object. For Yahoo! WS the structure of the JSON string is similar to the XML result but the difference between and attribute and element can’t be made. The following is a comparison of the XML result for the same call.

34.05217 -118.243469



      LOS ANGELES

      CA

      

      US

One way to get from JSON to a JavaScript object is to call eval(), with the string as argument. The following sample uses the Geocoder result to display LA’s Latitude and Longitude in an alert box. This is just static.

eval.html

This is nice but doesn’t do too much in the real world. The problem was to get the data from a Web service that is located on another domain imported without using a proxy.

The secret sauce

Adding the

Jason, my cube neighbor, wrote a really nice class that deals with the dirty work of adding, removing and making sure the tags are unique. If you open up the file, it even has a security warning from the inventor of JSON in there. Below is the code snippet:

jsr_class.js
function JSONscriptRequest(fullUrl) { // REST request path this.fullUrl = fullUrl; // Keep IE from caching requests this.noCacheIE = '&noCacheIE=' + (new Date()).getTime(); // Get the DOM location to put the script tag this.headLoc = document.getElementsByTagName("head").item(0); // Generate a unique script tag id this.scriptId = 'YJscriptId' + JSONscriptRequest.scriptCounter++; }// Static script ID counter JSONscriptRequest.scriptCounter = 1;// buildScriptTag method JSONscriptRequest.prototype.buildScriptTag = function () { // Create the script tag this.scriptObj = document.createElement("script"); // Add script object attributes this.scriptObj.setAttribute("type", "text/javascript"); this.scriptObj.setAttribute("src", this.fullUrl + this.noCacheIE); this.scriptObj.setAttribute("id", this.scriptId); }// removeScriptTag method JSONscriptRequest.prototype.removeScriptTag = function () { // Destroy the script tag this.headLoc.removeChild(this.scriptObj); }// addScriptTag method JSONscriptRequest.prototype.addScriptTag = function () { // Create the script tag this.headLoc.appendChild(this.scriptObj); }

Here is what the script does in some bullet points

Build request URL with input parameter and callback function
Build the script tag that contains the request URL
Add the new tag to the DOM tree
As soon as the tag gets addes, the WS request gets executed and what gets returned is the JSON response wrapped in a function call. The name of the function got specified in the callback parameter.
The response, which is a function call now calls the matching function with the JSON data as parameter. This is where the data can get extracted.
The script tag gets removed again

The sample I wrote for the Maps launch depends on a PHP proxy. I took that sample and wrote a version that uses JSON instead. Maybe it's because it's new, but for some reason I like the new version better.

All this is not just a hack that might not work again tomorrow because of a browser upgrade. Most of the dynamic ads use the dynamic script tag to display themselfes depending on the context. There are of course ups and downs for both technologies and not everyone agrees, but here are some points to think about.

Proxy method

More robust, error handling is easier
More control on the server side
It has some security implications on the server side as the proxy could be abused.
The server side can have additional functionality implemented that is hidden to the caller e.g. login, exchange secrets...

Dynamic script tag

No XML parsing necessary
Performance win
No traffic gets routed (and counted) on your end.
JSON converters don't know that they should define an array if they is only one nested element in a tag, even if the Schema allows 1..n
More cross-browser issues
Positive impact on rate limiting if it's done per IP
No need to set up a proxy

For more information about JSON, JavaScript and Web services have a look at our brand new Yahoo! Developer Network JavaScript Developer Center.

How to build a Maps Mash-up

Dan — Thu, 03 Nov 2005 07:11:00 +0000

Maps are cool! Not only do they provide better aerial images, what makes a map even more valuable is to add data to it. This HowTo describes how this can be done and how to overcome the XmlHttpRequest cross-domain restriction.

This post will discuss the following parts needed for the Mash-up:

Map API that lets you embed a map into your website
Data source
Ways to get around cross-domain restriction

There are a couple of choices for embeddable Maps out there. The “oldest” one is Google Maps, but there is also Virtual Earth, and today the new Yahoo! Maps launched! This is really good stuff. Yahoo! offers different ways of working with their Maps.

The simple API takes a geoRSS formated XML file and overlays the data on yesterday’s YMaps technology
There are three ways to interact with the brand new Flash Maps. One of them is JavaScript, making it really easy to write super slick applications
The last one is an AJAX API that works pretty similar to what’s already out there.

The sample is located at http://theurer.cc/maps/y/ and you can have a look at the JavaScript code by viewing the page source. It uses the AJAX API and demonstrates the following:

Geocode an address and display on the map
Overlay Traffic information like accidents and construction zones
Overlay local search information

The data sources used are three REST Web services that Yahoo also recently opened up. They are easy to program against as it is just a URL with parameters that returns XML.

Browsers like Firefox and IE enforce a cross-domain restriction, which does not allow HTTP calls from one domain to another. Since REST works over HTTP and the services are not hosted on the same server as the Maps application, we need to find a way to get around that. There are different ways to do that.

Software proxy
Apache mod_rewrite
Dynamically loading .js files via script tag and pass data instead of JS code.

The easiest approach is to write a proxy that lives on the same server as the application and just pass the Request and Response to and from the web server. This has a slight security risk, but the example shows how you could limit it. For our scenario it works perfectly.

Those were the basics. Let’s look at some code! The full source:

The appId in the script is included only for tracking purposes. Please replace it with your own. You can get an appid here. Once you have the code working, we can go ahead and add controls to the map.
//init map var ymap = new YMap(document.getElementById('mapbox')); //add controls ymap.addPanControl(); ymap.addZoomLong(); …

All that and more can be found in the GettingStartedGuide for AJAX. It gets a little more interesting when data sources come into play. For this sample, I demonstrate the easiest way to overcome the cross-domain restriction. A PHP file will serve as a proxy. Yes, this has some security implications but the proxy is limited to a request with a specific (Yahoo!) domain. For the use case of a free Web service, this is “good enough”. This is all that’s needed:

yproxy.php
header("Content-type: text/xml\n\n"); $url = $_SERVER['QUERY_STRING']; //strpos allows only calls to specified endpoint if ( strpos($url,"http://api.local.yahoo.com/") === 0) { $ch = curl_init($url); curl_exec($ch); } ?>

To run the proxy, the PHP Curl extension needs to be enabled.

Now that the proxy is in place we are able to make a REST call to api.local.yahoo.com. I want to follow what happens if the ‘AddLocation’ Button is pressed. The sample has an input box where the user can input an address.



 
value="5th Avenue,New York" size="50">

 - enter an address

Once the button is pressed, the function addGeocode() gets called. Yes, the name could be better… and what it does is assemble the REST query and call the function that actually calls the proxy.

var query = document.getElementById("geoquery").value; var uri = "http://api.local.yahoo.com/MapsService/V1/geocode? appid=dantheurer&location=" + query; callWS(uri, "geocode");

The callWS() does an XmlHttpRequest to the proxy with the REST URL as parameter and displays ‘Loading’ while it waits for the proxy to complete the request. The PHP proxy uses CURL to make the actual request, and the callback method parseResult() gets executed once the call has succeeded.
var url = 'yproxy.php?' + encodeURI(target); xmlhttp.open('GET', url, true); xmlhttp.onreadystatechange = function() { if(xmlhttp.readyState == 4 && xmlhttp.status == 200) { document.getElementById('result').innerHTML = ''; parseResult(xmlhttp.responseText, service); } else { document.getElementById('result').innerHTML = "Loading..."; } }; xmlhttp.send(null); }

From here, all that’s left is parsing the XML and adding the markers. In this sample I used the xmlw3cdom.js parser from SourceForge. It’s more than I need, but it makes things really easy. This DOM parser requires the following SAX parser xmlsax.js to deal with the XML object. I will not discuss how to parse the XML, because it’s easier to just look it up in the source file.

The results are used to create a Point and a String for the Smartwindow, which get passed to the createYMarker() method.

var point = new YGeoPoint(parseFloat(lat),parseFloat(lon)); var smart = "Long: " + lon + " " + "Lat: " + lat; var myMarker = createYMarker(point, i, smart);

It creates the Markers on the map.

var marker = new YMarker(point,'id'+number); marker.addLabel(number); YEvent.Capture(marker,EventsList.MouseClick, function() { marker.openSmartWindow(msg) }); return marker;

This is a pretty simple example but this can be the foundation for a much more complex mash-ups. Once this part is done, it only gets better.

Keep mashing!