Free APIs, online services to generate screenshots of web pages

I've graded the answers to this StackOverflow question about APIs & online services to generate screenshots of web pages based on whether the service can be used directly without any API key or require a key:

No key required:

• immediatenet.com
• kwout API (generating screenshots is not the primary intent of kwout so it doesn't directly give you an image to save)

Requires (free) API access key:

• Browshot API
• GrabzIt
• thumbalizr API (Simple syntax to use the API: http://api.thumbalizr.com/?url=http://mvark.blogspot.in&width=250)
• ConvertAPI (Free 500 one-time credits)

There are also online tools can generate screenshots of websites:

• Browsershots.org
• WebShots Pro
• ShrinkTheWeb
• Amit Agarwal's Screen Capture tool
• GrabzIt

Read More

Self-awareness software

A proponent of the Quantified Self movement and the co-founder of Foursquare, Naveen Selvadurai, has published a "personal API" consisting of his day-to-day personal details (related to his sleep, weight, steps, fuel/activity and checkins) as a step towards building self-awareness software. He calls it api.naveen. The data therein has been normalized from various sources: manual capture, nike fuelband, jawbone up, withings, fitbit, foursquare and various apps.

Image from the Human API website

The Quantified Self movement is based on the premise that by measuring and analyzing behavior, one can improve it. You’re less likely to splurge on french fries, say, if life is a game of improving your fitness and all the apps on your homescreen are keeping score.

This GigaOm article informs that there are projects like the Human API ("all human data streams in one open API") which help people find value in their personal data.

Read More

HOW TO show local time for a location based on its latitude and longitude

Here is a GitHub Gist snippet that shows how to get local time for a location based on its latitude and longitude using World Weather Online's Time Zone API & jQuery -

This can be useful in scenarios where you have to display time for a location relative to another.

Read More

The World Citizen News mash-up

Our latest app, World Citizen News gets you:

• the Twitter buzz
• weather forecast, local time & map for popular cities around the world
• images of selected location from flickr
• news based on the trending Twitter topics

It makes use of the following APIs:

• Twitter Trends
• flickr
• World Weather Online for the local time
• MapQuest 
• Yahoo Weather via YQL
• Google News

Whether you're a world citizen or a geography buff, try the World Citizen News mash-up to see what's happening around the world.

The following links explain how different features of the mash-up were built:

• Weather information for a place based on its WOEID using YQL & jQuery
• Finding the latitude, longitude of a location based on WOEID
• Local time for a location based on its latitude and longitude
• Display book titles in Amazon ads dynamically based on context or custom keywords

Read More

Yahoo! Weather report for India

In my last post I showed how to find weather information for a place based on its WOEID using YQL & Yahoo! Weather API. The Yahoo! API will get you weather info for ANY place that has a WOEID - including a country. How do you generalize weather conditions for a country as big and diverse as India?

The current conditions description (Smoke!) made me chuckle.

Also see:

• JavaScript: The Good Parts - photo gag
• "Had to scramble to get dressed when someone wanted video on a call."
• The trouble with Kindle is...

Read More

HOW TO find weather information for a place based on its WOEID using YQL & jQuery

There are many free Weather APIs that can fetch your weather details for a location but if you had to fetch it based on WOEID, you can turn to YQL or the Yahoo! Query Language & Yahoo! Weather API. Interestingly, unlike other free Weather APIs, YQL can also return JSONP results over HTTPS.

Here's the code (from my first GitHub Gist) -

The default unit of measurement of the temperature in the results returned by the API, is Fahrenheit. You can specify "c" or "f" for the unit parameter to get results in Centigrade and Fahrenheit respectively. Note that lower-case "c" or "f" has to be used.

Also see:

• Screen scrape with jQuery, AJAX, JSONP & YQL
• How can I bulk query the Yahoo Weather API for more than one location at a time?
• StackOverflow Yahoo Weather API, YQL Q & A

Read More

HOW TO find the latitude, longitude of a location based on WOEID

Yahoo! has devised a unique identifier called the Where on Earth ID, or WOEID - for almost any named place on the Earth. This is used by APIs like Twitter Trends, flickr etc.

However, a majority of geo-location related APIs require latitude, longitude of a place to provide their services. So if you're building a mash-up that has to deal with latitude, longitude and WOEID, you can resolve a WOEID to a place with this call:
http://where.yahooapis.com/v1/place/2507854?appid=[YourYahooAppidHere]

Replace, 2507854 in the above example with the WOEID of the location whose co-ordinates you want to find.

The results can be requested in XML or JSON format.

Read More

Search for free eBooks from the Google Books collection

I & my wife have a fascination for Web APIs that return results in JSON or JSONP format. We noticed that Google Books doesn't readily show if a out of copyright book is available for free download in PDF or EPUB format. You have to go through multiple steps using Advanced Search to find this.

Luckily, Google provides a Google Books API that exposes this information and it returns the results in JSON format. We wrote a web app that searches for free eBooks from the Google Books collection.

It uses a responsive HTML5 Boilerplate template generated using Initializr.

The H5BP template's view for Mobile

Some other interesting things we discovered while building this app:

• Results returned by Google Books API may vary by country. When a programmatic request is made to their book search API via the app, the same code will return different results for different countries. 
• Project Gutenberg doesn't have an API but they provide one huge file that contains the RDF data for all books. 
• Open Library has a RESTful API that provides useful information about books.
• Calibre is free and open source e-book software that can convert ebooks into many formats 
• Flipkart lets you download over 9000+ ebooks for free but you will need their app to view. 
• Programmable Web maintains a directory of Web APIs & tracks news about mashups built using those APIs. The site is a treat for developers who like Web APIs & creating mashups.

Read More

Firebug simplifies programming with JSON

The Developer Tools (F12 keyboard shortcut) of popular browsers show the JSON output returned by an API, like this -

Firefox add-on Firebug's JSON tab within the Net tab, presents it neatly like this -

This makes it easy to extract the specific details that you are after. 

The above view helps to visualize how to get to a flickr photo's id on making a call using the jQuery getJSON method to one of the flickr API services - data.photos.photo[i].id

Read More

HOW TO handle a API's JSONP response that uses a static or fixed or named callback function

Script and JSONP requests are not subject to the same origin policy restrictions. Most APIs that return JSONP data will generate a dynamic callback function name and they typically have a parameter like jsoncallback=?.

However some API's like flickr, Facebook require that you use the callback function name that they specify.

Here are a couple of ways that you can tackle that scenario -

window.fixed_callback = function(data){
  alert(data.title);
};

$(function() {
  $.getScript("http://api.flickr.com/services/feeds/photos_public.gne?tags=cats&tagmode=any&format=json&jsoncallback=fixed_callback", function(data) {
  alert('done'); } );
});

click to enlarge code snippet

Read More

A geography game made from a mashup of Picassa & Google Maps

Where in the World? is a simple game that is a mashup of Picassa & Google Maps. This geography game lets you view public photos stored by users around the world on Picassa and guess where they were taken.

There is no information on how exactly points are scored. The closer your guess is about where a geo-tagged photo is shot, the more points you win. I noticed that if your guess about the location is close to 150 kilometers, you get 600 points while there are no points if your guess is more than 5000 kilometers away.

Each round has 5 different photos and you can play unlimited rounds. There is hardly any element of competition. The overall score is not logged.

The game could have been better designed and also adapted as a mobile app.

GeoGuessr is a similar geography game. Wonder which came first?

Read More

HOW TO extract Wikipedia content through its API using jQuery

Wikipedia's API or more appropriately MediaWiki API,  lets you programmatically fetch content & metadata related to thousands of its articles.

I built a small custom Wikipedia search application - Q-Wikipedia, to get just the first paragraph of a Wikipedia article using jQuery with help from online resources.

The REST-based Wikipedia API has many configurable options. It's easy to select the options you require using the Wikipedia API Sandbox

As I need only the first paragraph I set the value of the sections parameter to 0 (default is all)

http://en.wikipedia.org/w/api.php?action=mobileview
&format=json&page="+ keyword + "
&redirect=no&sections=0&prop=text
&sectionprop=toclevel%7Clevel%7Cline%7Cnumber%7Cindex%7Cfromtitle%7Canchor
&callback=?

It has auto-completion of search terms, which was a breeze to implement using jQueryUI.

Super-script and a few other tags are removed from the Wikipedia content derived from the JSONP payload prior to injecting it into a DIV placeholder -

$('#wiki').html(json.mobileview.sections[0].text)
.find("dl,sup,.thumb,table.infobox,table.metadata,div.dablink").remove().end();

Try it out and let me know your feedback by posting a comment below.

Read More

Get fresh news fast from Twitter and Google News JSONP feeds using jQuery

Twitter is a good source for raw local news. Google News aggregates content from publishers around the world. Both of these services provide JSONP feeds.

Try out my search application written in JavaScript that gets the latest tweets and news from these JSONP feeds. As it is all in JavaScript, you can view the HTML source to see how it works.

Resources you may need if you want to adapt it for your requirement -

• Infinite scrolling with JQuery, AJAX & Twitter
• Google News Search JSON API Developer's Guide
• Twitter Search API Reference
• Using JSON to Exchange Data

Read More

Twitter Search Tricks

Did you know, you can search for tweets on a topic that convey a positive or negative attitude/reaction or pick only tweets that contain questions related to that topic?

Links within a tweet indicate there is some kind of a "story". If you are interested in looking specifically at tweets in you timeline that contain links, use this link in the address bar - https://twitter.com/search/timeline/http

You can also try this - https://twitter.com/#!/search/timeline/filter%3Alinks

On the other hand, if you're interested in tweets/conversations in your timeline without links, use this URL - https://twitter.com/#!/search/timeline/-filter%3Alinks

If you feel the count of followers is a measure of popularity of a Twitter user, you can search by tag on the WeFollow website to find folks who are are most influential.

The following search queries (annotated & highlighted for emphasis) from the Twitter documentation show how you can use other operators (from:, to:, place:, filter:, source:) and keywords while searching on the Twitter website -

• twitter search - containing both "twitter" and "search". This is the default operator
• love OR hate - containing either "love" or "hate" (or both)
• from:twitterapi - sent from the user @twitterapi
• to:twitterapi - sent to the user @twitterapi
• place:opentable:2 - about the place with OpenTable ID 2
• place:247f43d441defc03 - about the place with Twitter ID 247f43d441defc03
• superhero since:2011-05-09 - containing "superhero" and sent since date "2011-05-09" (year-month-day).
• twitterapi until:2011-05-09 - containing "twitterapi" and sent before the date "2011-05-09".
• movie -scary :) - containing "movie", but not "scary", and with a positive attitude.
• flight :( - containing "flight" and with a negative attitude.
• traffic ? - containing "traffic" and asking a question.
• hilarious filter:links - containing "hilarious" and with a URL.
• news source:tweet_button - containing "news" and entered via the Tweet Button
• India source:metrotwit - containing "India" and entered via the MetroTwit app

There are also other operators that you can use both through the site's search interface or programmatially using the Twitter API.

Also see:
HOW TO add hashtags while auto-publishing posts to Twitter
Archive tweets & favorites with Google Reader

Read More

HOW TO work with a JSON/JSONP feed locally

When you're working with a JSONP feed  like the one returned by Twitter Search API or Google News, the different results that show up each time may distract you from the functionality that you're building. You can instead work with a copy of the JSON resultset locally and hook up the code to the actual JSONP feed when you're done.

A JSONP feed is just JSON-formatted response wrapped in a function call. You can grab the JSONP content generated by an API by going to the Network Tab within Developer Tools (F12 keyboard shortcut) of Chrome or IE or Firebug within Firefox.

When a JSONP feed is called through jQuery, it will look something like this -
jQuery17109422175763174891_1337695981767({json})

To work with the dynamically generated JSON, we need the content within the brackets - {json}

Save that content within a text file & give it some name & a ".js" extension. To use the JSON content locally, specify the file name as the first parameter to the $.getJSON method -

$(document).ready(function(){
    $.getJSON('tsearch.js', function(response){
      for(var i=0;i<response.results.length; i++){
      $('#tweets').append("<li>" + response.results[i].text + "</li>");
    }});
});

The above example refers to  JSON returned by Twitter Search API.

Also see:

HOW TO show expanded form of shortened URL within a tweet using Twitter API 
HOW TO convert a RSS feeed to a JSONP feed on the fly

Read More

HOW TO visualize approximate radial distance from a fixed point on a map

The Twitter Location Search app utilizes Google Maps to show approximate radial distance from a specified point on a map

If you are an API developer, this tool can be useful to visualize approximate radial distance while dealing with location-specific queries.

The Twitter Search API supports a parameter called geocode that returns tweets by users located within a given radius of the given latitude/longitude for a given search term.

The parameter value is specified by "latitude,longitude,radius", where radius units must be specified as either "mi" (miles) or "km" (kilometers).

The maximum limit for the radius is not specified in the API docs but the Advanced Search feature on Twitter has 1000 as the upper limit.

Read More

Google Maps Driving Directions gadget - useful for a "Contact Us" page

Google organizes "the world‘s information and make it universally accessible and useful" but info about some of its own products may be a little difficult to find. If there was a reference to Driving Directions gadget & the Static Maps wizard on the Google Maps API Family page, they could be easily discovered. As these useful utilities exist on orphaned pages and are hard to find, many developers may go on to create their own components.

The Google Driving Directions gadget is a nifty utility that organizations can configure for their websites to show on a map where they are located. More importantly, it optionally lets users type their location & then shows the travel route visually & through textual instructions.

Read More

Say goodbye to Google Translate, Transliterate & ten other APIs

About a fortnight ago, Google highlighted on its official blog how "two clever Translate trends caught our eye—perhaps one of them will inspire you to come up with a fun Translate trick of your own.". Folks around the world were making fun remixes

First, some creative folks translated strings of consonants into German to create a new beatboxing tool. The phrase “pv zk bschk” didn’t initially make much sense to us, but a quick listen got us nodding our heads along to the beat.
Now it seems there’s a similar trend in Taiwan: using the spoken output of Google Translate as the vocals for self-composed songs or video spoofs.

A few days ago there was an announcement on the Google Code blog that the Google Translate, Transliterate & ten other APIs will be deprecated. The Google Translate API will be shut off completely on December 1, 2011 and the reason is "substantial economic burden caused by extensive abuse".  So what constitutes "fun Translate tricks" and abuse? Why not just debar developers abusing the API?

An aggrieved developer ranted thus in the comments section of that announcement -

you are supposed to be the smartest guys on the planet and the only solution you can come up is to shut it down? thats a joke, but the joke is on us, the developers.
with this move you just showed us what you really: a company that does not care about developers, you just want to become more like facebook and apple. oh what a great world we live in. more walled gardens. 

Also see - HOW TO monitor performance and availability status of public APIs & websites

Read More

HOW TO monitor performance and availability status of public APIs & websites

If you use Public APIs (like Bing Search, Google Maps APIs) in your applications, it helps to be aware of performance and availability issues that those services may face.

Thankfully, there are some free tools that monitor popular APIs & websites -

• Api-status.com tracks 40+ Public APIs every five minutes from one of its 49 global monitoring stations.
• Social Down Or Not tracks uptime and performance status of social networks
• Neustar Webmetics Lab project Crowdsourced Monitoring tracks the health of popular services by monitoring Twitter for mentions of downtime or performance problems. (broken)
• Google Apps Status Dashboard shows performance information of its consumer services as well as services for organizations using Google Apps.
• Facebook has a Platform Live Status page that shows current status & reports issues that arise.
• Amazon Web Services Service Health Dashboard shows up-to-the-minute information on service availability & also RSS feeds for its multiple services across continents.
• A free Pingdom account lets you monitor one website or server for uptime & response time. It provides 20 free SMS alerts while all email alerts are free.
• Windows Azure Service Dashboard provides status of its various services across its data-centers round the globe.

Also see:
How LARGE websites manage performance & scalability

Read More

Screen scrape with jQuery, AJAX, JSONP & YQL

Since reading this excellent article about scraping content from a Wikipedia page using Yahoo! Query Language (YQL) as a proxy for cross-domain Ajax, I'm hooked to YQL. YQL helps in circumventing the same-origin policy that prevents a script loaded from one domain from getting or manipulating properties of a document from another domain.  YQL has been around for about 2 years now & last year Yahoo introduced the capability to execute the tables of data built through YQL using JavaScript.

Ajax, jQuery, JSONP (JSON with Padding) & YQL make a heady combination - check Christian Heilmann's code samples.

Some facts about YQL from around the Web (work in progress) -
* YQL is a hosted web service that can scrape HTML for you. It also runs the HTML through HTML Tidy and caches it for you.
* It only returns the body content of the HTML - so no styling (other than inline styles) will get through.
* ...it treats the info on the web as a virtual table that developers can manipulate in a standardized way, regardless of the API that data came from.
* YQL understands and supports data sources like RSS, Atom, JSON, XML, CSV, HTML, Flickr, Yahoo! Finance, Weather, and so on.
* ...makes client-side mashups possible without using server-side proxies.
* Usage Limits:
Per application limit (identified by your Access Key): 100,000 calls per day
Per IP limits: /v1/public/*: 1,000 calls per hour; /v1/yql/*: 10,000 calls per hour

Also see:
HOW TO prevent screen scraping 
Google Spreadsheets functions for scraping external data

77

Read More