Enter your contact information.
Enter your project information.
Step 3Choose the PHP programmers you want to hire.
Compare Free PHP Quotes
Who Swears the Most? How Foursquare Used Hadoop to Find Out
*Attention PHP Programmers* - Add A Free Listing! Get more clients.
We told you who swears the most in their code, but what about in the real world? Foursquare, the location check-in service, has used its rather large dataset to graph the “rudest” places in the English-speaking world — Manchester, U.K. takes top honors.
While the results should be taken with a grain of salt — after all the swearing is limited to Foursquare users and there’s no hint of what constitutes a swear word — the methods Foursquare used to get the data make a great intro to the world of Apache Hadoop and Apache Hive.
Hadoop is an open-source MapReduce framework — a way of processing huge datasets stored in large server clusters (or grids). While MapReduce frameworks were originally introduced by Google (which has very large datasets to work with) they’ve since grown beyond Google and their usefulness isn’t limited to large companies with massive databases.
In fact, with Amazon’s Elastic MapReduce just about anyone can easily and cheaply run their own Hadoop framework and process vast amounts of data just like Google does.
Because word search processing is generally considered the canonical example of what makes a MapReduce framework useful, Foursquare’s blog post offers a good overview of how you can use MapReduce to mine through anything from large text documents to user-contributed data like the check-in snippets Foursquare is processing.
Foursquare’s server setup is specific to them, but there’s one key element that’s worth bearing in mind — store your Hadoop data well away from your production system. MapReduce doesn’t work at the speed of the web and you don’t want it dragging your site down.
In Foursquare’s case that means using Amazon’s Elastic MapReduce plus a simple Ruby on Rails server. The result is, as Foursquare Engineer Matthew Rathbone puts it, “a powerful (and cheap) data analysis tool.”
If you’re new to MapReduce and functional programming in general, read through the Foursquare post for an overview on how MapReduce is useful and then check out the Hadoop site, as well as this overview video from Cloudera.
See Also:
- Cussing in Commits: Which Programming Language Inspires the Most Swearing?
- XSS Vulnerabilities, Raw SQL Top List of Common Programming Errors
- Commenting Your Code — What’s Too Much, Too Little?
Tue, 01 Mar 2011 17:07:22 GMT
Tags: Frameworks, Location, Programming,
*Attention PHP Programmers* - Add A Free Listing! Get more clients.
Frameworks
Twitter Adds Responsive Design Tools to Bootstrap 2.0
Twitter's increasingly popular Bootstrap web development toolkit is just about to hit 2.0. With this release web developers will be able to use Bootstrap's new responsive tools to easily create flexible layouts for tablets and phones.
Behind the Scenes at Instagram: Tools for Building Reliable Web Services
Want to build a web service that sells for a cool billion dollars? Instagram's developers have some advice for you: Choose your tools wisely.
Location
Google Street View, Coming Soon to a Living Room Near You
It’s time for Google to rename its Street View feature. Google Maps’ Street View is no longer limited to streets, the company is now using tricycles to photograph off-road locations like the gardens at the San Diego Art Institute or Château de
OpenDNS and Google Working with CDNs on DNS Speedup
A group of DNS providers and content delivery network (CDN) companies have devised a new extension to the DNS protocol that that aims to more effectively direct users to the closest CDN endpoint. Google, OpenDNS, BitGravity, EdgeCast, and CDNetworks are a
Programming
Cussing in Commits: Which Programming Language Inspires the Most Swearing?
As any programmer can tell you, programming will make you swear. But did you know that writing C++ will make you swear considerably more than PHP or Python? Developer Andrew Vos was looking for a weekend project when he decided to grab some one million co
Amazon S3 Storage Now Handles Entire Websites
Cheap, cloud-hosted web servers are a key component of a distributed web. But sometimes you don’t need a server, you just need a cheap way to host your static files, like images and videos. That’s the gap Amazon’s S3 service has long fil
Professional website development and design, e-commerce solutions, specializing in the LAMP stack.
San Fernando Valley, California US
PHP/MySQL Developer
Zrenjanin, Vojvodina RS
We provide affordable php website design & SEO services to our clients. Our HTML is search engine friendly and builds a solid foundation for your SEO & marketing campaign. WE provide free estimates.
New Port Richey, Florida US
Perlman Creative Group is a Los Angeles based, full service graphic design firm specializing in marketing communication, brand development, logos, packaging, corporate identity and website design.
Marina Del Rey, California US
Serving North America, we offer high-end graphic, website, and print design services. We also offer 3D illustration as well as traditional illustration.
Livermore, California US
Slide Show Time: Building a Future-Friendly Web
Embedded above is an excellent presentation by Brad Frost. Below you can find a video that goes alon
Google Adds Google+ Info to Gmail Contacts
Google now includes Google+ profile information within contacts lists, both in Gmail and in the stan
The Un-Internet
The tech world is in an infinite loop. I’ve written about it so many times, but that’s
HTML5 Video on the Web Today
The hype surrounding HTML5 video may have died down, but as a recent report on the state of HTML5 v
Test Drive Firefox for Android Today
Mozilla has released Firefox for Tablets. The currently nightly build is still very rough around the
Article Tags
PHP Programmers Articles
Live Help Chat Software
Web Design Quote
Web Design