Reduce the Pain of Map Reduce

Find out how to painlessly write map reduce functions in my new blog posthttp://blogs.teradata.com/anz/reduce-the-pain-of-mapreduce/ 

Comments

Post a Comment

Popular posts from this blog

Why Study Shakespeare?

Published in  Sydney's Child,  August 2003 Many of us have mixed feelings when we think of studying Shakespeare at school. Such recollections can range from the depths of boredom and anguish to the heights of sublime joy and intellectual awakening. But regardless of our opinion of Shakespeare, we speak his language everyday. It is estimated that Shakespeare added around 1500 new words to the English language. Whenever we use our mind’s eye, to find method in someone’s madness, as they eat us out of house and home, because we thought they had a heart of gold and a spotless reputation but we were actually living in a fool's paradise; whenever we decide that discretion is the better part of valour or detect something in the wind; whenever we remark that brevity is the soul of wit, that  love is blind or caution someone that all that glistens is not gold or advise someone to be neither a lender nor a borrower;  from the salad days of youth, throu...
Read more

K Means Clustering

Image
K means Clustering is one of the simplest and most commonly used unsupervised clustering algorithms around. The general approach is as follows: Choose k centroids randomly. Calculate the distance from each point in the dataset to be classified to each centroid. Assign each point to the nearest centroid. Calculate the centroids of the resulting clusters. Repeat until the centroids don't move too much. Here is some R code which  generates a data set and implements the algorithm. Click here to see the animation. ########################################### # R code to implement k means classification ########################################## # NB - to make the animation - make sure you have ImageMajick installed from http://www.imagemagick.org/ ##########################################   # initiate libraries library ( animation )   # set working directory setwd ( 'C:/Users/RF186004/Desktop' )     ######################################### ...
Read more

Welcome the Edupreneurs

Published in Policy Magazine , Spring, 2005. A footnoted version of this essay is available on request. At the beginning of Pierre Boulle’s classic novel The Bridge Over the River Kwai, the commanding officer of the British prisoners-of-war, Colonel Nicholson, asks his Japanese counterpart, Colonel Saito, to reconsider his command that the British officers work alongside the other ranks to construct the bridge. Unbeknown to Nicholson, Saito has a deep seated antipathy to anyone questioning his authority. In a paroxysm of rage he beats Nicholson senseless and inflicts a punishing work regime on the rest of the prisoners. Saito’s reaction reminds me, in vehemence if not in kind, of the response of the education bureaucracy whenever the words “profit” and “education” are mentioned in the same sentence.  Teachers’ unions in particular are vehemently opposed to “edupreneurs”,   companies which wish to provide service for pr...
Read more