Posts

K Means Clustering

Image
K means Clustering is one of the simplest and most commonly used unsupervised clustering algorithms around. The general approach is as follows: Choose k centroids randomly. Calculate the distance from each point in the dataset to be classified to each centroid. Assign each point to the nearest centroid. Calculate the centroids of the resulting clusters. Repeat until the centroids don't move too much. Here is some R code which  generates a data set and implements the algorithm. Click here to see the animation. ########################################### # R code to implement k means classification ########################################## # NB - to make the animation - make sure you have ImageMajick installed from http://www.imagemagick.org/ ##########################################   # initiate libraries library ( animation )   # set working directory setwd ( 'C:/Users/RF186004/Desktop' )     ######################################### ...

My next talk on in database analytics

http://www.meetup.com/R-Users-Sydney/events/85449232/

Forthcoming talk

I will be giving a talk at University of Technology Sydney this Friday - see http://datasearch2.uts.edu.au/analytics/news-events/seminar.cfm for details.

Cloudera Hadoop Developer certification

Just passed my Cloudera Hadoop Developer certification exam - hooray!

Reduce the Pain of Map Reduce

Find out how to painlessly write map reduce functions in my new blog post http://blogs.teradata.com/anz/reduce-the-pain-of-mapreduce/  

Create a pdf data dictionary from a dataset

Here is some R code you can use to create a pdf data dictionary from a dataset. It's a quick and easy way to get an overview of a dataset. You need to have a latext program like MiKTeX installed to make it work.   You can see the sort of output it produces here . ####################################### # code to produce a pdf data dictionary of a dataset using sweave # # df: the data frame you want to create the data dictionary of # file_loc: the location where you want the resulting report stored # filename: the name you want the report to have # author: the name of the person who created the data dictionary # #######################################     data_dict <- function ( df , file_loc , file_name , author , num_lev = 2 ) { # initiate libraries library ( tools ) library ( ggplot2 ) library ( gplots ) library ( lattice ) library ( sqldf )   #Make some subsidary functions   # initial funtion to print without q...