Saturday, November 17, 2012

Black friday 2012 deals - google spreadsheet excel

Excel is a fantastic tool to search and maintain a list.. Instead of scanning through multiple print adds, a single spreadsheet can be a nice tool to compare and to find actual deals

I will keep this file updated until Blackfriday with latest deals

Created a spreadsheet of all popular mart blackfriday deals..

Only the popular deals ( about 500 items )


https://docs.google.com/spreadsheet/ccc?key=0AlP4buxzoaxsdExjMkM2WmJBcFZ5TlExTFMyWU1Oc1E


If you have certain product or store or category use this detailed list ( more than 5000 items )

complete list


https://docs.google.com/spreadsheet/ccc?key=0AlP4buxzoaxsdFBteHJwMjFlczVESUlTNXNNbEtuMXc

Google spreadsheet has a search filtering option for all the columns.. it is very easy to search and compare price of a particular item between different stores ..

Please download a local copy as XLS or pdf or any other format  from "file > Download as" 
using the menu bar..


Friday, November 2, 2012

top Travel movies -- My favorite genre

One of my colleagues , he is actually from Mediterranean . but he usually talks in detail about India, like he studied the country. Later he revealed about the travel he made in India during his early 20s with his friends. They had minimal money and certainly not enough to travel in planes and stay at palaces. This makes their travel intriguing and interesting.

Me.. I always made planned travels, maybe I dont have the heart to face the uncertainty of unplanned travels.. this uncertainty and exploration makes a travel movie very interesting..


Motorcycle Diaries
       this movie can easily takes the top spot of travel movies .. I like the south american scenic beauty and Che's friend who traveled with him. How about travelling a continent with a close friend on a bike ?


Zindagi na milega do bara (Hindi)
       Although it has the Hindi masala factor still an enjoyable travel movie. Friends exploring themselves on a aimless travel to spain.


Kikujiro (Japanese)
       What about travelling with a kid aimlessly . this is a funny and Im sure it touches everyone watches it, I think its because the boy's life resembles some of our childhood.


Le Grand Voyage (French)
       How special it is to travel with a father on his pilgrimage. I know the idea is no fun when he is alive , but I didnt have the oppurtunity to spend this quality time with my father . Im longing for a travel which I can never start.


Up
       Risking a travel at >60 for a petty promise he made to his wife as a kid.. Although this is animation this has lot of heart

 Into the wild 
       slow movie , can be a time for self exploration ... although I hate the idea of running away from humanity

127 Hours 
        Good music and plot

 Finding Nemo
        A fathers search for his son with the help of weird friend.. a must watch I would say

Thursday, November 1, 2012

Pentaho solution for Sqoop call with dynamic partitions

Pentaho big data release doesnt have a step for SQOOP as of this writing..
Simple solution is to use a "Shell script" stage to call sqoop..

But our requirement has a twist .. and most of sqoop users might have as well..
- capture incremental data for certain tables and keep it in a new hive partition
- Run the sqoop extract for a window of dates where each day's data goes to a partition (DAILY , HOURLY, MONTHLY etc)

Design


The solution is to use a DB for configuration and pentaho to frame sqoop calls

JOB


We are calling the pensqoop transformation to frame the list of sqoop calls to run
shell stage actually runs the sqoop script.. here check the "execute for each input row " so that this script will be called for each sqoop call framed in previous step.

TRANSFORMATION



1. get table configurations (like db connection, incremental column to use for incremental data capture etc)
2. switch the flow for DAILY partitions or FULL refresh
3. frame sqoop call using string manipulation at javascript stage
4. "copy rows to result" to send the output to calling job