Image Matching

I had been toying around with the idea on extending Video search to include the audio and video (image) data associated with the video in addition to the user specified tags. This is one of my weekend projects, others being the semantic representation of knowledge and android app development.
This weekend I spent some of my time get out a rough cut of the "Image Indexing using Color Correlograms" [the same with my notes] which could be helpful in my Video Search project. The principle behind an image correlogram [Correloation + histogram] is the spatial correlation between the image colors. A correlogram can be defined as f(c1,c2,k), where f is the number of pixels of color c2 around a pixel of color c1 which are at a distance of k from the c1 colored pixel.
The results looked promising. At present I used only the G channel to generate the correlogram, and I would be refining it over time.
Some results : I chose come images obtained by the query: road via google image search, calculate the correlograms for each and then used the metric described in the paper to find the closeness. The number adjacent to each image describes how close it is to Image 1. Lower the number the closer the image is to Image1

Image 1:

Image 2: 16.32

Image 3: 4.84

Image 4: 9.51

Image 5: 15.06

Image 6: 17.03

Image 7: 15.05

The closest images to Image 1 are Image 3 and Image 4, which looks intuitive.
Image 5 and Image 7 have similar correlograms which also seems fine.
But the observation that Images 2,5,6 and 7 are at almost equidistant to Image 1 is not very palpable. Especially as Image 2 is no way related to Image 1. On the other hand, it is not possible to draw inferences on the robustness of this approach with such a small set of test images. I'd be do some more research and come up with enhancements to get better results.

At present I just used the green channel of the image in the creation of the correlogram as histogram of the green channel is closet to the luminosity histogram. [ The reason for this is that the human eye is more sensitive to the color green than any other colors]. The current algorithm calculates f using all the pixels in an image and computing f is quite expensive O(n^2*d) . [The complexity has been mentioned to be O(n^2*d^2) in the paper and I will try to clarify this with the authors]. The space complexity is O(m^2 * d) [where m is the number of possible colors] Further improvements in time and space can be made by calculating the correlogram for select regions of the image (high gradient blocks).

btw, it took me quite some time to set up a development environment in windows.
I used the opencv sdk for reading the images, eclipse IDE with CDT, and mingw (gcc for windows)for the compiler.


Some Informative links that I followed:
opencv
http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/cvision.htm
http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html#SECTION00041000000000000000
http://www.xpercept.com/opencv.htm

eclipse cdt:
http://www.cs.umanitoba.ca/~eclipse/7-EclipseCDT.pdf

updates:
Don Dodge on video search

Autorickshaw economics

Today, while I was returning from a Shopping mall in bangalore in an Autorickshaw {Autorickshaws are 2 stroke, 3 wheelers which are a common mode of transportation in India. [Equivalant of taxi's in the US]} the autorickshaw driver said something to me in Kannada [local language]. I couldnt understand it, but, out of curiosity, I said "Nannage Kannada swapla swalpa gotthu; Hindi, English, Telugu". {this is what I usually say when someone talks to me in Kannada - I know very little Kannada. These are the languages I know}
The driver (about 25-30 years in age) then told me that he had been waiting for a passenger from 1 o'clock, and he did not get a single passenger till the time I asked him to take me home [It was 3:15. He had been idle for 2.25 hours!]. He also tole me that, he would have gone home if I hadn't boarded his auto.
The driver seemed friendly, hence I asked him some more questions. The information that I got is as follows:
mileage of an auto - 20 km per litre of petrol (gas). This costs him 2.5 Rs per km.
The standard charge is 6 Rs per km, thus he makes a profit of 3.5 Rs per km.
He travels close to 80 km every day [excluding the "idle" rides]. Thus, he earns around 280 Rs [7 USD] per day i.e. 2,600 USD per annum . That is pretty low as compared to and average engineer's salary [12000 USD per annum].
I can help these guys by getting them passengers. Using mobile phones to connect the prospective passengers to the idle auto rickshaws. I'll post the details of my idea soon.
Now, back to android!

I like this, I'd find it there

WHAT DO I LIKE:
1. I'd like to do something where I could always learn something new.
- Travel [learn history / geography / cultures]
- Photography
- Reading Papers / implementing them
- Reading Blogs / Google Videos

2. Something that I feel that I can make a difference
- Semantic web / agents
- 3D Reconstruction
- President [MEA events / publicity]
- SEEK Teaching

3. Something where I could motivate others / teach others
- SEEK
- TA ship
4. Something where I find solutions to real problems
- planning a tour
- building an mp3 jack to plug in the iPod to the car system
- finding out a way to have the minimum trips to dispose garbage

All this with enough money so that I could pursue my interests and hobbies [travel / photography / etc ]

Where can I find all these ?
Working on Innovative / hard (not necessarily) and open ended projects
- Research Labs in Academic Institutes
- Open Source Projects
- (Google) Research Labs [Rich]

What do I do to get what I want.

Personal Productivity

I thought that it was time to rethink over the way I work, prioritise tasks and implement them.
Hence, I read up a few things on the web.
Found a couple of good resources and suggestions.

I have bookmarked the links at:

http://del.icio.us/amirivija/Productivity

I also subscribed to a couple of blogs on productivity.

Here are a few notes I made for myself based upon what all I read:

Things to implement:
1] Keep a record of the start and end times of different activities
2] Cut down e-mail alerts. cut down IM. Look at the e-mails only 4 times a day. [Morning, After Lunch, Before Leaving - set aside time for rss feeds ets]
3] Clarify objectives, before starting any work
4] Break down the big problem into smaller ones

5] Work tends to expand in the time available.. [so compartmentalize the work] Hence, shrink the time you are going to work.. [ Plan to work only for 5 hours per day]

6] Stop multi tasking..

7] Do the task that gives the most benefit.

8] Join a group to keep you motivated
1] Apping group
2] Build an online programming network

9] Picture of a goal : expert computer science engineer

10] be patient

11] seek inspiration - blogs, people , articles

12] never skip anything two days in a row

13] Apply the ""do it, delegate it, defer it, drop it"" rule for all the stuff that' clogging your system

I also stumbled across the book "Getting Things done" by david allen. I would like to read this book too.

This one also has pointers to some good stuff
http://zenhabits.net/2007/02/beginners-guide-to-gtd/

And last but not the least, I reorganised my Google reader feeds

I am a Kinesthetic Learner

This is what http://www.metamath.com/multiple/multiple_choice_questions.html has to say about my learning style.

The results of Amirisetty Vijayaraghavan's learning inventory are:

Visual/Nonverbal 30 Visual/Verbal 28 Auditory 24 Kinesthetic 36

Your primary learning style is:

The Tactile/ Kinesthetic Learning Style

You learn best when physically engaged in a "hands on" activity. In the classroom, you benefit from a lab setting where you can manipulate materials to learn new information. You learn best when you can be physically active in the learning environment. You benefit from instructors who encourage in-class demonstrations, "hands on" student learning experiences, and field work outside the classroom.

Strategies for the Tactile/ Kinesthetic Learner:

To help you stay focused on class lecture, sit near the front of the room and take notes throughout the class period. Don't worry about correct spelling or writing in complete sentences. Jot down key words and draw pictures or make charts to help you remember the information you are hearing.

When studying, walk back and forth with textbook, notes, or flashcards in hand and read the information out loud.

Think of ways to make your learning tangible, i.e. something you can put your hands on. For example, make a model that illustrates a key concept. Spend extra time in a lab setting to learn an important procedure. Spend time in the field (e.g. a museum, historical site, or job site) to gain first-hand experience of your subject matter.

To learn a sequence of steps, make 3'x 5' flashcards for each step. Arrange the cards on a table top to represent the correct sequence. Put words, symbols, or pictures on your flashcards -- anything that helps you remember the information. Use highlighter pens in contrasting colors to emphasize important points. Limit the amount of information per card to aid recall. Practice putting the cards in order until the sequence becomes automatic.

When reviewing new information, copy key points onto a chalkboard, easel board, or other large writing surface.

Make use of the computer to reinforce learning through the sense of touch. Using word processing software, copy essential information from your notes and textbook. Use graphics, tables, and spreadsheets to further organize material that must be learned.

Listen to audio tapes on a Walkman tape player while exercising. Make your own tapes containing important course information.

to READ

http://code.google.com/edu/content/submissions/uwspr2007_clustercourse/listing.html

Another Idea - Speech recognition

2-3 years ago I had looked up some work on speech to text conversion.
The approaches were based on Neural networks, and I did not find them helpful.

I dont know the as it is now, but here is the Idea that sprung to my mind yesterday.

Towards Semantic web. One step at a time

I was traveling back from my hometown to bangalore when I came across this good science fiction article which revolves around semantic web and embedded devices.
Just reinforcing my vision of a world with information at the time we want and at the place we need. Basically it is the availability of information when needed.

I had been reading a few papers on Information retrieval, the semantic web [collaborative filtering/ social networks]. Few of these papers have fascinated me and I believe that they are going to make the world a easier place to live in.

Here is my iota of contribution towards this end:
This is a continuation of my efforts started here.

I implemented the concepts mentioned in the paper above to come up with a program capable to developing a concept map for a small document corpus.

Here is the concept map:


I used the Stanford NLP parser to extract the nounphrases from the document.
JGraphT to represent the graph (I used the DirectedGraph) and JGraph to display the graph.

Lots of improvements to be done:

1. At present all the document processing and generation of the concept map is done online. That sucks up a lot of memory. Hence I'll have to use persistent storage store the nounPhrase - Document association, the Document information, the nounPhrase - nounPhrase association. I am planning to try hsqldb for this. It supports inmemory databases.
   
2. Need to speed up the execution by having some parallel processing. I will try getting help from a professor at IISc [SERC] for this.
3. I need to add the relationship between two entities (alongwith their weightage). For example: Google has a relationship with yahoo. Right now my program just tells that there is an association. but what type of association [I can be competitors/ successful startups/ young CEO's/ web companies/ great places to work at/etc etc]
4. Optimise the code.
   

And I want to learn developing applications for embedded devices too.
I will start with image processing in mobile phones. I bought a second hand nokia 6600 for this.
The two thing I have in mind for mobile applications are:

• An image processing program that will tell me the destination of the bus when I click the photograph of the destination board of the bus (which is in Kannada) so that I dont have to rush to the bus to ask "Bhaiya, majestic jayega?"
• An image processing program (again) which converts my notes (which are quite random) into a good power point presentation (It should at least capture the shapes and the content)

Hmm.. seems that I talk a lot. Let me get back to work!

Distance Learning MS in CS

This one looks good:

http://www.grad.iit.edu/bulletin/programs/cs.html#mast-sci-comp-sci

Cost: 20 760 U.S. dollars = 8,47 ,796.79 Indian rupees

Programming core courses
CS 522 Data Mining
CS 525 Advanced Database Organization
CS 529 Information Retrieval
CS 540 Syntactic Analysis of Programming Languages
CS 546 Parallel Processing
CS 551 Operating System Design and Implementation

Systems core courses
CS 542 Computer Networks I: Fundamentals
CS 544 Computer Networks II: Network Services
CS 547 Wireless Networks
CS 550 Advanced Operating Systems
CS 555 Analytic Models and Simulation of Computer Systems
CS 570 Advanced Computer Architecture
CS 586 Software Systems Architectures

Theory core courses
CS 530 Theory of Computation
CS 532 Formal Languages
CS 533 Computational Geometry
CS 535 Design and Analysis of Algorithms
CS 536 Science of Programming
CS 538 Combinatorial Optimization

The germ

The 16th International World Wide Web conference was held at Banff, Alberta Canada from May 8 to 12th this year.
The web has revolutionalized the way information flows, the way people learn, the way people do their everyday things, the way people socialize and a zillion other things. In short, it has revolutionized the way we live.
And each year the www conference sets the stage to accelerate this revolution in the years to come.

Here are a few papers that I'd like to read up in the limited time I have.
I collected these docs by searching for pdf site:http://www2007.org/ with google.
I have skimmed through the abstracts of the interesting papers of the first 90 results
I found around 29 papers interesting.

Track: Semantic Web (2)

1] Analysis of Topological Characteristics of Huge Online *

Social Networking Services

http://www2007.org/papers/paper676.pdf

2] Combating Spam in Tagging Systems [Stanford]

http://www2007.org/workshops/paper_97.pdf

Track: Search, Information organization, retrieval and processing (17)

1] Navigation-Aided Retrieval

http://www2007.org/papers/paper162.pdf

2] Supervised Rank Aggregation *

http://www2007.org/papers/paper286.pdf

3] Functional Faceted Web Query Analysis [NSU, Singapore] *

http://www2007.org/workshops/paper_44.pdf

4] Efficient Search in Large Textual Collections *

with Redundancy

http://www2007.org/papers/paper800.pdf

5] Formalization, User Strategy and Interaction Design:

Users’ Behaviour with Discourse Tagging Semantics

http://www2007.org/workshops/paper_30.pdf

6] Sponsored Search with Contexts [upenn]

http://www2007.org/workshops/paper_83.pdf

7] Towards a Semantic Knowledge Base for Yeast Biologists

http://www2007.org/workshops/paper_130.pdf

8] Detecting Near-Duplicates for Web Crawling [google] *

http://www2007.org/papers/paper215.pdf

9] Robust Methodologies for Modeling Web Click [yahoo]*

Distributions

http://www2007.org/papers/paper056.pdf

10] Random Web Crawls[criteo]

http://www2007.org/papers/paper339.pdf

11] An Adaptive Crawler for Locating Hidden-Web Entry Points [Utah]*

http://www2007.org/papers/paper429.pdf

12] Web Projections: Learning from Contextual Subgraphs of the Web

[Microsoft, cmu]*

http://www2007.org/papers/paper551.pdf

13] The Discoverability of the Web [yahoo]*

http://www2007.org/papers/paper592.pdf

14] Extraction and Search of Chemical Formulae in Text

Documents on the Web [PSU]

http://www2007.org/papers/paper100.pdf

15] Summarizing Email Conversations with Clue Words [ucb, cancada] *

http://www2007.org/papers/paper631.pdf

16] Answering Relationship Queries on the Web [IBM]*

http://www2007.org/papers/paper058.pdf

17] Do Not Crawl in the DUST: Different URLs with Similar Text

http://www2007.org/papers/paper194.pdf

Recommender systems, Collaborative filtering (4)

1] Improving Ontology Recommendation and Reuse in

WebCORE by Collaborative Assessments

http://www2007.org/workshops/paper_8.pdf

2] The Complex Dynamics of Collaborative Tagging [Princeton] *

http://www2007.org/papers/paper635.pdf

3] Scaling Up All Pairs Similarity Search [google] **

http://www2007.org/papers/paper342.pdf

4] Applying Collaborative Tagging to E-Learning

http://www2007.org/workshops/paper_56.pdf

Data Mining: (2)

1] Towards Domain-Independent Information Extraction

from Web Tables

http://www2007.org/papers/paper790.pdf

2] Page-level Template Detection via Isotonic Smoothing [yahoo]* omi

http://www2007.org/papers/paper588.pdf

Miscellaneous but interesting (4)

1] Optimal Audio-Visual Representations

for Illiterate Users of Computers

http://www2007.org/papers/paper764.pdf

2] A Mobile Application Framework for the Geospatial Web

http://www2007.org/papers/paper287.pdf

3] Communication as Information-Seeking: The Case for

Mobile Social Software for Developing Regions [Washington]

http://www2007.org/papers/paper669.pdf

4] Tag-Cloud Drawing: Algorithms for Cloud Visualization

http://www2007.org/workshops/paper_12.pdf

Know thy self

I came across this blog post and found it very insightful.

I could relate myself to the author of this post.

A few points that I found interesting. [verbatim from the post]:

-
Beware the traps of justification and attribution when thinking about your life. Many people justify past decisions because they want them to make sense and feel better.

^-- This is what I have been doing the past one year! Fooling myself.

Yes, most clouds have silver linings. But sometimes getting hit in the head with a shovel is just getting hit in the head with a shovel. It’s not fun, there’s not much educational value, and your life really is better without it.

-
I really don't know what I want.
The author gives simple and effective tips to understand what you want:
1. A feel good list <-- In this, you note down the events that made you feel good. eg: understanding the paper on semantic search. eg: Explaining Sudhir on why I feel that semantic search would be better than textual search. eg: Telling a colleague that google patent search would be better than delphion.
eg: Helping a colleague finding a solution to the problem he was facing.


2. An ideas list <-- In this you note down the ideas that come to your mind. The author mentioned sometimes he gets a great idea in the sleep and he just gets up and notes that idea down. This happened to me a couple of times :). 3. Finally, get external inputs. Understand your personality and get inspired. I took up this personality test on: and found that : I was and ENJF http://typelogic.com/enfj.html

Extraverted	Intuitive	Feeling	Judging
Strength of the preferences %
33	38	12	11

I'd like to read it again.
In short, it says that people like me like to find solutions, look for greener pastures, help others and are pedagogues of humanity.
That is true I believe, but it is for the people around me to tell me that.

Here is something from a test that I took up previously.


More good talk. From the same blog:


This talk is good:

Richard St. John: Secrets of success in 8 words, 3 minutes

• Do what you love: Passion
• Determination: Hard work, focus, push yourself, get good in one thing
  
• Make something useful: Ideas, serve others
• Keep your chin up: Persist

I need to look at the highlighted points

One needs to persist in the face of failure and other CRAP:

• Criticism
• Rejection
• Jerks (or another word for a non-nice person)
• Pressure

Personality development for Kids

Here is something I wrote on a piece of paper.. This was an effort for the kids of SEEK, VISHWAS, Bangalore. A part of it has been implemented

Syntactic to Semantic Web. The search engine that "Understands" what you want!

I have been looking for current work on text mining, knowledge representation/extraction from unstructured data. I have bookmarked

I found this paper particularly interesting:
"Knowledge Discovery from Semi-Structured Data for Conceptual Organization"


The authors talk about

• creating a concept map (a graph of co-occurring [in other terms, related] concepts [noun phrases]) from a corpus,
• then extracting cliques of a particular concept [using a topological sort],
• and then assigning the documents to that particular clique

What you get a the end is the mapping of a document to concepts. In short, what are the noun phrases that describe the document.
These noun phrases may not always contain all the noun phrases that occur in the document [eg. red soil, which may have occurred in one of the documents containing the concept "rose", but it did not co-occur in any other document containing "rose"]
On the other hand, these noun phrases may include some concepts which were not obvious from that particular document [eg. a document containing "school kids" may not contain the concept "chocolate", but these two concepts have co-occurred in a significant number of documents in the corpus]..

I liked the paper, the concept is very similar to what we do in real life.
We have concepts stored inside our brain [connection between neurons?]. To extract the concept map stored in your brain, you just need to think about all the things that come to your mind when you think about a concept say "s e x".
When ever we come across something new (a new concept), we just associate it to an already existing concept map in our brain. These concepts are in turn linked to information about them (related documents?)

We can even take this a step ahead by assigning weights to the edges [based upon how frequently they co-occur, label the edges[and nodes] with all possible verbs that connect them ]

btw, the authors the Stanford Parser to extract the noun phrases.

you can try it online here.
Moving from syntactic to semantic.. arent we?

why are we [Indians] prosperous but not happy

This post gives a good idea..
I'd like to read it once more in my leisure and ponder over it.

http://harmanjit.blogspot.com/2007/04/notice-period-to-god.html

Portal for Kids

I feel that this idea would be a hit..

resources here
http://www.memory-key.com/Parents/strategies.pdf
http://www.journal.naeyc.org/btj/200407/OnlineAndPrintArtResources.pdf
http://www.surreymuseums.org.uk/somethingelse/psycho.htm

Cognitive Sciences

I dont know what that means.. but some of the things sound interesting..
and maybe complex..

will just make a note of few things that I come across..

Please correct me if I am wrong..
Wordnet is a collection of synsets.. collection of groups of words which relate to a similar concept

Some resources:
A course at MIT OCW [mentioned in my previous post]

Some Papers:
http://www-tsujii.is.s.u-tokyo.ac.jp/~yusuke/paper/eacl2003.pdf
http://www.d.umn.edu/~tpederse/wnsim-bib/

MIT OCW. Real Cool Studd here

I feel that I need to constantly keep learning..

understanding new things has always facinated me..
So has looking at the same thing from a different point of view.


Now and then, I take up a topic which it vaguely related to what I know or what I can comprehend easily and try to understand it.. But such efforts have not led to something concrete till now..
The good this is that I learn.. what next? What do I do with the things I learn?

cut the crap..
Here are a few courses that I'd like to look at in the coming months: [Apart from occasional Google Videos]

• Natural Language and the Computer Representation of Knowledge, Spring 2003
• Operating System Engineering, Fall 2006
• Introduction to Algorithms (SMA 5503), Fall 2005

Will keep on adding to this list over time

Google Tech Talks I'd like to see

Update [ Aug 5, 2007]: These are the ones that I would like to add - From the Google Scalability conference - from glinden
The Google tech talks are quite informative. Ranging from mysql query optimisation to psycology and economics.

Here is the list of videos that I'd Like to watch over the next few months...

http://video.google.com/videoplay?docid=236331448076587879




Building GUI applications with Python, GTK and Glade
http://video.google.com/videoplay?docid=5838951374743244232

Haiku: The Operating System
http://video.google.com/videoplay?docid=236331448076587879

Joomla!
http://video.google.com/videoplay?docid=7035976321528074794

7 Ways To Ruin A Technological Revolution
http://video.google.com/videoplay?docid=-8538240380002213699

Should Google Go Nuclear? Clean, cheap, nuclear power (no, really)
http://video.google.com/videoplay?docid=1996321846673788606



Comparing American and Chinese Negotiation Styles
http://video.google.com/videoplay?docid=4634363396909200237

GeoServer and Architectures of Participation for Geospatial Information
http://video.google.com/videoplay?docid=-3923618086187138447



Search Engine Strategies - Eric Schmidt interview
http://video.google.com/videoplay?docid=-5651532968895500419

Spatial Query Processing Utilizing Voronoi Diagrams
http://video.google.com/videoplay?docid=-2755539754474649930



A Live Motion Portable 3D Video Camera
http://video.google.com/videoplay?docid=-3656494784112768834



Toward the First Revolution in the Mind Sciences
http://video.google.com/videoplay?docid=983112177262602885

You Are What You Say: Privacy Risks of Public Mentions
http://video.google.com/videoplay?docid=6474169875352273382



Code Generation With Ruby
http://video.google.com/videoplay?docid=1541014406319673545

Peer to Peer Web Search with Minerva
http://video.google.com/videoplay?docid=8710122769175704670

Journey Towards The Center Of The Earth
http://video.google.com/videoplay?docid=5797001505961854840



Near-optimal Monitoring of Online Data Sources
http://video.google.com/videoplay?docid=-2567976561168591576



The New Digital Sky
http://video.google.com/videoplay?docid=4364162621957872851



Python 3000
http://video.google.com/videoplay?docid=-6459339159268485356

How can we better understand customers?
http://video.google.com/videoplay?docid=1932851428624192110

Some Python Integrated Development Environments
http://video.google.com/videoplay?docid=8158216898634409900

15 Views of a Node Link Graph: An Information Visualization Portfolio
http://video.google.com/videoplay?docid=-6229232330597040086



Competing and Collaborating in China with Bi-Cultural Competence
http://video.google.com/videoplay?docid=6672770494265798071

People to People Lending with Prosper
http://video.google.com/videoplay?docid=8082730452536323779



Becoming a Software Testing Expert
http://video.google.com/videoplay?docid=6852841264192883219



Impact of Technology on Reducing Poverty and Alleviating Social Issues in India
http://video.google.com/videoplay?docid=-6671271293722797569



Introduction to Perl: The friendly programming language
http://video.google.com/videoplay?docid=-876009974056913377



Building Large Systems at Google
http://video.google.com/videoplay?docid=-5699448884004201579



Leveraging India As India Stands Up
http://video.google.com/videoplay?docid=2089147957292061314

Strike Up The Brand: How to Design for Branding
http://video.google.com/videoplay?docid=5654878583447435228




Team Server / Ajax Development with IntelliJ IDEA
http://video.google.com/videoplay?docid=-5548138713037707664

The Next Fifty Years of Science
http://video.google.com/videoplay?docid=-6119231548215342323



Our Lives, Our Facebooks

http://video.google.com/videoplay?docid=3910777240176719644



django: Web Development for Perfectionists with Deadlines
http://video.google.com/videoplay?docid=-70449010942275062



Biofuels: Think Outside The Barrel
http://video.google.com/videoplay?docid=-570288889128950913



Urban Sensing, Social Networking, And The Third Thing
http://video.google.com/videoplay?docid=7829327427683569548



Predicting bugs in code changes using SCM information
http://video.google.com/videoplay?docid=7802818288058377867

General Purpose, Low Power Supercomputing Using Reconfiguration
http://video.google.com/videoplay?docid=-4969729965240981475

The Reiser4 Filesystem
http://video.google.com/videoplay?docid=6866770590245111825

Stanford Experts on Climate Change and Carbon Trading
http://video.google.com/videoplay?docid=2226061573523196174

Food for the soul

Nowadays I feel that I am being overly disappointed with myself and with the things around me..
I told my friend about this and he told me that I tend poke my nose into other affairs..[ The things that are not in my "Department"!].. and that I tend to judge others..

I guess thats true..

I feel that one way to get rid of this disappointment and "blue" feeling is to look at some practical spiritual quotes.. I chose tao te ching.. and buddhism

Here are some quotes applicable to me that I collated

--

Therefore, stubborn people are disciples of death, but

Flexible people are disciples of life.

-Be impeccable with your word.

Speak little

Bold action against others leads to death

Bold action in harmony with Tao leads to life

-Don't take anything personally.

Be wary of both honor and disgrace

Honor is founded on disgrace

and disgrace is rooted in honor

Both should be avoided

Both bind a person to this world

Words born of the mind are not true

True words are not born of the mind

Those who have virtue do not look for faults

Those who look for faults have no virtue

-Don't make assumptions.

Through the course of nature

Muddy water becomes clear

Through the unfolding of life

Man reaches perfection

Stillness benefits more than action

Silence benefits more than words

Rare indeed are those who are still

Rare indeed are those who are silent

Rare indeed are those who obtain the bounty of this world

-Always do your best.

She who gives herself to this path

is like a block of wood

That gives itself to the chisel

Cut by cut it is honed to perfection

Only a student who gives herself

Can receive the master's gift

Buddhist Values

1. Right Viewpoint - Realizing the Four Noble Truths
2. Right Values - Commitment to mental and ethical growth in moderation
3. Right Speech - One speaks in a non hurtful, not exaggerated, truthful way
4. Right Actions - Wholesome action, avoiding action that would do harm
5. Right Livelihood - One's job does not harm in any way oneself or others; directly or indirectly
6. Right Effort - One makes an effort to improve
7. Right Mindfulness - Mental ability to see things for what they are with clear consciousness.
8. Right Meditation - State where one reaches enlightenment and the ego has disappeared.

Buddha Noble Truths

1. Suffering

There is suffering, dukkha.

Dukkha should be understood.

Dukkha has been understood

2. Desire

There is the origin of suffering, which is attachment to desire.

Desire should be let go of.

Desire has been let go of.

3. cessation of suffering

There is the cessation of suffering, of dukkha.

The cessation of dukkha should be realised.

The cessation of dukkha has been realised.

4. The eight fold path

--

I am gonna take the printout and keep it in my drawer and have a look at it now and then..

How good it would be if we had some "reminder" system that would just pop out an applicable wise saying based upon a situation..

dream along

list

1. ST wildfire - Market, competitor survey
2. Portal for farmers - move it ahead
3. test humaa, be in touch with v Fernando
4. Improve intonation
Download, test adn choose onr speech analyzer.
5. get GRE podcasts/wordlists for the mobile
6. e-bike
7. webhosting and deploying my web-harvest application
8. goals, points for ems discussion, fix the tcp

Improving intonation, accent reduction

I need to improve the "impact" of my delivery. This was one of the development needs that my manager has in mind for me.

Thats true, even if you have stuff, and you are not able to convey it in a manner people are impressed, most of the hardwork goes waste..

My voice is monotonous - no up's and down's in the frequency.. It is a drone.. When you listen to me, you would feel that you are listening to a truck that speaks :)..

I also need to improve my accent [So that my US counterparts get me properly]

I need speech processing software to analyse, improve and monitor my voice.
http://linguistlist.org/sp/SearchWRListing-action.cfm

http://www.sil.org/computing/speechtools/download.htm

Free Speech analysis softwares: http://www.personal.rdg.ac.uk/~llsroach/phon2/freespeech.htm

It is known by: [Google search keywords]

voice training - this applies more to singers
speech therapy
improve intonation
speech training
speech analyzer
...

I will keep this post updated with my progress.

I should be done by end of March [Thought it is a continuous improvement process :)]

btw.. Clipmarks is a good mozilla addon to mark the pages that you would like to store..

Improving intonation, accent reduction

I need to improve the "impact" of my delivery. This was one of the development needs that my manager has in mind for me.

Thats true, even if you have stuff, and you are not able to convey it in a manner people are impressed, most of the hardwork goes waste..

My voice is monotonous - no up's and down's in the frequency.. It is a drone.. When you listen to me, you would feel that you are listening to a truck that speaks :)..

I also need to improve my accent [So that my US counterparts get me properly]

I need speech processing software to analyse, improve and monitor my voice.
http://linguistlist.org/sp/SearchWRListing-action.cfm

http://www.sil.org/computing/speechtools/download.htm

It is known by: [Google search keywords]

voice training - this applies more to singers
speech therapy
improve intonation
speech training
...

I will keep this post updated with my progress.

I should be done by end of March [Thought it is a continuous improvement process :)]

btw.. Clipmarks is a good mozilla addon to mark the pages that you would like to store..

The best way to improve upon your speech is to be conscious of your speech.

Listen to good speakers[auditory training].
-> Knowledge@Wharton [Management podcasts], BBC and a whole lot of other podcast sites are good sources for materila to listen.

Record your speech [Impromptu speech, reading from a textbook, etc]
Anf then listen to it.. where are the places where you do not sound good, the places where you may be misunderstood, places where you stumble, etc

Note down the areas of improvement.

Practicing in from of the mirrir also helps. Pay special attention to lip movements [In my case, I do not flex my lips to the extent that is needed..]

And.. you need patience.. one cannot perfect it with a day/week etc.. it is a continuous process..

Gene Expression data analysis

Paper: Graph-based iterative Group Analysis enhances microarray interpretation
Rainer Breitling,Anna Amtmann,Pawel Herzy
src: http://www.biomedcentral.com/1471-2105/5/100

Gene Expression data analysis :
[from micro array data of a partcular cell state (during a particular phase in the cell cycle/diseased (brain tumor)/healthy) which we want to analyse]
microarray data only gives the magnitude of cDNA's of different genes

One needs to derive the relationships [cause -effect relationships]

• metabolic pathways
• signallin pathways
• protein interaction maps

Identifying subgraphs (that form a community? [PNAS art]subgraphs having high clustering coefficient )

bigraph - Graph with two types of nodes

Good information on the existing methods of Gene Expression Analysis [And gene expression data too]
http://smd.stanford.edu/cgi-bin/search/QuerySetup.pl

good presentation : http://genome-www5.stanford.edu/help/TUTORIALS/SMD_Analysis.ppt

Slide 6 : Factors for Measurement/Process errors
Data smoothened out using normalization : doping controls

Imp: Clustering algorithms
good example begins: slide 21

Heirarchical clustering/Self organising maps

gene Expression microarray data converted into n dimensional vector

x[i] = log(ratio)[i] i=1 to n

How close the two genes are is defined by the distance between those vectors
Metrics:

1. Euclidiean - the distance between the two vectors in space v-u v and u are ntruple vectors
2. Manhattan - http://mathworld.wolfram.com/TaxicabMetric.html
3. Pearson correlation coefficient : It measures the tendancy of both the variables to increase and decrease together.<-- This is also a part of datamining

• This can imply that the change in them is the effect of some other vector
• or both the vectors influence each other [infinite loop till saturation?]
• http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient

4. Cosine Similarity? [this is used in document similarity] : It is the cosine of the angle between two vectors.

Cluster the expression data [patterns] based upon the similarity[ metrics given above]
Similar to Kruskal's Minimum spanning tree algorithm.

this can lead to erroneous results (why?)

SOM's (self organising maps) <-- This is also a part of datamining
This is a single layer feed forward network connected to a grid of 2D/3D vectors.

Why dont I use edge betweenness based algorithm? [PNAS: community structure in social and biological networks]

Tools:
PAJEK
SMD - Stanford dataset

Good links -

I would like to have a collection of links..
The links which I would like to read at a later point of time..
Bookmarking is one way..
It is a bit difficult to sift thru them.. especially if you have a lots of them..

This is my bundle of good links:

Inspirational posters:
http://www.businessballs.com/free_funny_inspirational_motivational_posters.htm

[Mar 27 Update]
http://admissionsourcehelp.blogspot.com/2007/03/computer-science-admission-requirements.html
http://www.slideshare.net/DERIGalway/conor-hayes-topics-tags-and-trends-in-the-blogosphere
http://webcamp.org/SocialNetworks/
http://csclub.cs.sjsu.edu/wordpress/?p=64
http://pimm.wordpress.com/tag/diy/
http://www.hung-truong.com/gradschool/2007/03/

[An informative paper on C++.. from the author of C++ itself!]
http://www.research.att.com/~bs/oopsla.pdf

[Apr 14 Update]
http://edcorner.stanford.edu/index.html

[Apr 16 Update]
Graph Algorithms
http://docs.google.com/Doc?id=w.dgmrhftr_0ctq6mw

e-bike?

My Office is around 15 kilometres from my residence.
And I am fed up with the traffic jams here in Bangalore, the periodicity of buses.

So I said, lemme try going to the office on a bicycle.
I tried it on the weekend.. and guess what it just took me 10 more mins than what the bus takes! [8:25 AM to 9:25 AM]..
But I was too tired and coming back from the office at night [10 PM] was miserable..

Bicycle is good.. but I need a power assist added.. Pedal whenever you can.. and if you dont feel like it, just turn on the motor and there you go!

It is not difficult to add a motor to the bicycle.. just a few hand calculations, few trips to the workshop and a few instructions to the workman.

Google is my friend:
This is a read good website:
http://www.peltzer.net/ebike/
www.estelle.de

The commercial e-bikes
http://www.electric-bikes.com/

I had seen a person use a 0.5 HP DC motor, 24V batteries.. the guy above uses a 1HP [746 Watt],3000 rpm motor.
btw, TXU Turbines are 1000000000 Watt [:)].. A thousand Megawatt

Say I want to go at a speed of 20 kmph and the wheel diameter is..
I need to do the calc and come out with the spec for the

motor,
batteries,
charger,
the speed reducer,
the motor speed controller,
dimensions of the supporting structure,
etc

Motor would be a major cost here I believe, Need to ask is some shop here or look for some flea market in case a new motor is too costly [A motor from a washing machine would do]

btw a new e-bike costs 16000 INR (The pamphlet said that it costs only 80? paise per km)
Will do some more research and come up with the design..

I want to be at the office tomorrow morning at 6:00 am.

Hope that such wishes get fulfilled in the near future :)

Update 1:
I bought a second hand DC motor[1200 INR], 0.5 inch [thickness] 18 and 38 teeth sprockets, a chain [all three costed me 768 INR] and two small metal sheets for housing the motor.. Will upload the photos of my endeavour later.
I need to go to the workshop to bore the sprockets and to drill them metal sheets and assemble all of them and get my bike ready [:)]
getting work done at the workshop is tiring.. so I would do this manufacturing/assembly thing the week after next..

My friends are excited about the effort!..
Will keep you updated

Network Analysis ...

Need to Carry my thesis work forward..
My thesis was on Study of hub neighborhoods in some networks

In the thesis, I had come up with a few metrics to find the interconnectednedd between two hubs in a network.

http://www.genetics.ucla.edu/labs/horvath/CoexpressionNetwork/

I am not able to figure out what should I work on..
Will have to decide it by tomorrow..
so much to do. so little time!

More choices is good.... and bad too!

A Paradox? probably yes.
Just had a look at the Google video by
The Paradox of Choice
by Barry Schwartz

Where he talks about the dark side of having more choices..

Paralysis: The choice make becomes overwhelmed by the deluge of choices in front of him. Thus, he either postpones his decision or does not make any decision!

Different types of choice making strategies:

Maximiser: These people look at all the options and make the decisions.. but these people are more unhappy [According to the speaker].. I feel that this is true in practice too!
Satisfisor: These people keep on trying the alternatives till they get the alternative by which they are satisfied. Their choice may not be optimal... {because they have not looked at all the options].. But in practice they are more happy than Maximisers!

Principal-Agent Problem:
In this case, the agent does all the work of choosing.. thus insulating the stake holder from the pain of making decisions.. The agent does not need to be a guru... if he has some knowledge of the products, he can come up with good choice for the stake holder. Both are profited.. both are happy..

The Speaker also said that Google was the part of the solution of this "choices" problem.

Where are our Energy needs taking us: A summary of Newsweek 2007 contents

I started going through the articles and realised that I would be able to appreciate the implications of Energy supply/demand on various sections better if I had some background of Macroeconomics. I googled and got a fairly good introduction to Macroeconomics.
Here's the presentation on Macroeconomics. If you already have an idea of what/how/why of GDP, Inflation, bank interest rates, oil prices, etc.. then u can skip the macroeconomics presentation.
web.syr.edu/~bjdempse/Introduction%20to%20Macroeconomics.pdf

Create > New > Playlist

It has been long since I have learnt any new songs on my guitar..
I need to learn new songs to play on the guitar..

Here's the list of the songs that I would be learning:
These are the the songs that are

1. Kya Mujhe Pyar Hai - Woh Lamhe


2. Woh Lamhe - Woh Lamhe


3. Show (Hey Oh) - Californication


4. Allah Ke Bande - Kailas Kher


5. Dil Kya Kare Jab kisiko.. kisise pyar ho jaye


6. Neele neele ambar pe

7. Listen to your heart - Avril Lavigne


8. One Tamil Song


9. One Telugu song


The Songs that I already know are:

I need to improve upon retention of lyrics,
Areas of Improvement
1. Knocking on heavens door - Bob Dylan - Lyrics
2. Californication - RHCP - Lyrics
3. Dum Maro Dum
4. Papa Kehte Hain - Lyrics
5. Hotel California - Lyrics, Lead
6. Yahan Kyun jee rahen hain [my composition]
7. over the mountains of the seas - Lyrics
8. Open the eyes of my heart - Lyrics
9. This one for the people - Lyrics

Will try to buy the Sony Ericsson Protable speakers. So that I can have the percussion and background music.. the portable ones (These run on the phone battery only)
or the ones that need the power supply.. I need to see how heavy they would be on my wallet [:)]

TRIZ - Become an Edison!

"Make significant contributions to IP" was one of my long term goals that I put in my ems [ an internal resume where we list out our contributions to the business, strengths, development needs and our future plans]..

No the question is.. How do I make a "significant" contribution to IP [Intellectual property].. I am a bit creative.. and creativity coupled with science and technology leads to new ideas, new inventions and new technology breakthroughs. One way to generate a new idea to improve something (say coming up with a disruptive technology) is to learn more about the technology, and keep on brainstorming (with yourself) until u get an idea. And what do I do about a technology that I am not expert at.. I would have to become an expert in that field first.. and then think about the "creative" idea..

I dont have the luxury of that time.. Though I have the luxury of resources to become an expert in the technology I want to genetate intellectual property [Steam Turbines.. I dont want to be more specific]

I searched for something on innovation in our library .. and came across TRIZ.. it sounded cool :)
just googled for TRIZ (note that google is a word in the english dictionary. GOOGLE has become analogous to search)
TRIZ I learnt... was a method for inventors to come up with new ideas to solve their problems.

Genrich S. Altshuller, the father if TRIZ, used to work in the patent team of the Russian Navy.

He
Today I had a chance to look into the details of TRIZ.
Basic philosophy is to convert your problem into an abstract one.
find the abstract solution for the abstract problem [from a similar problem which has already been solved ]
convert the abstract solution to the solution of your problem..

This document gives a good idea of triz [Along with a good example of a beverage can desin optimisation]
http://www.mazur.net/triz/

An algorithm to solve your problem [i.e. move your system towards ideality] would be.
1. Identify the technical conflicts in your problem [Abstract]
2. Look at the TRIZ matrix and find the possible solutions [Abstract]

Convert the abstract solutions of step 2 into the solutions specific to your problem..

More when I file my first patent disclosure :)

Friends now : Friends hereafter ..

Just wanted to record where the people whom I know are right now and how do they choose the road on the crossroads.. and where would those Roads take them.. just a small study..

Name: Education :
Current state : Possible next state..
Changes of state

1. Durgaprasad: BE Mech/ MSc Infosys [BITS Pilani] 2005
Microsoft Hyderabad, development[Mobile operating systems]

2. Srinivas : BE Mech/ MSc Infosys [BITS Pilani] 2005
MS Mech, Stanford. [1st sem CFD work. switching field to design. As CFD jobs require only PhD's]
MS->job->MBA

3. Gautam. BE Mech, BITS Pilani 2004
MS Materials Science. Among Class toppers. Applied for a Ph.D. (more inclined towards biology/ humanities).. Likes to influence people around him

4. Bharath BE Mech, BITS Pilani 2004
MS Mech. Looking for a job rite now

5. Chandana BE Mech, BITS Pilani 2004
MS Mech, Purdue

6. Divya (violinist) BE Chemical, BITS Pilani 2004
Did internship at JFWTC in summer hols. Completed MS in 3 sems [no final yr project]
currently doing a job in an Environmental Science company?

7. Dinakar BE Mech, BITS Pilani 2004
MS - No contact

8. Narayana Mech 2004
MS - No contact

9. Adi..

Address book is a better way to maintain this info I guess..

GRE Resources

I this post I would be adding resources that would aid in the preparation of GRE..

http://www.cs.ucr.edu/~sdixit/gre.htm

The coming year

There are three segments which I would be targetting
1. MS
2. EEDP
3. GE

Main focus would be:

1st half:
GRE, EEDP, POM, DREAMS

2nd half:
Knowledge acquisition, career shift (maybe)->depends on 1st half

The action items would be:

Jan -
1. Read about GA, ANN, and how it has been applied to optimisation problems. [//]
2. Apply GA to the design optimisation problem [FW 4]
3. Talk to IISc prof about the problem - Possibility of a paper. [FW 3]
4. GRE - wl [1 to30]

Feb -
1. Look at how you can improve the optimisation algo. Apply various optimisation algos to your design problem and benchmark performances. [ FW 7]
2. GRE - wl [ upto 60]
3. DMAIC ss [FW 8]

Mar -
1. GRE - wl [ pass 1 finish]]
2. Present in DREAMS

Apr -
1. GRE/EEDP - start trying for paper publication
2. GRE - pass 2, analytical writing, essays, etc

May-
1. EEDP [Revise CDC's]
2. GRE final pass

Jun-
1. GRE
2. EEDP

Second half:
DP's, Datamining

Research topic for Data Mining+Collaborative Filtering+NLP

I am thinking of MS as a safety boat in case the ship that I am sailing in sinks :)
Moreover the pattern of MS is also gonna change.. So I feel that it is always better to have the score in hand.

I am planning to do an MS in computer science

I havent done any major projects in computer science..
To get into the top colleges [with aid] I feel that they would be looking at my inclination towards research..
I am planning of doing some project this year [and publish a paper.. maybe]

I am thinking of doing a project in collaborative filtering (involving NLP, Datamining)

Looking for some research topics..
Literature survey documented below

Searching in
ACM library, Google, Google Scholar, and a collection of search tools by Todd Veldhuizen.

Need an ACM account.. will finish the topic selection by sunday
[update - 5th Jan]
hmm.. friends say that I am becoming unstable day by day.. some even coined a noun for me - lahari baba.. the fickle one.. and here is another case of this behaviour..
I am thinking of a topic that involves [Artificial Intelligence - > [ANN or Genetic Algorithms] and design optimisation]..
One topic that I thought of is "Multi objective design optimisation of Steam turbine bucket and wheel using genetic algorithms" (This topic is research oriented, aligned with my work at office and also requires less effort)..
Now I need to look up what genetic algorithms are in detail.. I have a fair idea about them.. They operate on the principle of natural selection.. picking up the best qualities of the previous generation, generating new offspring with these qualities and then replacing the least fittest of the current population with the newly generated offspring...
I need to look at how I need to plug in the GA into the already existing program structure..

Books to read this year..

I am looking forward for a great year ahead..
A year that would catapult me to great heights..
I am in the process of making a roadmap [both personal and professional] for the coming year.

Reading books (I am targeting 6-8 books this year) is a major item in the list.
The topics I am lookin at are
1] Innovation
2] *[Data Mining/Knowledge discovery/Collobarative filtering] **NLP& Aboutness
I feel that * and ** would be a good combination so that the text information can be represented in a structure understandable by the * algorithms

text --NLP*--> [ structured and representation] --User Data, CF,DM algos**----> good recommendations


3] Program Design : Algorithms , Design Patterns
4] Entrepreneurship [web based]


Here are some of the books that I came across in our library:
I am looking at amazon.com for the book reviews. I would writing my notes after each book.
Hope I come up with the final list by saturday..

------------------------------------------------------------------------------------------

INNOVATION:

Innovation on Demand: new product development using TRIZ Fey, Victor.
Rivin, Eugene I. [I feel that this book would be better. as it is written by mech engg profs maybe I can relate to the concepts better]
658.57522 FEY P05
2005

40 Principles : triz keys to technical innovation Altshuller, Genrich [This is book by the formulator of TRIZ]
620.0028
2001

Notes: TRIZ looks like a good concept.
>> TRIZ (some russian word):
TRIZ also evolved by transferring strong principles from one field to another.
TRIZ is also used in DFSS (for solving new problems using 6 sigma methodology)

This image compares TRIZ with other problem solving techniques

Biomimicry : innovation inspired by nature Benyus, Janine M.
600 BEN N97 [Looks like some stuff that they show in natgeo/discovery]
1997

--------------------------------
DATA MINING

The elements of statistical learning : data mining, inference, and prediction Hastie, T.
Tibshirani, Robert.
Friedman, Jerome.
006.31 Hastie, 2001
2001


Data mining : practical machine learning tools and techniques with Java implementations Witten, I. H.
Frank, Eibe. [This is a must for any one planning to learn/apply data mining. Associated with WEKA]
http://www.cs.waikato.ac.nz/~ml/weka/book.html
006.3 Witten, 2000
2000


Advances in knowledge discovery and data mining Fayyad, Usama M.
006.33 Advanc, 1996
1996 [This one looks like a good book]

Came across this nice pdf by usma fayyad and co
http://www.idi.ntnu.no/emner/it3704/lectures/papers/aimag-kdd-overview-1996-Fayyad.pdf

------------------------------------
ALGORITHMS

Learning to classify text using support vector machines: methods, theory and algorithms Joachims, Thorsten
005
2002
Algorithms in c,part1-4 : fundamentals data structures sorting searching Robert Sedgewick
005.133
2001
Multi-objective optimization using evolutionary algorithms Deb, Kalyanmoy.
519.3 Deb, 2001
2001

Machine learning Mitchell, Tom M.
006.31
1997
---------------------------------
NATURAL LAnguage processing

Foundation of Statistical Natural Language Processing Christopher D. Manning and Hinrich Schutze
410.285
2002 [This one looks like a good one]
The companion site for this book : http://nlp.stanford.edu/fsnlp/

Moreover this book is being used in the following Universities:

UPenn CIS530, UPenn CIS639, Berkeley SIMS 296a-4, Johns Hopkins: current (Eisner) and previous [lots of great slides by Jan Hajic!], Brown CS241, CMU 11-682, CMU 11-761, Stanford CS224N, Oregon Graduate Institute CSE580, Ohio State, U Chicago, Purdue [lots of slides], Tufts, Minnesota 1, Minnesota 2, SUNY Albany, San Diego SU, Mississippi State, UCLA.

Speech and Language Processing: An Introduction to Natural Language Processing,Computational linguistics, and Speech Recognition Daniel Jurafsky and James H. Martin
410.28 DAN
2000
-------------------------------------------

Books That I zeroed in on:
First Half:
4. Artificial Intelligence [Genetic Algorithms, ANN and optimisation methods]

Second Half:
1. Innovation on Demand: new product development using TRIZ Fey, Victor.Rivin, Eugene I.
2. Data mining : practical machine learning tools and techniques with Java implementations Witten, I. H.Frank, Eibe.
3. Foundation of Statistical Natural Language Processing Christopher D. Manning and Hinrich Schutze
5. [Design Patterns]
6. [Web entrepreneurship]

[update Jan 7]
I would be quite busy for the first half of this year.. GRE/EEDP.. hence, I would be stressing more on knowledge acquisition in the second half.