Showing posts with label information search and retrieval. Show all posts
Showing posts with label information search and retrieval. Show all posts

Saturday, July 07, 2007

Towards Semantic web. One step at a time

I was traveling back from my hometown to bangalore when I came across this good science fiction article which revolves around semantic web and embedded devices.
Just reinforcing my vision of a world with information at the time we want and at the place we need. Basically it is the availability of information when needed.

I had been reading a few papers on Information retrieval, the semantic web [collaborative filtering/ social networks]. Few of these papers have fascinated me and I believe that they are going to make the world a easier place to live in.

Here is my iota of contribution towards this end:
This is a continuation of my efforts started here.

I implemented the concepts mentioned in the paper above to come up with a program capable to developing a concept map for a small document corpus.

Here is the concept map:


I used the Stanford NLP parser to extract the nounphrases from the document.
JGraphT to represent the graph (I used the DirectedGraph) and JGraph to display the graph.

Lots of improvements to be done:
  1. At present all the document processing and generation of the concept map is done online. That sucks up a lot of memory. Hence I'll have to use persistent storage store the nounPhrase - Document association, the Document information, the nounPhrase - nounPhrase association. I am planning to try hsqldb for this. It supports inmemory databases.
  2. Need to speed up the execution by having some parallel processing. I will try getting help from a professor at IISc [SERC] for this.
  3. I need to add the relationship between two entities (alongwith their weightage). For example: Google has a relationship with yahoo. Right now my program just tells that there is an association. but what type of association [I can be competitors/ successful startups/ young CEO's/ web companies/ great places to work at/etc etc]
  4. Optimise the code.
And I want to learn developing applications for embedded devices too.
I will start with image processing in mobile phones. I bought a second hand nokia 6600 for this.
The two thing I have in mind for mobile applications are:
  • An image processing program that will tell me the destination of the bus when I click the photograph of the destination board of the bus (which is in Kannada) so that I dont have to rush to the bus to ask "Bhaiya, majestic jayega?"
  • An image processing program (again) which converts my notes (which are quite random) into a good power point presentation (It should at least capture the shapes and the content)
Hmm.. seems that I talk a lot. Let me get back to work!

Sunday, June 03, 2007

The germ

The 16th International World Wide Web conference was held at Banff, Alberta Canada from May 8 to 12th this year.
The web has revolutionalized the way information flows, the way people learn, the way people do their everyday things, the way people socialize and a zillion other things. In short, it has revolutionized the way we live.
And each year the www conference sets the stage to accelerate this revolution in the years to come.

Here are a few papers that I'd like to read up in the limited time I have.
I collected these docs by searching for pdf site:http://www2007.org/ with google.
I have skimmed through the abstracts of the interesting papers of the first 90 results
I found around 29 papers interesting.

Track: Semantic Web (2)

1] Analysis of Topological Characteristics of Huge Online *

Social Networking Services

http://www2007.org/papers/paper676.pdf

2] Combating Spam in Tagging Systems [Stanford]

http://www2007.org/workshops/paper_97.pdf


Track: Search, Information organization, retrieval and processing (17)

1] Navigation-Aided Retrieval

http://www2007.org/papers/paper162.pdf

2] Supervised Rank Aggregation *

http://www2007.org/papers/paper286.pdf

3] Functional Faceted Web Query Analysis [NSU, Singapore] *

http://www2007.org/workshops/paper_44.pdf

4] Efficient Search in Large Textual Collections *

with Redundancy

http://www2007.org/papers/paper800.pdf

5] Formalization, User Strategy and Interaction Design:

Users’ Behaviour with Discourse Tagging Semantics

http://www2007.org/workshops/paper_30.pdf

6] Sponsored Search with Contexts [upenn]

http://www2007.org/workshops/paper_83.pdf

7] Towards a Semantic Knowledge Base for Yeast Biologists

http://www2007.org/workshops/paper_130.pdf

8] Detecting Near-Duplicates for Web Crawling [google] *

http://www2007.org/papers/paper215.pdf

9] Robust Methodologies for Modeling Web Click [yahoo]*

Distributions

http://www2007.org/papers/paper056.pdf

10] Random Web Crawls[criteo]

http://www2007.org/papers/paper339.pdf

11] An Adaptive Crawler for Locating Hidden-Web Entry Points [Utah]*

http://www2007.org/papers/paper429.pdf

12] Web Projections: Learning from Contextual Subgraphs of the Web

[Microsoft, cmu]*

http://www2007.org/papers/paper551.pdf

13] The Discoverability of the Web [yahoo]*

http://www2007.org/papers/paper592.pdf

14] Extraction and Search of Chemical Formulae in Text

Documents on the Web [PSU]

http://www2007.org/papers/paper100.pdf

15] Summarizing Email Conversations with Clue Words [ucb, cancada] *

http://www2007.org/papers/paper631.pdf

16] Answering Relationship Queries on the Web [IBM]*

http://www2007.org/papers/paper058.pdf

17] Do Not Crawl in the DUST: Different URLs with Similar Text

http://www2007.org/papers/paper194.pdf

Recommender systems, Collaborative filtering (4)

1] Improving Ontology Recommendation and Reuse in

WebCORE by Collaborative Assessments

http://www2007.org/workshops/paper_8.pdf

2] The Complex Dynamics of Collaborative Tagging [Princeton] *

http://www2007.org/papers/paper635.pdf

3] Scaling Up All Pairs Similarity Search [google] **

http://www2007.org/papers/paper342.pdf

4] Applying Collaborative Tagging to E-Learning

http://www2007.org/workshops/paper_56.pdf

Data Mining: (2)

1] Towards Domain-Independent Information Extraction

from Web Tables

http://www2007.org/papers/paper790.pdf

2] Page-level Template Detection via Isotonic Smoothing [yahoo]* omi

http://www2007.org/papers/paper588.pdf

Miscellaneous but interesting (4)

1] Optimal Audio-Visual Representations

for Illiterate Users of Computers

http://www2007.org/papers/paper764.pdf

2] A Mobile Application Framework for the Geospatial Web

http://www2007.org/papers/paper287.pdf

3] Communication as Information-Seeking: The Case for

Mobile Social Software for Developing Regions [Washington]

http://www2007.org/papers/paper669.pdf

4] Tag-Cloud Drawing: Algorithms for Cloud Visualization

http://www2007.org/workshops/paper_12.pdf