Semantic DataBase

Reasoning Engine Logic

2011-01-06T21:17:00.002+10:00

Its been quite some time since I did a blog post. It took me a while to understand what I want to get out of my reasoning engine. After spending considerable amount of time to understand how reasoning engine should work, I came up with these (mentioned below) points to summarize how the reasoning engine should behave.

How to Select the entities for Reasoning? This includes the mention of what kind of entities we are interested in, what is the selection criteria for those entities to be filtered from others in the knowledgebase etc.
What information is needed from those objects? Not all attributes of the entities are useful in output result. We need to be specific about what attributes we are interested in from the selected entities.
How the details needs to be presented in output? The representation of output object (entity).
What level of detail is good enough? We need to define a boundary where we can say the detail presented is sufficient enough for us to proceed further.
What can be derived directly and what can be derived indirectly?

That's all for now. Will try to post regularly now (one of my new year resolution). Until Next Time... !!!

Semantic Search Criteria

2010-01-27T21:46:00.002+10:00

In Previous posts I mentioned Semantic Search and the starting point for Semantic Search. But as we all agree that search needs some sort of criteria to filter the results. In this post I am going to put down my thoughts on what I see as a criteria for the Semantic Search.

Semantic search is a result of conversation between user (human or machine) and the machine. The conversation is two way where the requester sends out search criteria and machine brings out a possible set of results that matches the criteria. But there is a twist to this process. The search criteria for semantic search is not straightforward like normal search. There are more to the search criteria than a query condition. For Semantic Search:

We need to mention whether the condition must be meet or it is not important to fulfill the condition.
In case there is a contradiction (with another condition) which one would take precedence?

Also the output must contain the relevancy factor with respect to the criterias entered by the user. There should be some sort of indication as how many of optional and additional criterias have been meet.

I am still yet to work out how the criterias will be evaluated against the objects available within the knowledgebase. Will be posting my thoughts on this as I progress further.

Until Next Time....!!!

Search : Past, Present and Future

2009-12-02T20:31:00.004+10:00

In my previous post on semantic search I discussed the drawbacks of current searches and also mentioned that it takes average 3 google searches to get the desired result.

Recently I read a paper on evolution of search 3.0. This paper described how the search has evolved over a period of time. This is what author has to say in the paper:
"In the coming third decade of the Web, Web 3.0 (2009 - 2019), there will be another shift in the search paradigm. This is a shift to from the past to the present, and from the social to the personal, and from the generic to the precise."

In short the next generation of search will be returning results based on the information supplied by the user. This means user's data has to be available to the search engine or user will publish a personal information (virtual card) along with every request they submit. These details will be metadata driven and will be used by various search engines to filter the search result and tailor it to suit user requirements matching his expertise level.

What it means is each content that is published on web must publish the metadata that describes what the content is. The metadata must contain sufficient details about the content and must be in a form that it can be interpreted by search engines. But metadata is just one side of the story. The search algorithms must be modified to make use of this metadata and produce the results considering the (published) user information. While some searches will be locations independent, there will be few searches that need to be location sensitive and the results must be valid in current location of the user.

Until Next Time...

Representing Uncertainty

2009-11-11T20:16:00.005+10:00

In one of my earlier blog post Is AI a Possibility I discussed the need of a 3rd state. For past few days I am thinking about scenario where just returning a Boolean value ie True or False is not good enough.

Normally a function evaluates to either "True" or "False" based on whether the attributes of the entities meet the conditions defined in the rule or not. But it may happen at times that the entity does not contain the attributes required by the rule to evaluate it properly. Then in that case we need to have a 3rd (Not Available) and 4th (Not Applicable) state as rule outcome. When a function returns Not Available then the it implies that the entity does not contain the attribute needed for the rule to execute or process the object. On the other hand Not Applicable means the rule does not apply to the type of the entity in context.

So in total we have 4 return values for the function:
- True
- False
- Not Applicable
- Not Available
A Boolean value (outcome) is not the possible solution here. So we need an alternate representation here for the function result. All functions cannot evaluate to True/False. For those rules (functions) that cannot be evaluated we need to find out the what was the state (Not Applicable or Not Available). When we apply multiple rules to the same object in a sequence (workflow) the outcome is a set of conclusions. But the conclusion must include which rule was evaluated and which ones could not be evaluated.

I am still puzzled as how to represent the two more state considering that internally everything is represented as either 1 or 0 and that does not leave room for representing uncertainty.

Until Next Time....

Representing Frequency

2009-10-26T21:58:00.000+10:00

While reading about Tree of Porphyry (proposed by Ramon Lull in 1272) I learnt about the 10 questions that can be asked to any entities. But one thing that was missed out of this list is representing frequency. Suppose some process A takes place every 2 days. So we need to find a mechanism to represent the repetition and the frequency at which this occurs. Ramon Lull describes When as the question that can represent the date and time related attribute of the object.

What I propose is extending the 10 questions as listed in Tree of Porphyry and adding another question to the list How Often. The purpose of how often is to represent the frequency of a repetitive attribute of the object. It will have few sub attributes like a Value (How Much) and the Unit (What Kind). Together these will describe the nature of the repetition.

Until Next Time...!!!

Finding the Right Data Structure for Knowledge Representation

2009-10-21T21:50:00.005+10:00

The most commonly used data structure today is the Row-based data structure where in one row represents details about an instance of an entity type. But in real-world the representation of an entity is not a flat structured. An object's representation contains several sub-attributes that those sub-attributes may have their own sub-attributes that make up the entire object (its attributes). But if we go by the row-based representation of the object we cannot represent the sub-attribute and their relation with the main object.

Consider an example where a Person has FirstName, Surname, Home Address (Street, Suburb, State, Post Code) and Work Address (Company Name, Street, Suburb, State, Post Code). If we use a row-based representation here then we find that our records look like this :

FirstName, Surname, HomeStreet, HomeState, HomePostCode, CompanyName, WorkStreet, WorkSuburb, WorkState, WorkPostCode.

The limitations here is unless we uniquely name the Street, PostCode, State attributes for both Home and Work Address we will not be able to distinguish their real meaning. On the other hand consider a Structure like this:
Person
  - Name
    - First Name
    - SurName
  - Home Address
    - Street
    - Suburb
    - PostCode
    - State
  - Work Address
    - Street
    - Suburb
    - State
    - Post Code

By looking at this structure we can easily tell that the home Address is made up of 4 attributes and Work address is made up of 5 sub-attributes. These in-turn can have their sub attributes as well that will define them in more detail.

It is evident that the hierarchical data structure provides more flexibility and room to grow than the flat row-based structure for representing a real-world object.

Until Next Time...!!!

How do we achieve Artificial Intelligence?

2009-09-15T15:37:00.000+10:00

Artificial Intelligence, I am sure many readers are familiar with this buzzword created by research groups around the world not so long ago. Where we started dreaming about many things a machine can do that we do in our day to day life and will in turn make our life simple and easy. But what happened to most of those projects, they are either shelved or have very limited usability in our day-to-day life. Though there are few outcomes that we did find useful.

When I read about the artificial intelligence and where it went wrong, I ask a question as what went wrong? Where did it all go wrong?

Lets define the intelligence. The intelligence is art of making best choices based on what we know (or rather don't know). But what determines whether we know something or not. It is our ability to recall something we learnt in past. Learning is associating facts to a context. Context define how the entities are being linked together. The linking does not have to be static.

So in a nutshell, in order to build a system that can:

Understand the context in which a particular fact is stated.
Retrieve the most appropriate rule that can be applied to the available facts i.e. show some sort of intelligent behavior.
The retrieve operation depends on how the raw data is structured.

In my opinion it all comes down to how the data is structured (represented) and the reasoning mechanism that works on the data.

Until Next Time..

Starting Point for Semantic Search

2009-08-04T09:31:00.004+10:00

In my previous post on Semantic Search, I discussed about what is Semantic Search in general. One of the ideas that is revolving around is how to make the search efficient.

At an average it takes 3 google searches for someone to find what they are looking for. This is mainly because the google search engine has to scan their index table and it brings out all the documents that matches the keyword, of course ranked by the google page ranking algorithm also known as PigeonRank.

But if we keep the technology aside, then there are two possible ways we start searching for something.

When we know what we are looking for. This is the simple and straightforward case where we are very well aware of our needs and we often get result faster.
But there is this other situation when we don't know what we are looking for. We just have knowledge about few attributes, characteristics of the object we are searching.

A Semantic Search will have to operate taking both into account. The search engine will have to rank the results based on the criteria matching.

Until Next Time....!!!

Information and Knowledge

2009-07-13T16:32:00.002+10:00

Does Information mean Knowledge or vice-versa? This is often the topic of discussion when I happen to talk about knowledge and its role in Semantic Web. More often than not we confuse information with knowledge.

So what is information? It is the smallest detail we have (fact in the system) about an entity. Today is Monday is an information, But Monday is first day of work week is not. In western countries first day of week is Monday but that is not true for middle-east, their week begins on Sunday.

So how do we define knowledge? Knowledge is the interpretation of information. In our previous example, Monday is the first day of the week or not depends in which country's context we are discussing Monday. We can define knowledge as Information related to a context. If not attached to a context, the information does not convey any meaning and that means the information is of no use hence cannot be classified as knowledge. So for an information to be classified as knowledge we must have the context attached to it.

Until Next Time...!!!

What is Semantic Search

2009-07-01T08:51:00.004+10:00

If I ask this question in a group of 10 researchers, I will have 11 correct answers of this (including one of mine) for sure. But the question will remain unanswered as what exactly is "Semantic Search"?

During Semantic Technology Conference 2009, this topic was debated among search biggies as what exactly is next direction of search ? One thing that came out of the discussion is page ranking and keyword search is definitely NOT the way the searches are going to work in future. One of the most probable future for search engine is "It will be more like a conversation with the user".

Currently the search engine work in "Tell and show the result" mode. The user enters the keyword and the search engine dumps all possible matches (based on page rank, keyword matches etc) to the user. Most of the time the results do not make any relavance to the search intention at all. A conversation style of search seems to be more appropriate where the user refines their search criteria with continuous interaction with computer. This resembles more of how we search information in our day-to-day life. The search session will be more like a brainstorming where the user will feed more and more information about what they know about what they want. The system will then return a set of most relavant searches to the user. The user then will add more details of what they want and the steps will be repeated till the user finds what they want. At any point in time, they would also like to go back and start from scratch.

One of the positive step in this area is Semanti search engine. This search engine provides a list of suggested categories where a particular keyword is associated. This is just the starting point for semantic search. The semantic search engine should be able to let the user feed more (unlimited) details of what they know about what they are looking for. These details (provided by user) could be directly related to the result they are looking for or it might not describe the result at all. Knowledge reasoning is a major influencer in the search process. Reasoning is the core of the semantic search process and will determine how accurate the search engine is. While the search engine is designed for accuracy, the efficiency will have to be compromised at least for a while. One of the possible scenario these search engines will have to handle is percentage accuracy of result based on the search criteria. The more matching result should be displayed on top of the list.

So what is your idea of Semantic Search?

Until Next Time...!!!

Search Engines

2009-06-29T09:11:00.001+10:00

Search is the next big thing in the World Wide Web and every big player is trying their best to capture the bigger market share of the search today. Recently I had a chance to learn about Microsoft's new Search Engine Bing and came to know that the product bing is a result of M$ buying a company that was working on Semantic Search.

With my limited interaction with Bing I did not find any WOW factor with it. The accuracy of result is bit better than google search result, but still it misses out quite a few search results (what google brings back) with high level of relevance. But it promises to be a good alternative for google search.

I also tried out Wolfram Alpha's Computational Knowledge Engine. In my opinion that is not a search engine, the Wolfram Alpha is more like a knowledge engine, that brings back facts when we submit a query. I tried to ask few questions related to Biotech and other science subjects but the result were quite disappointing, as sometime it showed me the share prices of the company and sometimes it did not bring anything at all.

The search engine technology is still far from being matured and it will be sometime before we see a fully matured search engine that can answer most of our queries. We need a true semantic search if we want to build a search engine that is helpful to the users. In the next post I will discuss more about how a true semantic search can be achieved.

Until Next Time....!!!

Is AI a possibility

2009-03-14T17:38:00.003+10:00

In one of my previous post I discussed whether the World is Ready for AI? Today I was reading this post with after a long time (more than 2 yrs) and asked myself this question whether we have made enough advancements in the comupting to make AI possible in near future. We have had faster computers, greater RAM and storage available on our laptops over last two years but we are still far from having the computing framework that can support AI.

The basic of computing is a bit that has two states 0 and 1. What it translates to is the computer always has a state of certainty ie whether it has something or it does not have. On a contrary our normal intelligence works on few more states. We operate on 3 states.

We know that we know.
We know that we don't know
We don't know that we don't know.

This third state of ours is what makes us intelligent and gives us the power to reason given a situation. Human intelligence operates mostly out of this 3rd state of mind. The moment we know about anything that falls in 3rd stage it moves to either 1st or 2nd category.

Our brain operates in a 3-dimensional space and that's what provides us the flexibility to process similar data differently. But on the contrary computers operate in a linear space and that limits the processing capability of the computers. A simple example is for computer a glass of water is a glass of water no matter how many times we feed this data in, but for humans the first glass of water is life saver (if we are thirsty) but the same is not true with the 30th glass of water if it is drunk in succession. The 30th glass may become a burden to drink. So the same data is interpreted differently here in case of humans.

What we may require is to think about the fundamental aspect on which our computing is based at. The basis of computing is 0 and 1, but we may need to think about a state where the computer can be in May Be state ie somewhere in transition. Once we have this third bit discovered and our machines are based on that, we may be able to feed consciousness and that will lead to natural intelligence in computer.

Until Next Time.

Object Structures and Descriptions

2009-03-07T16:50:00.002+10:00

While reasoning an object we often fall into a trap of thinking about the attributes and methods of the object (in a typical Object Oriented way). But reasoning about an object goes beyond the attributes and methods. While reasoning an object we need to consider the following:

Object fall into categories. eg. My car is a Hatchback. My Pet is doberman. etc But then we also have instances where an object is part of multiple categories like I am an Employee, Blogger and a Husband.
Categories can be more general or more specific in nature eg. Physician and Surgeons are types of Doctors, A Father is a parent etc.
In addition to generalization being common for categories with simple names, it is also natural for those with more complex description. A Contract employee is an employee. A family with at least one child is not childless etc.
Object have parts and these parts have multiplicity of 1 or more. Books have Title, Humans have 2 arms, Cars have 4 wheels etc.
The relationship among an object's parts is essential to its being considered a member of the category. A pile of book is not same as catalog of book.

These are few things we need consider while deriving a framework for knowledge representation. Then there are additional complexities added to it as if the same word is used as noun or pronoun. He is Helium or He refers to another person depends on the context where the word is used.

Until Next Time....

Solving Problems in Semantic Web

2009-02-28T16:53:00.005+10:00

It's been quite sometime since I am hearing about Semantic Web and what it can do. It's been the original idea of the World Wide Waste (WWW) since the day one. But somewhere down the line we seem to have side tracked from our original path. But the question everyone is asking is :

In-spite of spending so much in terms of manpower and research funding why this problem is still a PROBLEM?
Why we are drifting away from the actual problem i.e. the Semantic Web?
How much time it will take before Semantic Web becomes a reality?

The answer is simple because nobody is trying to solve it. We have vendors whose sole purpose is to push and promote their tools and technologies. We have committees that work on the standards but somewhere they get side-tracked. Nobody is working towards the big picture where we can see all the pieces of puzzle fitting together. But everybody is working on small-small pieces and then calling it a step towards the Semantic Web. But that is not the approach that will work in this case. To achieve Semantic Web we need to :

Identify what is needed to make it work. We are better of going away from current format of Web as trying to extract meaning out of Web (today) is like trying to extract water from stones.
Bring a team of experts who then will sit together and get their heads aligned in one common direction.
Build teams that work towards solution aligned with the top-level goal.
Build the tools and technologies that is solely designed to build the Semantic Web.
Last but not least Start Afresh.

I am sure if we take this approach we will have Semantic Web as a reality not just another science fiction.

Until Next Time....

Reasoning Mechanism and Project Halo

2009-02-16T08:27:00.002+10:00

After a long silence I am back again to unleash the ideas through this blog. A while ago I had a look at the Questions of Project Halo and that made me thinking as how can I represent chemical elements and work out a general mechanism to represent elements, compounds and chemical reactions.

What I found that using the classification described in Tree of Porphyry it was easy to describe them all and also that made the whole scenario simple to explain. Basically the chemical reactions has 3 core parts. Chemical Elements (including molecules), Compounds and the Chemical Reactions. For a chemical reaction to proceed we need to either elements or compounds or a mixture of both. The result is again a compound, element etc.

In the future posts I will discuss the procedure I am following to tackle the elementary level problems in chemistry. I would love to hear from the readers if they have a problem scenario in mind they want to discuss and then we can work out how to solve this using the reasoning mechanism I am working on. The more complex the problem is, the better it will be for me work out an appropriate way to solve the same.

Until Next Time....

Facts, Beliefs, Truths, Goals, Statements

2008-11-20T22:25:00.004+10:00

Off-late I've been busy reading about Semantics of Facts and Goals etc. There are many things in our day-to-day life that we come across. These are Statements, Facts, Goals, Beliefs, Truths, Obligations etc. Most of what we know or don't know fall in any (or more) of these categories. This made me thinking as what is this all about? What is the common thing among all these? What differentiates them? and many more questions pertaining to this topic.

Recently while researching on these topics I could draw these relations.

Statement is the common ancestor of Goal, Beliefs, Obligations, Truths, Facts etc.
Except Goal everything works is valid for a given time (has time component) and place.
Expression of goal needs two states for the same object. One is the initial state and one is the State of the object at any given time t when we claim the object has achieved the goal. The statement that an entity has achieved its goal is always in comparison to the state of entity at the time when the process to achieve the goal begun.
Beliefs change over period of time.
Facts are discovered not invented. They are present whether we know about them or not.
Facts belong to Closed-World semantics.
Beliefs belong to Open-World semantics and as discussed earlier they are constructed and can be destroyed as well.

I still have long way to go on this way. Any pointers from readers would be good.

PS: Its been almost 2 months since I had my last post on this blog. But I plan to come back in full swing soon and start posting regularly.

Until next time... !!!

Predicate and URIs

2008-09-01T21:59:00.003+10:00

While going through different papers I realized that almost every research paper has different functions that do some common operations and work on two objects. Say isMotherOf(x,y) could be mother(x,y) and so on. Now the questions arises that if there are many such predicates exist in different knowledge base then how will someone know that they are related and they virtually do the same thing.

The Predicates are like function or relation-builder as they establish relationship between two objects. So we can very well say that the predicates are the base for our reasoning mechanism or determining what the two objects are all about. It also plays an important role in the context definition of the two objects.

This gives me a strong point to put forward the predicate being a URI so that two knowledgebase if they are using the same predicate or relation then they do mean the same thing. Using this principle a Reasoning Engine can be developed that can extract the meaning of the statements / facts etc in the system. If we design a system on this principle then the amount of ambiguity we need to deal with will be less.

The RDF facilitates the Predicates to be both URI and simple text, but in long run the simple text might look flexible or simple to work with but more people will use URIs for the predicate in RDF. Removing the ambiguity once and for all is a distant dream though. Even though we follow these principles, we will still have some duplicate predicates in the system. But somewhere along the line we need to develop a mapping that will assign aliases for both the predicates and based on popularity the less used one can be phased out in due course.

Until Next Time.....

Entities and Representing Facts

2008-07-21T20:57:00.003+10:00

In my previous post I started a discussion about Inferencing and Facts. The biggest challenge we have is how to represent these facts in the computer system. Last week I was reading the paper "A Library of Generic Concepts for Composing Knowledge Bases" by Ken Barker, Bruce Porter and Peter Clark, Proceedings of K-CAP 01, October 22-23 2001.

In this paper the authors make a classification between Entities and Events. Entities are things that are and Events are things that happen. Events are states and actions. States are static situation brought about or changed by actions.

To elaborate further on this Entities are the state information or the facts. Actions apply on those entities. So if we revisit our earlier post the facts can be represented as attributes of an object. Like Sun rises in east can be represented as Rising Direction (Predicate) attribute of Sun (Subject) with a value East (an instance of type direction). Similarly we can also represent other facts mentioned in the earlier post.

Any second thought????

Until Next Time ....

Inferencing and Facts

2008-06-05T20:55:00.003+10:00

In my previous post I started a discussion on Reasoning and Inferencing. Going by very definition of inferencing it is an act of attaining conclusion based on certain facts. But what is fact? The facts could be

Universal Truths like Sun rises in east, it is winter in Australia.
Statements about an object instance. Toyota Yaris, YRS Rego ABC 123. Here we are considering only about one car.
A general statement about all objects of one type. Toyota cars are better than Honda in terms of easy maintenance (I am not going to start car manufacturer war here).
Statement applicable to more than one type of object. If battery is down then none of the battery operated or petrol vehicle will start.

But the actual problem is not what the fact is. The problem begins when we want to store the fact for computer systems to understand and reason. We need to store them in our KnowledgeBase which is nothing else but a collection of facts about one or more entities. To store facts in KB we need to have few issues sorted out like:

How do we represent the facts in Computer System?
How do we link the facts to the entities they describe about?
How do we retrieve the facts and relate them to the entities?
How do we find all the facts that are known about an entity?

There are many such buzzing questions which needs to be answered before we go ahead with building a system that infers these facts. I would like to get an opinion from the readers as what their opinion is.

Until Next Time...!!!

Reasoning and Inferencing

2008-05-22T09:50:00.003+10:00

Whenever I read about a journal or article on Artificial Intelligence, it sound like a science fiction to me. Okay okay I may not be up-to-date on my knowledge about what is happening in AI field. But more often than not what I find is most of the scenarios described in those fiction (?) are related to inference.

I read somewhere long back that Inference is the act of attaining a conclusion based on certain facts already present in the system. What are facts? In my opinion it is do to with the statements presented before us. But does the computers understand the statements as we do? I guess not. Then in relation to computer system the facts are the object, and their attributes. So we have few objects and their state information (as attributes) and we need to deduce a conclusion from that. How do we do that?

In order to combine these facts the system needs to have certain ability (A set of rules which will let us combine these facts together and infer something). This ability could not be anything else but Reasoning. By Reasoning we mean semantic relationship here. As we discussed in earlier posts that we need to have proper annotation in order to do Semantic Search and establish semantic relationship among entities in the system.

I guess I am getting more and more philosophical on this topic. I remember in our childhood we used to have a phrase. More Study More confusion, Less study less confusion, No Study NO Confusion :).

I would love to hear from readers about their opinion on relationship between Inference and Reasoning.

Until Next Time... :)

Can we beat Google for Web Search?

2008-05-14T14:46:00.005+10:00

Sounds like the most difficult question we faced so far? In Today's world we are relied on google so much that we are not in a position to think that we can have a day at work without using google search and also that there is a better search engine than google. Google has created so much hype around and has made us dependent on itself that we benchmark every new search engine against google. Including those who existed before google like yahoo, altavista etc.

But while google is becoming powerful with each new application it releases, and upgradation it does to its search engine there are still few important points which is missing in google search engine which happens to be the core of google.

Most of us who are interested in Search Engine and how it works have read the paper published by Page and Brin on original google search engine architecture and also the initial version of page ranking algorithm. But over last few years they have believed to change the original page rank algorithm. There are few problems with this search engine.

The page rank considers based on Words and the documents.
The google search is based on current web. Whereas the web is growing and evolving with every passing minute. The paradigm of World Wide Web is Persistent Publish and Read. Which holds good to an extent but the web we are looking at today is evolving. We are not in the era of one publisher and many readers but today we have more content producers than readers on web.
The page ranking algorithm uses the index table and the crawler (software) traverses through the links available on page to navigate to next page and so on. The philosophy what google and many other search engines have adopted is to represent the pages as set of nodes (or documents) connected to each other by a static link (HREF). They see it as some sort of tree structure. Whereas the web is not exactly like that. There are pages that do not have any link at all no incoming and outgoing link. Such pages are left behind by google search. An example is my poetry page which is very much hidden from the google search. Though its been on web for almost 3 yrs now. The google crawler managed to reach the main page of my homepage but could not get to the poetry page as there is no link to poetry page from the main page.
We do not maintain a registry which is based on relevance for the web pages outside the page. The google search engine uses the keywords found in the page while indexing. But there are chances that a page which is relavant might not contain the keyword at all.
Though Google is planning to use Latent Semantic Indexing for its next upgrade for page ranking, the accuracy of result is still doubtful.

This was about the problem, but then what is required to beat the google search engine? As discussed in my previous post on similar topic. I stressed the need for a Semantic Search for the web. The semantic search is missing in google search engine and unless that is made available the google search (and for that matter all other search engine) will still give us the irrelavant results (in abundance) when we query them.

Until Next Time... :)

Why Semantic Search?

2008-05-09T09:45:00.006+10:00

In my previous post I discussed search engines in general and also how do they build the index table which is the core of any search engine. One thing which became very clear after these studies that the search engines available today are very limited when it comes to functionality. The keyword search does not leave much room for returning the relevant result. In google if we enter Paris Hilton as a search keyword we also get Hilton Hotel in Paris returned as search result that too on the first page. But the search engine there is not able to distinguish that we are not looking Hilton in Paris but Paris Hilton celebrity. On the other hand if we enter Hiton Paris we also get Paris Hilton in our search result. One way or the other the search result we get is not relevant to what we are looking for.

Last night I was reading about Latent Semantic Indexing (LSI) and that did give some hope. I found this page at SEOBook explaining it in a much simpler way about LSI. There are other references as well but this is one page which other than Wikipedia that explains it in a layman's term.

But the million dollar question we are faced with is whether LSI is going to take away the pain of going through irrelevant search results when we query the search engine. In my opinion that is still not very clear. As the algorithm of LSI is still based on the keywords found in the document. And that is not going to take the pain away unless we use the semantic search. But then why semantic search?

Semantic search as most of us know is based on the meanings conveyed by the objects. The term meaning has more depth than it appears from surface. The semantic search is not new, its been there for centuries now. In ancient times philosophers have given the mantra to the world as how to perform the semantic search. Its just that only a handful of people (technologists) today take the pain to read through those literatures. What the current search engines fail today is to restrict the result to what the user wants. We are allowed to input only a bunch of keywords.

In case of Semantic Search the driving factor is context as different terms (or concepts as John F. Sowa describes it) have different meaning or interpretation depending on where they are used. If we build a search engine around these philosophies then we can definitely achieve semantic search (upto a great extent).

Until Next Time... :)

Reusing Ontology

2008-04-28T22:17:00.003+10:00

In one of my earlier post I had put emphasis on Why we need a Common Ontology. Over last couple of days while reading through different papers and books I came across few cases which explains why we need a common ontology.

One of the basic idea of Semantic Web is to allow user to reuse an existing ontology if it meets our needs. Alternatively in the worst case we should be able to use a part of it to fulfill our requirements.

At the same time if every user of the web starts to build their own ontology then there would be no common language and shared understanding about anything. There would be no interoperability of any kind between the two agents using those ontologies. There would be no global processing possible either. The exchange of message will not take place and different machines cannot interpret the messages either.

Thus without ontology reuse the very basic idea of Semantic Web is void. Reusing ontology to an extent is even more important than reusing a URI.

Until Next Time.....

Building Index Table

2008-04-23T21:36:00.005+10:00

In my previous post we discussed about Search Engines in general. We also discussed that there are few basic functionalities of a Search Engine. They are:

Building Index Table
Performing the Search
Building the Result for us

Building Index Table
In this post we will predominantly focus on how the Index Table is being built. The index tables are the heart and brain of a search engine. The process of building index table begins much before the search engine is live. This is an ongoing process which begins much before the search engine is made available and continues till the search engine exists. In a way we can say that the process of indexing determines the quality of result from the search engine.

The indexing is done by a piece of software called Crawler aka Spider. The crawler as the name suggest crawls on the web page and collects virtually all the information it can from the web page. The input to the crawler is the main URL of the web page. Once the crawler receives the URL it performs the following:

Build an Index Table for each and every word on the Page. Since the word may appear more than once in the document it stores the word, the URL and the number of occurences of the word in the document. This is being done for almost all the words found on the page.

Once the crawler is done with building table for each and every word on the page it then navigates to the first link which happens to be a URL again and crawls the new page. ie it performs the similar activity what it did before ie building an index table for each and every word on the page. At this point in time there are two situations possible.

It encounters the word that is not part of the current (or previous) document in the index table. So it just adds the new word to the index table along with URL and number of times the word is found in the document.

b) The word already existed in the table and in that case it locates the word in the index table and adds the reference to second URL where the word is found to it. Also the number of times the word occurs in the document.

Once the crawler is finished with the current page then it moves on as described in Step 2.

If there are no unvisited link found on the current page then it will go back to the previous page and will start from next link found on the page and will repeat step 2 and 3.

The flaw of this method is that the web is infinite and practically the step 2 - 4 will never finish. The best possible outcome of this procedure is a fraction of web pages are indexed today. Google which is assumed to be the most powerful search engine can index only 1-2% (approx) web pages on the World Wide Web.

This is not the efficient mechanism to build the index table. There has to be a limit where the crawler has to stop going further down the hierarchy and crawl to other pages in the list (in the original page. In the future post of the series I will bring out the discussion on different approaches to crawl the pages. The choice has to be made whether to go for Dephth-First or Breadth-First. For now we assume that the index table is being built and the search engine is ready to perform the search operation.

Performing Search Operation
The index table is used when we type in the keyword to perform search operation. In simplistic term it goes through the index table and searches for the keyword and the document (URL) where it appears and then builds the list of documents to retrieve. But as I said earlier this is the simplest case. The actual result building mechanism has much more to it than just retrieving the documents and presenting it to the user.

In the next post in this series I will bring the perspective of how the result is being built and shown to the user. What affects the page rank and few more details about the same.

Until Next Time... :)

Search Engines

2008-03-25T21:14:00.000+10:00

After a long gap I am posting something to my blog. Well there has be numerous activities and the most important among all those was getting married last month. The whole February month was filled with travel, meeting family and friends etc. Finally the day when I realized, the holiday was already over and it was time for me to come back to real world. Well I am back now in real world and will be posting something interesting as I discover something on the way of my research on the semantic web.

The most common use of internet today is for searching. The idea is to locate and access information or resources on the web. For example finding out more about the Formula 1 cars etc.

Today the search engines are based on Keywords. They can retrieve documents which contain the given keywords. As long as they given document contains the keyword it will be included in the search result and later shown to the user. The current web then passes on the pain to read and interpret whether the page makes any sense to the user or not. To understand this let us see how search engines are constructed. In this and few more upcoming posts I will be discussing in detail about the search engine and why they function how they function. In this post I will focus primarily on the problem (as described earlier) and some more detail about the search engines.

Today the contains hundreds of millions of web pages. To locate few handful of paged we might be interested in among those hundreds of millions of paged we use search engines like google, yahoo etc. They are the most popular search engines besides others like Altavista, Live Search etc. Inspite of the differences claimed by them a large part of it still remains the same. The fundamental of a search engine building remains the almost the same.

In the future posts will discuss about how the search engines are constructed and why they can do only the keyword search. The next post will be based on creating Index table which is being used by the search engines.

Until Next Time....

Work on Annotation Found Elsewhere

2008-01-28T19:10:00.000+10:00

Recently during my regular research work on annotation I came across some work done by W3C on Annotation. I was surprised to see that the work on annotation was quite active back in late 1990s.

Annotation project using RDF at W3C:

WWW9 Developer's Day presentation
paper submitted to UIST 2000
RDF Description Services (discussion note for the RDF Interest Group).

some interesting work elsewhere:

Dan Connolly's Javascript code for excerpting from the Web.
The Crit service, based on CritLink (opensource Perl software) is one of the longest established online annotation servers.
Netscape's What's Related service uses RDF to send 'related link' annotations to browsing clients. See also Mozilla documentation.
USC Annotation Technology pages include a set of specifications for annotation software design and a Java implmentation, Annotator.
Aggregating Recommendations using RDF, Libby Miller.
NCSA Hypernews
ComMentor: Scalable Architecture for Shared Web Annotations as a Platform for Value-Added Providers, at Stanford
Cornell Annotation System
Web Interactive Talk claims to be 'the first dialectical discussion system'. Here is a An Overview of 'Web Interactive Talk', the "first dialectical discussion system" (1994)
A Protocol for Scalable Group and Public Annotations, by Daniel LaLiberte and Alan Braverman of NCSA
Public Annotation Systems, discussion and implementation by Wayne Gramlich at Sun.

I hope you enjoy reading these links as much as I did.

Uniqueness of Entities

2008-01-06T23:04:00.000+10:00

I was having a discussion with a friend of mine about what makes an object (entity) identifiable. The conversation started with different attributes of the object and then we got into a situation where we had to distinguish two objects who had similar attributes.

More often than not in an Enterprise System we are faced with a situation where two objects come across having similar attributes, which primarily (on a higher level) identify them. When we are faced with such situations then the only way out is to identify another attribute attached to the object which is bound to be unique. In a database application we have the primary keys generated by a sequence generator which guarantees its uniqueness. One the commonly used real-world example is Social Security Number in USA. The social security number is bound to be unique. Another unique attribute is the Credit Card number which is supposed to be unique as well.

In light to the above example the question we have is what is the unique identifier of the object. Is that something attached as as attribute to the object or it is the one which defines the object (rather the object itself). The more we think and discuss about it the more we come to a conclusion that there is no fixed rule as such. Both object being its own unique identifier or the object having an attribute which could uniquely identify it have their own pros and cons. COM uses GUID as a unique identifier for the objects created. A GUID is likely to be unique even though it is generated at many computers simultaneously and is done for years without any interruption.

I personally kind of find myself torn between the two approaches. Object being its own unique identifier has the benefits like the two objects can be identified unique even though they have no attributes to identify them. Whereas having an attribute which makes an object unique is that we can always work out a better combination (of attributes) if the current one is no longer able to gurantee the uniqueness.

I would like to hear from our readers what approach they would prefer in such scenarios and your experience as well. Hope this year brings lots of Joy, Happiness and good times to all of us.

Until Next Time... :)

The Issue of Annotation

2007-12-23T06:57:00.000+10:00

A while ago I was doing searches on Google Scholar, ACM and IEEE databases to find some good references on Web Service Annotation. To my surprise I could find only few papers which had some emphasis on annotation. It made me think whether the issue of annotation is really an issue or not. I could find lots and lots of work on WS Composition, Automated Discovery, Workflow etc but annotation was something left behind by many researchers around the world. To me annotation is a research problem because:

With the growing popularity of Web services, there arise issues of finding relevant services, especially with the possibility of the existence of thousands of Web services.
Web Services are typically used as part of larger Web processes that result from Web services composition. Current Web Service standards have focused on operational and syntactic details for implementation and execution of Web services. This limits the search mechanism for Web services to keyword-based searches.
With the growing number of Web Services the current search mechanism (keyword-search) is not going to be feasible way to find Web Services. As the keyword search will return virtually every service which has the keyword mentioned as part of the description. The user will then have large number of services to filter from (most of them will be irrelevant though).
Semantically described services will enable better service discovery and allow easier inter-operation and composition of Web Services.

Considering the role annotation is going to play in the problem space it is a need of the hour to find a suitable annotation mechanism that will be solve or eliminate the problem to an extent. I would love to know what you think about the annotation and its role in Semantic Web Services. In future posts I will discuss about different annotation models.

To every visitor of my blog. Wish you Merry Christmas and a Happy New Year.

Until Next Time....:)

Evolving Knowledge

2007-10-26T12:11:00.000+10:00

In one of my earlier post this year I discussed about atomicity of knowledge in itself. Then in subsequent posts I had an attempt made to relate the knowledge representation to stars. Also had the types of knowledge discussed a while ago. But they are just the bits and pieces of the knowledge is and what we call knowledge. In this post I am going to discuss about the evolving nature of knowledge.

"Change is the only constant thing in this world", Most of us have heard and read this line at different texts, blogs, literatures etc sometime or other in our life. But what do we mean by change here. Are we talking about something totally different than original or an evolution of what was originally available. The evolution at times could be n folds and the next evolved outcome may look like a different thing altogether. But as a species of this planet we have been evolving for billions of years now.

The evolution of life on this planet required billions of years for its first steps (primitive cells, DNA), and then progress accelerated. During the Cambrian explosion, major paradigm shifts took only tens of millions of years. Later, humanoids developed over a period of millions of years, and Homo sapiens over a period of only hundreds of thousands of years. With the advent of a technology-creating species the exponential pace became too fast for evolution through DNA-guided protein synthesis, and evolution moved on to human-created technology. This does not imply that biological (genetic) evolution is not continuing, just that it is no longer leading the pace in terms of improving order (or of the effectiveness and efficiency of computation). This is how we evolved from a primitive cell to the fully functioning (well almost) human being.

When we talk about knowledge. It takes a considerable amount of time to build the first bit of knowledge as that lays the foundation for future developments. One of the requirement of the knowledge is it should be evolving continuously with what is available. One similar example which relates to this nature is Fractals, to be precise Probabilistic Fractals. The Probabilistic Fractals the probability of each generator element being applied is less than 1. in this way, the resulting designs have a more organic appearance. So going back to use this theory in context of knowledge evolution the new piece of knowledge will appear more intelligent than the original piece which created it. The principle of Probabilistic fractal is already in use by computer chip makers. The computers are used to design the next generation chip which is faster and better than the chip which is used to design the new chip.

So the question which arises at this point in time is:

What is the best way to represent this tiny bit of knowledge which can sustain growth at any scale?
What are the other factors we must consider while designing this first bit of knowledge?
Is there a way we can predict the growth path or evolution of knowledge and can influence its direction?
What are the best practices we can or must follow to avoid and reduce chances of errors?

The problem is out there in front of research community. So I assume that there are few people who know the answers as well or at least have made some progress to find the answer. Any thoughts here?

Until Next Time... :)

Semantic Web and MVC

2007-10-20T22:30:00.000+10:00

A while ago I was going through a discussion on Semantic Web and then came across a good post on Web evolution and Model View Controller (MVC) paradigm.

Web 1.0: Only the View is available, the Model is completely missing. The computer can't interpret anything. The computer does what it is asked to do be right or wrong. No wonder the computers were called dumb terminals in the beginning.

Web 2.0: APIs are coming, Controllers give partial accesses to the Model. The computer starts to enjoy. We have the ability to participate in interactive discussion. Facebook, Orkut, Blooger etc are few to implement this. We have been given limited write access on Web. We have started to appreciate this and success of Wikipedia is an example of how much community has appreciated this concept. We are both producer and consumer. The traditional methodology of one producer many consumer does not hold good anymore. Everyone participates equally in the evolution and development of a concept (wikipedia), work (open source) or a process.

Web 3.0: The Model is largely opened, we are finally able to read and write the World Wide Database. The computer can't wait! We are given full access to write based on the access level we have. We are part of a big universe but still manage to keep our own little universe isolated and whenever we want we become part of the big picture. Its an intelligent me part of the We.

Check out my earlier post if you want to know what after Web 3.0

Until Next Time ... :)

Why we need a Common Ontology

2007-10-01T22:29:00.000+10:00

In my previous post I started the discussion whether we really need a common ontology? To go into detail of this first we need to find out how ontology is helpful in Semantic Web. To understand it better lets take a business case.

There is a Business A, Who deals with Selling Laptop and their Warranty. Business A sells buys the laptop from Business B and Extended Warranty from another Business C. Now lets consider a Customer D places an order for a laptop with extended warranty. In this case Business A needs to do the following:

Check with B whether they have particular laptop model in available in Stock.
Check with C if they offer Extended warranty for the laptop model (selected from B).
Combine the prices for both the services, Add its own markup to the price and generate final quote for the customer.

Now lets assume that the customer has decided to go ahead with the quote and place the order. In such case A has to place an order with B and also an order with C. The payment will be made to A by the customer which in turn will be routed to B and C as per their quote. When the parts arrive at A, it has to be bundled together and shipped to the customer.

This might look like a simple business case but behind the scene there are lots of activities which take place and most of them are too complex to handle. Also there are non-functional requirements like Data Integrity, Security etc come into picture which the users (customer) is not aware of many a times.

Also there are different document formats, types etc come into picture. Each vendor have their own format to accept the messages and the response is also sent back in a pre-defined format as set by the service provider.

Coming back to the topic how the Ontology is going to be helpful in such a scenario. The Ontology can apply to both the Document Type and Semantics.

Documents Types here refer to the structure of the business data that are communicated between the trading partners. Documents Type refers to Concepts, Attributes of concepts and relationship between concepts that are ideally managed by and represented within ontologies. Ontologies can in a very precise manner define as well as manage concepts, attributes and relationships between concepts.

The Semantics here refers to the correct population of attributes with correct domain values. Not every value that an attribute can contain is semantically correct. Like the model number attached to Laptop in the business scenario discussed must belong to a laptop. If a desktop model number is used instead that will be correct as far as syntax is concerned but it will not make any sense in context of laptop.

The Ontology Server can be used to store and access the domain and application concepts, like product name, country where it is available, warranty applicable etc, represented in form of ontologies. Since all these different attributes together must belong to an entity which is not only syntactically correct but also each attribute has some sort of relevance to other attribute (as discussed earlier). For this to happen in reality, they all must have knowledge about each other (at least what matters to them) and the cross-verification must be achievable.

A common ontology server comes handy in such cases as all entities and their attributes belong to one common hierarchy and they can seamlessly integrate and refer to each other.

Note to the readers: I would like to hear from you as what you think about it. If you want to discuss this offline, do not hesitate to drop me an email and I will be happy to discuss it with you.

Until Next Time.. :)

Web 4.0

2007-09-12T22:42:00.000+10:00

In one of my earlier post about What is beyond Web 3.0, I did mention that after Web 3.0 it is going to be Web 4.0? I also mentioned there that Web 4.0 will be about man-machine communication. There have been few discussions since then as what is the meaning of man-machine communication and how different it is going to be in future.

Before we discuss this lets take a look at what is been done so far.

Web 1.0 Theme was You. You produce content, You manage the server, You are the producer and I am the consumer.
Web 2.0 Theme is We. Lets discuss, Lets interact, Lets build our network. In this both the parties actively participate.
Web 3.0 Theme is Me. Its all about ME, my Environment(s), my personal space, people who can see me etc. We are talking about personalizing the Web (and Internet) to suit our individual needs.

So the question arises here is what is left for Web 4.0 to achieve. Well as I mentioned earlier it is all about Intuitive Man-Machine communication. The communication is not just via the command mode, but its based on the thought process generated within us. Sounds like a Science-Fiction isn't it? Well to your surprise many pieces of this puzzle are already in place. Its just that we need to start putting them together and tweaking them to suit each others need. How are we going to change human? We don't have to, thats the best part of this.

The Sigularity Institute is working on similar research projects for advancement in Artificial Intelligence. But that's not the whole point here. That just solves one side of the problem i.e. making machine understand the language. But we humans must have a way to translate our thoughts into that form.

I am sure many of us must have heard or gone through ECG test sometime in our life. We all know what it does. My thought of building such a communication channel starts from ECG sensors fitted to human body when Wearable Computer becomes a common man thing. Using those sensors we can capture the signals generated by human. The generated signals then will be converted to Bluetooth signal by attaching a small device to the computer we wear. We can pair the human bluetooth device with the computer (machine) and establish a communication channel between them.

Of course there will be requirements to build interpreters which will translate the ECG signals to appropriate software code and vice versa. Did I say vice-versa? Yes I mean it. Recent studies in medical sciences have proven that if we generate the similar impulse in our brain we get the same muscles stimulated as though it is happening in front of our eyes. Watch the movie "What the Bleep Do We Know" and you will understand what I am talking here.

Sounds like a whole new channel opening. Well this is going to be the future of communication. I am going to come back with some more thoughts on this and other related technologies.

Until Next Time... :)

The Need of a Common Ontology

2007-09-05T22:54:00.000+10:00

With lots of talks around about Semantic Web, one thing that stands out among all the terminologies used is Ontology. In my earlier post, What is Ontology, I touched upon the basics of Ontology and discussed only the basics of it. Towards the end of the post I proposed to build a common Ontology Server.

While reading Guizzardi and Guarino and after understanding about ontology in bit detail. I went back to read Knowledge Representation (John F Sowa) Chapter 2. But in this process what come out is a belief that if the Semantic Web is to become a reality and ubiquitous then we must have a common way to represent the entities. Which brings back to the need of a Common Ontology Server.

Going via the definition of Ontology given by the philosopher Willard Van Orman Quine the fundamental question of Ontology is "What is There?" and the answer which came back is "Everything". Now if something has to accommodate everything then it has to be be designed to keep every other interpretation of entities in this world.

Then the question arises is: Do we Really Need a Common Ontology?
Before we answer this question, we must ask what we will be loosing if we don't have a common ontology? In absence of a common ontology we might not be able to represent an entity in an uniform way. Uniform way here means by following a common structure and pattern. The rule layer (as part of Semantic Web stack) will not be able to operate properly. This in-turn will make other processing complex and tedious.

In search of solution I came across the paper "Discovering Semantic Web services with and without a Common Ontology Commitment. Jorge Cardoso. Proceedings of the IEEE Services Computing Workshops (SCW'06), 2006". This paper talked about the discovery of semantic web services without having a common ontology to represent them. But then there are few limitations of this methodology as well and it is not suitable for the original vision of Semantic Web.

I will keep working on it and see what I come across next. If you have any thoughts on this drop me a line and I will be happy to hear from you on this.

Until Next Time...:)

Normalization Where to Stop?

2007-08-18T00:06:00.000+10:00

One of the most important thing we learn in our DBMS subject is Normalization. Whether you are a data modeler, DBA, or SQL developer, normalization is one of those topics we all learn. We learn this either at work or during our formal IT degree. We are taught that there are 5 different normal forms and what goes in where etc etc. But how much do we use them?

Take a look at most production databases. The best you will find that the database has been implemented using Third normal form (3NF). Very few databases reflect higher normal forms, such as Boyce-Codd normal form (BCNF), the Fourth normal form (4NF), and the Fifth normal form (5NF). So, why don't most database designers go beyond the 3NF?

I am not going deep into the normalization and their definition. A google search will bring out all those definitions so I opted to keep the definition out of this post.

How far should you go with normalization?
To be sure, each progressive step may impact upon overall performance. I have seen normalization taken to absurd lengths. In one of the recent discussion one person came out with the idea of different financial document types. As though the world of accounting is going to change. I had to remind him that Accounting is just the recording of historical events :)

A while ago I was reading an article and there the author mentioned about the normalization. Over a period of time I learnt to ask few questions before I decide how much normalization is required.

What is the nature of the system. Is it an OLTP or OLAP system?
What is the nature of DB Query. Are they mostly Insert or Retrieve?
For Part of DB where the inserts are more, its better to have the Data in 3rd normal form.
For system where Retrieve operation is more than Inserts, 2nd Normal form is the best.

Where you draw the line in the sand is ultimately up to you, but you will be better equipped to draw it with a sound understanding of the various normal forms and the risks of not going far enough. And not to forget the business requirement, after all its "Not Your Software" its the users who will be using it not the developers who write it.

Why most designers don't go beyond the 3NF?
There are few factors which affect the level of normalization we choose for the database. Most of my decision to design the database in past were based on the answers I got back after asking these questions:

What is the insert/retrieve ratio?
What is the database used for Transaction Recording/Processing or Decision making?
What is the response time we are looking for in case of Insert, Update, Delete and Retrieve?
What is the estimated peak transaction load on the database also the off-peak hour transaction load.
What is the database deployment strategy Centralized or Distributed?
What is the transaction control strategy. Whether it is Single-Phase commit or Multi-Phase commit?
Is there a temporary Cache implemented or required?
Do we need to maintain the user session or user interaction with business layer is stateless?

Asking these questions does give an overview of how the system will look and to suit the need bes the appropriate strategy can be formed.

Sometime in coming days I will write a post on how to optimize database and what are the common mistakes database designer make. Keep watching this space for the post.

Until Next Time... :)

Mandatory and Essential

2007-08-16T11:20:00.000+10:00

Last night when I logged onto yahoo messenger, I saw this message as one of my friends status message "What is the difference between Mandatory and Essential". At first I thought what's big deal? Are they different? Are they not same? At one point I went to an extreme thinking that his Phd course has taken its toll and he is about to get into second phase of PhD degree where one becomes grumpy, tired and insane :) But the more I started to think on this the more it became evident that Mandatory and Essential are NOT the same thing. They are two very-very different words when it comes to their real meaning.

I remember doing a post earlier this year on Atomicity and Knowledge Representation, I discussed about Mandatory, Additional and Optional Attributes of an object. While I am not going to go deep into philosophical aspect of it, I would rather stay in context of Software Engineering and Semantic Web.

Mandatory:
Mandatory Attributes signifies the bare minimum required for existence. It is the basic definition of any thing(object) in this world. In reference to my earlier post on Atomicity it is the Mandatory Attributes of an object. The bare minimum requirement for any object which exists in any space is Type and a unique attribute (most of the time it is Name) which distinguishes the object from others of its kind in the same space or for any other space where the object is likely to be found.

Essential:
Essential attributes of an entity (object) is the must have attributes for an object in any context or where it has to be used or play an important role. More often than not it is also the mandatory attribute of the object. Referring back to the post I did earlier this year, I see Essential Attributes are nothing else but the Additional Attributes of an object which is required for an object to participate in certain activity or to be used in a context.

In context of Knowledge Representation we can always rely on Mandatory Attributes to be present where Essential Attributes are (due to their contextual nature) guaranteed to be available only if we are considering object in a given context.

Would love to hear your thoughts on this as what you think the Mandatory and Additional attributes are. Whether you see them being same or different, if yes then why?

Until Next Time... :)

What is beyond Web 3.0

2007-08-07T23:23:00.000+10:00

Recently there have been talks about what the Web 3.0 should be and what it will do etc. Go to facebook, myspace, orkut or any other site where techies come together and most of them you will find talking about Web 2.0 or Web 3.0. But among them there was one gentleman who said that Web 3.0 is going to be about Relationship Economics. It is the user who will drive the next generation of advancement. Lets see

Web 1.0 and 2.0 has been predominantly technology oriented. Starting from a command line browser to pull texts of a server by passing the URL to building websites to facilitate social interaction the journey is been quite long (almost 17 yrs) and in this journey we discovered many technologies and we left many of them. Some of them we still keep with us even after years and some we left along the way.

Web 3.0 is about deep and meaningful relationship, effective networking. If Web 2.0 is about breadth i.e. adding more and more feature and technology advancement, Web 3.0 is about depth i.e. creating an understanding as why and how we can leverage upon them.

Most significantly, Web 3.0 is the turning point where the 100 year cycle of transaction economics is superseded by the next great cycle, usually described as relationship economics. Web 3.0 transcends and includes both 1.0 and 2.0 to provide the platform that provides lift off for this great shift in economic, organisational, social and cultural structures. Web 3.0 is likely to be born out of a major crisis affecting current structures. If you are part of Web 3.0 you will be carried by a great wave. If you're not, you may find it difficult to survive.

But then then thought came to me what is going to be after Web 3.0? Well how about Web 4.0? Seems like a light bulb glowing :) but yes what advancement we will seek in Web 4.0. The way I see is Web 4.0 will go one level further and take the man-machine relationship to the next level. The machines will be able to understand human language and the there will be a change in our behaviors towards machines and vice-versa.

If we call Web 3.0 a paradigm shift then Web 4.0 is going to be the Dimension Shift. I would love to hear your thoughts on this.

Until Next Time... :)

WWW - Past, Present and Future

2007-08-03T23:56:00.000+10:00

Lately there have been lots of talks going around about World Wide Web. Some people are giving their best shot to describe what the web is today and where it is going in future.

When I started to look at where the web started back in early 1990s and where it is today and how it evolved. I found the whole phenomenon quite interesting and the trend with which it is developing is quite interesting too. In this article its my attempt to capture the Past, Present and Future of World Wide Web. The special emphasis is put on the future trends of Web and how I see it evolving.

Past
It was back in 1991 when Sir Tim Berner Lee along with his colleague Robert Cailliau and Necola Pellow developed the first web browser. This was a command line interface which could read the content from a given location (Servers). But that was just the beginning of a whole new era for the humanity. These browsers were capable of pulling out content (text only) from a server.

It wasn't until 1993 when Marc Andreessen and Eric Bina invented the "" tag for HTML which revolutionized the way web pages look and allowed developers and artist to unleash the creativity. In the same year MIT developed the technology to index and count the web servers which took the architecture of Web to next level.

Present
Today when we talk about Web we talk about many things like Dynamic Web Sites, Blogs, Animations, Social Networking etc. In a nutshell we have everything we need to suffice our requirements. If we have to summarize the Web today we can describe following characteristics for the Web today.

Portability: Today we are not just limited to browse web through our desktops. Instead we access web from our mobile devices, hand-held devices, TV, Digital Camera. Also the choice for input device is not limited to keyboard and mouse. We also use Stylus, voice recognition etc to interact with web.

Diversity: When first started web pages were available in only English. But today we have many languages in which there are web pages available. Google Search Engine is available in 100+ languages and its increasing. Though the percentage of pages in English is still more compared to other languages but, other language web pages are increasing their share of pages on Web.

Distributed Technologies: Web is becoming distributed now. We have technologies like Web Services which allow the user to share the applications over Internet (not Web). We can not build a new application by using services which is already out there available.

Collaborative: In today's web if we have to name one technology which is been evolutionary it is collaboration. Wikipedia is the largest online encyclopedia available today and contains information about almost everything. The wiki is the best example of what collaboration can produce.

Social Networking: Today we have websites which facilitate social networking and allow users to interact with other users of the site. MySpace, Orkut, Facebook are few in this list who have got a larger share of this market. There are other sites like LinkedIn where one can create their professional network. So in a nutshell the web is not just publish and read.

Consumer and Producer: Today the user is not just the consumer of the information published on the web. The user is also the publisher and consumer at the same time. A web user can publish his thoughts / ideas etc in form of a blog which may include some of the texts which is been published at some other site or contains a link to an external site.

Future
With so many technology choices available today, one might as well start to think what new we need on web? We have the tools and technologies which are pretty much sufficient for our requirement. The current buzzword in Industry is Web 2.0 (which is the technologies available today) and Web 3.0.

The evolution of Web 2.0 will lead us into Web 3.0. It has enabled the users to start thinking in a new paradigm (which is really an old paradigm) collaboration. This was the original idea why World Wide Web was created in the first place. Web 3.0 (aka The Semantic Web) - is really just an extension of the original collaborative concepts which was the main driving factor for the World Wide Web. But it makes the data connected in a sense that is more relevant to the user i.e. based on Contextual.

With the advent of Web 2.0 we have seen the rise of user generated Taxonomies, Folksonomies, and other related Information Architectural Methodologies. However they are still limited as in 2.0. The classic example is the taxonomy of one source does not equal to the taxonomy of another source. Though with the growing usage of open API's we are starting to see the taxonomies shared on a collaborative level between systems. In the Web 3.0 conceptualization we will see the data is shared not through API's, but through the structure of the data, i.e. through the meaning and context of the data.

In the Semantic Web the data (entities) will be linked to each other at a deeper level which is most likely to be in their raw form. The linking among the data is not going to be pre-defined rather it will be based on the few aspects like Context (in which they exist), the meaning, any other similar behavior they might exhibit. According to few I had discussion with in past they feel that there will be some kinds of standard emerging to link the data. But as we have witnessed in past the standards are not always the best approach either. More often than not those standards are heavily influenced by vendors and their proprietary technologies.

But then the question arises is how will this work out? and how will be able to achieve the Semantic Web? Whether we ever be able to achieve it or not. One of my friend (Nathan McCosker ) with whom I regularly discuss on Semantic Web thinks that it is going to be an iterative and evolving process. Initially there will be few groups coming together to form a standard what we may like to call Web 3.0 but then there will be further corrections and the standard will evolve and we may as well end up versioning them as 3.1, 3.2, 3.x etc.

But this is not where it ends. Current research works have no (or very less) emphasis on what the user experience is going to be with Semantic Web. The research is useful and accepted by masses only when it is done for the masses. The research labs around the world have started to throw different technologies like RDF, RDFS, OWL, RUL, RVL etc in the basket but they have not touched the nerve i.e. the problem they are trying to address or the pain they are trying to relieve. The technology is not the bottleneck here. Today or tomorrow the experts will find the technology solution to make the things happen, but the biggest hurdle is why? As in "The Matrix" Movie they always talked about the reason/purpose for anything to happen we need to apply the similar logic here and find out why we need the Semantic Web and when (not if) that is defined it will not be a theoretical discussion at all.

Suggested Reading

Those who are interested in knowing more about Web 3.0 and beyond would like to read through the following links.

http://www.iht.com/articles/2006/05/23/business/web.php
http://www.pcmag.com/article2/0,1895,2102852,00.asp
http://www.androidtech.com/knowledge-blog/2006/11/web-30-you-aint-seen-nothing-yet.html
http://www.roughtype.com/archives/2006/11/welcome_web_30.php
http://www.alistapart.com/articles/web3point0

Conclusion
Semantic Web is becoming next buzzword in the industry and without knowing what it is in detail many people have started to claim their product being suitable for Semantic Web etc. So far it is been purely theoretical in approach and those who are claiming that their product is based on Semantic Web technologies are still on the way. They are not there yet.

Until Next Time.. :).

Interoperabiility

2007-07-17T14:49:00.000+10:00

Interoperability is one of the buzzword that with the Advent of web 2.0 is become famous day by day. Every now and then some company announces that their product is fully-interoperable etc. Slowly it is becoming one of the most abused terms as well, like many other terms associated with Semantic Web like Ontology, Taxonomy etc.

According to Wiki the term interoperability with respect to software is defined as, the capability of different programs to exchange data via a common set of business procedures, and to read and write the same file formats and use the same protocols. (The ability to execute the same binary code on different processor platforms is 'not' assumed to be part of the interoperability definition!)

In a simple sense the term interoperability can be defined as Ability to work with each other, Without knowing the details of how it works. But it is easier said than done. We do have some standards like Web Services(WS) which can very well be useful to achieve interoperability but at what cost? If we use WS then we are tied to the input and output message structure and given the case sensitive nature of XML it becomes a nightmare at times (at-least to debug).

But at the same time if we can achieve the persistent publish and read mechanism for the systems then it should not be a problem as I see. What is required here is:

A common structure at least in theory which could be understood by an application at runtime. What I mean here is we should have the data and data structure (metadata) also persisted somewhere. So that application can pickup metadata and parse it to understand the data. Upto an extent XML, XSD or RDF and RDFS can solve this issue. The XML (along with its subsidiary technologies) is the best bet here though there are numerous overheads associated with it.
URI (URL) based access of data. So that one application can save the data to a location which is picked up by another application. Also the consideration must be made to ensure that it accidentally does not erase data written by another application if the space is shared by many applications.
Some way to notify application that data is persisted so that another application can read the data. Alternatively the message for recipient can be written in a common pool which the recipient inquires periodically to see if it has an incoming message. A Message Queue implementation (JMS, MSMQ) will be handy here.
A parser at both Sender and Receiver end to parse the metadat and then interpret the data. For reasoning the metadata and data both OWL-S can be used. Not sure how much helpful it will be to use Description Logic here though.

I guess this is just the beginning and I am going to collect all these bits and pieces sometime to bring them together.

Until Next Time.... :)

Graph Theory and Semantics

2007-07-10T10:15:00.000+10:00

Last Weekend I spent most of my time revisiting Graph Theory which I studied first in 2001 during my MCA course and since then had forgotten quite a lot of it. At that time I did not find this topic that interesting and studied just for the sake of clearing exam. But last week I was going through some links on Wikipedia and it struck to me if I can use Graphs to show the relationships among objects in a system where the objects are attached to each other based on some sort or relation.

Going back to what John F Sowa described in one of his book Knowledge Representation about concepts and relations. In my opinion there lies the potential of using Graph Theory to list the relationships among different concepts existing in the same space.

Each Vertex of the graph could be the concepts as defined by Sowa and the Edges of graphs could be the relationships between the two concepts. The Edges can be uni-directional or bi-directional depending on what kind of relationship the two concepts share. The immediate benefit of this as I see is, it will facilitate the visualization of the system where each of the object exist in isolation.

While this is the thing that came to my mind when I had a read through the Graph Theory last week, I can see it being applicable to more areas as far as semantics is concerned. I am expecting by this weekend I will have a good amount of information as whether and how much graph theory can help in semantic space.

If you have similar thoughts would love to hear your views on this as well.

Until Next Time... :)

Off Topic - A Mysterious Visitor

2007-07-09T12:11:00.000+10:00

Hello to all the readers and visitors of this blog. I thank you for the time you spend reading through the thoughts I put in here.

However recently while looking through my visitors log I found that there is a Mysterious Visitor to the blog who is located in LA, California and (s)he loves the blog so much that he makes it a point to visit at least 4-5 times a day at times. I am yet to get the complete detail of this visitor and just wanted to let him know that I am very close to know who you are. But before I do that it will be good if you can let me know little bit about yourself. I will be happy to exchange thoughts with you.

My email id is located on top left of the blog page and last time I checked it worked fine so that should not be a problem either.

Little busy these days reading graph theory and its application(s). I am trying to find a possibility to represent a semantic relation using graph theory. I will be back with regular content in couple of days again.

Until Next time.. :)

Intelligent Reasoning - Demystified

2007-07-04T20:01:00.000+10:00

Often in the texts of AI, I come across this term called Intelligent Reasoning. Different texts give their own definition of this term. More often than not the questions related to Intelligent Reasoning comes back to me.

What is Intelligent Reasoning?
How to define Intelligent Reasoning?
What is the requirement for a reasoning to qualify as an Intelligent Reasoning?

In quest to find answer to my questions, I first looked at different sources for the definition of meaning hoping that adding intelligence to the process of reasoning is the right answer. But after reading tons of text I understood reasoning as :
A way or process to base our beliefs, actions, philosophies, concepts etc. This is in layman's term what I understood about the intelligent reasoning. There are resources available who had more complex definitions than I can understand.

Now that I understand what the reasoning is, the questions about the Intelligent Reasoning again popped up. I looked back at the questions once again and here is the answer I came up with. This is related in context of Artificial Intelligence and applies to machines only :)

What is Intelligent Reasoning? A process to establish a statement, a theory, belief or action in a way as it is performed by human beings.
How to define Intelligent Reasoning? We have various ways of looking at things, 1) the way they appear or 2) the way they function. While the first approach focuses on Why and What, the second concentrates more on How?. Using these as a mechanism to based a concept could be closely defined as an Intelligent Reasoning Mechanism.
What is the requirement for a reasoning to qualify as an Intelligent Reasoning? Any reasoning mechanism which is built on the top of existing facts and which co-relates statements, facts to deduce another fact/statement which is not obvious from the fact could be classified as intelligent reasoning. Say for example given the statements like

A is Brother of B
C is Sister of B then we conclude that
C is Sister of A

Here in this example above the third fact is derived on the basis of first two facts. The mechanism to reach to the conclusion (3) is classified as intelligent reasoning.

AI being a relatively new field of study, the intelligent reasoning derives its meaning from various other streams like Mathematics, Psychology, Biology, Statistics and Economics who have contributed a lot towards distinguishable notions of what constitutes intelligent reasoning.

The thought of learning in depth of Intelligent Reasoning came to mind while I was working on Ontology server and its architecture for one of the ambitious projects of mine which is receiving great feedbacks from whoever has heard about it. The idea is maturing as I am digging it more and all the bits and pieces like AI, Intelligent Reasoning, Ontology, KR, Semantic Web Services are coming together like different pieces of same puzzle. I will keep putting my ideas and thoughts here as I always do but if you want to be part of this, pls feel free to contact me. I am always reachable for discussion :)

Until Next Time... :)

What is an Ontology?

2007-06-22T20:40:00.000+10:00

While reading about Semantic Web we come across one common term always, The Ontology. What I found that every site and author gives a different definition of ontology. The best of that I liked was what was given by Tom Gruber "An ontology is a specification of a conceptualization".

I tried to find the answer at few more places as what the ontology is all about. There were few interesting answers I got:

Webster Dictionary
1 : a branch of metaphysics concerned with the nature and relations of being
2 : a particular theory about the nature of being or the kinds of things that have existence

David Koepsell, Center for Commercial Ontology:

“Ontology is the very first science. Ontology involves discovering categories and fitting objects into them in ways that make sense…. When we make a list of things to do, or of records and books we most want to buy, or videos we intend to rent, we are categorizing-we are engaging in rudimentary ontology. By prioritizing items in a list, we are assigning relationships among various things. Ontology can be relatively simple, or it can be quite complex.

Ontology becomes more complex, and even daunting, when we begin to grapple with large domains of objects with complex relationships among them. For instance, anyone who has attempted to outline the processes and components of even a relatively small enterprise has experienced the brain-cramps that can come with complex ontology.”

Wordnet from Princeton University defines ontology as
1. (computer science) a rigorous and exhaustive organization of some knowledge domain that is usually hierarchical and contains all the relevant entities and their relations
2. the metaphysical study of the nature of being and existence

There are many more definitions available on net and one is bound to get confused when he/she comes across so many different kinds of definitions of the same word. But what actually the Ontology is? The question still remains unanswered or not convincingly answered.

From my study about ontology I found that It is explanation about the things as they exist in the real world. I am not after another definition here but just making an attempt to simplify for my understanding. But at the same time one of my associates think that Ontology is a Reasoning mechanism and it is representation of the logic.

What I am looking to work out is a generalized way to define the ontology which is universal in nature. I am sure many out there will disagree with me on this point that there could be a Universal Ontology. But my question to those people is why it can't be? What is so difficult about ontology which is stopping it from being represented in a general form.

In one of my earlier post on Knowledge Representation I mentioned that Knowledge Representation makes and ontological Commitment. If that is the case then each object in this world is part of one or more ontology. And if an object can exist in more than one ontology then there has to be some sort of similarity between two of them. Just thinking on those lines I guess it is possible to have a set of general ontology defined for all kinds of objects in existence today.

For those who are reading, I would love to know your view on this.

Until next time... :)

Dynamic Composition using Semantic Web Services

2007-06-14T13:12:00.001+10:00

As per Wikipedia the workflow is defined as: Workflow at its simplest is the movement of information and/or tasks through a work process. More specifically, workflow is the operational aspect of a work procedure: how tasks are structured, who performs them, what their relative order is, how they are synchronized, how information flows to support the tasks (wordflow) and how tasks are being tracked. As the dimension of time is considered in workflow, workflow considers "throughput" as a distinct measure. Workflow problems can be modeled and analyzed using graph-based formalisms like Petri nets.

Workflow is an intergral part of today's business applications. Where we perform a chain of activities like Order fulfillment, Payment Processing, New Recruit etc. Today SOA, BPM and many others are trying to address the same problem domain.

With Semantic Web Services in place creating a Workflow dynamically seems to be a possible task. Here are few reasons why I see that a possibility:

Ontology Based: Every Service makes an Ontology Commitment. i.e. it finds its place in the Ontology Structure.
No Fixed Messaging Structure: This means each service is capable of using a common understood format for input and output messages.
Meaning Based: Each publisher will publish details along with the Service which conveys meaning of it. This makes the Discovery process more accurate. The only problem will arise if two or more services are published with the same Semantic description. In that case another parameter like region, cost or any other such parameters will be required to make the choice.

Considering these I am in the opinion that dynamic composition is a possibility using Semantic Web Services. In future posts I will elaborate more on this and also the way I see it all coming together.

I am right now in the process of working on a model for Ontology Server. I would love to hear from you this. If you have any thoughts / ideas on Ontology Server I would like to hear from you.

Until Next Time ... :)

Introduction to Semantic Web Services

2007-06-06T22:03:00.000+10:00

In one of my earlier post I talked in brief about the Semantic Web Services (SWS). Today I was reading another Conference paper on Semantic Web Services titled "Approaches to Semantic Web Services" and there I came across few interesting facts about the same.

Semantic descriptions of Web Services are necessary in order to enable their automatic discovery, composition and execution across heterogeneous users and domains. Current technologies for Web Services provide description at the syntactic level. This makes it difficult for the requesters and providers to interpret or represent non-trivial statements such as the meaning of input and output parameters or any constraints that apply to them.

By applying a rich set of semantic notations that augment the service description we can relax this limitation described earlier. A Semantic Web Service is defined through a service ontology, which enables machine to interpret the capability of the service as well as makes it easy to understand the domain in which the Service is applied to.

Semantic Web Service infrastructures can be characterized along three orthogonal dimensions viz usage activities, architecture and service ontology. These dimensions relate to the requirements for SWS at business, conceptual and physical level.

Usage Activities define the functional requirements which a framework for SWS is ought to support.
The Architecture of SWS defines the components needed for accomplishing these activities.
The Service Ontology aggregates all concept models related to the description of a SWS and constitutes the Knowledge-Level model of the information describing and supporting the usage of the service.

The activities required for running an application using SWS includes : publishing, discovery, selection, composition, invocation, deployment and ontology management. In future posts we will discuss each of them in detail.

Until Next Time...:)

Relationship of Semantic Web Services

2007-05-13T21:44:00.000+10:00

Today I was reading a paper on Semantic Web Services titled "Semantic Web Services Composition Using AI planning of Description Logic" by Lirong Qiu, Fen Lin, Changlin Wan and Zhongzhi Shi. This paper explained in detail as how a Semantic Web Service works and also their composition using First Order Logic language. Below is some of the useful thing I found in that paper.

Relationship of Semantic Web Services:
There exists relationships between different Web Services when considered Service Composition. The Assumption is being made that Si and Sj is the sub-service of Services S. Both Si and Sj provide different kind of services. In light of that the relationship between Si and Sj can be identified as:

Independent Relationship: Si + Sj = Sj + Si which means each sub-service is independent of other and the order of execution of these two sub-services do not affect composition services.
Identical Relationship: Si = Sj, which means the two services seems to be doing the same thing but they have few differences in terms of their attributes.
Conditionally Identical Service: Si =~ Sj, this means Si can provide the same function as Sj in Some situations.
Substitute Relationship: Si (less than sign) Sj, which means Si and Sj can be substituted for each other or they can swap their roles anytime without affecting the overall functionality.
Conditionally Substitute Relationship: Si <= Sj, Which means Si and Sj can be substituted upon fulfilling some conditions.
Overlapping Relationship: Si (X) Sj, which means there is some commonality between the two services. While composing this overlapping or common part must be excluded.
Pre-Requisite Relationship: Si -> Sj, which means that one service has to finish before the other starts or in other words, the execution of Sj depends on successful completion of Si.

I am still working on how to make the Semantic Web Services automatic discoverable, while it is not easy but I guess it is not impossible too. With the advent of OWL-S it should be easy. I am yet to dig deep into it though.

Until Next Time.... :)

How To Be a Programmer

2007-05-12T12:43:00.000+10:00

These days I started to use a new tool which integrates well with my Mozilla Firefox brower. It is called Stumble Upon. This tool helps me looking for sites based on the interest (I need to define them first after I sign up with Stumble Upon). While browsing through programming Category yesterday I came across a very good site which says How to be a Programmer. Well it might sound like one of those cheap trick sites, but it is not. Its a lengthy article to read and takes about an hr to read and understand.

There are two major skills identified for the programmer (or developer)

Personal Skill
Team Skill

Each of them are then further divided into sub sections. While the first skill deals with what the person must have in terms of skills required like Technical, Aptitude etc. The second skill focuses more on what it takes to be part of the successful team ie correct estimation, managing self time etc.

In a nutshell its a good read for all those who aspire to build their career in Software Industry and want to grow. It is recommended to read for those who are in Software Industry as well. So that they can identify their gaps and make sincere effort to fill them.

Until Next Time..:)

Service Hierarchy

2007-05-10T09:13:00.000+10:00

It was during my regular read from home to work when I came across this idea of Service Hierarchy in an Enterprise Application (for that matter any application). I was reading the book "Extending Web Services Technologies - The Use of Multi-Agent Appproaches" published by Springer.

While in general sense the Services (Web-Services) are classified as Simple and Composite, but in reality there are many sub-classifications possible. On top level there are only two types but the nature of complexity varies in Composite services.

In the diagram above I tried to represent how the services (functionalities) build on the top of simple ones in an Enterprise Information System. The simple services like Create, Save, Update and Delete are at the bottom of the hierarchy. The layer 2 services use the basic ones to meet their functional requirement. Layer 3 builds on the top of layer 2 and so on.

While simple services are piece of software which performs a smaller task. A composite service is the one which is build by assembling and accessing multiple services (possibly from multiple service providers).

This draws our attention to next issue which is Service Composition and Service Oriented Computing (SOC). In the next post will discuss in detail about the Service Oriented Computing.

Until Next Time....:)

Relating Stars to Human Knowledge

2007-04-28T20:36:00.000+10:00

Last night I was out for a workshop and there post dinner we all were interacting and sharing our ideas and thoughts what interests us. After some time for some reason I separated myself out of the crowd and was back into the thought process which I like to get into. I was looking at the sky and then all I could see was stars. Some bright and some not so bright. This triggered a thought as do they also represent the way knowledge is stored in human brain? This post relates back to the previous post where I put my thoughts about a Self Learning Systems. In this post we will discuss about how the knowledge can be accumulated over a period of time.

The accessibility of any piece of information in our brain has a factor of when it happened or when in past we came across the incident. Like the stars in the sky, there are few bright ones and few not so bright ones. Each star appears to us with a different intensity and brightness. The one which appears brightest among them are the ones who are closer to us. The classic example is Sun. Same way the things which we remember easily and better are the events which happened in recent past and as they day passess by our memories of the same also starts to fade.

Another similarity of Knowledge and Stars is their lifecycle. The stars are born, they grow to their peak capacity and then they die and become a dwarf star or black hole. The similar things happen to our knowledge and learning as well. We Learn and the knowledge stays with us for a while and after sometime it fades away (if we don't use it) and vanishes. But as it happens with stars they get transformed into a dwarf or black hole, the knowledge we accumulate moves to a non-accessible or not so easily accessible location in our brain. But it never vanishes. It remains there.

The advantage of using a computer system here is that we have access to all those details uniformly no matter how old they are and when they were used last. In the last post of mine when I talked about the Self Learning System, I mentioned about the able to learn from user input. The system must ensure that the knowledge, data, query result etc are accessible with a reference to when was that acquired and when was it used last (with an option to view the log as how many times the same piece of knowledge was used).

How can this be made possible? This is the million dollar question one will ask? Well to answer that we need the following:

A computer program to capture the User input Query.
A Service to query and retrieve all the different possible ways to solve the problem from available vendors or service providers.
A mechanism to store the query criteria, output and the reasoning used to arrive at the result for future purposes.
A mechanism to log the actions in the system and show them to the user when the need arises.

In next few posts I will attempt to draw an architecture of how this kind of system may look like.

Until Next Time... :)

Self Learning Systems

2007-04-19T18:17:00.000+10:00

It was yesterday around lunch time when I went out to buy myself lunch I got this thought. Been working in Software Development for more than 11 years now, I sometime get this feel that computers are nothing more than a dumb orderly. I still remember when someone (don't remember the name) explained me the Expansion of COMPUTER (Completely Obedient Machine Provided User Types Everything Right) first time. Not sure how correct this expansion is but it sounds very much correct in context of computers.

Coming back to the idea what I had. I was thinking if we can put in a place a mechanism to store the last user response to a certain user query; I am sure we turn this dumb machine to something more intelligent. Say for example I query for Pizza Order for Morningside, Brisbane with delivery option available after 10pm. There are different pizza shops in the same suburb. They will publish their services in regard to which suburb they can deliver, if they have a time limit for delivery etc. First time my computer does not know about which vendor to choose. But say I select a pizza vendor I know is good and place next request to get the menu from the vendor. After selecting the Pizza from the menu I place the order and pay by my visa card. Next time if I place the same request it should be able to query the vendor and get the menu in front of me from the vendor. At this point the computer the intelligent system will be able to tell me that last time I ordered the Chicken Pizza and paid by my visa card. Whether I would like to continue with the same or want a different pizza or want to pay by cash or some other card.

Where I am coming from is. The system in this context is not just able to obey my order but is also able to suggest me based on the previous decision I made. So next time on the system is exhibiting the intelligence and knows my preference as well.

In order to build such a system we need:

Client Agent which will store the user query and also the results along with a validity (if applicable).
Server Agent with whom the client will be interacting and passing on the query for which the client agent does not have the answer.
Global Service Database where every vendor's services are listed.
Query Server to perform query based on the parameters supplied. Also queries should be based on the meaning. Not just the attributes. eg. If I am after the pizza shop in Brisbane, Australia it should not bring back the results for Brisbane, California.
The queries should be based on Ontology so that more meaning can be added to the Search criteria.
The services published also have some sort of Ontological Commitment so that it can be retrieved by the Query server.

If we have similar kind of infrastructure in place. I am sure we can achieve an intelligent computer (and a computer network in-turn) and realize the vision of Semantic Web.

Until Next Time...:)

Developing Intelligent Applications on Web

2007-04-17T23:11:00.000+10:00

I was reading a book on Creating meaning on Web and sharing information on Semantic Web. While going thru pages I came across one good extract from an article from Fensel and Brodie, 2003. They mentioned a 3 level solution to the problem of developing intelligent applications on the web.

Information Extraction: In order to provide access to information resources, information extraction techniques have to be applied providing wrapping technology for a uniform access to information.

This directly corresponds to different approaches which is taken to access the information and interpret them.

Processable Semantics: Formal languages have to be developed that are able to capture information structures as well as meta-information about the nature of information and the conceptual structure underlying an information source.

This here refers to the recently developed specifications like XML, RDF, OWL in conjunction with Syntactical and structural approaches.

Ontologies: The information sources have to be enriched with semantic information using the language mentioned in step above. This semantic information has to be on a vocabulary that reflects a consensual and formal specification to the conceptualization of the domain, also called an ontology.

This directly corresponds to the recent efforts made by scientific community to enrich the Semantic Web in terms of Meta Annotation and Term Definitions.

While much has been talked about the later two. The first element i.e. the information extraction is still a puzzle for many people in the scientific community. This is due to the heterogeneous nature of the web. Where the heterogeneous not only in terms of data, but also in terms of the logic to interpret the data and the context in which the data element is being used.

To retrieve the information semantically we need to build the data in semantic way. Unless we adopt to a standard which is sufficient enough to describe the smallest entity in the system and can be scaled to the largest entity using the same mechanism, the achieving true semantic is almost next to impossible. In one of my earlier post about Atomicity I had explained how the principle of atomicity can be scaled from an atom to a large scale enterprise database. May be we can leverage on the same concept and build something which is good enough for the Semantic Web.

Until Next Time...:)

Information System Design

2007-04-15T21:57:00.000+10:00

Lately I've been busy acquiring in-depth knowledge of Web Services and why do we need them. In this journey I came across few good things and few not so good things. My primary concern was always the design issues which forces one to use Web Services and on the top of that the architectural concerns, constraints etc. Not to forget the Business Requirement which adds to the constraints we already have to deal with.

In this process I came across a very good which described about the Basics of an Information System design. Lets first get our head around what an Information System consists of. In a Typical information system we have:

Client Layer: The front-face of the application as seen by the client. This is typically a HTML page rendered by the web browser.
Presentation Layer: The logic which is used to render the data and build the presentation for the client. The best example of this is XSL rendering which is being used in eBay web pages.
Business Logic Layer: Set of programs which process data entered by user as per the business rules defined in the system and then pass on to data access for storing. When retrieved the raw data from the data access layer these programs are responsible for converting them into a format which is meaningful to the user.
The Data Access Layer: The data access layer manages the way data is stored in the system. It exercises the Normalization / De-Normalization principle, Establishes relationship among data elements before persisting them into the data store. This understands the format in which the data is being stored.
The Resource Layer: This is the core of an information system. The data which is stored in Database, File Systems, Active Directory etc form this layer of the information system.

Having known the different parts of an information system we then go for designing how the information will flow and how the different parts of the system will interact with each other. There are few different approaches which is put into practice by architects and designers around the world.

Top Down Approach:
This is the most common approach which is followed by the architects and designers around the world. In this methodology the Functionality or the Business Requirement drive the system design and their components (abstraction) etc. Here client is the center of focus always. The system is designed to keep client in mind, how will they interact with the system etc. The name Top-Down approach comes from the way the design of the system evolves. The designer first sees how the client is going to use the system ie Presenation Layer, then they design the Business Logic or Application Logic components of the same and then at the end the resources (data store) is finalized.

The steps involved in Top Down approach is:

Define access channels and client platforms
Define presentation formats and protocols for the selected clients and protocols
Define the functionality necessary to deliver the contents and formats needed at the presentation layer.
Define the data sources and data organization needed to implement the application logic.

This approach of design has few advantages like it emphasizes the final goal of the system, and at any point in time it can cater to changes in both functional and non-functional needs of the system. But on the other side it can be applied to only the system which is developed from the scratch.

Bottoms Up Approach:
The Bottoms Up approach of system design is not a preferred choice of the designer to start with. It occurs out of necessity than the choice. There are systems which exists in this world serving their original purpose with great accuracy and the organizations rely heavily onto it for their day to day business function. But these systems have become legacy system as they are not able to cater the needs of requirements than they are originally designed for. These systems have a fixed set of functionality defined and the development effort to make changes in the system to cater the new requirements are not viable option for the organization. Under such circumstances the Bottoms Up approach is used.

The Steps involved in Bottoms Up approach is:

Client Defines access channel and Client Platforms.
Examine legacy system and any other resources and the functionality they offer.
Build a wrapper around the existing resources and also build an adapter to provide a consistent interface to the outside world.
Build adapters around the application logic so that it can be used by any client and access channels and protocols.

As stated earlier this design approach is not by choice. Rather it is dedicated by the need to integrate the services provided by legacy systems. This is one area where much of spending in today's world is happening. Most of the buzzwords today like Web Services, SOA, EAI all are primarily targeting in this area.

Until Next Time...:)

My Wishlist for Delphi Compiler

2007-03-26T10:54:00.000+10:00

Today I was discussing at work about the problems we are facing with Delphi as an IDE, Language, Development tool. One of the problems I see with Delphi relates to the way it handles Abstract methods. This forced to think what I would like to have in Delphi Compiler.

Enable me have annotations in the classes instead of keywords to override methods.
There should be a way to declare a class abstract like in Java. The user program should not be able to instantiate the abstract classes.
A class with minimum one abstract methods should be forced to declare as abstract.
Like DotNet we should be able to make a Class / method final. So that nobody can extend it and in-turn override the behavior of the method.
There should be some way to warn user that an overridden method has empty implementation. May be an optimizer feature but this will be really helpful.

There are few more features (related to IDE) which would be nice to have:

Keyboard shortcut to see all the open files in one popup etc like eclipse.
Keyboard shortcut to go to method or class declaration. Just like F3 key in eclipse.
A Javadoc kind of feature where the user can attach some sort of documentation with the method and that can be picked up by the Code-Insight. So that we know what this method is doing. The current mouse hover does not give sufficient details as we cannot write documentation for the method to be picked up by any other class.
Also a scrapbook kind of feature in Delphi IDE will be cool to have where we can write couple of lines of code and then just test it. Something similar to what WSAD has in it. This will save time to test few lines of code we want to try in our code.

I am not sure if any of the reader has a contact with Delphi Team but it would be desirable to have these features in Delphi which one time was my favorite language which Borland team managed to destroy over a period of time.

Disclaimer: This post has nothing to do with the Semantic Database or Semantic Web Services as such.

Until Next Time...:)

Problems With Web Services

2007-03-22T14:18:00.000+10:00

In previous posts I'd been talking about why Web Services are not for Web and how Semantic Web Services promise to solve the problem etc etc. While I was reading about WS and listing down what are the problems we have with the technology, I came up with a list of issues which needs to be addressed any technology which claims to be superior to Web Services.

Web Services are one of the best means to share applications in today's world. It frees one from worries of Applications, Infrastructure, Platform, Language, Upgradation etc. But while they are great for ordinary use. They do have few issues which need to be resolved to enable them serve their purpose.

Web Services today have problems like Automated Discovery, Choreography and Composition. Not to mention the security aspect this is not even thought about.
Currently web services use the infrastructure provided by the World Wide Web (WWW) but they do not actually operate on web. The World Wide Web is built on the top of the Infrastructure provided by Internet. Web Services operated on Internet infrastructure but do not work on the core principles of World-Wide-Web.
Web services also lack the capability of asynchronous communication or callback functionality.
Web services are very rigid in terms of messaging syntax. They have a fixed format for input and output messages.
There is no way to determine at execution time as which web services to pick up and which one serves the purpose. As web services do not work on Semantic structure (meaning) and they work on Syntactic structure. What this means is, they are very much discovered early or have a static binding. There is no dynamic discovery in Web Services
Web services also lack the capability of asynchronous communication or callback functionality.
Due to the Black-Box nature of Web-Services we often don’t know what is happening inside. By looking at a Web Service Description we can get information about the input parameters (message format) and Output Result (output message format) but we do not get any information about how it is doing what it is doing. In a nutshell the transparency is not there at all.
While using Web Services we assume that the output we are getting as result is correct. Since we don’t know (rather most of the time we don’t care) about the internals we cannot really verify the steps (logics) used to derive the final result from the input parameters passed to it. So verifiability is another concern we have in regards to Web Services. That affects the reliability on the Web Services as well.

I guess I'd been very negative about Web Services here. But then it is a great way to take our existing system functionality to a next level. Next post on will start touching on what is required to have a Semantic Web Services.

Until Next Time... :)

Introducing Semantic Web Services

2007-03-14T13:38:00.000+10:00

In last few posts I was concentrating mostly on Web Services and what they are not. Last night I was reading and Article Approximate Reasoning and Semantic Web Services and there I happen to read about what it takes to create a Semantic Web Services. According to one of many definitions the Web Services can be described as systems with a number of features:

They are identified by a Uniform Resource Identifiers (URI)
Their public interfaces and bindings are defined and described using XML;
Their definitions can be discovered by other software systems;
Other systems may interact with the Web Services in a manner prescribed by their definitions using XML-based messages conveyed by Internet protocols.

The proposed definition of the new concept Semantic Web Services has few changes and enhancements when compared with the definition of the Web Services. Semantic Web Services are system that:

Are identified by a URI.
Whose public interfaces, bindings, some behaviour and various other features are defined and described using Semantic Web language. The Semantic Web Language here refers to OWL, DAML-S etc.
Their definitions and other descriptions can be discovered by other systems.
Other systems may then reason about and interact with the semantic web services in a manner guided by their definitions using xml-based messages conveyed by internet protocols.

In the light of those definitions, it can he stated that the biggest advantage of the Semantic Web Services is their flexibility. The interacting system do not have to know everything about each other before they start interaction. They "learn" about their specifics and behaviour at the time they interact. The concept of Ontology and its application to represent information ahout data and services give such capabilities. Each service in the fiamework of the Semantic Web Services is represented by:

Profile ontology, called ServiceProfile, is a description of a service.
Model ontology, ServiceModel, is a description of a process itself and its composition.
Grounding ontology, called just Grounding, describes mapping of service elements to WDSL messages.

This is the first of the many posts yet to come related to Semantic Web Services. Keep watching this space for more on this topic in days / weeks / months to come.

Until Next Time...:)

Global E-Business

2007-03-12T23:44:00.000+10:00

There was a time when Business used to take place in person. ie people used to discuss among themselves and finalize the deal. While this was the best way for doing the business, it had few drawbacks as well. It was time consuming and one was limited by the time and resources available.

The times have changed and we moved to E-Business now. The transaction which used to take place over phone or in person started to happen on internet. While this enabled us to do transaction with any other business located in any part of the world, it introduced quite a few complexity in the whole process. But this is not what the main theme of this post is.

The Vision of E-Commerce
The original vision of E-Commerce was to enable anybody to do trade and negotiate with anybody else. But is that really possible in current scenario? Lets see what it takes to have such a system in this world:

A Machine Interpretable format or A system must be in place which can deal with numerous and heterogeneous data formats and numerous and heterogeneous Business Logics. A Strong decoupling is required for various applications/components which realizes the E-Commerce application.
Some sort of semantics is needed in this heterogeneous environment which will allow mediation at the conceptual level. i.e. a Strong mediation service is required which allows a system to communicate with another system. Here we are talking about machines to communicate with each other without human intervention.

Is Current Technology Sufficient?
In todays world we have Technologies like Multi-Tier and Distributed computing, Web Services, Web based shopping carts etc. These tools and technologies help people to do business over web. We call this E-Commerce. But they are nowhere close to realizing the vision of E-Commerce. These technologies while they work great with known systems, they fail to mediate between two different systems. Also the systems today are not able to handle the hetrogenity in terms of data and business logic.

Web Services as a technology has made lots of promises to the businesses worldwide. It had what it takes to become the most acceptable solution for E-Commerce. But since its advent it failed to realize the vision. I will discuss about this in one of my future post. In my previous post about Are Web Services Really Web Services, I had pointed out what is missing in today's Web Services.

What is the Solution?
Semantic Web as a technology has lots of promises to the world. Semantic web promises to make the content of world-wide-web Machine Readable and Machine Processable. Semantic Web involves Ontologies which is one of the most abused term (from its original meaning). Researchers around the world are working day and night to make the web contents machine readable and machine processable by developing Ontologies. At the same time Web Services community also worked on to improve the Web Services Standards. But so far not much of work is been done to combine the best of two and come up with a powerful solution.

Semantic Web Service is one such solution which promises to enable E-Commerce in its original form and realize the vision of E-Business.

In my upcoming posts I will discuss about Semantic Web Services in detail. Starting from conceptual level to the more detailed level architecture and implementation.

Until Next Time.... :)

Are Web Services really Web Services?

2007-03-11T08:04:00.000+10:00

In my previous post regarding Web Services I mentioned why besides their name Web Services have nothing to do with the Web. In this post we will discuss them in detail as why it makes me and everybody else who researches Web Services to come to this conclusion. While Web Services as a technology is been great for ordinary users there are several disadvantages which becomes obvious when the researchers look at it.

Current Web Services Technology
Current Web service technology is based on tightly coupled message exchange. The sender and receiver has to be online during the whole communication period. It will fail if one of them is offline. This situation is similar to the situation faced by scientist community in the pre-Web age, where one had to send an e-mail message to a scientific colleague, asking for a specific paper or piece of information. With the introduction of web it became simpler. In the current Web as an infrastructure for humans, this pattern of communication has widely been replaced by persistent publication and asynchronous retrieval. We no longer request the biggest share of information in lengthy, synchronous communication cycles with the originators, but fetch it from persistent sources, e.g. the scientist’s personal Web page. We don't have to be online all the time to ensure that whoever wants our page can get it. We can just publish the article on a web page and let others download it from there. We argue that this has significantly contributed to the success and scalability of the Web, since it freed the sender from the need to handle individual requests, and it made resources identifiable and thus referable.

Imagine what will happen if Web was not there today. In such a situation one will have to handle individual requests from all your scholarly colleagues who visit your Web site. You might soon become the main bottleneck for the dissemination of your own ideas. The shift from information dissemination based on message exchange not only made the Web scale tremendously, it has also sped up the dissemination process.

On the contrary when we compare Web services with this important principle of the Web it becomes very obvious that Web services are not following the core idea of the Web. It is true that Web services use the Internet infrastructure as the transport medium. However, that is more or less all they have in common with the Web. The idea of publishing data and accessing it asynchronously is lacking. Web services failed to work in asynchronous mode. Instead of publishing the information based on a global and persistent URI, Web services establish (1) stateful conversations based on the (2) hidden content of messages.

The negative effect of such distributed applications that communicate via message exchange is that they require a strong coupling in terms of reference and time. This means that traditional Web services require that the sender and receiver of data:

maintain a connection at the very same time, that they
agree on the data format, and
know each other and share a common representation.

Therefore, the communication has to be directed to a particular service and is synchronous as long as neither party implements asynchronous communication (and jointly agree on the specific way this mechanism is implemented).

The two major criticisms around Web services today are about the improper usage of URIs and the violation of the stateless architecture of the Web. It is one of the basic design principles of the Web and REST architecture to not provide stateful protocols and resources.

In real-world(where we live in) this means that when sending and/or receiving SOAP messages, the content of the information is hidden in the body and not addressed as an explicit Web resource with its own URI. Therefore, most Web functionality dealing with caching or security checks is disabled, since using it would require parsing and understanding all possible XML schemas that can be used to write a SOAP message. Thus, when a stateful conversation is required, this should be explicitly modelled by different URIs. Moreover, referring to the content transmitted via an explicit URI in an HTTP-request would allow the content of a message to be treated like any other Web resource.

Summary:
In this post we discussed how Web-Services failed to realize the Web part of its name. What is missing ie stateless architecture and using URIs to refer to the resources. In next post we will discuss what is Semantic Web Services and how they are better than the Traditional (syntactic) Web Services.

Until Next Time....:)

Web Services are NOT for Web

2007-03-10T11:37:00.000+10:00

Yes this is right Web Services are NOT at all for Web. Last few days/weeks I spent reading through different journals, conference papers and articles to understand what Web Services are all about. At the end when I was writing down my concluding notes on what I read and what I understood. I came up with the conclusion that Web Services miss the Web part and are better of called as Internet Services.

Internet and Web
The first question that comes to anybody's mind is that are they two different thing? I mean Web and Internet aren't they same thing? To a novice reader there is no difference between Web and Internet. Web today is the major means of publishing and accessing information. Starting as a closed network to exchange emails and share files for scientific reasons, today it has become a global media used for information dissemination and information access within little over a decade.

Internet is the infrastructure and the TCP/IP stack we are talking here. It is the backbone of the World-Wide-Web. It handles all the communication and the data transfer. Web is just an application of the Internet. There are other applications like File Transfer, Voice and Data transfer etc. All of them use the infrastructure provided by Internet.

Where is the missing?
Web Services technologies today are far from being in a state where the automated and seamless integration is possibility. There are known problems in the Web Services arena and researchers around the world are working on to overcome those problems.

Web services are mainly based on transient message exchange and not on persistent publication of data. As a matter of fact, Web services are far from using the Web as means for information publication and access. Instead of following the ’persistently publish and read’ paradigm of the Web, traditional Web services using WSDL and SOAP establish a tightly coupled communication cycle, most frequently using a synchronous HTTP transaction to transmit data. URIs, which are meant as unique and persistent identifiers for resources are used only for the identification of the participant, whereas the information exchanged is hidden in the SOAP message.

In next posting I will discuss how the Web Services as a Technology do not fit the specifications of the web. There will also mention why apart from name Web is not there in Web Services at all.

Until Next Time ...:)

Semantic Web and Semantic Web Services

2007-02-22T20:10:00.000+10:00

Last night I was reading this paper from Hepp about Semantic Web Services (SWS). The paper tile is Semantic Web and Semantic Web Service: Father and Son or Indivisible Twins? This was published in IEEE Internet Computing March-April 2006.

While this paper starts with giving numbers about recent Semantic Web conferences in terms of attendees, industry participation etc. The best thing which I came across in this paper were the facts about Semantic Web and Semantic Web Services. In this post I am writing down what I learnt after going through this paper.

Myths about Semantic Web: While it is widely believed that the next generation of web will have the contents which can be interpreted by computers. The intelligent search engines will be based on meanings which will produce the search result as per the relevance not just the meta tag. This on one hand while looks promising it also is based on several myths:

Everything is Available on Web: Many people today believe that all the information we need in todays world is available on World Wide Web. While this sounds true but far from being real. I come from India and there are many places one can visit and which is of historical importance. But there is almost none information available about them on web. Most of the hotels in the big cities like Bangalore, Delhi, Mumbai does not have the booking or enquiry facility on web. In Order to get any information about booking rate etc one has to pick up the phone and make few phone calls. And they are not 1-2, rather a majority of other businesses also have the similar kind of presence. They are known to locals and people who visit there quite often but they don't have any presence on web.
The Business Systems, Web is not Stateless: A fully compliant web-application should not change its internal state as a result of a http read of an available data. In other words unless the transaction occurs the behavior of a web app should not change at all. But as per one survey almost all business applications do not comply to this standard in todays world. In business world, there is always scarce for resources and that drives the competition today. For example if an airline has say X number of seats for a special fare on some routes then if there are X+1 people querying at the same time the last person who runs the query does not get the discounted fare as the system thinks that the prices are not available at all. Note that its just query and the transaction is not made yet. This while represents the real world scenario, it puts a big question mark on the query results received by two people who are looking for the same thing at the same time.
Annotation of Data Vs Annotation of Functionality: There are evidences of work being carried out to annotate dynamic web contents. But the hard reality is the data what is being generated on a web page comes from an underlying database. The business data is very volatile in nature and annotating that will never give a correct result. So annotating data is virtually impossible. But there is still something which can be done. A function can always declare what are its pre-condition, its logic (as what it does) and post-condition ie how the system's state will be affected as a result of function execution. This feature is already available and one can use Web Service Modeling Ontology(WSMO), OWL-S or Semantic Web Service Language to achieve the same. Also SparQL provides capability to query RDF databases.
The Matchmaking Process: Well it has nothing to do with the matchmaking advertisements on the web these days :). In any industry all the data is not available to everyone. A car dealer never reveals what the best price they can offer on cash purchase. Same goes with any other business. The business never reveal all the secrets and offers on web because fear from competitors. Now this brings a problem in the system. Suppose a car dealer wants to give a 5% discount on cash purchase. Now when a customer is looking at prices on web he cannot find the option whether he/she wants to buy it outright (cash) or on finance. And he will have no clue about the best price he can get. Web in its current form cannot recognize whether the visitor making enquiry is a genuine buyer or just another visitor. This matchmaking process is too complex. It may sound simple in real world but a computer program doing it requires high degree of complexity built in it.

No Semantic Web Without Semantic Web Services: The time is coming where most of the businesses are publishing their services over web in form of Web-Services. This is a much cleaner approach than directly publishing data on the web. In order to transform web into a Semantic Web we need much more than just the static page annotation. We need a dynamic mechanism to retrieve the data based on the current context. Semantic Web Services are promising to provide that missing capability to the web. In todays world the number of materials available on web on semantic web is very less compared to Semantic Web. A Google Statistics for search results on different topics are listed below.
Semantic Web - 20,600,000
Semantic Web Services - 777,000
OWL Ontology - 150,000
OWL-S Ontology - 828
OWL-S Web Services - 230
WSMO - 245,000
WSMO Ontology - 589
WSMO Web Service - 841

These are the search results as of today when I performed the google search with whole search keyword within quotes.

What it indicates is that the number of research papers being published on Semantic Web Services is very less compared to Semantic Web. I guess this calls for research community to increase the participation in the field of Semantic Web Services. We as a research community should think about how we can make Semantic Web Service more reliable. What is the feasibility of Automated Discovery, Composition and Orchestration. Whether a lightweight option is available. It is not just any other AI research topic. The researchers also have to consider the market factor and how businesses operate in real world.

Until Next Time...:)

SWS - Web Services Revisited

2007-02-18T23:59:00.001+10:00

Today while reading a paper on Semantic Web Services I just did a google on Web Services. Guess what? Google search result returned 1,250,000,000 ie 1.25 billion pages. Well I cannot blame google here as every third page(technical) in the world talks about Web Services, SOA, REST, UDDI etc :). Sometimes I wonder how it is going to be developing applications in 5-10 yrs time. We will be developing services and that will be sold by the companies worldwide.

More than enough can be found about what the Web Services are and how important they are in current world etc. So I would rather not go into that detail and concentrate on bullet-points instead.

What makes a Web-Service special? This is the question many people asked me in the initial days of Web-Services. Well we all (most of us) come from an OO Programming background where we learn to use (rather re-use) classes. Web-Services take this concept of re-usability to the next level where we can re-use the application as it is. In real world OO can be explained as talking to someone about how the car looks like and WS is actually handing over the car-keys to him.

What was there Before Web-Services?
Before we got acquainted with Web Services we had few technolgies in past. The computer system started with a piece of monolithic application and then we moved to 2-Tier, 3-Tier and n-tier application where we created abstraction or layers in between logical separation of application components (eg Presentation, Business Logic, Data Access etc). When we moved our applications to n-tier we came across several technologies which helped client application to commonicate with server component.

Before we reached to Web-Services we had Remote Procedure Call(RPC), Common Object Request Broker Architecture (CORBA), Component Object Model(COM) and Distributed COM (DCOM), Remote Invocation Method(RMI), Messaging(IBM MSMQ) etc. The common thing among all these technolgies were that they viewed application as components and their primary focus was to use component's functionality in a way that it could be independent of Hardware Platform, Location, Language, Operating System etc. Upto an Extent they did succeed in achieving the same.

Technology Enablers for Web-Services: A while ago there were talks about Application Service Provider (ASP) model. The idea was to enable organizations renting their applications from ASP vendors and not to worry about hardware purchase, software license, upgrades, availability, scalability etc. Though this idea died prematurely, the Web Services are expected to bring renaissance to the ASP model. Lets have a look at the technologies which enable Web-Services.

Extensible Markup Language(XML) is to data of Semantics what HTML is to rendering of Web documents. XML does not impose a set of pre-defined tags like HTML does. The beauty of XML document is that it is hierarchical in nature. A XML document is called Well-Formed if it confirms to the basic syntax rule of XML and is valid if it is Well-Formed and confirms to the rule of DTD or XML schema.
Web Services Description Language (WSDL) is used to describe Web Services. The WSDL document provides enough information about how to interact with the Web-Services. The WSDL document contains Five Elements three abstract elements and two concrete elements.

The Abstract Elements define the interfaces. Types are embedded XML schema where data types are defined; messages describe details of methods and their parameters; portType defines operation in terms of input/output messages.
The Concrete Elements on other hands defines physical properties; binding provides protocol information for the operation and service address provides URI for locating a service.

The Simple Object Access Protocol (SOAP) is an XML based communication protocol and encoding format for inter-application communication. This is backbone of Web-Services. SOAP consists of following parts:

Envelope header which specifies application level requirements, digital signature
Envelope body describes message content and processing instructions. It also contains application specific data like method name, parameters and return values.
Encoding rules which is used for application defined data types based on W3C XML schema specifications.
The RPC conventions specifies a convention for representing Remote Procedure Calls and responses.

Universal Description Discovery and Integration (UDDI) registry is a collection of information on all the registered web services. Its main purpose is enabling dynamic discovery of Web-Services. It is a free public registry. Vendors publish their services and consumers search for Web Services they need. The UDDI has three major components:

white pages which contains address, contact details and known identifiers for the Web Services.
yellow pages contain industrial categorization of the Web Service based on standard taxonomies.
green pages contain technical information about services.

How are Web Services Used?

Web Services are provided over HTTP protocol. When a Software is available as Web Services it takes away the issues like platform, hardware, Operating system, programming language etc from the organization using it. This opens a whole new way of developing applications. In future the applications will be developed by selecting the appropriate Web Services published by different vendors and one application to tie them all together.
Large corporations have systems which is been in places for many decades. These systems are running on mainframe and developed using COBOL etc. But even today they are the backbone of the organization. But in todays world the demand has been raised to access those applications over web. Web services are used here to provide a wrapper for the legacy system using Web Services framework.
In the global economy an organization needs to lias with multiple vendors, customers etc and most of the time they are spread globally. Web-Services are useful there to develop an interorganizational system which can interoperate with applications of Vendors, Customers etc.
Business Process Integration (BPI) is a methodology of coordinating and executing sequence of steps to accomplish a Business task. Each step could be a different application which is located at either Customer or vendor site or is internal to the organization.

This is about the Web-Services in a nutshell. Next post onwards will start putting my thoughts on Semantic Web Web Services (SWWS).

Until Next Time...:)

SWS - The Initial Impression

2007-02-18T16:26:00.000+10:00

Last week or so I started to get acquainted with Semantic Web Web Services (SWWS). I was drawn to this topic when during one of my digging (information browsing) session I came across Web Service Modeling Ontology website. The concept did fascinate me and I started to collect materials from different technology and research portals and collected lots of materials on the same topic.

In order to do that I had to re-visit the Web-Services once again. This time I was hoping to find something which was missing when I had a look at WS last time. But to my disappointment during last 2-3 yrs the Web-Services as a Technology (rather Specification) did not show much improvement. The problems like Orchestration, Choreography, Transaction Control, Authentication etc still persist and I am not sure how they are going to be resolved in the near future.

While in first glance it seems to be promising. Yet so far I am yet to find a paper which describes how we tackle the problems faced by Web Services as a Technology.

I have started to dig more into this area hoping to come out with some solid reasoning and integrate the same with my work on Knowledge Representation. Keep watching this space for more on this topic.

Until Next Time....:)

Types of Knowledge

2007-02-17T22:21:00.000+10:00

I was reading the book Knowledge Representation, An AI Perspective and there I came across a very simple but interesting classification of Knowledge. The knowledge at top level can be classified into two Categories:

Domain Knowledge: This deals with the finer details about the domain an application is targeting to. The program knows which domain it is being used for. So the program has the details about the domain it is operating within.

The first type of domain knowledge is Structural Knowledge. It denotes the types of entities which are relevant to the application domain. In a Retail domain entities like Invoice, Receipt, Customer, Supplier needs to be mentioned as they are part of the knowledge within the domain. There are also occurances of hierarchical Knowledge eg Cash Receipt is a kind of Receipt and so is Cheque Receipt.
The Second type of domain knowledge is Relational Knowledge. This deals with relationship between the entities which are distinguished in Structural knowledge. In retail domain attaching Customer to an Invoice and Supplier to a Purchase Order is an example of Relational Knowledge. They help in deriving some logical conclusions based on the relationship among the entities like if the Invoice is not paid after the due date is over then the Customer Account is in Arrear.

Strategic Knowledge: Domain knowledge is good as long as we collect the data and represent it in meaningful form. But in order to make some meaning out of the domain knowledge the Strategic Knowledge is being used. This knowledge is particularly used for solving problems within a domain. Say for example in the retail domain if a customer is been paying in time and last couple of invoices are due for payment. The supplier might still decide to sell him goods looking at his/her payment history.

While the distinction between Domain (Structural and Relational) and Strategic knowledge exists more at and abstract level. More often than not the domain knowledge is tightly associated with Strategic Knowledge.

This was first analysed by Clancey(1983) and Wielinga and Breuker(1986). The practical importance of this classification is often seen while developing expert system. There the human expert is first interrogated in order to understand how human expert solves problem in their area of expertise. The methods applied by experts are often driven by the domain knowledge they acquired over the years. This in turn creates a high-level of dependency between domain knowledge and strategic knowledge making it hard to be distinguished as two different.

While domain knowledge are very much fixed in nature, the strategic knowledge are driven by human factor as well. The human factor plays an important role in Strategic Knowledge. In order to develop a system for strategic purposes we need to have some sort of human factor incorporated. The best could be using domain knowledge to present Multiple Options to the user and let the user choose which one suits best or override all the options with the totally new option.

Until Next Time.... :)

Changing Track for Sometime

2007-02-11T22:43:00.000+10:00

Last week I came across a very good technical paper while googling for Web Services and Semantic. This paper was about Using Semantic Web Services. I also came across this site called Web Service Modeling Ontology which did fancy me and made me visit this site again and again over next few days.

At this point in time I am planning to change my track for a while and nail down the details of the Semantic Web Services before coming back to KR again and continuing from where I stopped.

At first instance it appears very interesting topic to dwell into.

In coming day / weeks I am going to put few posts about what I find in Semantic Web Services.

Until Next Time..... :)

Applying Semantics in Real-Life

2007-02-10T11:20:00.000+10:00

Today morning I was replying to an email from a friend of mine in India. While replying to her email I did remember doing different kind of analysis (which I normally do) based on the persons details. Actually its very interesting thing to do in its own right. You come across various facts about the person which he/she is not aware of.

Using First Name, Last Name etc One can perform Name analysis of a person. As it is being believed that The name shapes a persons behavior. Similarly there are many other analysis on name can be performed.
Using Date of Birth, Place of Birth, Time of Birth etc one can perform all sorts of Astrological calculation and find out more about the person.
Education Details gives an indication of what the person's interest is. The interesting fact is the percentage of marks in individual subjects gives an inclination towards person's interest.
The list is too long about the different kind of analysis we can perform on attributes (details) from one person.

But one you may ask how does this relate to my current work. Well its no different as I see. Going back to what Semantic Database is all about. Its an attempt to link information based on its meaning. This is how I see this being applied in real life. Say for example I have a Person object in my database. This is how different kind of details we can derive from a person's individual attribute.

Using Date of Birth, Date of Anniversary we can schedule our system to send an automated greetings on those dates. This is particularly useful for a CRM application.
Using Address we can calculate shipping cost for an item one buys from other supplier. This detail can be used for sending promotions to the person in that area.
By analyzing persons Salary we can calculate taxes.

The World-Wide-Wait(WWW) is too big for one person to find all the information by him/her self. Millions of pages contain information which can only be understood by human being. There is no way in current scenario we can make computers understand those informations. To be frank Computers are still dumb :). In one of my earlier post on World-Wide-Web I had discussed why it is difficult to link bits of information today.

But I guess we had enough discussion about about the problems. The time has come when we need to work on solution part of it. This is how I see it being a reality.

We need to identify an object as collection of objects, ie we should be able to break a big object into small-small manageable pieces (object).
We need to develop a capability in the system to process each of the small objects in its own right of one big object as a whole.
We need to have a system capable enough to take a small object (even though it is part of a big object) and perform actions on it if asked for.
The system should be able to consolidate all the processing results into one final result keeping the individual existence of results intact.
We should be able to identify these elements in the system and also be able to attach a processing logic to the same.
The system should be able to provide a facility to create a workflow where we can group the processing logics and transform them into a big one.

There are many more things which could be added to the list (above) and the result will take us to build a system which works on the basis of meaning.

Until Next Time....:)

KR as a Set of Ontological Committments

2007-02-03T09:44:00.000+10:00

In the previous article we discussed about KR As a Surrogate. In this part we are concentrating on how Knowledge Representation affects the Ontology.

When we choose a representation; we often remove few aspects of the object and choose only those which makes sense in the current context or which is relevant to the situation and seems to represent the object itself. Which in a sense is like making the Ontological Commitment. These commitments are essence of the whole representation mechanism.

While the basic intention for all reasoning machines remains the same as to reason the objects (entities) from the real-world. But we know for the fact that the real-world is complex and not organized as our machines are. So in order for the reasoning mechanism to decide what we (and the reasoning machine) needs from the real-world object and what can be ignored.

At what Stage the Commitment begin?
The ontological commitment is unavoidable and it is present in almost all form of reasoning be it logic, rules, view points or any other things in this world which supports reasoning. The only difference here is each of these present their own view of what they think is important and which requires our attention. But then again as we discussed in previous post, every representation ignores some facts and makes some assumptions. So the best we can do is to have a best or most accurate guess to start with. No matter what we do we end up making a commitment (even a small amount) to the bigger ontology here.

Multiple layers of commitments
The moment we start representing the knowledge a set of commitments are made. As we move on additional layers of commitments are added on top. We start KP with the best guess (not accurate) about how we see the world. As frames suggest prototypes and taxonomies but do not tell us which things to select as prototypes rules suggest thinking in terms of plausible inferences but dont tell us which plausible inferences to attend to Similarly logic tells us to view the world in terms of individuals and relations but does not specify which individuals and relations to use.

Commitment to a particular view thus begins the moment we make our choice to how we see the world and additional layer of details are added as we move along.

Until Next Time...:)

KR as a Surrogate

2007-01-23T21:44:00.000+10:00

In the First Part of this series; I touched upon briefly what the Knowledge Representation (KR) means and what it reflects. In this post I will be touching upon KR as a surrogate.

Before we dwelve deep into this lets find out what the reasoning is. Reasoning is a process that goes on internally; while most things it wishes to reason about exists only externally. A person who is writing program thinks about Class, Procedures, Functions etc while these things exist in external world only.

If we say that KR is a surrogate it raises couple of questions; (i) About its intended identity and (ii) about the semantics of the representation i.e. some kind of mapping of Surrogate with its real world equivalent. In this we elaborate on what details we publish and what details we omit from the real-world equivalent of the knowledge we are representing. Say we want to map an Employee (Entity) to its real world equivalent (Human Being) then, in this process we omit few details like parents name, place of Birth, time of birth etc of the person.

In this process we are omitting few details of the person object when representing in the system. As far as practicality of the completeness of information about an object is concerned; it is virtually impossible and under no circumstances we can claim that an object (from real-world) can be represented anything else but by itself. An object is true representation of itself rest every other representation is incorrect as they contain some assumptions about the object and some attributes which may or may not exist at all.

While it may not sound the best way to represent a real-world entity as surrogate. But there are few advantages we see for KR as a surrogate.

It treats both Tangible and Intangible object equally.
It allows both the kinds of objects to be reasoned and exist in the same space. It treats both the objects which exist in real world and the objects which exist in beliefs to be reasoned in the same way.

This leads to a conclusion that imperfect surrogates are inevitable. When we talk about this; with reference to what we mentioned earlier. We learnt that in the process of representing real-world entity we omit some details or we abstract a few. Which is different from the original object itself. Which also leads to another conclusion that no matter what reasoning we apply to the object the final result is always going to be incorrect as the entity we are reasoning about is not represented correctly to start with. Here also the level of inaccuracy in the final result may vary depending on various factors either known or unknown. But we always try our best to ensure that the level of inaccuracy is low while representing the entity for the task in which the entity is going to play a role. This takes us to another conclusion that if the reasoning of an object is long enough and broad enough it is guaranteed to result in an error.

Another point to note here is that irrespective of whether the reasoning is sound or unsound we at the best can only ensure that the error in representation is not due to the reasoning mechanism we applied to the object. But that does not leaves us to take the problem in hand casually.

At the end before we choose a representation of an object (from the real-world) we must evaluate the cost and benefit of the representation and the reasoning mechanism applied to it.

While this sounds like an interesting thing to read. But going forward I look for a general mechanism to represent every measurable thing in an object. i.e. developing a structure to represent an object so that we can accommodate all possible attributes of an object which can be seen or measured. While this may sound like a promise but in future post I will be touching upon creating one such framework where the object from the real-world can be represented in machine form.

Until Next Time...:)

Atomicity and Knowledge Representation

2007-01-21T00:03:00.000+10:00

In my previous post on Atomicity I asked few questions about the atomicity as a concept itself and how does it relate to the Software Engineering. Also in my previous post on Knowledge Representation I discussed 5 fundamental things about Knowledge Representation.

It was last week when I was talking with a friend whom I happen to meet first time and it was a planned meeting which was scheduled for just 30 mins but then as it normally happens with me an interesting discussion last longer then anticipated. The meeting went on for about 1.5 hrs :). I know I am terrible in managing time especially when it comes to technical discussion.

While we were discussing about atomicity, there are few points that came out during that and one of them was how the smallest possible piece of knowledge can be applied to the big picture.

The Atomic Structure:
If we look at the structure of an atom (Except Hydrogen). It has a Nucleus which determines the Atomic Number and Atomic Weight for the element. Then it has a Electron(s) circling around in the pre-defined orbit.

Using the same concept in Semantics if we look at the object (for that matter any object) which exists in this world. We will find that it has two different kind of attributes (i)Key or Principal Attributes and (ii)Helper or Supporting Attibutes.

Key Attribute(s) Determine how the object can be represented in the world. The attributes for an object which fall in this category are its Name(Value), the Type(family) of object. What it means to define an object in its basic form or the basic object contains what family it belongs to and what is its value.

Supporting Attribute(s) help object being part of another object or how it can help object being part of Some Transaction or Process. Like for a Product in a Retail System its Unit Price is the supporting attribute which adds a whole new meaning to it and is being used when we put that product on an invoice, order or quote.

Optional Attribute(s) are another kind of attributes we find for an object which if added adds more meaning to it but its absence does not hurt either. A Slideshow or Thumbnail image, Video, a GI information about the product could be classified as as optional attribute. Which when present adds more detail to the Product but if they are not present the product object can still be used. They also do not participate in any transaction etc, they just give a more clear picture about the product.

But then the question comes is it is okay for the individual objects (entities) but how do they fit in the big picture? Lets visit any Database and check out how it relates to the concept we just discussed. In a DataBase we have a DB Schema which has Tables, Procedures, Triggers, Indexes etc. The Tables are the essential part of a DB Schema, Procedures and Triggers are the supporting elements of the DB Schema and Indexes form the optional elements of the Database.

During the discussion we also realized that if we find the best way the smallest Object(Entity) can be represented in the System, then we can then use the same philosophy to represent the Bigger Entities (Database Schema) and then using multiple Schemas we can represent the System and Since we build the System on a common principle we can have the computer systems talking to each other without any human intervention.

The idea on paper looks good and as simple as ABC but in order to make this a reality we need to go through Dimension Shift (Higher than Paradigm Shift). This calls us to change the way we look at things, the way we analyze the events, things etc around us, the way we channelize our thoughts.

Until Next Time...:)

What is Knowledge Representation?

2007-01-09T19:50:00.000+10:00

A Happy New Year to those who are reading this. May this year brings success in all your endeavors.

A while ago in my post relating to Knowledge and Its Representation I left with few questions as how to represent a knowledge. While I spent a fair amount of time reading about knowledge and its representation. Few days back I came across the paper What is Knowledge Representation by Davis, Shore and Szolovits from MIT AI Lab and Laboratory of Computer Science. I was confronted with few things:

Knowledge Representation(KR) is a Surrogate. A substitute for the thing itself, used to enable an entity to determine consequences by thinking rather than acting.
KR is a set of ontological commitments. ie It makes an attempt to answer the question as "In What terms I should think about the World"
It is a fragmenting theory of intelligent reasoning represented in terms of the three components mentioned below

The representation's fundamental conception of intelligent reasoning
the set of inferences the representation sanctions and
the sent of inferences it recommends.

It is a medium for Pragmatically efficient Computation ie the computational environment in which the thinking is accomplished.
It is medium of Human Expression ie a language in which we say things about the world. The point to note here is that we are talking about the universal language here.

After reading journal while most of my questions were answered upto an extent, but the desire to know more made me read this entire journal 3 times so far. I am on my way to dig more on each of the point mentioned above and will be putting here one by one in future posts.

Until Next time...:)

Resolutions for the New Year

2006-12-30T20:02:00.000+10:00

Its time of the year when we take a stock of what we had planned for the previous year viz a viz what we achieved. What we left on the way and where we gave up. But while on one hand we talk about what happened in the past we also make resolutions for the new year. While the new year eve celebrations are just under 48 hrs away we already have list of things TODO in the new year. While I was looking at what I am planning to achieve in the world of Semantics Database I can see that in the coming year I need to:
- Work on Finding the best way to represent the dynamic data.

- How to get over the problem of merging two different Taxonomies and Ontologies.

- How to represent the Knowledge (we gather over a period of time) and subsequently build a Knowledge Base.
- How to Import the Knowledge from an existing Knowledge Base.
- What logic we need to apply to process the data which is not represented in linear form like we get from a SQL based RDBMS.
- How do we ensure that the Process Result are kind of understood by the computer.
- How do we manage the interaction between two or more agents and define a Trust Relationship among them.

There are lots of things which I am planning at this moment to do.

Recently I was contacted by a someone from other part of the world and he suggested me to have a post on Semantic Approaches from an Industry Vertical Perspective. Same in the line of what ACCORD does. This is one more task I am adding to my list of TODO for the year 2007.

I am not sure how the list will appear one year from now when I am doing the stocktake once again. But I am sure of one thing that the list will not be the same as it is today. There will be few new items added, few removed of become obsolete. But yah the New Year is bringing lots of promises with it and I expect to have a busy year like It was in the year gone by.

Have a Wonderful and Prosperous Year Ahead...

Until Next Time..:)

Holiday Wishes

2006-12-23T18:39:00.000+10:00

Wishing Everyone a Merry Christmas and Happy and Prosperous New Year Ahead.

While this year is about to come to an End. I look back and retrospect what I learnt during this year. While there were ups and downs, rights and wrongs, successes and setbacks. But at the end it was one of those years of my life which brought lots of opportunities to learn something. The new year eve which as per calendar was around 51 weeks ago seems like something which happened yesterday. The Antarctica Flight trip was one of the most memorable experience I had this year. Added to that meeting few wonderful people on this planet. I am not going to put their names here :).

As far as Semantic Database is concerned. Last week I got a new concept of building a Database which supports Semantics rather than making it a business logic of an application. I had an initial discussion about that with few friends and they seemed to have liked it. It is just a concept which I am penning down. If you are interested in knowing what I am talking and also want to be part of that, mail me your gmail id and I will invite you to collaborate on the document.

Until Next Time.. :)

Why Semantics?

2006-12-19T20:43:00.000+10:00

I was asked once by a friend of mine as what is Semantics? I gave a very short explanation as what I am trying to achieve. But then the flow of question started as:
- Why we need such a thing?
- Why we need semantics?
- Are current technology not good enough?
- Is this going to be just another jargon?

To understand this lets go back to where we started. We refers to human beings. Going few hundred yrs back in time if we look back at where we were. Human beings were living in a closed society, We were not interacting with other culture, countries much like we do today. Only few peoples were able to travel to different places meet different people etc.

The same thing happened to computers as well. When the first computer was invented it was mere a device to perform calculation. Over a period of time it was transformed into a Stand-Alone system. Then came network, followed by internet and world-wide-web. This changed the way computers work, they interact, they collaborate.

Today Human beings are traveling to different places, interacting with different cultures, learning / adapting different environments. We are able to communicate with other cultures, peoples without any boundary or restriction. In a nutshell human intelligence has increased.

Then the question which we need to ask here is what does computers do? They perform our day-to-day tasks. They perform tasks on our behalf.

So for them to keep doing what they were designed for (i.e. performing tasks on behalf of human being), their intelligence level has to be increased. This is not an option anymore. It has to be done otherwise computers will not be able to meet their primary objectives.

The human intelligence has increased, computers intelligence has to be increased. Computer need to understand the human way of interpreting the data, understanding the data. Unless we put this sort of intelligence in computes they will not be able to perform the job on our behalf.

Linking back to one of my earlier post about Semantics, Its the study that relates to signs of things in this world. This is the way we human beings operate. If we make the computers understand the way we do, they can definitely perform the kind of tasks we do.

At the end in my opinion, Semantics is not just another buzzword. It is the way the computers are going to understand the things in future. It is more to do with philosophy than the computer science itself.

Until Next Time...:)

Is World Ready for AI?

2006-12-11T23:04:00.000+10:00

Last week I was discussing Artificial Intelligence and the Semantic Web (W3C roadmap) with few friends of mine. Every body started to describe the kind of work they are doing and how they see the whole thing will flow in future.

In the middle of this discussion It just strike in my mind, Is the world ready for that kind of sophisticated technology. We are talking about data being located on internet and one can get complete information about where he/she is located on his/her palmtop.

There were few things which came out during the discussion:

1. There is a need of such a technology which can make use of the data stored on array of servers dispersed geographically.

2. There is a requirement for such an agent who can read those data, especially the sensitive data about an individual.

3. There is a requirement of an authentication mechanism where an individual can gain access to his data and provide controlled access to others. It is desirable to have the customization feature based on an individual user or a group of users (boss, friends etc).

4. There is a requirement of an uniform way to represent the data. Here I see XML as a potential solution. Refer to my earlier post Representing Knowledge Using XML where I've put forward a case to use XML as one of the way to represent knowledge (data).

5. There is a requirement of allowing the agents (Agent of user who requests and Agent of user who processes the request) to choose the best possible algorithm for producing the output. But at the same time the room for human intervention should also be there.

The list goes on and on. But at the end when I ask a question is the world today ready for such a sophisticated computing mechanism? Unfortunately the answer I get is we are not there yet. We might have to wait for few more years before we can start seeing these (listed above) being part of normal human being.

Until Next Time... :)

Representing Knowledge using XML

2006-12-06T14:20:00.000+10:00

Yesterday I was reading a book on XML and structure of a XML document and then suddenly it striked me that, it can be used for representing any kind of data.

1. XML is a metalanguage that allows user to define markup of their document using tags. User can define their own tags and are not bound by the number of pre-defined tags / keywords as in HTML.

2. Nesting of Tags introduces structures. The structure of documents can be controlled using DTDs or Schemas. This can help in verifying the data before it goes for processing.

3. XML separates content and structure from formatting. XML is meant to carry data only and leaves lots of rooms for programs to present it in a different way. Using appropriate XSLs we can generate from an input XML a PDF, CSV, HTML and many other outputs.

4. XML is the de facto standard for the representation of the structured information on the web and supports machine processing information.

5. XML supports the exchange of structured information across different applications through markup, structure and transformations.

6. XML documents can be queried using XML Query language.

While XML has lots of advantages it also has few drawbacks as described below:

1. Nesting of tags does not have standard meaning. They don't always represent the correct structure.

2. The semantics of XML documents is not accessible to machine, to humans only. Machines just parse the XML documents as Child Nodes or Attributes etc. They do not carry any special meaning for machines to understand.

3. Collaboration and exchange are supported if there is an underlying shared understanding of the vocabulary. XML is well-suited for closed collaboration, where domain or community-based vocabularies are used. It is not so well-suited for global communication. As long as the participating organizations use the same schema they can share the XML document or interpret them. There is no such thing as global representation etc.

While the number of benefits appear to be outweighing the drawbacks. I guess still there is lots of work which needs to be done in order to use XML as a way of representing data in a way which can be understood by machines.

A common semantics or grammar is required to do so. What is your opinion?

Until Next Time :)

Today's World-Wide Web

2006-12-05T13:19:00.000+10:00

The World-Wide Web (or World-Wide Wait) has changed the way we communicate with other human beings and the way business transactions take place in this today's world. While this has changed the way we see the world. It has also changed our perception towards computers.

When computer was invented first it was mere a computing device used for numerical calculation. But since then they've evolved over a period of time and today they are not limited to numerical calculation. Today they are primarily used for information processing, databases, text processing, games, scientific calculations to name a few.

Coming back to WWW, in its current form it contains informations which can be consumed only by human being. No matter whether we look at a static HTML page or a dynamic web page where the information is retrieved from database, they are presented in a form which can be understood only by human beings. They have no significance for machines or computers.

The typical application of today's web includes surfing and browsing for information, making purchases online, connecting to other peoples etc. But if we look at these processes the main thing which comes out of it is, they are all driven by human although there is a software which supports all these activities. But they are limited to things like saving and retrieving data, establishing connections, routing to different webservers, establishing communication with other servers around the globe etc.

But these all have their own limitations and the biggest limitation is they have the pre-defined set of instruction as and how to do. The highly-sophisticated machines cannot decide their own course of action. They cannot process the information on their own.

In next in the series we will discuss what are the problems faced by the Current Web architecture.

Until Next Time.. :)

The Joy of Being a Developer

2006-11-21T13:38:00.000+10:00

Being in this profession for almost 11 years now, Sometimes this question runs through my mind as who are we? As it appears or as the world thinks of us, We (Developers) are a bunch of very very analytical and logical thinking guys. We try to analyze all the problems in the world in a logical manner. But nobody cares or gives a thought about the creative side of ours.

As a part of the developer community, I often think why we develop? There are thousands of reasons for many developers around the world. For a big chunk this is to earn for living. For few of them they develop to earn extra cash which can give them a rich lifestyle. I develop because I like to do it. Its not something which I was forced to do rather I love sitting in front of the comp and writing code.

But for most of us we are missing the fun of being a developer. The real fun of being in this profession is the creativity. I am sure we all must have jumped with joy when our first Program (Hello World) worked properly. But then on I personally experienced the satisfaction when a C-code of 4 line could restart the old DOS PC and we cried our heart out (with anger) when we found that the program which is crashing is because of the poor code written by the Tool Developer itself (which we relied on). Because those lazy programmers just wanted to get the job done.

More often than not we don't realize the amount of satisfaction we(developers) could get out of our profession by changing the way we look at our job functions little bit. Why not take the new task as a challenge for doing it better way than just doing it. The challenge could be anything, writing a code with fewer key-stroke, writing an optimized code etc. While this generates the excitement while doing the regular work, it also increases our Knowledge-Base.

All it calls for is some sort of out of box approach towards our regular job. The Joy of Being a Developer :)

Until Next Time.. :)

Sun OpenSources Java

2006-11-15T09:40:00.000+10:00

These days almost all developer sites have only one News on their front page. Sun Open-Sources Java. I am sure as a reader you must have also heard / read this by now.

Available today are the first pieces of source code for Sun's implementation of Java Platform Standard Edition (Java SE) and a build-able implementation of Java Platform Micro Edition (Java ME).

Details are available at: at http://www.sun.com/opensource/java.

In addition, Sun is adding the GPLv2 license to Java Platform Enterprise Edition (Java EE), which has been available for over a year under the Common Development and Distribution License (CDDL) through Project GlassFish(TM).

Sun is expected to release a buildable JDK in the first quarter of 2007, following established free software community practices for licensing virtual machines and their associated libraries.

What does this mean to the developer community? Now we as a Java developer will have chance to participate in evolution of Java platform. This was a much awaited move from Sun and the talks were on for quite sometime now.

Until Next Time...:)

Human Brain and Quantum Physics

2006-11-13T22:23:00.000+10:00

A while ago in office I was having a discussion related to Quantum Physics and modeling the human brain. It was a good discussion where Brian was explaining his experiences with modeling and representing realities in terms of partial differential equations.

Human brain is a collection of nerves tissues and they have a very unique way of storing information and retrieving it. Considering the fact that the number of such tissues are in order of billions and there is a mesh kind of structure they form. But then again as we study more about humans and the way human brain work. Few questions pop up immediately:
1. How does human brain stores information?
2. What is the format in which the information is stored in the brain?
3. How does it relate one information to Other?
4. Why do we remember something which happened long back but forgot something which happened recently?
5. Why it is easy for us to work with images and pictorial impression and difficult to do lengthy calculations?

There are many such questions which can be asked about it. But one of the question which I am very interested (personally) in knowing is how does it relate one form of information to another?

While studying books on Knowledge representation I did come across few pointers which tell me how the informations can be inter-related. At this moment I am not in a position where I can comment more on that. But as I progress further on this topic I will keep posting what I learn.

There are many such questions I am coming across these days. As I am getting more into the world of Semantics and ways to interlink informations with each other too many HOWs and WHATs are popping into my mind.

Until next time :)

Knowledge and Its Representation

2006-11-09T23:49:00.000+10:00

It was a while ago when Someone asked me whether I know about something (related to the topic we were discussing at that time). It immediately triggered a question in my mind as:
- What do we mean by Knowledge?
- How can I say I Know something?
- Is Knowledge something we can relate to in Real-World?
- How do we represent a Knowledge?
- Is there any standard way to represent Knowledge?
- How will computers represent Knowledge?
- Can Knowledge ever become obsolete? If so then can we still call it Knowledge?

What I understand is that Knowledge:
- Is not a Physical Object
- Is an abstract product of the human consensus
- Cannot be measured
- Never becomes obsolete, even though we don't use it. The Knowledge about not using a Knowledge is again a Knowledge
- Can have different representations in different language

Many people talk different thing about representing knowledge. But in a nutshell it is a medium of human expression, medium of effective computation.

What I feel is there has to be a universal way to represent knowledge. Sign language (Semiotics) have managed to achieve this upto an extent but still lots of work needs to be done in this area.

Until Next Time...:)

Back To Brisi

2006-11-03T14:55:00.000+10:00

Back online after 2 weeks vacation in India. Had a good but hectic holiday back home. Was home after 5 years for Diwali celebration with family :).

Will Start posting again soon. So keep watching for this space.

Until Next Time ... :)

Definition of an Entity

2006-10-11T11:36:00.000+10:00

Quite often I run into a situation where I ask myself:
- Why I need to define a particular object?
- What I neet to add to the object to make it complete?
- What will make it complete?
- What is mandatory for an object to have?
- What can be treated as optional?
- How do I classify what is mandatory and what is optional for the object?

My approach towards solving these question is what is the bare minimum requirement for the entity to exist. Like for a person to exist in this world we need First Name and Last Name. Rest other details attached to the person is based on where we need them. But name is something which is unique to the person in the system.

But then again it is also driven by where the entity is used. A person's date of birth and Joining Date is mandatory if the person is part of an organization. But they are optional if the person is being treated as Supplier or Customer.

I am still exploring on this topic and hoping to find an answer soon.

Until Next Time :)

Application of Mathematical Logic

2006-10-09T08:24:00.000+10:00

In my last post I left with two topics one of them is Application of Mathematical Logic. As I understand it leaves many people wonder what is the Mathematical Logic and how is it related to Computers and for that matter Semantics?

Application of Mathematical Logic = Applying Mathematical Logic or Usage of Mathematical Logic. In order to understand what it means it is better to understand the Mathematical Logic First.

Mathematical Logic is a specialized sub-field of mathematics where we deal with Formal Systems in relation to the mathematical concepts like Sets, Numbers and computations etc. It has an important role to play in the Study of Foundation of Mathematics. It is not much of of the Logic of Mathematics rather instead it is Mathematics of Logic. It deals with those parts of the logic which can be modeled and studied mathematically and also includes area of pure mathematics like Model Theory and Recursion Theory.

It was a while ago when I was discussing the same topic with Nathan and there we were trying to find one common language which is being understood by both human and the machines. While Humans can understand different languages the computers can understand Machine language well. Things we humans find cumbersome to do like Calculating large numbers, analyzing large volume of data etc are cakewalk for computers but things what we find easy like Face recognition, Voice Recognition are pretty difficult and at times nearly impossible for computers to perform.

So where does it leaves us. If we go back and have a look at the Applied Science everything revolves around mathematics. All the complex calculations have their mathematical into it. A while ago I was watching Matrix and the idea of representing human structure as Matrix really hit me hard. I started to think is it really possible to represent a complex structure like human in terms of Matrix? Well nothing seems to be impossible and if that is the case then Mathematics is the language which can bridge the gap between Humans and computers.

Going back to what Semantics "is the formal science of the conditions of the truth of representations" Which also can be related to mathematical equations. I have a strong feeling that mathematics is one of the ideal mechanism to represent the truth. Which will also mean that It can be used to bridge the gap between human and computer's interpretation of the Logic.

I am still in the process of exploring how the mathematics can be used for writting applications which makes human language understandable to computers.

Until Next Time... :)

So What's big Deal about Semantics

2006-09-28T21:03:00.001+10:00

More often than not I have to face this question from my friends as what I am doing these days and what kind of project I am working on. It use to be a joke among our friend circle in India as "What is the Similarity between a Beggar and a Software Professional? Whenever they meet they ask only one Question Which Platform?". I guess more or less we Software professionals are like that, especially those who fall under one special category called GEEKS :)

But the next moment I am faced with another set of questions. I tell them that I am working on Semantic Database. This starts a new set of questions. Its a new area of computer science and is not known to many IT Professionals still. Few of them even went to ask me which company's product is Semantic Database as they were confused with normal database and also why did I not try oracle database than going for Semantic Database. This reminded me of early days (1996) of my career when I started working on Delphi and whenever I told my friends that I am working on Delphi the first question they asked then was "Is it a Database?"

In my Last Post I left with a Question "What is Semantics?"

As per Wiki "Semantics refers to the aspects of Meaning that are expressed in Language, Code or other form of representation". Semantics is the study that relates signs to things in the world and patterns of signs to corresponding patterns that occur among the things the signs refer to.

Then the Question comes is What am I doing with Semantics. How is Semantics related to Computer Science. In computer science, Semantics is considered in part as an application of mathematical logic. Semantics reflects the meaning of programs.

Here We come across few new Terms like
- Application of Mathematical Logic
- Meaning

We will deal with these in Subsequent posts.

Until Next Time...:)

Role of Primitives in Semantic Language

2006-09-25T22:51:00.000+10:00

Its Monday evening and I am back to my books. After a weekend away from technical reading I am back to normal reading now and also discussing online with my mate Nathan about Semantics and the philosophical concept of Semantics.

In search of finding the root of semantics or what is required to make something semantic. I was digging into books and articles and in one of the articles from one of the famous author so far (John F Sowa), I came across something which is so good to be true. He explained what it is required for a Natural language or First order language to be semantic.

Every Natural languge or the First order logic deals with five primitives. They are called semantic primitives because they go beyond syntactic relations between signs to semantic relations between signs and the world. Any notation that is capable of expressing these five primitives in all possible combinations must include all of FOL as a subset. As an example, the WHERE clause of the SQL query language can express each of these primitives and combine them in all possible ways; therefore, first-order logic is a subset of SQL. Different languages may use different notations for representing the five primitives:

1. Existence. In most natural languages, existence is implied by mentioning something. For emphasis, languages also provide an explicit existential quantifier such as the word some. In the algebraic notation for logic, existence may be expressed by an explicit symbol, such as $. In SQL, existence is stated implicitly by mentioning something or explicitly by using the keyword EXISTS. A real World Example could be something like "There is a Person". This statements emphasizes on existence of the person.

2. Coreference. To say that two different signs refer to the same thing, natural languages use a variety of methods, both explicit and implicit: pronouns, determiners, inflections, and forms of the verb be. Most linear notations for logic use variables and the equal sign, and graphic notations use connecting lines or ligatures. Like other linear notations, SQL uses variables and the equal sign. In real-world we can equate to something like "He is my Friend". Here we are referring to someone as a Friend.

3. Relation. Content words in natural languages express some information about at least one entity, known as the referent of the word, but they may also relate or imply other entities as well. The verb give, for example, refers to an act of giving, but it also implies a giver, a gift, and a recipient. In SQL, relations are called tables. A Statement like "Michael Schumacher Drives Ferrari". Here the relation between Michael Schumacher and Ferrari is being established.

4. Conjunction. In both natural and artificial languages, conjunction may be expressed implicitly by making one statement after another or explicitly by a word like and or a symbol like Ù. SQL uses the keyword AND. Like Samir Lives in Australia and Possesses Indian Citizenship.

5. Negation. All natural languages and most versions of logic provide words, inflections, or symbols to express negation. The biggest variations from one language to another are in the methods for distinguishing the context or scope of what is negated from what is not negated. SQL uses the keyword NOT with parentheses to show scope. e.g. He is not Working.

In our day to day life we make use of these primitives numerous times. And they have become part of ours. This poses a very legitimate question is "What is Semantics?"

It is coming in next post..

So What's the big deal about Atomicity

2006-09-19T15:14:00.000+10:00

Often I have to face this question as why do we cry so much about atomicity? What does atomicity has to do with Software Engineering? When I started to find out more about "Atomic Theory", I came across a term called Atomism in philosophy. According to Wikipedia (http://en.wikipedia.org/wiki/Atomism), The Atomism is defined as "all the objects in the universe are composed of very small, indestructible elements - atoms. Or, stated in other words, all of reality is made of indivisible basic building blocks. The word atomism derives from the ancient Greek word atomos which means "that which cannot be cut into smaller pieces".

Now the question comes back to how it can be related to Software Engineering. To answer this question lets look at a Software System. A Software System comprises of many small - small independent functional modules. Each module performs a specific task.

While designing a Software System if we break a big system into smaller and independently managed modules who can work in isolation our task becomes simpler. The whole system can then be developed easily and tested in its own right.

Atomic theory also implies that a system is whole of its parts. It means the systems behaviour is the combined behaviour of all its parts or sub-systems.

Until Next Time...

Semiotics and Semantics

2006-09-18T12:11:00.000+10:00

I was reading an article from John F Sowa on Semiotics and Semantics and there is somethign I found very interesting in that article. It goes back to what semiotics and semantics is all about.

The study of signs, called semiotics, was independently developed by the logician and philosopher Charles Sanders Peirce and the linguist Ferdinand de Saussure. The term comes from the Greek sêma (sign); Peirce originally called it semeiotic, and Saussure called it semiology, but semiotics is the most common term today. As Saussure (1916) defined it, semiology is a field that includes all of linguistics as a special case. But Peirce (CP 2.229) had an even broader view of that includes every aspect of language and logic within the three branches of semiotics:

1. Syntax. "The first is called by Duns Scotus grammatica speculativa. We may term it pure grammar." Syntax is the study that relates signs to one another.

2. Semantics. "The second is logic proper," which "is the formal science of the conditions of the truth of representations." Semantics is the study that relates signs to things in the world and patterns of signs to corresponding patterns that occur among the things the signs refer to.

3. Pragmatics. "The third is... pure rhetoric. Its task is to ascertain the laws by which in every scientific intelligence one sign gives birth to another, and especially one thought brings forth another." Pragmatics is the study that relates signs to the agents who use them to refer to things in the world and to communicate their intentions about those things to other agents who may have similar or different intentions concerning the same or different things.

What I found is Semantics is all about how we represent real-world into the world of computers. It is all about how the smallest piece of information is represented in the whole universe of data and how it fits in the context of the big picture.

Why RDBMS is not suitable for Semantic DB

2006-09-14T10:08:00.000+10:00

Since the day I started working on Semantic DB, I found that there is a need for a database which is very flexible and not rigid like RDBMS. I know many people will argue why I say RDBMS is rigid. Here is why I see RDBMS not working for me when it comes to store objects in semantic way.

1. It has a very fixed Structure. Define once and let all the objects fit into the structure.

2. The data is stored in a contiguous block in the database.

3. The chances of duplication is higher. I mean to say that a value cannot be uniquely identified in the system.

I guess I am looking at the wrong technology to solve my problem.