Structure The Web And Improve The Web

Most people in SEO have heard the term "semantic search" - this means that a search engine is for a specific niche with the aim being that you get more relevant results for your searches.

Let's say you search on Google for "Derby" - you could mean the Kentucky Derby, Derby hat, Derby in England, hotels in Derby and so on.

So there's a whole lot of people creating search engines for these niches - this is a "good thing". We've looked at it ourselves and have a few test sites out there but the problem we have is that it's great that we have context for our results but what about the results themselves?

If I'm looking for Derby hats, I want to see pictures and prices, if I'm looking for Derby in England, I want to see the local council site, the football club, the university and so on.

So you're looking at totally different results pages for each niche. This means you need a purpose written crawler to get the information you need.

The problem then is (bear with me here) that every website is built differently. So let's go back to the Derby hats example - the crawler needs to be able to identify products, pictures and prices and produce a nicely structured result. This is almost impossible with a generic web crawler. To approach each and every retailer to get a feed would be quite some task so you're left with creating a specific crawler for each site.

Now this isn't a problem in itself - it's not a huge job to get structured content from most websites but there could be 20,000 Derby hats websites - to write a crawler for each site is now a BIG job.

For me, this is why semantic search is always going to be limited. Until websites adhere to standards of some sort (ideally XML or meta tags) then there will always be this problem.

Putting content into meta tags is not that big a deal. Various government websites should adhere to certain standards which mean that data can be derived from a page such as source, author, FOI status and so on. If someone came out with a meta standard for website content then this would really shake things up.

Imagine a whole load of meta data with product title, description, price, pictures - a crawler could simply pull that information down and display nicely formatted results.

Google, Yahoo! and MSN/Live Search (whatever) are ideally placed to put such a standard forward. If Google said "if you put these tags into your pages then we'll rank you on X" then you can imagine thousands of webmasters reacting immediately.

It's no small task but it's do'able (it's a word) - over to you Google - "organize the world's information and make it universally accessible and useful." (one of Google's mission statements)

You can read more about this on my Squidoo lens.

Dan Frost
CEO of Inter Advertising Ltd and co-founder of Crawl Score

Drastically Improving Aircraft Performance Using Acoustic Waves
Is it possible to send sound waves under an aircraft wing to thicken the air and allow an aircraft better buoyancy? Can we thicken the air around an a...

What is the ‘Strong Matrix’ in Business?
Earlier than 1970, large businesses were most typically formatted in a silo design, where workers were divided into departments that report to a line ...

Matrix Organizational Structure - Optimize Cross-Departmental Relationships
Before the 1970s, big corporations used to work in "silos". This denomination refers to groups of workers who functioned in separate spheres and answe...

Success - Three Questions to Choose Your Peer Group
Do you have the experience of being hanging around with the r wrong people? There is a strange feeling of known you are notr in the right place. Even ...

Management Of Change - Keep Things As They Are
What would be more difficult: to stop smoking in a smokers environment or to quit when everybody around you continues with their same habits? Is it mo...

Make Your Site Search Engine Friendly
Current cut throat cyber world doesn't allow your business's online existence as it snatches away the targeted traffic that your website needs in orde...

Various Steps For The Website Optimization
1:-Keywords analysis: There is a very first step in the website optimization is to collect the relevant keywords for your website that can define the...

Change Versus Improve - Two Sides of the Same Coin
Always a good question to ask when dealing with a possible change is: Why? Why dont we improve instead?We can not.Well, we have tried this for the las...

Google Pagerank Algorithm Explained
Google uses the PageRank algorithm to rank pages in their search engine results. The algorithm is based on analysing link structures: each link can be...

Good Organisational Structure Enhances Infrastructure
A person who has a lazy, slow-moving gait tends to look less commanding than someone who walks with a good posture that exudes confidence. Likewise,...

How To Structure Your Website For The Search Engine Ranking
A good search engine ranking result may be the most sought after goal of any website owner. Just building a website doesn't automatically result in vi...

JetBlue Discount Airline Tickets Going in for the Kill Shot
With Delta in bankruptcy we see the more successful discount air carriers going in to the kill. JetBlue is expanding, buying planes and opening routes...

Black Belts Within A Company’s Structure
InvestmentThe immediate investment costs of training existing employees may be more than the monthly fees charged by an external Black Belt, but when ...

Mortgage Loan Rip - Off - Facts To Know
The Facts Surrounding Mortgage Fraud and Predatory LendingPredatory lending occurs chiefly in the sub prime mortgage market, where most borrowers are ...

Can An Article Directory Improve My Search Engine Rankings?
Keyword-Rich ArticlesYes, of course they can. The problem is, as with all other internet marketing methods, they are not always executed correctly. Fr...