Chasing The Search Engine Robot: Ed Stockwell
Search Engine Optimization(seo) aka Internet Publishing

The days of the tips and tricks are over. Sorry, all the sites that are relying on old games to manipulate robots have ended. Search engines, aka robots are getting more sophisticated by the day. What works for one completely obliterates another. It is simply impractical to maintain pages for each robot. The fact of the matter is that robots are becoming much more adept at searching out content. And that content better be correct and fresh. Yes, grammatically correct and in some proper form.

The fact of the matter is, if it has a link, a title, keywords, content and relevancy; it works. Tricking robots by using ghost pages has been gone for years and now so is loading up pages with keywords. What matters most is that the pages on your site contain quick and accessible content for the user. Links are helpful, but even the reciprocal link is being scrubbed by many in the SEO world. It comes down to the fact that what is on your page matters. How often that content changes and how relevant it is; MATTERS! If you haven't changed your home page, navigation, items... basically the CONTENT on your pages; well... your site just might not pop up as #1.

This leads to an interesting conundrum. What are we to do other than rely on robots for traffic. Oh, the marketing question. SDI has integrated into it's systems the ability to create tangible materials to use in your stores in order to promote and drive traffic. These case cards and shelf talkers work. There is also the question of just how much "shameless self-promotion" you and your staff is doing. If you are relying on robots to drive all your traffic, you will be at the mercy of the random hit. Instead of being up in the office be on the floor. I have a friend in the business that takes "shameless self-promotion" to an art form. He is a walking, talking billboard for his business. If you speak to him, I'll give you a matter of a minute or less before he mentions his site. He has been doing this for 18 years now and has even branched out into writing books.

SDI is continuing to evolve and improve it's websites in order to keep pace with technology. It is up to you to make your site, aka your business relevant to your clients. SDI is implementing new measures in order to make links and content more accurate and more relevant. The argument has been made that the use of dynamic pages reduces your sites ability for a robot to properly catalog your site. This is simply not true. In fact, the robots prefer a dynamic page structure. The robots view this as a statement of competency in publication and technology. There again, if the dynamic rich content is not dynamic, rich or relevant the robot lowers your rank. We have all seen sites come and go from the #1 position on SERP's. The site is hit by the robot then 90 days later it is off the map. Dynamic, rich, relevant and it hit what SEO's call the "sandbox". The site hasn't moved as far as the content published is concerned and it goes right to the back of the pack.

Note: If you use a keyword on your page, that content must appear on the page. If it doesn't, your page rank will suffer. Filling up the meta keyword section, meta descriptions, etc. with non-relevant content is a "no-no". The robots are wise to this trick and lower your page rank significantly. The axiom that less is more really rings true in this case and the axiom that being direct and to the point is everything.

When it's all said and done, you have to be an expert and be on top of your game. If you sleep more than 6 hours a day you are not on top of your game. If you want that coveted #1 slot, your golf game and social life are going to suffer. SDI makes the recommendation that your site be updated no less that once every two weeks. Remember, we are dealing with some really smart people here. If your entire site changes everyday, well that isn't good either (unless it's supposed to). If you have a section on your site that is called wine of the week, that page had better be updated every week. If it isn't, the robots will know it and your sites rank will suffer. Make your changes consistent and in a pattern. Robots like patterns. You can actually tell robots when to come back for more content.

SDI is using new technology that self spiders our sites and provides the information directly to Google and other robots. This is new technology and SDI is in the testing phases right now. This web search and site mapping arsenal of tools will run automatically on all SDI sites to update and publish your content directly to robots. SDI will inform you when these changes will be invoked.

As an aside... the new content management program is complete and is being released to all clients. This new tool contains a wide variety of features that will allow you not only to link your data to Digital Wine Archive content but also fine tune your data in ways that were previously impossible. Requesting content has never been easier as you may now request content on a winery, not just a single product. Once that winery is entered and our spider catalogs the winery; future updates and new releases will automatically hit your database. The process to link client content to harvested data takes approximately an hour. Initial sweeps have linked 480 items out of 3000 items. Items are flagged as "questionable" and you must go through those items, clean up your data and try linking again. Using the content scrubber tool you can effectively reduce the number of questionable items and increase the relevancy and accuracy of the content on your site.

When all is said and done there is really only one thing that matters. Your site, how you manage, market and update it. This is not and never has been a business that you put out there and the sales just pile up. That was the attitude with most ".com" companies and they all went belly up. I have clients that update their pages on a regular basis and clients that don't. The ones that do have more products in the #1 slot on SERP's than the ones that don't. Once again, the key is regularity and consistency. By nature and by their invention, robots don't interpret. They are programmed to recognize patterns and then make correlation within and between those patterns. This concept is in theory simple but in application once you add human interaction and manipulation becomes very complex. Make your site alive and relevant to the subject you wish to promote or sell. In the long run the cream always rises to the top!

Ed Stockwell is President and founder of Stockwell Design Inc (SDI). SDI has been creating and developing web-sites and data mining robots since 2000. Sites and data systems geared particularly for the wine trade and a system called "Digital Wine Archive" are but a few of SDI's arsenal of products. You can contact SDI at 973-300-1333 or visit SDI at http://www.stockwelldesigninc.com.