Do You Even Crawl Bro?
Making Sure a Search Engine Can Even Fucking Find You
Why was this assignment so stressful?
You need to show up as a search engine result to promote your business? A website can be an excellent way to create organic growth for your business because it increases exposure to potential customers.
Yet something isn’t right. You’ve got your website, but no matter the search engine, or your word choice, no one can find it. What’s going on? It may be that the spider bots that index the internet can’t read your website. You’ll need to clean up and optimize your webpage. This will make it easier for the bots to index your site.
First, make sure you are writing in the simplest code, such as HTML. The easier it is to read the more likely it will be found and show up in search results. That means no Flash, AJAX, or complex Java script menus. It’s alright to use these in bits and pieces, but it is more complex to read, and takes more time to load, so don’t use it for your entire site, or the more important pieces of information. Some bots will skip over it, and other bots will leave it if it takes too long to load. The more user-friendly your site is for humans, the more likely a bot will queue it. After all, the spider is trying to find a site that is useful for a human. Large image files can also slow down a bot. Simplify video, audio, and other images to attain maximum loading speed.
Speed is our next target. If the page takes too long because of errors or large files, it will drop it from the results, but if you’ve done all of the above, and you’re still not showing up, it could be a server problem. In which case, you’ll need to fix the server, or find a new server provider. Otherwise, no matter how clean your site, traffic will remain minimal.
Next, links. Creating links inside your website improves crawlability. The most information is on the first page, but the next page has some really useful info. Connecting them allows the bot to connect too. Making the links contextual, rather than sitting at the bottom of the page, will also help.
Moving on to the importance of your URL. Once more, keep it simple. Don’t let your URLs get out of control and become especially long (Dynamic URL). This applies to Session IDs as well. Bots see the Session ID as repetitive, and thus, useless.
Finally, once you have done all of the above you can top it off by submitting an XML sitemap. A site map can help show the bots more about your website that they may have missed. Do not do this first. Sitemaps can also deter spider bots if it is not done correctly, negating its purpose. You can also request Google to do a test crawl of your site before going live to fix anything you missed.
You’ve been told before that bots look for key words in a search and that it will find the site that has the most of that key word. That is true, but if it occurs too frequently, search engine spiders know what you’re doing, and will consider it a useless website. Don’t put down a paragraph of one word at the bottom of your page or behind a picture, it doesn’t work.