We got into some of the more technical goods from the paper yesterday, but figured these would also be an worthwhile — or at least more enjoyable — read. Friday and all.

1. “Wow, you looked at a lot of pages from my web site. How did you like it?” – people encountering a crawler for the first time

They note that they received almost daily emails from people either concerned about copyright issues or asking if they liked the site after looking at it. For many people with web pages, this was one of the first crawlers they had seen.

“It turns out that running a crawler which connects to more than half a million servers, and generates tens of millions of log entries generates a fair amount of email and phone calls. Because of the vast number of people coming on line, there are always those who do not know what a crawler is, because this is the first one they have seen. Almost daily, we receive an email something like, “Wow, you looked at a lot of pages from my web site. How did you like it?” There are also some people who do not know about the robots exclusion protocol, and think their page should be protected from indexing by a statement like, “This page is copyrighted and should not be indexed.”

More innocent times.

2. A billion web documents predicted by 2000

“It is foreseeable that by the year 2000, a comprehensive index of the Web will contain over a billion documents. . . The goal of our system is to address many of the problems, both in quality and scalability, introduced by scaling search engine technology to such extraordinary numbers.”

Now in 2018, there are reportedly 130 trillion documents on the web — an extraordinary number indeed. And sure enough, their search has scaled to meet it.

3. Google took up 55 GB of storage

“The total of all the data used by the search engine requires a comparable amount of storage, about 55 GB.”

Now, Google is 2 billion lines of code. As noted by one of their engineering managers in 2016, the repository contains 86TB of data.

4. “People are still only willing to look at the first few tens of results.”

Please note: “tens.”

They write about the need for more precision in search. Remember the days when people regularly clicked past page 1?

5. Percentage of .com domains: from 1.5 to 60, to now 46.5

They note how “commercialized” the web was already becoming, leaving search engine technology “to be largely a black art and to be advertising oriented.”

“The Web has also become increasingly commercial over time. In 1993, 1.5% of web servers were on .com domains. This number grew to over 60% in 1997.”

According to Statistica, the number of .com domains is down to 46.5% as of May 2018.

“With Google,” they wrote, “we have a strong goal to push more development and understanding into the academic realm.”

6. “There are two types of hits: fancy hits and plain hits”

After going into some technical detail about optimized compact encoding, they reveal that they’ve their complex compact encoding preparations are categorized simply — endearingly — into fancy and plain.

7. Already defending user experience in anticipating search

From the start, it seems Brin and Page fought for users to not need to excessively specify their queries in order to get desired information. They wrote:

“Some argue that on the web, users should specify more accurately what they want and add more words to their query. We disagree vehemently with this position. If a user issues a query like “Bill Clinton” they should get reasonable results since there is a enormous amount of high quality information available on this topic. Given examples like these, we believe that the standard information retrieval work needs to be extended to deal effectively with the web.”

It’s interesting that this was so clearly in their thinking from the beginning. At last week’s Search Summit, Googler Juan Felipe Rincon said, “The future of search is no search, because search implies uncertainty. Instead, it will be about how you populate something before someone knows what they don’t know.”

8. There was a typo

In the second paragraph of section 3.2, they write “Couple this flexibility to publish anything with the enormous influence of search engines to route traffic and companies which deliberately manipulating search engines for profit become a serious problem.”

Did you catch it? The verb should be, “companies which are deliberately manipulating search engines become” or “companies which deliberately manipulate search engines become.” Of the utmost gravity, we know.

Just goes to show that even if an incomplete verb phrase won’t keep you from doing some pretty cool stuff in the world. And of course, that even the best of us need editors.

9. Search Engine Watch shout out

We tweeted this yesterday, but felt the need to share again for extra emphasis. Our very own Search Engine Watch was cited in the paper, stating that top search engines claimed to index 100 million web documents as of November 1997. Been a fun 21 years.

10: They chose these photos

Happy Friday, everyone.

Links you build require action on your behalf, while links you earn happen without any work specifically toward acquiring links.

Let’s take the common example of content. You’ve created new content and with that investment you’ve used precious time, resources and personnel.

Your wonderful content — that you’ve invested time and hard work into — is ready for unveiling to the masses. What’s the next step?

Well, with any content there’s the natural promotion channels that should always follow publication:

1. Promote through social media
2. Contact anyone featured or mentioned in the post
3. Leverage relationships
4. Utilize paid promotion (depending on budget and content — could include things such as paid tweets, Facebook ads, etc.)

Those are natural steps any content creation should involve, customized to the content of course. For link earning, there’s no additional steps beyond that. You simply wait for the links to roll in after publication and promotion.

Earned links, on the other hand, are others recognizing your greatness and awarding it appropriately.

Personally, I don’t depend upon others to recognize and reward my hard work. The world we live in is simply too fast-paced, too self-centric. If you want someone to recognize your value, you have to promote that value. Intelligently.

For those interested in building links, there are a few additional steps after publishing content:

• Identify audiences who will appreciate your content
• Identify sites where that audience spends time
• Identify pages on those sites where a link would make sense
• Contact the webmasters of those sites, informing them of:
     o The fact that your content exists
     o The value of your content to their audience
    o The suggested page where a link would make sense
    o A request for feedback regarding the content
• Follow up with webmasters as necessary

Bear in mind this is specific to content link building. Building links is only limited to creativity and there are plenty of ways to build links without content.

Will you get links without going to the extra steps? It’s entirely possible, especially if your initial promotion gets solid traction.

But for every link you earn, there are more links that you deserve, but that your promotion won’t reach naturally; links which you could secure with good old-fashioned elbow grease and targeted promotion/outreach.

So which method sounds better to you, earning or building? Why would you invest in content creation if you’re not going to do everything in your power to promote it? What would keep you from finding relevant websites and encouraging them to promote it to their audiences via links?

How did we arrive at the trend of “link earning,” and why does link building sometimes sound like a dirty word?

The answer: Penguin.

Google’s Evolution and Algorithms

Google based its algorithm around links back when it was still known as Backrub and just a project Larry Page and Sergey Brin were working on in grad school at Stanford.

Google’s reliance on links is no secret — everyone in search knows it and you can still go read the original abstract on Stanford’s website.

That was the founding of Google — a search engine that recognized a link as a vote of confidence and trust between websites. A link as a signal of authority and relevance.

Over time, Google has evolved, grown and adapted to the web. However, one thing hasn’t changed: links are still a big signal in its algorithm.

There are other signals, too, of course — Google’s quick to mention over 200 — but the fact of the matter is few things move the needle like links.

SEOs notoriously exploited that in the past. Links became shortcuts to quick and easy rankings, leading to manipulation and bad results. Google had to step in, and step in they did, in a big way. Specifically, with Penguin.

Penguin, however, didn’t reduce the effectiveness of links, or reduce links’ role in Google’s algorithm. Instead, Penguin detects low-quality and manipulative links and works to either discount or actively punish those links.

Unfortunately, links are a complicated and confusing subject, especially to those who aren’t savvy in SEO. Everyone understand what a link is, but few understand how Google actually works.

Many business owners and marketers are unsure of the difference between a decent link and a link Penguin will punish.

Why The Term “Earning” Has Merit

Google has successfully (or largely so) ended the ability to spam links for rankings.

Considering it’s been 11 months since the last Penguin refresh, which is necessary to recover if the algorithm is impacting your website, your website has had zero chance of recovery for nearly a year.

A year with significantly reduced search traffic could very well mean the difference between a successful business and a failed business.

Have I mentioned that Google doesn’t let you know if you’ve been impacted by Penguin? The only way to know is to monitor when your search traffic dropped and approximate that with a Penguin update.

So you have Penguin, which will:

• Impact websites it deems to be building low-quality, manipulative links.
     o Google won’t tell you which links these might be, although they’re quick to warn away from “low-quality tactics” which you may or may not be using for SEO.
• Remain in place for uncertain, but large amounts of time.
     o Google’s less than clear about when it will update, or any timeline in general.
• Require hours upon hours of link removal requests, along with a “machete” for disavows.
     o Google wants to send spammers a message and make the penalty hurt.

The fact of the matter is Penguin is a looming, dire threat to any business looking to engage in SEO, particularly if they don’t understand the ins and outs of the industry.

Any link or tactic that bears even a hint of manipulation or low quality has a very dark stain at this point. It’s simply not worth the risk. And that stain has rubbed off on the term link building.

You see, we need a way, a term, to differentiate between link building activities pre-Penguin and post-Penguin.

Hence the term “link earning.”

People want to be so far on the side of safe, so removed from anything that has any chance of trouble, so it is 100% “natural” that they only want links people point at their website without prompting.

The problem with this?

For every link you “earn” there are five more links worth building. And link earning gives the wrong impression.

The Real Issue with the Term Link Earning

The real issue with the term link earning is the mentality it inspires. The if-you-build-it-they-will-come attitude.

Any marketer can tell you there’s just no such thing. Just look at the amount of noise online (courtesy of Domo):

There is such an incredible amount of noise. Thinking that you can simply earn your way through it is setting you up for disappointment.

Links are important to SEO and subsequently search engine traffic. No one can dispute that with a straight face.

And I’m not saying that you should work to build links no matter the cost. What I’m saying is that you should look to build links that you deserve. Good links, that make sense.

If you buy into the concept that you can and should stick to “earning links,” you’re hindering your own ability to work effectively. You’re unnecessarily handcuffing yourself in fear of Google.

Matt Cutts, head of webspam at Google, said himself there’s nothing inherently wrong with link building, in both an interview with Eric Enge and more recently in June at SMX Advanced.

There’s nothing wrong with earning links — but you shouldn’t be afraid to build them, as well. Especially if you’re creating something of value and deserve attention. Put in the extra work to get the most out of your website. We are optimizers, after all.

Google’s Matt Cutts is the superhero of combating webspam. Certainly he doesn’t work alone at the offices of Google; he has what is known as the webspam team. But this isn’t “The Avengers“; Cutts is the star of his own movie.

Of course, there are more people fighting webspam. Duane Forrester is one of those people; he’s the Cutts of Bing.

Forrester’s name recognition is much less widespread than Cutts’. This is simply due to the fact that Google hoards such an inordinately large share of the search market. This also might be due to the fact that Cutts releases an average of two to three videos per week, videos in which he takes the time to answer burning SEO questions. Even though he is a regular at conferences and is hardly averse to doing interviews, Forrester simply doesn’t have Cutts’ visibility.

A lot of what Forrester says can get lost in the shuffle, but every now and again he will make a bold pronouncement that pricks our ears.

What he said on May 9 of this year is as good an example of that as any. Let me draw your attention to myth number seven in particular.

• Links are all I need

While important as a vote of confidence for the content they point to, there is simply so much link spam these days that it’s tough to know where to turn. Obviously buying links is a dead end, and it doesn’t matter how you split this hair: sharing, encouraging, incentivizing, buying – it’s all the same. You want links to surprise you. You should never know in advance a link is coming, or where it’s coming from. If you do, that’s the wrong path. Links are part of the bigger picture. You want them, but you want them to be natural. If an engine sees you growing tem naturally, you’re rewarded with rankings. If they see you growing them unnaturally, you’re rewarded with penalties.

First off, let me say that I concur that “links are all I need” is a ludicrous myth. I myself am a link builder, and I certainly can’t overemphasize enough the value that link equity provides in attaining visibility in the SERPs, but they are far from the only ranking factor. There are some 200+ other signals to consider.

And yes, there’s undeniably a hefty amount of link spam out there, the kind of stuff that strengthens the principle of “this is why we can’t have nice things.”

‘Surprise Links’

This is about where Forrester and I stop agreeing. The following segment of this myth simply left me confounded.

You want links to surprise you. You should never know in advance a link is coming, or where it’s coming from. If you do, that’s the wrong path.

Every now and again, you will get links that surprise you. It’s happened for our company site several times, and it’s a great feeling when it happens.

Here’s the catch-22 of this, however: these “surprise” links are only possible if your site already has some visibility. And without link building, the pursuit of relevant and authoritative links, attracting visibility is difficult.

Here’s something that I’ve learned during my time as a link builder: link building is no longer just SEO; link building is also promotion.

Link Building Is Promotion

This wasn’t always the case. Link building used to be a much more technical trade than what it is today. But the “e” in SEO often stands for evolution, because is it is an ever-changing industry.

SEO and link building have slowly but surely been integrated into the proverbial marketing funnel. Most comprehensive marketing strategies include an SEO component today.

This was most likely inevitable. Marketers are having to adapt to new channels provided by new technology.

Television reigned for several decades, but don’t let the present moniker of “the golden age of television” fool you: that’s more to do with the quality of the programming than the amount of people watching. In fact, a lot of these amazing new shows aren’t being watched traditionally, as Netflix has birthed the cultural phenomenon of binge-watching.

Succinctly said, we live on the Internet now, and even though the death of television advertising is often overstated, marketers are allocating more funds toward Internet marketing. Even though the two have a few differing characteristics, it’s still all about disseminating your brand message and name.

Links are a powerful form of marketing, because a link is an endorsement from another brand. It’s another site in your niche lending you authority and recommending you to the surfing public.

This is untrue of television advertising. The average watcher understands that a 30-second spot on “The Big Bang Theory” has been paid for. Yes a network can and sometimes does execute editorial authority over the brands that are not allowed to advertise on the network, but for the most part they will feature the advertisements of anyone willing to pay the price.

The Importance of Links

Links don’t work that way. Let me rephrase: links shouldn’t work that way.

The following are examples of link schemes which can negatively impact a site’s ranking in search results:

• Buying or selling links that pass PageRank. This includes exchanging money for links, or posts that contain links; exchanging goods or services for links; or sending someone a “free” product in exchange for them writing about it and including a link

Google founders Larry Page and Sergey Brin made links the most powerful signal because they appreciated the value of a link, that a link was an endorsement from site A to site B. That’s why you want the links. It isn’t only because you want to increase your rankings, but also because a link is a message that you are trustworthy or offer a unique value.

Links are an incredibly powerful branding tool. The more users see that other authority sites are willing to link to you, the more likely those users are going to believe you are such an authority.

But links don’t just come to you, and if your site is relatively new, no one is going to find your site in a search engine if you don’t have any links. You have to do the work to get your name out there.

This is link building.

Why It’s OK to Pursue Links

An email pitch to a webmaster is no longer just a plea for that webmaster to lend you a little link equity. More importantly, it’s a promotion of your site, a call-to-action that your site is a valuable resource to users/readers of her/his site.

This is why I find Forrester’s proclamation so egregious. Why is it so wrong to notify another webmaster that you have a page or asset on your site that may contribute to the value of that person’s site? Yes, the end goal is to increase your own visibility, but if you’re adding to the user experience that search engines like Bing are so eager to preserve, that seems like a good thing to me.

And as I said before, it’s simply foolish to think links should only come as a surprise. A majority of Web traffic is directed by search engines. These engines depend on links in order to determine rankings for search terms. But if a webmaster for a burgeoning site isn’t allowed to promote her/his site for links, how does that site have any hope for ranking?

Thankfully, Cutts acted as a voice of reason at SMX Advanced earlier this month:

Q8: Is link building just dead? You keep saying a new tactic is dead, or must be nofollowed, is it really you just don’t want people to try to build links at all?

Cutts: No, link building is not dead. And a very small percentage of links on the web are nofollowed. There’s a lot of mileage left in links.

(Danny clarifies that he doesn’t mean links in general, but instead people building links).

Cutts: Duane Forrester had an article on Bing’s Webmaster Blog that said you should never know a link is coming – that’s the wrong path. That’s going a little bit far. Really, do compelling stuff and your links will take care of themselves – if you’re doing great stuff you won’t always know about the links you will get. Add value. “It’s easier to be real, than it is to fake real.”

It’s clear from this interview that the goal should be to create compelling content and do what you can to add to the user experience. Some links will come to you “by surprise” – and you can track those incoming links fairly easily – but it isn’t wrong to approach other webmasters in your niche to alert them of your valuable content. Just don’t abuse that privilege and teeter toward spam.

As marketing evolves and moves more toward the Internet, search engines must be able to accommodate. After all, sites like Google and Bing contributed to the rise of the Internet.

Link building isn’t just for the socially isolated hackers anymore; it’s in the boardroom meetings of major conglomerates. You would think that Bing, a subsidiary of a major conglomerate themselves, would be able to appreciate that.

So I’ll do you a favor, and I’ll avoid trying to tell you that “content marketing is dead.” We’re nowhere near that point yet.

An incredibly small percentage of marketers have even reached the point where they’ve been able to execute it properly, and I’d be very surprised if the approach has already peaked.

And yet…

We’re digital marketers. Our livelihood is played out on a battlefield where not even the ground itself stays the same from one year to the next. To remain competitive, we need to stay at least one step ahead of the curve, and just as importantly, think two or more steps further down the road.

It’s with that in mind that I can see a future, a relatively near one at that, where what we call “content” today will not be king. I know that this next step exists, because the most powerful companies on the web have already taken it.

It’s time to talk about the elephant in the room. It’s time to talk about that next step.

Applications, the Future of Digital Marketing

In 1996, Stanford PhD students Larry Page and Sergey Brin built a search engine as a research project. By 1998 it was already getting talked about on top sites like Salon.com, which said it gave better results than Hotbot and Excite.

At that time, Google had acquired a grand total of $100,000 in funding and had virtually no marketing in place. By 2004, Google handled 85 percent of the searches on the web.

In January 2004, Harvard student Mark Zuckerberg created a social networking site for Harvard students. Its growth continued to include members from all Ivy League schools. The site had a million users by the end of the year, with only a $500,000 angel investment.

These sites were doing very little marketing at all, and certainly not “content marketing.”

A quick look at the most linked to sites on the web reveals that essentially every single one of them is a tool of some kind. Facebook, Twitter, Google, YouTube, WordPress, Wikipedia, and LinkedIn simply don’t owe their success to content marketing. To pretend that they did would be disingenuous.

They owe their success to innovative (and not necessarily complicated) software ideas. They have come to dominate the web because they are more than content. They are interactive. People use them.

And let’s not forget about Amazon, the site that consumers now turn to even before Google when they want to research a product. Amazon was just one of many ecommerce sites in the 1990s, and the media didn’t think much of it. But then Amazon started inventing things. Recommendation engines. The ability to look inside a book before buying it. An API. It was this that took Amazon from a site that didn’t turn a profit in its first four years to the most wildly successful ecommerce site on the web.

The future of the web is only going to get more tool-based as users switch over to mobile.

Here’s a jarring statistic from Flurry: users spend 80 percent of their mobile time in apps. Digital marketers are devoting 100 percent of their time to something that users are only devoting a fifth of their mobile time toward.

Flurry doesn’t expect the web to die as a result of this. Instead, they expect it to start looking more like apps.

Oh, and did I mention that in 2011 people were already spending more time in apps than on the web? That’s everywhere, not just in mobile. And, again, that was 2011.

Are you ready for this change? Because it’s coming either way.

Tools to Connect

“OK, so people are using apps more than the web,” a Mr. Straw Man says. “Who cares? Look how much of that time is devoted to Facebook. We can’t cut into that!”

Actually, Mr. Straw Man, believe it or not, Facebook use may be in decline, at least in the first world.

Facebook has already publicly admitted that teens are using their social network less than they used to. In fact, research suggests that Twitter is now more popular than Facebook among teens. But what may be more interesting is the surprising rise of niche social tools, mostly for mobile.

For a while it looked like the social space was completely packed and there was no room for new players. But now we’re seeing things like:

• WhatsApp, the world’s most popular messaging app, currently has 350 million active users. That’s more than Twitter. WhatsApp is also working on an API that will also allow users to share content with each other, but in a private setting more intimate than Facebook. It’s one of many messaging apps which, together with texting, take up way more use than Facebook.
• Snapchat, a messaging app that friends use to send selfies to each other, which disappear shortly after, has 5 million monthly users.
• Vine, the social platform with its six second videos, has at least 40 million users, and has continued to grow despite the introduction of Instagram’s video feature.

“OK, but still, those are platforms dedicated to social. We’re not social networks.”

Well, here’s some more news for you, Mr. Straw Man. Did you know that 65 percent of American social media users have read an online message board in the past week? That almost half have read one in the past 24 hours?

That this makes online message boards more popular than blogs?

In fact, Reddit, which is essentially a message board, had 90 million unique visitors last month, making it roughly as popular as Pinterest.

On top of that, free forum platforms like moot are taking forums in a more modern direction.

I’m not suggesting that installing a pre-packaged forum platform on your site should be the beginning and the end of your approach to this new era of social tools. I am suggesting that doing so is probably the bare minimum if you change nothing else, and if you want to matter in five years.

As humans, we remember and care about experiences far more than things, or content. We will always remember and treasure a vacation more than, say, a movie (unless it’s “Star Wars”). The marketers of the future need to create interactions. Users experience them, and so they become memorable. Peer reviewed research suggests that this is the only way social networks create sales anyway.

While it can be useful to do this on a platform like Facebook, it’s far more useful if it takes place on a platform of your own, where the call to action isn’t far away, the topic of discussion is related to your product, and there are no distractions.

Your tool doesn’t have to be the next Facebook. It just has to connect people. If it’s also innovative enough to pick up some press coverage, that’s all the better.

Tools to Create

What might be even more memorable than an interaction? The act of creation.

You could say that YouTube, WordPress, and Blogger are some of the most linked to sites on the web because of their content, but that would be missing the point. These sites dominate on the web because they make it extraordinarily easy for people to create something that they otherwise couldn’t, at least not without a server and some programming knowledge.

We’re only just now seeing the beginning of what tools like these can accomplish. Social networks, for the most part, don’t really facilitate the act of creation. But if you’ve taken a look at the stuff that gets posted there recently, I have a feeling images like these will be familiar:

Image Credit: someecards

Image Credit: Imgur

Someecards sees roughly 100 million impressions each month, and Imgur gets about 70 million unique visitors each month. All these sites really do is make it easy to post text on an image and share it in a way that looks a bit better than something you would make in MS Paint.

The fact of the matter is, users are very hungry for tools that help them create something to share with their friends and family, and they don’t have many to choose from. Most just do a Google image search and post something that looks original to them, because nothing else is easy enough.

There’s a wealth of opportunity here.

Just take a look at some of the most popular iPhone apps:

• Elf Yourself: An app from Office Max that you can use to make an elf version of yourself.
• Perfect 365: People use this to give themselves virtual makeovers.
• Emoji: To create images made out of smileys.
• InstaCollage Pro: For creating collages.

All of these apps have tens of millions of users, and of course I’m just glossing over Instagram, which offers filters that have made the image sharing tool attractive to a base of 150 million active users.

The important thing to realize about tools like these is the fact that you don’t just reach the people who use the tool. The end product is likely to end up on social networks, where it will be shared and seen by a much larger audience.

Perhaps even more importantly, if you empower your users to create something of their own, they may start to develop a common language and form a culture around your brand as well.

Farewell to the Static Web

Embedded Vines, tweets, and Facebook posts, Pinterest style infinite scrolling, Amazon style recommendation engines, and advanced message board technologies are only the beginning. The web may have started as a series of web pages with little more interactivity than the ability to click a link, but this is changing.

The sites with the best tools have almost always dominated the web, but this is changing from something we expect from elites to something we expect from every corner of the net. As we move away from the PC and onto our mobile devices, we increasingly find web browsers dull and boring compared to the immersive experience of apps.

Apps are the future of marketing. Don’t say I didn’t warn you.

Google quietly made a huge change to its core search technology in the past month, one that affects about 90 percent of searches worldwide. Calling it Hummingbird, the technological update of Google search is designed to handle more complex queries.

Calling it the culmination of 15 years of work, the news was revealed by Amit Singhal, Google’s senior VP, one day before Google officially celebrates its 15th birthday. He spoke at a special press event today in the garage that Susan Wojcicki rented to Larry Page and Sergey Brin when they started Google in Menlo Park, California,

“Our algorithm had to go through some fundamental rethinking of how we are going to keep our results relevant,” Singhal said.

Beyond announcing that Google search is now be better at understanding concepts, relationships between concepts, and more complex questions, and despite numerous questions from reporters, Google wouldn’t give any more details about how Google Hummingbird actually works, according to various reports.

As TechCrunch explained:

The main focus, and something that went repeated many a time, was that the new algorithm allows Google to more quickly parse full questions (as opposed to parsing searches word-by-word), and to identify and rank answers to those questions from the content they’ve indexed.

Caffeine, which rolled out in 2010, was designed to improve indexing speed and provide users with fresher results.

Search engines – webmasters and search engine optimization (SEO) professionals follow their guidelines for the highest possible rankings on them; paid search marketers pay to be featured on them; and users turn to them when they’re searching for answers, information, or entertainment.

Search Engine Watch has been covering search engines since June 1997 and has watched the industry evolve to its current state. Over time, many search engines have come and gone, as users have spoken with their keyboards (and literally with their voices – thanks to voice search technology).

In recent years, search market share has remained mostly unchanged – for much of the world, it’s Google followed by every other search engine (in the U.S. the “Big 5” search engines consist of Google, Bing, Yahoo, Ask.com and AOL, which combine for hundreds of billions of searches every month). Meanwhile, many of the players have consolidated or have become footnotes in history.

What follows is an overview of today’s major global search engines, with some history and explanation of why each one is important to webmasters, marketers, and users. We’ll then review some of the top directories.

Major Search Engines

Google

Started in 1998 as a university project by Stanford University students Sergey Brin and Larry Page, Google is now the dominant search engine by no small margin and that didn’t evolve slowly.

In fact, in June of 1999 Netscape Search was updated and AOL/Netscape search began to be powered by Google bringing their search volume to approximately 3 million per day; huge for the time.

On June 26, 2000 Yahoo Selected Google to provide its organic search results (replacing Inktomi) with its impressive index of more than 25 million pages; again, huge for the time.

Google has since become synonymous with the word “search” and as most of us know, is often used in place of the word. Don’t know the answer? Google it!

The continued strength of Google as a search provider is based on a large number of factors and won’t be debated here, save to say, they have successfully provided the results people are looking for in a manner those searchers either enjoy or are comfortable enough with not to switch to a different provider.

Google continues to tweak their search algorithm multiple times per month and adjust the layout of their results to test for improved visitor experience and advertising revenue.

The majority of Google’s revenue is derived from their AdWords and AdSense programs. In fact, advertising accounts for more than 95 percent of Google’s earnings. If there is a weakness in the Google model this is it; they need to tweak their layout and results to promote the paid avenues of their offerings. This gives advantages to other engines who may have revenue generation strategies outside of search.

Bing

Bing was launched in May 2009 as a fundamental upgrade from Microsoft’s previous efforts into search, MSN Search.

Since the launch of Bing, Microsoft’s share of the search marketplace has more than doubled. Add to that the deal between Microsoft and Yahoo for Bing to power Yahoo’s organic results and Bing powers over 25 percent of search.

With the Microsoft/Yahoo alliance also came the affect that Yahoo’s paid search platform would be used to power both Yahoo and Bing’s paid results. While this may not seem like a big deal on the surface, it is actually huge.

Where once business owners and marketers had to consider whether it was worth the hassle of managing both a Bing paid campaign (for the significantly lower traffic they yield over Google) and also make the same call on managing a Yahoo paid campaign – the two now are manageable in one convenient location, significantly reducing the time it takes to setup and manage.

This of course makes the cost for these campaigns less expensive and when you combine that with Bing’s increased market-share then they’re in a position to take some of the ad dollars from Google (or at least, gain some for themselves).

Yahoo

Yahoo is an interesting search engine and one which, until recently, I had a very hard time taking seriously.

Once upon a time Yahoo was a major leader in the search field but has been in decline, making bad decision after bad decision, announcing layoff round after layoff round and making what can only be described as one of the worst business decisions in history when they turned down a takeover from Microsoft valued at $33/share. Yahoo shares dove after that and have never been anywhere close since.

From that point, until 2012, it seemed that every piece of news from Yahoo was bad news, until July 16 when the announcement came that they had snagged Marissa Meyer from Google to become CEO. This was the first move they’d made in a long time and had people wondering if this might just be the breath of fresh air and change of direction that the company needed.

From reviews of all hires and selling key properties such as Alibaba to putting their own search technology back on the forefront; Yahoo has maintained its position as one of the top three search engines, despite not producing their own organic results.

Other Major Search Engines Around the Globe

Google dominates the U.S. and most of the world – but not everywhere. Yahoo and Bing have had about the same luck (zero) making a dent in Google’s search market share on other continents, but a couple of search engines in other countries have managed to stay ahead of the Mountain View, California-based search engine. If you’re from these countries or interested in marketing to them – pay attention.

Baidu

In China, Baidu is the major player with more than three of every four searches conducted on their engine.

To say Baidu blends organic with paid search is misleading, they use a hybrid approach wherein they have pay for performance (P4P) results (users bid to have their websites place at the top of what would appear to be the organic results).

In addition, Baidu offers PPC which, similar to AdWords, is displayed at the top or right of the standard results. One could argue that the existence of the PPC-like results further confuses the users clicking on the standard results area into believing they are organically generated.

While some investors consider Baidu to be overvalued as a stock, their earnings are consistently high. For companies looking to market into China, understanding Baidu is crucial.

Yandex

Yandex is the primary and most popular of all Russian-language search engines with significant market dominance in Russia.

On October 1, 2012, Yandex launched their own browser and mobile app to keep their position secure against Google, their only real competitor in the space.

Yandex’s advantage in Russian seems to be based on an algorithm that performs much better in understanding the unique syntax used and integrating that into the consideration of what type of results the user is likely looking for (for example – is the search string a question or simply keyword entry).

Directories

Directories are an interesting topic. Do they carry weight? Can they hurt your rankings? Should you even bother? The answer here is yes, yes, and yes.

This section will mainly focus on general directories, but the end of this section does include a few tips on how to find niche directories (or even other general directories) and how to determine if they are worth getting a listing on.

Yahoo

The Yahoo Directory was started in 1994 under the name “Jerry and David’s Guide to the World Wide Web” but in 1996 became Yahoo. At the time Yahoo was primarily a directory with search functionality and (interestingly) neither SEO nor Internet Marketing were even categories at the time.

Through the late 1990s Yahoo pushed to become a web portal and in 2000 even signed a deal with Google that would see Google power Yahoo’s search functionality. Their focus at the time was to acquire users through acquisitions such as GeoCities (RIP), bringing more people into their portal and keeping them there. Unfortunately Yahoo! didn’t have the same user loyalty that Apple does and the walled-garden approached failed as users Googled their way out of the Yahoo network of sites (ironically right on Yahoo’s own properties).

All this said however, they still provide a solid directory (back to their roots). The cost is a non-refundable $299 review fee.

BOTW

Best Of The Web may be my favorite of the general directories due in no small part to the fact that they allow for a permanent listing. The directory was founded in 1994 as a listing of the best of the web (seems to be the year of directories) and actually gave out a series of awards (take a peek, it’s interesting to see what types of things won back then). That lasted until 1998 at which time the site lay dormant until purchased in 2002 at which time it became a general web directory.

BOTW is a human edited directory. They will decline your listing if they don’t like the site. A submission is $150 annually or $300 for a permanent listing.

DMOZ

No list of directories would be complete without DMOZ. DMOZ was founded in June 1998 as Gnuhoo. It was purchased by Netscape in October of the same year at which time it became The Open Directory Project. By April 2000 it had surpassed the Yahoo Directory in number of URLs in it’s index and currently sits at about 5.2 million.

For those in the industry long enough to remember, DMOZ suffered a catastrophic failure in October of 2006 at which time they had to display a backup version of their directory. This wasn’t remedied until December and new sites couldn’t be suggested until January. This is he time when it seemingly became increasingly difficult to get a listing in DMOZ as any editors seemed to have found new things to do with their time.

It is still possible to get a listing in DMOZ. For the 10 minutes it takes, it’s well worth the time and it’s free to submit. (Tip: try to submit to a category that has an editor.)

Business.com

Business.com was started in 1999 as a search engine for business and corporations. They came close to bankruptcy during the dot-com bubble bursting but after major layoffs and restructuring they became profitable once more in 2003.

Business.com is focused on business-to-business resources (so take that into consideration when thinking about submitting. The cost is $299 per year and all submissions are reviewed manually.

As with Yahoo and BOTW, the fee is non-refundable if your site isn’t accepted. You’re paying for the review, not the link.

Honorable Mentions

Moving past the major players, there are also a number of other good general directories. These directories have all survived many updates including the Penguin and Panda rounds.

Remember, though, link building is about balance. You don’t want to submit to a bunch of directories and consider your job done. A better strategy would be to bookmark this page, submit to a few and as you’re building more links using different strategies, add a directory or two mixed in with the rest.

• Jayde – Submission is free.
• Ezilon – $69 annual fee of $199 permanent.
• Alive – $75 annual fee or $225 permanent.
• 01 Web Directory – Free submission option or $49 one-time for a guaranteed 3-day response time.
• Aviva – $50 annually or $150 permanent.
• SunStream – $29 annually or $49 permanent.
• Wow Directory – $43 for a permanent listing.’
• GoGuides – $69 one time.

Again, this list only contains consistently solid general directories.

Directory Guidelines

You’ll want to also look at niche directories (which may well hold more weight than any of the general directories above), but you need to be careful. There are many horrible directories out there.

Here are a few directory guidelines to follow that universally apply:

• Is the submission a guaranteed placement? If a directory will list you automatically (with or without a fee) then it’s not an editorial link and either doesn’t carry weight or likely won’t in the near future. It should be avoided.
• Do they require a link back? If they do (even for their free listings when a paid is available), it probably should be avoided.
• Is their PageRank 3 or below? Yes, it’s an old metric, but is still helpful to gauge general site health. A directory with a PageRank of 3 or less will, at best, pass virtually no weight; at worst, it’ll cause you problems. Generally, you should only look at PageRank 3 directories in the case of niche directories; with general directories, don’t even consider anything less than a 4.
• Common sense. Ah, the toughest one because our brains can trick us into seeing what we want to see. When you look at a directory (or any other link source for that matter) you have to ask yourself, “does it make sense that this link should pass weight to my site?” If you can honestly say “yes” to this then it’s likely a good link.

A Final Warning

The saying “don’t put all your eggs in one basket” comes into play here. Once again, directories can provide good and relevant links to your site (and hey, even some traffic) but a solid link profile contains variety.

Never put all your energies into one single link source. If you find a lot of great niche directories, put them all on a list and add a couple each month while you’re engaged in other strategies to help remind Google that you’re not a one-trick pony. You have good content liked by directory editors, bloggers, social media netizens, and others.

The whole question was raised by a series of correlation studies by Searchmetrics, Moz, and others that found “amazing” correlations between Google +1’s and higher search rankings.

These studies prompted Google’s Distinguished Engineer Matt Cutts to find the politest way to debunk the idea that more Google +1’s lead to higher Google rankings.

This, in turn, prompted a serious discussion about the difference between correlation, causation, and coincidence.

And this is what prompted Stone Temple Consulting to conduct a study of the impact of Google+ shares on search rankings. This is not a correlation study, but an actual measurement of causation. And the most surprising finding was: “Google Plus Shares did not drive any material rankings changes that we could detect.”

This means that Cutts was telling the truth. However, in this industry, even this finding will probably prompt yet another round of comments and debate.

Eric Enge, president of Stone Temple Consulting, provides the gory details of his new study in an article that appeared on his website this morning, “Measuring Google Plus Impact on Search Rankings.”

Enge also spoke at SES San Francisco last week and provided some hints about today’s findings.

In addition, Search Engine Watch interviewed Enge – under embargo – about his study, which had three major goals: to see if Google+ would drive discovery, indexing, and ranking.

In terms of discovery, Enge said, “In my opinion, it is highly likely that Google+ drove discovery of the content.”

In terms of indexing, Enge found that Google+ shares probably drive indexing, as well. However, of the six articles tested (three test pages and three baseline pages), all six articles initially appeared in the Google index 10 days later.

“We saw no evidence of Google+ shares driving ranking,” Enge said. “Once we saw that a page was indexed, we were immediately able to find search queries for which the page ranked. However, this does not mean that the shares were driving ranking. As per the original Sergey Brin – Larry Page thesis, each page on the web has a small amount of innate PageRank. This PageRank by itself might cause a page to rank for certain types of long tail queries, even in the absence of any other signals.”

For example, one of the test pages had 10 search phrases. As the chart below shows, there is no pattern to the rankings since the page appeared in the Google index:

Here’s my complete interview with Enge, which is available on the Search Engine Watch channel on YouTube:

In addition, Enge will be holding – appropriately – a Google+ Hangout on Air entitled “Documented Google Plus Impact on Search Rankings” this Thursday at 4 p.m. ET to discuss the study’s findings. Joining him to discuss the study will be an all-star panel that includes Mark Traphagen of Virante, Pete Meyers from Moz, Joshua Berg of Google Plus SEO and SMO, and Marcus Tober of Searchmetrics.

Google announced its intention to issue Class C stock last year, however the Brockton Retirement Board and shareholder Philip Skidmore sued the firm, claiming that the firm’s co-founders Sergey Brin and Larry Page engineered the stock split in order to keep control of the firm. Google has now settled the lawsuit before the case was about to go to trial in a Delaware court.

Google’s rocketing share price that currently stands above $850 highlights the demand for the firm’s stock, but diluting the Class B stock would mean that Brin and Page would also lose some of their voting influence in the company as the pair own over 50 percent of all Class B shares, with each share having 10 votes per share. Page and Brin therefore wanted to issue Class C shares, which would carry no voting rights.

Google’s legal agreement stipulates that the firm will have to pay Class C stockholders if the shares are worth less than the price of the Class A shares. The firm has also agreed that if the Class C shares are between one and five percent lower than the Class A shares it will offer a percentage of the difference to Class C shareholders in cash or additional shares.

Despite Brin and Page having founded Google, with various early investors taking a chunk of the firm, they have only around 15 percent of Class A shares. Page and Brin want to keep control of Google, but because it is a public company the only way they can do so is by keeping significant voting rights through Class B stock ownership.

With Google seemingly having overcome the hurdle posed by Brockton and Skidmore, the path now seems to be clear for it to issue new Class C shares.

This article was originally published on the Inquirer.

Forbes has released its annual list of the world’s billionaires. While most of the richest people in the world remain the same, one noteworthy change is that Google co-founders Larry Page and Sergey Brin are no longer tied for net worth.

Page, now CEO, and Brin debuted on the Forbes 400 together in 2004 tied at $4 billion and I believe have been consistently at the same net worth level since on Forbes’ annual list of the richest Americans, which is released in September, as well as this list.

However, among the world’s billionaires, Page, 20th on the Forbes list with $23 billion net worth, is slightly ahead of Brin and his $22.8 billion net worth, good for the 21st spot. The duo were tied in 24th place last year, with $18.7 billion.

Once again, the rich got richer overall, with Forbes reporting a record number of billionaires (1,426) and record net worth ($5.4 trillion).

Here’s a quick rundown of search, social, and tech notables on the list.

• Jeff Bezos, Amazon CEO, was 19th on the list with $25.2 billion net worth.
• Steve Ballmer, Microsoft CEO, 51st, $15.2 billion.
• Mark Zuckerberg, Facebook CEO, 66th, $13.3 billion.
• Eric Schmidt, Google’s Executive Chairman, 138th, $8.2 billion.
• Robin Li, CEO of Baidu, ranked 172nd, $6.9 billion.
• Reid Hoffman, LinkedIn Executive Chairman, $3.1 billion.

Google began with a couple of guys and an idea. Two Stanford graduate students, Larry Page and Sergey Brin, were thinking about how to make a better search engine.

At the time, this seemed about as good as an idea as trying to come up with another fast food restaurant that was going to dominate America. It certainly seemed like there were enough search engines already, and a powerful few maintained a stranglehold on the market.

But the search landscape was far from perfect. Searchers weren’t getting consistently relevant results, and those results typically took a long time to load.

Yet most people didn’t notice or mind these problems since online search seemed to be such a miraculous improvement over what had existed before: calling the theater and spending ten minutes listening to a recording of movies listings and show times; or lugging out the Yellow Pages and phoning 12 bicycle shops to see if they had a 24-inch unicycle tire in stock.

Online search, with useful information at your fingertips from the privacy and convenience of your computer, seemed like a miracle. Page and Brin assumed there were better ways to serve it up.

From Keyword Stuffing to Off-Page Factors

The big problem with search engines, prior to Google, were the rules those engines used to decide what page to show the searcher. The search engines would look at the pages, read the words on the page and say, “OK, so that’s what this is about.” The more times the keyword appeared on the page, the more relevant the search engine assumed that word was for the searcher.

Website owners quickly figured out that getting a top ranking on the search engines for important keywords was crucial to their success. Rather than thinking about the experience of the end user (their customer), they lured prospects to their pages by manipulating the search engines.

A search for “24 inch unicycle tire” would return the pages that contained that phrase the most. Search engine optimization (SEO) became an arms race to stuff as many keywords into the web page as possible.

As you can imagine (or perhaps remember), search engines rewarding “keyword stuffing” meant that searchers were landing on pretty dismal web pages.

In this face of this less-than-optimal search experience, Larry and Sergey asked a crucial question: in other settings, how do people decide if a source is relevant?

They applied their question to the world of academia: How do researchers know when a particular paper, book or study is really important?

How Scholars Vote

Scholars vote for important works by citing them in their own work. The value of a study can be determined by how many times it’s referenced in subsequent work.

If a work is published and never gets referenced, it probably isn’t of great value. In contrast, a book or paper that appeared in the notes and bibliographies of other academic publications – that must be an influential work.

Google was built on this insight and analogy. Page and Brin devised what they called “off page factors” to determine whether a web site or a web page was relevant for a particular search term. A web page would be deemed important and relevant by Google and would rank highly for a keyword if a lot of other web pages linked to that page using the keyword as the visible part of the link.

For example, a post on a blog read by many unicycle enthusiasts that links to unicycles.com/tires as follows:

“… I got a great deal on a 24” Panaracer unicycle tire, so I was able to refurbish my unicycle…”

…would help that page rank highly in Google for the keyword [24” Panaracer unicycle tire].

The more “popular” those linking web pages are, the more popularity they confer upon a page they link to. Like how you can identify the popular kids in high school because they hang out with the other popular kids.

For the first time, the top positions on search results pages were assigned based on merit as determined by a broader community.

In a matter of few months with no advertising, Google eclipsed the other search engines and become name brand of search. We don’t Lycos, we don’t Webcrawl, we don’t AltaVista; we Google. The improved search experience made Google the overwhelming winner.

What does that fun history lesson mean?

Moral 1: Your Ads and Website Must Support Google’s Prime Directive

Google understands that its prime directive is to quickly give searchers relevant results. When they launched AdWords in 2002, they applied the same principles to advertising that they did to their “real” content. Ads that aren’t relevant, or annoy searchers, threaten Google’s dominance by diminishing the search experience.

Google discovered that search is different from newspapers or television. People don’t pick up newspapers or flip through channels on their television hoping to find an ad for unicycle tires.

When people search online, on the other hand, they are almost as likely to be looking for a sponsored listing as a free one. Whatever scratches their itch, whether it be organic or paid.

Unlike typical advertising media, Google allows and rewards ads that provide a high quality experience for searchers. Google measures this in two ways:

• Does the ad match the search intent, as represented by the search query?
• If someone clicks that ad, do they get a high quality experience on the website?

Only when the answer to those two questions is yes will they pay attention to the auction that determines placement.

Google cares about the experience of the searcher much more than they care about you giving them a dollar in exchange for a click. A click today means a dollar today, while a satisfied searcher means many more dollars tomorrow and beyond.

Even though you pay Google when you use AdWords, you aren’t their customer. The satisfaction of the searchers who never give Google a dime is the cornerstone of their vast empire.

Moral 2: Stay Ahead of Changes by Understanding Google’s Motivation

The second relevant aspect of Google psychology is this: Google understands it rose to prominence out of nowhere really quickly, based on a slight edge. They had made, initially, a slight improvement to the search experience.

Therefore, everyone at Google has profound paranoia drilled into their DNA: they think of Google as being one tweak away from being out of business completely.

Just as their rose to dominance based on a slight edge, lots of people are out there trying to create better search engines, aiming to eat Google’s lunch.

Page and Brin ran the company from a friend’s garage for six months, so they know that a good idea is more powerful than lots of funding and huge a staff.

Google has never stopped experimenting to improve the search experience. That’s why thousands of Google advertisers woke up one morning in summer 2006 to discover that all their five cent keywords now cost them $10. Google had determined that their ads represented a poor quality experience for searchers, so it penalized the advertisers until they improved that quality.

And pretty much every month over the past 7 years, Google advertisers have been surprised and shaken by “Google Slaps”: changes in policy and algorithm that kick them off the first page of search results.

If you don’t understand the direction in which they are moving, which is ever increasing relevance and quality of search experience, you’re likely to wake up one morning sputtering in indignation at what they did to your account.

Once you grok the basic Google concept (a win for the searcher is a win for Google), AdWords becomes not easy, but simple. Write ads and create landing pages and websites that serve your prospects’ first and your business second.

The best way to “game” Google is to pretend that Google doesn’t exist and treat your prospects they way you’d like to be treated.

Image Credit: Danard Vincente/Flickr