RelevantText.com

SES NY Day 4: Meet the crawlers

March 20th, 2008 Jason Posted in Google, MSN, SES, Yahoo, conferences, seo | No Comments »

Session brief:

Representatives from major crawler-based search engines cover how to submit and feed them content, with plenty of Q&A time to cover issues related to ranking well and being indexed.

Yahoo!, Google, and MSN all put themselves on the podium and each ran through an overview of their webmaster tools. Anyone who is already using these tools and/or keeping up on the industry blogs probably didn’t get much new out of it, though I did catch just a couple tidbits:

• Yahoo! will now accept the robots.txt sitemap.xml URL being on a different domain. I think this is pretty new, and useful for anyone who may have issues with getting things hosted on their corporate servers.
• The protocol for supplying a Google news feed is different than the standard sitemap.xml protocol. I hadn’t looked into this much in the past, so that’s probably not new, but it’s good information to have heard.

For once, the Q&A was the heart of the session, and after a bunch of standard “I have this very specific issue with my site” kind of questions, I couldn’t resist the opportunity to try raising a ruckus with a meaty one. I wanted to know how the engines were currently viewing the use of the rel=”nofollow” attribute on internal links to a website’s own pages. Though Matt Cutts has gone on record about it several times, I think there’s still confusion, so I asked. The responses:

• Sean Suchter of Yahoo! said they are not using a nofollowed link for calculating the “link quality distribution” score for a page. He did not specifically say, “don’t use it on internal pages,” but he did specifically say, “I would be wary of using it for sculpting pagerank.”
• Evan Roseman of Google did a marvelous bit of dancing and basically said “go find Matt’s post,” but also said that there were some situations where an internal nofollow might be appropriate. He did not comment on the use of it as a sulpting/siloing device. (The post in question is, I believe, on Matt Cutt’s blog here, but there are other more recent – and not always clearly consistent – quotes on SEOMoz, SEORoundtable, and Dave Naylor’s blog as well. In fact, there’s a lot of opinion out there, if you do as Evan suggested and Google: Matt Cutts nofollow )
• Nathan Buggia from MSN dodged entirely and simply said he thought there were better uses of your time and money than worrying about it. Which, interestingly, is pretty much what Matt said in Dave Naylor’s post, and with which, honestly, I have to completely agree…though he didn’t, technically, answer my question.

I was a bit amused that Evan referred to the question several times as a “pretty advanced topic.” Sorry, man, the SEO 101 session was three days ago, and I didn’t cross the ocean to go to it.