Comments on: Taming the robots

By: edith

edith — Thu, 29 May 2003 15:53:15 +0000

I did figure it out, thanks. Doesn’t it suck how the syntax is different for every frickin’ thing you have to do in this world?

By: austin sipes

austin sipes — Thu, 29 May 2003 15:30:40 +0000

You probably figured this out, but I forgot the colon after "Disallow" in the example I sent you. Sorry ’bout that.

Great blog, BTW, and great music. If you get a chance and want to listen to some good indie label stuff, check out http://www.ibrecords.com. If your own music is any indication, you’d like e_o’s album, in particular.

By: edith

edith — Thu, 29 May 2003 08:14:11 +0000

THANKS, I’ll try that. I’d rather do it the legit way instead of having to delete posts that are attracting too much attention. Especially since I run my own cgi search engine — if people REALLY want to find something they should look there anyway instead of trusting what Google says.

By: austin sipes

austin sipes — Thu, 29 May 2003 07:59:46 +0000

You disallow certain files the same way you disallow whole directories. Just use this syntax:

Disallow /directory/filename.html

Disallow /filename.html (for URLs on the root)

Hope this helps.