Spider Hunter

The core of the 404 Trap is parsing the environment variables to figure out what the user is looking for and then display that data. I have a few different ways that I have been doing this over the years, but I really have settled on one way in particular. I design my sites so […]

The basic principal of a 404 Trap is that the pages you are displaying to your customers do not exist. (I love this part of the conversation, it confuses so many people.) For the 404 Trap to work you need the web server to start generating a 404 Error message, which in the web server […]

Why? Search Engine Optimization silly! I’m not 100% sure of the status of this right now, but most search engine spider do not index dynamically generated pages. Google’s Googlebot does, but they even admit that this is limited in the amount they will index. So what is the answer? Make your dynamic pages look like […]

Some of the people who know me from the old days are groaning right now. Saying under their breath, not 404 Traps again … Yup, it’s time to start talking about 404 Traps … again. Also, for some of the more recent readers, these blogs give me a place to jot down my ideas and […]

Thinking back on it I most likely should have only talked about ipconfig and left the route command to more advanced areas, but oh well. Starting in Windows 2000 ipconfig is the main tool to configure your IP protocol stack. In Windows 98 there is a graphical tool called Win IP cfg, but it seems […]

I’ve been doing this so long now that I forget that some people may not be sure how to track down information on an IP address or even why you would want to. Over the years I’ve written internal scripts that I use on my own server that makes it easy for me to see […]

Yesterday I spent some time getting the framework for the RSL site up and running. It’s done well enough to call it a Web site, but it is really still in a BETA state. In any case the real conversations that I will be having about the development will be over there from now on. […]

I got the website up and running at www.realtimespiderlist.com It’s a silly little site at this point hat just barely describes what I want to do with the site. I got the DNS server powered by DBJ DNS (TinyDns) working and now it is down to the real work. I need to write the code for the […]

Yesterday I registered the domain name www.realtimespiderlist.com, then spent some time working on a dedicated server to do the DNS lookups for the RSL. I’m hoping to have something in a BETA form before the weekend, but we’ll see how the rest of my week goes first All in all it looks like this […]

I was in Yosemite all weekend and during the drive I was thinking about how I would like to implement the Real-time Spider List. First off I found that TinyDns from D. J. Bernstein at http://cr.yp.to/djbdns.html . This is the same guy who started Qmail as well, at least that is the way I remember […]