User Controls
Bounty: Kill SpectraL
-
2018-03-25 at 12:18 AM UTC
Originally posted by Speedy Parker That is absolutely BS.
http://www.robotstxt.org/robotstxt.html
That's how it was explained to me.
and this says Robotstxt there is a robot.txt
edit in
Yes it says robots.txt (Im guessing its the same or I am right) and it's saying just that.. it can block a crawler. but it's not explaining all of it's abilities on this site.
Its the same thing when you pull your pud and climax halfway through before mom or dad comes busting through the door to tell you dinner is on. -
2018-03-25 at 1:05 AM UTC
Originally posted by BeigeWarlock That's how it was explained to me.
and this says Robotstxt there is a robot.txt
edit in
Yes it says robots.txt (Im guessing its the same or I am right) and it's saying just that.. it can block a crawler. but it's not explaining all of it's abilities on this site.
Its the same thing when you pull your pud and climax halfway through before mom or dad comes busting through the door to tell you dinner is on.
Pud puller! -
2018-03-25 at 10:53 AM UTC
Originally posted by BeigeWarlock That's how it was explained to me.
and this says Robotstxt there is a robot.txt
edit in
Yes it says robots.txt (Im guessing its the same or I am right) and it's saying just that.. it can block a crawler. but it's not explaining all of it's abilities on this site.
Its the same thing when you pull your pud and climax halfway through before mom or dad comes busting through the door to tell you dinner is on.
A robot.txt file can not do what you described regardless of how bad it was explained to you or how bad you misunderstood the explanation. -
2018-03-25 at 11:07 AM UTC
-
2018-03-25 at 11:17 AM UTCWhat the fuck? Aren't you supposed to be a millionaire? You pledge $3.50?
Jesus Christ. -
2018-03-25 at 11:20 AM UTClol Speedy Parker's definition of how a robot.txt works. lol
-
2018-03-25 at 11:40 AM UTC
-
2018-03-25 at 11:50 AM UTC
-
2018-03-25 at 11:59 AM UTC
-
2018-03-25 at 12:10 PM UTC
-
2018-03-25 at 12:45 PM UTC
-
2018-03-25 at 1:06 PM UTC
-
2018-03-25 at 1:18 PM UTC
-
2018-03-25 at 3:56 PM UTC
-
2018-03-25 at 4:11 PM UTCGrimace vs Speedy Parker. Last man standing. Who will win? Stay tuned next week for the answer.
-
2018-03-25 at 5:52 PM UTC
-
2018-03-25 at 5:53 PM UTC
-
2018-03-25 at 7:43 PM UTC
Originally posted by Speedy Parker A robot.txt file can not do what you described regardless of how bad it was explained to you or how bad you misunderstood the explanation.
Really.. well then. You're suggesting it was those at archive (or someone who worked there) that did it. I was trying to give them the benefit of the doubt. and that's how it was also verbally conveyed to me.
But if you're the expert, then I stand corrected on that. That makes it even more involved though. If it was a sort of retro-pull or retro delete thing, then it wouldn't of been so deep rooted. but now it's clear someone physically removed it from the server.
They had enough problems as it was with the feds didn't they? When Arron Shwartz got into trouble trying to gather Public Domain Literature for Archive. commits suicide thinking the feds were going to put him away for the rest of his life (Which is why I kind of hate google that much more) -
2018-03-25 at 7:54 PM UTCThose pages WERE archived on archive.org for some length of time. Very strange.
-
2018-03-25 at 7:57 PM UTC
Originally posted by -SpectraL Those pages WERE archived on archive.org for some length of time. Very strange.
I could add another scenerio
Someone broke in. The owners or people who run it, never new.
anything is possible. Physically broke in or hacked it. but perhaps a forensic data person could figure it out. Assuming they didn't switch out HDD server. which they probably will say they did.
Edited in: if someone hacked in and removed.. I wonder if the original owner or current owner (because it was purchased by archive) had also made hard copies.
