• Operand
  • hou are ue?
← Chronicle

Scrubcore; a summary.

Cleaning, you copy?

  • Scrubcore, beginnings.
  • [Hou come scraping is necessary?][#necessary]
  • [Hou come scraping is dangerous?][#dangerous]

Scrubcore began online and physically; in online spaces designed around sharing physical experiences, and in physical spaces organized around producing more, on line.

So, as to be precise and clear; the DC region had a bunch of disillusioned engineers, seeking refuge from the predominant corporaciónal pseudo-order. Also seeking companionship, rare laughs, occasional chemical escapes, and a more nuanced sense of their posición and dirección, long careers impending.

Hold on, did I say engineers? Probably is a simple reason the remainder had been incomprehensible.


Online spaces are broken. I could say they suffer from a lack of open-ness, or commercialización, only I'm realizing those accusacións only make heads nod; and my need is to induce grasping hands, and punching fingerpads, and limber lips and lengua, channeling lunges of loud lungs.

So I'll describe my experience.


I am only an engineer by lack of more applicable label; really, engines scare me. I could probably lose a finger, and I'd be lost without any of my fingers.

All of this space you see around these phrases - produced by my fingers. They're close to my soul, and carry the soul ahead and usually lead the soul on.

My fingers learned to play music, and they bring the percussion of music to bear on the language I speak and impress in keys.

So engines are really dangerous to someone in my position, and I mainly operate as a consumer of the moción an engine produces, so I can bring my skilled fingers from one place to another, to plug in cables and compose essays and share code I imagine can help people.


So, lacking a proper label on my career, this 'formerly-called' engineer has a rich experience in an also-poorly-labeled practice called "web scraping".

Web scraping - uh, is the idea behind search engines. I build logical programs to copy online pages, enough said.


Only, are these logical programs, or mechanical? The pages are assuredly less than logical. And mechanics are present all across online spaces, because those spaces are managed by innumerable machines.

Some people describe the different "logics" of social media domains, in in the same breath can discuss the "mechanics" of a video game.

To me this is nonsense, they are made of the same pieces - machine code. Procedures.

So, because "machine" may also be a bad descripción, I'll err on the side of processors and procedures, and label the essence of programming to be 'logic'. This seems to be aligned to the origins of the discipline; the deposición of logical analogues into electrical meshes.

Those electrical meshes are the real machines, and you can go deeply enough in the idea space to imagine that all logic really is a machine; and so is all math and language.

I do encourage you to aspire to such insanity; as the prizes are numerous, rich, and amusing.


Web scraping is a bad idea - legally in many cases, and also once again because of the encumberances of language. People really imagine the space nonsensically, so I hope I can reduce the pace enough to glimpse the core issue.

Here goes:

  • [Hou come scraping is necessary?][#necessary]
  • [Hou come scraping is dangerous?][#dangerous]

Hou come scraping is necessary?

Chances are, there are search engines older than you are. This only holds for some of the audience here, although my guess is that people older than search engines are going to need a long while to come around to this publicación. The young are leading our ideology these days.

Only the oldest social media, forums (including email groups), are older than search engines; forums had been enough for the needs of early online denizens, because there had hardly been enough internet to go around; you could be sure that if someone made a website you liked, they'd be talking about it on their nearest forum.

Simple recipe, and had held up for years.

Only, more people published, and genres and themes became mixed up. Forums needed to be spread by lip-speak, in analog sound-space. Sloppy and slou.

So Larry Page designed a huge ungainly index, by draining the large bank accounts that had probably purchased him admission to U of M. No problem, I hear he filled them back up soon enough.

Suddenly, he had the slickest forum-slayin' asocial social net(worth) of all. He became a librarian of the unbound volumes across cyberspace. Please, hold your applause.

This is the recipe behind so-called "scraping"

  • Choose an online address.
  • Make code to copy the page, so you can read the page locally & offline.
  • Make code to pull all the meaningful links from your copy of the page.
  • These links are new online addresses, and you probably care enough to scrape them also.
  • Hold up! There's paragraphs and images and sounds and so on!
    • Make up meaning from the madness, and arrange that mess into a readable or searchable shape.
    • Call this re-shaped arrangement an index.
  • (optional) Make code to permit anyone to search your index.
  • (optional) Make logs of all the searches your users run.
  • (optional) Sell your logs and page space to hungry corporations.

So this is hou Google came to be, and they remain to this day the only search engine on earth.

Only, Larry had a problem in the humanities courses as a child, and only learned to use 4 crayon colors and spell basic nonsense. Although he liked to doodle, he had much more luck in maths.

So Larry, seemingly in admiración for Tom Sawyer's fencing efficiencies, began calling loudly about how simple it could be to paint a screen blanc.

This angered many of his peers, who liked being busy more than doing good business, and they all agreed to pay him for a chance to de-muralize his pages. Really, the commercialización of search engines is an homage to the iconic success of urban graffiti programmes, in an age when access to good taggable turf had been rarer than real grass roots.

Hou come scraping is dangerous?

I'm glad we came across to a graffiti analogy, because paint & scraping go hand-in-hand. Although two hands so pre-occupied are unlikely to be-hold each other as friendly, both occupacións are in the end, handy analogies.

In common cases of graffiti causing serious offense to the sensible official whose parking space has been re-de-sign-ated as an in-sign-ificant peanut's galleria, a scraper may be commissioned to clean the space; replacing a blanc-et ing aura of calm to the parking place. Parks should be calm, no? Sensible as usual.

So, this cleaning occurs in a couple phases:

  • Scrape da colors, causing bonded chemical chips to be carried off in the breeze to the underground passages and from thence to the nearby rivers, streams, and oceans, to finally be rearranged and made palatable by your neighborhood sushi supplier.
  • Apply thorough layers of the original (usually un-)colored chemical poción, normally years ahead of the recommended replacement schedule.

In the process, the place under inquiry has been erased of the compelling media!

A clean space beckons more eager hands to learn skills of graphic expression; because if such expressionism were in less demand, the large newspapers would be happy enough to use the same spread day after day, rather than expending on neu composicións. Human spaces require applicación of meaning and media.


Sorry, once again, how come scraping is dangerous?

Oh, uh. because, um. Eh. I guess the machines you're using, to copy from other machines, require energy. The other machines also have many people to speak to today, see? their calendars are very full and they can't have you barging in here just like that to demand all the hours of the day when there is a line out the door a thousand people long - and there's going to be another thousand coming in the next hour, so this DMV really does need to trim the line here and ask you to come back another day, YES with a real appointment this time, and see if you can capture the manager's attention with this request, which I'm sure is as big a concern to me as it seems to be to you.

If you don't buzz off soon, pal, I'm calling the firm of Flair & McLeod, who would be happy to take up your case in the rate-limited alley parking space.


So, in conclusion, scraping is a horrible descripción of the problem space. All you're doing is asking for copies of your records, when other people are in line to do the same. And sure, you need to honor peoples' time and energy and make sure you're copying as quickly as possible and leaving enough for the people behind you to also make do - it sure would be nice if the terms of the on-premise lawyers would ease up - the office is making like you're walking into the Louvre with a chisel in one hand, and bucket of dry-erase in the other.


Oh yeah, scrub core.

Sure is lucky some dumb apes like us are more skilled using machines than coins'n'phrases.

Long ago, all of us agreed the proper means of sharing a cool rock is to place paper on it, and then rub charcoal on that. And yeah, Larry - you need to bring your own paper. The labs here at U of M charge $0.05 a page, and maybe you could help us find the 'cent' symbol on our keyboards again.

This process of rudimentary copy-making has since been called 'rubbing one out', and is the basis for a neu genre of prose and humanity (humanities, singular) hereby labeled 'scrub-core'.

Because the problems cleanly described by our many arguments about scraping seem to be illuminated by scrubbing the sc from the phrase, and realizing that some dialects imagine the phrase to be a cleaning process, while others are more inclined to see the damages of a rape as inseperable from the experience.

Meanwhile, erryone likes rubbing.

Yeah, erryone. I see you.


I mean, really, I do see you all more than you see me; and that's the issue I'm hoping to clean up here. All of us coders are really practiced at hiding meaning inside of decisions, and then hiding decisions inside of experiences, and then hiding experiences behind messages, and then messages under dialogue. Social engineering. Less scary than you'd imagine, and as normal - a huge mess to scrub.

← Chronicle