A shape keeps recurring across my projects: something exists, it is useful, and it will not tell you how it works. I wrote a keygen for Balsamiq Mockups as a research exercise — a C# study of how its license keys were derived, with nothing distributed. I built a scraper that pulls country statistics out of Wikipedia infoboxes, which encode the same fact in fifteen different ways. I built a serverless job that backs up my SoundCloud playlists, which meant prying a client ID out of a page because there was no documented way in. Three unrelated projects, one activity underneath: watch what the system does, form a theory, test it, write down what you learned.
We talk about reverse-engineering as though it belongs to security researchers and people cracking DRM. But if you integrate with anything you did not build, you are doing a version of it constantly. The docs are out of date. The error messages lie. The field marked required is optional and the optional one is load-bearing. Every integration is part archaeology, and the ones that go badly are usually the ones where someone assumed it would not be.
The method is not sophisticated. You change one input and watch what moves. You keep the network tab open and read every request the page makes, not just the one you were expecting. You save a response before and after some action and diff them. When an endpoint rejects you, you read the rejection as data — it has just handed you a rule you did not know. Most of the effort is narrowing: this field matters, that one is decorative, the order of these two calls is load-bearing and nobody wrote that down.
The part people skip is writing it down as you go. A theory you hold in your head decays between sessions; a theory in a text file, next to the request that proved it, survives. Half of my scrapers are really that file with code wrapped around it — a running list of "if it looks like this, it means that," accumulated one surprise at a time. When the source system shifts six months later, that file is the difference between an afternoon and a rewrite.
You also learn where to stop. You do not need a complete model of the system, only enough of one to finish the task in front of you and to notice when an assumption breaks. Chasing full understanding of something you will touch once is its own kind of waste.
Some of the work where this mattered most is work I can't lay out here — done for other people, under terms that keep the specifics private. The technique was the same as everything above; only the write-up is missing.
It is worth leaning into because it is the outsider's way in. You do not need permission, a partnership, or a support contract — you need patience and a tolerance for being wrong a few times before you are right. Once you are comfortable with it, a whole category of "we can't, they don't have an API" quietly turns into "give me an afternoon."
There is a line, and it is rarely blurry in practice: studying how something works — a licensed copy on your own machine, a response in your own browser — is not the same as breaking into a system you are not allowed to touch, or handing around the thing you built from what you learned. I stay on the near side of that. Everything above is about making permitted things legible, not making forbidden things reachable.
The foundation is laid; now it's time to build.