Public Learning

project-centric learning for becoming a software engineer

Week 5: A Recap

API Under A Microscope

While I felt that I had a sufficient grasp on the basic idea behind my project as it was laid out in my post last week, I was in for a few surprises this week. It actually turned out that I hadn’t considered quite a few key aspects. That was expected, but it made me hesitate in the first days of my project work. I basically went back to the drawing board, spending quite some time on actually planning out all the moving parts. I even drew a big, two-page flowchart to try and gain a better understanding of how a client request to my API is served.
As I was gaining an understanding, more questions arose. Most of them I just put into my short design document and a Trello board until I find a better way to organize my project work. In trying to figure out the answer to questions like the choice of a data store, I figured that I should probably not try and solve every problem in theory. Instead, I opted for an iterative approach and started to actually write code. At the core of my project stands the API backend server which is supposed to handle all API requests semi-intelligently. So my plan is to create a working prototype of that system, not yet caring about any authentication, persistent data storage or any of the other dozens of details.

Come to REST

Actually dealing with that crucial piece of the project made me realize that my goal of handling requests without having any API schema in place will most likely only work for a happy path use case. API design is obviously not a pure science and it doesn’t get better that the term RESTful gets thrown around even for APIs that are clearly not compliant to REST. I read through all of Roy Fielding’s original dissertation about REST and had a hard time connecting enough of it to what REST seems to describe nowadays. It has certainly nothing to do with HTTP verb methods - or HTTP in general. I actually have to confess that I am still not 100% sure what REST really means. Fielding emphasizes the hypermedia approach which is something that most API’s seem to outright ignore. The GitHub API actually implements that idea well, with each response sending links to relevant further actions: if you do a curl https://api.github.com you’ll get a response object with all actions available and that continues through all subsequent requests as well. This is honestly something I haven’t noticed a lot so far, but it’s essential to the idea of REST.
Another core aspect to an API is the resource as the central item and how each action is either used to retrieve the current state of a resource (like a list of the current users) or to transition that state (e.g. by deleting a user). The fact that we ubiquitously represent these actions through HTTP verbs is funnily never mentioned in Fielding’s description at all.
But that’s where the difficulties begin: a resource can be a set of entities (like a list of users at the endpoint /users) or the single authenticated user (like in the GitHub API, available at the endpoint /user). Both require a different set of actions, with one being a collection and the other a singleton. This is something that is hard to automatically handle with a set of rules like I intend to create. It gets even worse with a lot of bad practices often found in the API wilderness: /login and /logout are often used as endpoints that only allow a POST request to perform the appropriate action, but they are most definitely not resources, they instead represent a single action. There is no way this behavior can be handled with the same rules like a user resource would.
Long story short: while my automatic API route handling can facilitate its intended results, it will do so only when a strict resource/collection-based approach for endpoints is used. So the API will handle paths as if they follow the notation

/api/<resource>/<item-identifier>/<sub-resource>/<item-identifier> ...

Anything else would be guesswork, but it also limits useful behavior to this strict focus on resources. This is unintuitive to develop for, because how would one map the logout action to this ideal? As a PUT request to an authentication resource? A DELETE request to a session resource, coupled with an authentication token? Both ways would make for an awkward API interface.
So the feature to manually define responses for specific endpoints will no longer be just a stretch goal to aim for, but a central piece of my project. Without it, the API’s usefulness would be severely limited, though that means that the frontend portion to define those rules will become a critical piece with all its accompanying features (e.g. user and password management).
The Design of Web APIs Cover Another result of those considerations is that I’m reading through The Design of Web APIs (Manning, 2019) to gain a better understanding of the place API design is at right now. I figured that if I want to provide some semi-intelligent route handling, I should at least have a passable understanding of current best practices.

Getting My Hands Dirty

I somehow dreaded having to write the first line of actual code. I still dread it, but I got around that fear by starting to write a throwaway prototype for the actual API. There’s only so much planning possible, and getting my hands dirty helped me to identify lots of details I’ll have to think about eventually. I almost automatically added the CORS package, but is that really all there is to it? What about someone making an OPTIONS request to check for CORS availability beforehand? Writing code forces taking considerations at even the most basic level, and no amount of planning can give all those insights.
So, that is where I’m at right now. Unfortunately, a minor health issue substantially cut my productivity down to a bare minimum during the last four days, resulting in both a disappointing amount of time spent this week as well as a lack of progress on my prototype. But I’m still well on schedule and will hopefully continue with full force this week.

Summary

😷

Time spent this week: 22 hours