Public Learning

project-centric learning for becoming a software engineer

Week 17: A Recap

On Post Mortems

I really like reading technical post mortem blog posts. Post mortem translates to “after death”, but in one of its current meanings it refers to the detailed analysis of a technical incident: be it a service outage, a data loss or an exploding rocket.
Post mortems, especially when they’re coming from the affected organizations themselves, provide a fascinating insight into how obscure bugs can sometimes cascade into major problems. One of the most famous incidents of the last years was probably Cloudbleed, in which an HTML parser used by Cloudflare (the web infrastructure behemoth) would accidentally leak memory content it shouldn’t be able to access. Cloudflare’s post mortem shows an engineering culture of transparency and is a fascinating read. As is the post-mortem of Gitlab’s data loss from three years ago, showing a dedication to not blaming a single engineer (even if he was technically responsible), but instead focusing on the underlying process that allowed such an error to happen in the first place.
There is much to learn from these reports about companies and the difficulties of engineering at large scale (and if you’re as fascinated by them as I am, you can find a list of interesting post mortems here).

Now why am I writing all this? Well, I wouldn’t call it a post mortem, but i’m going to write about how I misjudged my approach to the 200 OK frontend and have essentially wasted about two weeks of work. I am going to go into some detail as to how it happened, mostly to reinforce the intended personal learning effect. Fundamentals First was one of the guiding principles of Launch School‘s education, and I seem to - at times - seek shortcuts to circumvent the need for strong fundamentals. Not a good idea, as you might guess.

Hook, Whine, and Tinker

I have written at length about my approach to the frontend portion of 200 OK. For the longest time, it was a bothersome part of the whole project. I didn’t have much love for frontend work and just wanted a quick way to build the necessary dynamic user interface. That was also the reason to change my initial plan of building a React SPA: I didn’t want to have a long initial learning phase and dive into a topic that I wasn’t excited about. Just using vanilla JavaScript would have meant researching a proper way to handle complex UI state and DOM changes: doable (and potentially very interesting), but also time intensive.
That’s when I decided to leverage the power of Preact and its JSX-equivalent HTM: with a minimal footprint and no changes in my dev and deploy environments, I could hack together some React code and be done with it.

The first major decision was choosing the new React hooks syntax for writing components. I still think it’s a good idea to not rely on React’s old class-based syntax, because Hooks clearly are the future of React. But whatever minimal amount of React knowledge I had was in the class component system (through some experiments a few years ago and my horrible, half-baked Hacker News client built in January).
The difference between class components and functional components in React is actually a fundamental one, but that’s where I made my first mistake. I thought I could just build upon my knowledge of a class component’s lifecycle methods but write functional syntax instead. So I set out and used the two basic hooks, useState and useEffect just like I would use this.state and the lifecycle methods. I thought that while useEffect had a completely different syntax, I could still replicate things like componentDidMount and componentDidUpdate with it and build a hybrid of the few old React idioms I knew and the shiny new hook syntax.
This was the worst misjudgement. useEffect (and React Hooks in general) are fundamentally, conceptually different from lifecycle methods. And some of the intricacies of useEffect are not described very well. This ties into another problem that I have mentioned before: learning resources for hooks are sparse. And the tutorials that are there fall into one of two categories:

  1. Tutorials made for absolute beginners: those are the usual posts on Medium or that show a small, happy-path application that leverages hooks in a superficial way. Those tutorials stop before stuff gets really interesting, only demonstrating oversimplified use cases for methods like useEffect.
  2. Tutorials made for people very familiar with React class components. Even the official React docs spend a lot of time explaining different ways to achieve certain class syntax idioms with hooks. I’ll be honest, most of those analogies didn’t make much sense to me as my React class syntax knowledge was spotty at best.

So I was happily writing hook syntax as I had learned it from some random tutorials, and some of my intended functionality was easy enough that the code just worked. The dashboard and the request/response debugging tool were pretty basic, with limited amounts of internal state and side effects. At that point, though, I had already noticed another flaw in my approach (that I have also mentioned in a blog post before): the lack of debugging possibilities. My minified package that included Preact and HTM did not work with the debugging library, making the official Preact Developer Tools unavailable to me. I knew those from their React equivalent: it’s a nifty way to inspect your internal component’s state and props and see changes propagating through that component tree. None of that would help me now, so I had to rely on console.loging my way through any visible errors. At one point I wrote a small mixin that would log a component’s state variables and props (plus timestamps), but having all this information cluttered inside the browser’s console window was more confusing than helpful.
Yet, things worked for some time. Until I started work on the biggest frontend feature: the route customization.

Programming By Coincidence

The route customization feature is still simple enough: you see a list of all your custom routes and select one (or create a new route). Then you specify an endpoint (like /a/custom/route) and pick which request methods that route should respond to (GET, POST, PUT or DELETE or any combination thereof). For each method, you can specify some JSON with which the response is populated. You click save, it gets sent to the internal API and processed, then you test it and are happy! When I was creating an initial sketch for the UI, it seemed straightforward: a text input, four textareas with their respective checkboxes (to toggle that request method type on or off) and a submit button, all placed next to a list of existing custom routes.
So I started hacking away, but soon stumbled upon some small problems when trying to sync the state of those textareas across multiple components, especially when toggling those textareas on and off in the meantime. I did not yet see that I had some fundamental misconceptions about the inner workings of React hooks, so I hacked and slashed at my code until it was working, at least barely.
I even thought I was almost done, completely ignoring two warning signs: the first one was that I could constantly see a few more component renders than I anticipated. I ignored that. There was no visible jankiness, so who cares if the component sometimes renders twice in quick succession? And I had a persistent bug with the toggling of those individual textarea elements. Sometimes, they would seemingly toggle on strange events not related to their code. Little did I knew that there was an absolutely fatal misuse of useEffect right under my nose that could explain all the bugs and unnecessary rerenders.

useEffect, at its core, is simply a hook for any side effects and it exists to synchronize those effects with Reacts data flow. A perfect example is fetching data from an API: once a component is rendered for the first time, you set the loading state to true and issue a request to some URL. Once the response is received, you set the loading state to false and either populate a state error variable (if the response is negative) or a data object (if it was successful). All this gets wrapped into a useEffect function. That way, you can inject external data into the flow of React props and state.
Besides receiving that logic inside of a function, useEffect takes a second argument: a dependency array. And if you don’t know much about React, this is confusing as hell. Basically, every reference inside that array gets diffed to its previous state for each invocation of the function component. If it has changed, the effect gets invoked as well.
That’s how it is possible to try and achieve what componentDidMount did in old class-based components: by using useEffect(() => { // some effect code ... }, []), the empty array of dependencies will never change and the effect is only executed once (after the initial render, just like with componentDidMount).

But there is so much more to it than that. A few days ago I read through Dan Abramov’s blog post on useEffect (and it’s more of a mini book than a blog post with its recommended reading time of 45 minutes. It took me about three hours to completely work through everything written there). It was an insightful read, and has cleared up many confusing aspects of both useEffect usage and how hooks in general work. This and a few other articles on Abramov’s blog should be linked prominently on the official React docs, because they clear up many confusing aspects. The thing with React hooks (and I have come to really like their way of working) is that doing things wrong does not throw any errors. You can shoot yourself in the foot and still write perfectly valid syntax that at first glance even seems to work. That’s what happened to me, until some of the nasty bugs gnawed their way through the thin veil of working code.

Failing Forward

So last Wednesday afternoon, after my toggle checkboxes went haywire, I decided to reset my approach. I went back to reading about the React fundamentals (with Abramov’s writing clearly among the best resources I was able to find). Today I also started slowly rewriting every piece of functionality for 200 OK in an unstyled testing environment, using create-react-app. That way, I can use the React Developer Tools to clearly confirm that stuff is working as intended. Plus, I can leverage the support from many ESLint rules specifically created for hook-based React syntax to not accidentally use another anti-pattern.
Once the basic functionality is working, I am going to rewrite the JSX parts to HTM and use it with my small Preact package. That way, I can continue without complicated build steps and an enforced SPA approach, but develop with all the comforts that a modern React tool chain provides.
I could have done this two weeks ago, but sometimes it takes a stark reminder of why hack and slash coding is almost always a bad idea. And that frontend development is nothing that can be taken lightly. Well, learning in public does also mean failing in public. But it’s only in vain if you don’t learn from it.