This model has five levels of maturity and capability. Successful software can be made at any level, but it's more likely with a more mature software process. What I personally add to this is the concept that the level a software project should have is related to it's actual maturity. In other words, one is not seeking maximum maturity day on, that's developed iteratively. Less mature levels are appropriate for less mature code. We apply our full set of practices incrementally, over time.

The levels are as follows:

1. Initial - Otherwise known as heroic, where problems are daringly resolved, hopefully, through long hour antics and fire fighting incidents. A mature system should not be at level one, but a research or demonstration project may be.
2. Repeatable - The programmer's practices, such as installing software or giving a demonstration, are repeatable. This implies some notes and instructions, especially assuming repeatability by a team. However, some things are still only in the brains of the engineers working on the system. Some practices are ad hoc, the way a particular team naturally interacts, and will not translate forward easily as the team grows or transitions.
3. Defined - The repeatable processes are fully defined. In this day and age this definition is used to automate repeatable processes as through CI/CD, but also processes like sprints, right-sizing them, team size, creation, and breadth.
4. Capable - At this level there are metrics for the repeatability of all processes, from technical things like system deployment in the cloud, point capacity per sprint, and any other sprint metrics. This allows the team to accurately say how long a particular project will take and accurately schedule it in calendar time.
5. Efficient - All systems are continually refined according to metrics. These metrics will also be continually refined in terms of end results, like the popularity of the system with users, or costs. The metrics themselves thus become a focus of refinement as they now can drive the rest of the development process, but cannot be relied on forever. Over time the limitations of some metrics become clear, or the terrain changes requiring different metrics.

When starting a project, level 1 and 2 can come more or less simultaneously by documenting things, always... in other words by keeping notes and publishing them.  Comments in code can be the first notes, then that can be moved to READMEs, then also to wiki's and other document repos. When installing a system, always make notes, even the most crappy notes, so that next time you can read them. Seeing what they lack creates a better next draft. After a few times the instructions for the repeated activity will be complete. If the process is inherently complex, making the notes and documentation complex, this indicates that you should change the system to make it easier to document.

This initial level is appropriate to proof of concept, prototypes, initial development for libraries and tools, done by a single person. Level two projects emerge as projects are shared with other developers, the notes published, and the system can be reliably deployed and developed.

Level 3 should exist no later than the beginning of a team development effort, especially since we have tools such as kanban issue systems, sprint workflow tools, and the like, and what such tools provide is specifically level 3 definition. They are places to execute the defined actions, guiding all improvements and problem solving effort.

The fact is, a functional application, such as an internal tool or small site, can operate sustainably at level 3, but will falter when a major problem arises, which it eventually will. But some organizations will prefer to wait until that time, and then do a spike, and rebuild their system, and that effort can succeed at level 3 b/c of the freedom to just write new code and abandon old.

Serious software and enterprise software (even smaller scale) must operate at Level 4. Without the measuring, when the "major problem" arises mission critical software failure is likely, and at scale this cannot be solved with a spike in a reasonable time period, and the system is too complex to rewrite greenfield, and there would be no way to measure the results of the new effort in comparison to the last. Level 4 must be in place long before its needed.

If at Level 4, you should absolutely begin to act at Level 5. The metrics are there for you to have a basis against which to improve, or measure degradation as the software increases in capability. Ultimately there is little reason to have metrics unless you use them, and if you don't improve the metrics themselves, your gaze is fixed and isn't taking into account the changing environment which leads to the major problems we are trying to avoid.

Thanks for reading.

In practice, each system of practices in a workflow will obviously have its strengths and weaknesses. Only systems with well known catastrophic weaknesses, such as "waterfall", are thought of as invalid. Centralized designs catastrophically fail at scaling. However, the flaws of waterfall and centralization can be mitigated, so removing some of the practices and creating new practices that address those weaknesses is never off the table. Some special projects and some particular cases may lead us to seek some of their practices for a particular project.

Let me point out specifically what the reader may have already noticed, I am mixing engineering management issues with engineering architecture issues. This is intentional, because the two are intimately related. The engineering management practices used to develop software leaves its mark inside the software, and leads to many of the characteristics, laid down in sedimentary fashion in the source code. Nevertheless, as I continue, I will be focusing more and more on the analytic engineering abstractions used to understand particular software systems, leaving you to relate them back to workflow design.

I analyze all software systems using a set of abstractions into which I decompose them. Another truism, we all have no choice but to do this, but what is the set of abstractions? This truism applies to any existing systems and subsystems, be they libraries, processes, classes, or executables, and to any system I will build. I do something similar with workflow methodologies. For example, I decompose a team's particular use of Jira, or other work tracking software, into my own project tracking methods.  I map it into my system, but I don't change it. My own system is just a set of elements to which I can break down the other system, like analyzing water and finding it's hydrogen and oxygen, related by covalent bonds.

I recognize, and the reader should consider, that the system being decomposed to elements will have richness that is not obviously represented directly in the decomposition.  The behavior of water is not obviousl from the behaviors of Hydrogen and Oxygen on their own. Their relationship transforms their behavior into a very different observable behavior when connected in a system. The decomposition is just an additional way of understanding it, translating it, into more fundamental laws of computation. They allow doing local work in the system.

My fundamental system is un-opinionated, like organic chemistry, allowing all possible system behavior, but it is very reductive.  A particular system always is opinionated, it is specifically itself, and makes decisions about which ingredients to include and what relationships to coax them into. A cake and a jam can both be understood in terms of organic chemistry and seen as particular implementations of it. If an organic chemist encounters the most amazing food they've ever had, having never encountered anything like it, they can be amazed how such a thing is possible. When they take that food to their lab and decompose it, they will nevertheless discover the basic elements of organic chemistry or it would not be edible. They will discover the enzymes, emulsions, and particular chemicals. This increases their understanding of the particular system that surprised them with new systematic behavior.

You can describe baking cake in terms of chemistry and emulsions; the "opinionated" part is what makes the result a cake instead of an industrial solvent. Decomposing a problem domain into my own model helps explains the problem domain for me, clarifying any inherent complexities. In my model there are general purpose software artifacts that can be applied at this elemental level which at the higher level may seem opaque or unsolvable due to assumptions upon which the system relies.

In workflow, I maintain my model for what's going on in the software in parallel with any workflow system like Jira. Jira and the like are communication software for having a conversation about workflow. I'm certain good managers and engineers maintain their own models of project status throughout a project. If they didn't, they wouldn't be able to improve the workflow they implemented in Jira, or define it in the first place.

For the programmer or logician of any sort, abstractions are like tools in a workshop. They are used to break down ideas into parts, or combine them into systems. There are an infinite number of ways to equip a workshop, but there are patterns. The spectrum of tools you'll likely need or find helpful for a type of task bears relationships to all possible tools. A wood shop for making furniture will bear many similarities to other furniture workshops. It will even bear noticeable relationships to a workshop for making doll house furniture. Woodworking shops bear some similarities to a metal machine shop. Japanese gardening tools and European gardening tools cover the same spectrum of tasks, some are interchangeable with each other, but some are special.

The abstractions below are the primary tools of my workshop. Of course I have a lot of more specialized tools, including fantastic software created by thousands of fellow programmers. But all these other tools are decomposable into these principles and can be built from them. I think of all particular or specialized tools (i.e. opinionated software) in terms of how it can be decomposed into the following elements.

Unit of Computation and Flow of Control

The unit of computation is the function.

To execute the function, we hand it the flow of control. That of course gives the function control, and the flow of control flows downward in the function, and when sub-functions are called, deeper in the stack. The currently running function is the control, to the function it calls, and its caller is its controller. Having said this, in practice some routines specialize in controlling their called functions, and are called controllers in the design. Other routines specialize in performing the system logic (i.e. data reduction and business logic) , and these are called workers and services.

The flow within the function could be entirely sequential, it could branch, it can loop and so on, skipping around the source code. Nevertheless this is all powered by a downward flow through the source code. The body of the function is executed from top to bottom, from caller into to callee, deeper down through the callee, and eventually back to the caller to move downward again. Embedded in the flow of control in the body of some of the functions are the computations. The computations are the explicit data transformations that take place. Functions always transform data. To return nothing is to transform your input into nothing.

Lest you think this is a philosophy purely from functional programming, no, it's merely noting that other abstractions, like "classes", can also be thought of as made of functions. I am not a functional programming purist, merely a functional realist. Object oriented class declarations declare a set of functions. It provides syntactic sugar for easily bundling functions together in a tiny library, and sharing state. The shared state can be freely and multiply instantiated, allowing multiple alternates states handled simultaneously, and provides a way to pass this library as a single reference, supporting things like injection of complex dependencies.  That's super useful, and is a set of functions coupled together. Often, in a regular old standard function library the library keeps state, the pattern does not require OO syntax, it's just a lot easier and better looking. A class is just a useful syntactic sugar from the compiler to make working with this miniature library, the object instance, easier to manipulate and easier to write.

Decomposing things into functions.

A function always owns exactly one flow of control, which to the function is like its flow of consciousness. The flow is always at one particular point of the source code of the function at any given moment in time. That is analagous to what the function is thinking at a particular moment. It will share that flow of control with other functions, no doubt, the other functions it calls. Even the lowest level functions still are made of function calls, system calls, or inside the kernel, device driver calls. Even though at some point a function turns into voltages and a circuit,  from the programmers point of view it's functions all the way down.

A process is always started by calling a single function. A thread is created by calling a function in a way that gives it a new flow of control of its own. As the initial callee, it's now "controller" for everything that it calls, and everything called by the functions it calls, everything beneath it in the stack. In principle, all these subfunctions are "under it's control". However, it has temporarily given up control, giving it to the callee. Though it will eventually return to it's caller, for now, it decides when to return or just loop forever, and it's caller is dormant. It can do things to never return to the controller, such as call process exit, or throw an exception which may skip the caller on the way up the stack. Note: in most  if not all languages and systems the caller always has ways to intercept such attempts to never return control, such as catching the exceptions or similarly intercepting a process exit event.

All programmers will know what I'm saying, but probably have not narrated it this way to themselves. I hope you understand my perspective and can see how it fits  with or against your own way of decomposing programming. For example, you may not want to consider the caller technically "dormant" b/c maybe that's a poor synonym for "paused". Perhaps you see functions as conceptually properly objects, so everything is objects to you is a lower level decomposition. I'm eager to hear such feedback. A large part of my dedication to my own decomposition techniques is the pragmatic value it's generated for me in my work, allowing me to understand anything throughout the stack of tools in distributed systems.

When a message is  sent to another server, the sender is calling the receiving routine that will process that message. There is always a routine that accepts that message after all. Generally it will be a routine that knows how to dispatch the message further. Obviously there may be many functions handling many messages. Somewhere after delivery is a function that has essentially been called by the caller in the other process. The caller and callee still share understanding of the input data schema, but what they don't share, is a common flow of control. The sender does not have to be paused/dormant/blocked.

Loose vs Tight Coupling

Functions have callers. The relationship between the caller and the function is a coupling. The coupling can be loose or tight. A loose coupling is easier to deal with. There is give and take in it. In some cases a tight coupling is needed, where two systems each have orthogonal purpose, but still cooperate closely to implement the features of a complete system. Tight coupling can be more computationally efficient, use-of-memory efficient, that is, optimized. It's fair to say when possible we prefer loose couplings, for they can more easily be made to degrade gracefully.

There are two factors contributing to the tightness of the coupling, shared data schema, and shared flow of control.

Functions calling each other in a single thread are tightly coupled in flow of control. The advantage of this coupling is they know they are not fighting each other to access data, and so no mutual exclusion systems are needed. Things run efficiently with zero race condition collision between the functions. Also, it's unavoidable that many functions are run with the same control flow and from the same scope, as functions are made of function calls.

A function sending a message to a function in another thread is more loosely coupled to it. Because of the intermediary messaging system, little is known about the other function's flow of control. It may be in another thread or even another process. It could even be executed later in the same thread such as with async functions. It may be broken up into a number of messages that fan out and are handled by multiple functions across multiple processes, coupling the caller indirectly to multiple callees simultaneously.

The caller and callee, wherever is runs, are still directly coupled by the shared data schema of the data passed between them. Both must understand the data schema with some degree of compatibility. If they both have to have the exact same understanding about the whole body of data passed, that would be more tightly coupled. If each only had to understand small parts of the data passed, the functions are more loosely coupled. An example of the partial schema would be a caller that passes a whole object, such as a user object, to a function that only understands one part of the schema, such as where in the shema the user's name located. Thus the callee doesn't care about the rest of the schema, just so long as the user's name is in the expected location.

A good example of the tightest possible data schema would be two functions communicating by a unique binary protocol. In cases like this a list of long integers stored as bytes may not be compatible when the same code is run on different processors with different byte ordering.

In general, the more one function has to know about its caller's or callee's data schema the more tightly they are coupled by data schema. For example, if a calling function must call specific other functions to prepare the data to pass to function G, that's a tighter coupling than if it can just call G without preparation, given the data it started with. If function A called function B and B calls C with the same input A gave it. This couples A to C in that case, but ironically, A and C might be only very loosely coupled to B! If B's job is passing data from A to C, then all it has to know is enough to pass the data, which even in the binary data example, means a pointer and length.

You can empirically test how tightly coupled functions are, in a real system, by trying to change one of the functions, and seeing how many other functions have to change with it. Therefore, a general concept of coupling is to ask, "if I change function G, is it likely I will have to change functions that call G?" and "if I change function G, is it likely I will have to change functions that G calls?". The more function that have to be changed along with G, the more tightly coupled the functions are.

Another way of looking at the same issues is to ask "how easy is it to refactor this set of functions?" To the first few orders of accuracy, the looser the coupling, the easier it is to refactor the whole set. However, taken to extreme, the looser coupling will not actually make refactoring easier. This is because the loosest coupling evolves toward highly configurable systems, which are useful in reducing direct coupling. This can become overgeneralization which actually introduces configuration complexity which might in fact make refactoring difficult due to too much indirection. This simply reminds us that in spite of "preferring" loose coupling, in fact, what is desired is optimal coupling. The preference for "loose coupling" comes historically from the era of mostly-only tight coupling, which was inevitable before general data encodings were available.

The ideal design suggested by this interpretation is one in which functions perform well isolated responsibilities, and thus don't have change because another function changes. They are ideally plugged into the system in such a way that changes that don't change their responsibility don't affect them.  If adding a new feature, only callers that want to use the new feature have to change. If changing how a feature is accomplished (or fixing a bug), then callers don't have to change how they call the function.  

Optimized systems often use tight coupling to solve complex problems and present loose couplings to the rest of the system (e.g. to other processes in a distributed system).  A point I'm trying to make clear is that there are potential reasons to do any conceivable type of possible coupling, via control flow and data schema planning, given a problem domain where it's advantageous. But at the same time, there is a general case for the vast majority of software in which an array of mostly loose couplings is used between functions regardless of if they are subroutines, are in the same thread, in another thread, or in another process, on another machine.

Every software system is a set of choices made about functions and how they are coupled by data and flow of control sharing. For me all the answers to questions about the swarm of needed functions and their optimal coupling come from the problem domain itself. Certain patterns are called for, and the problem domain is what calls for them.

The subtitle on the cover of Martin Fowler's book Refactoring, Second Edition 2019, originally published in 1999, implies that Refactoring is about "improving the design of existing code". Further inside, in the preface, he says refactoring is "the process of changing a software system in a way that does not alter the external behavior of the code yet improves it's internal structure". A couple paragraphs further on he provides the motivation, "with refactoring we can take a bad, even chaotic, design and rework it into well structured code". He argues that as we build a system, we simultaneously learn how to improve its design. We can alter the code to the shape of the improved design using a catalog of reusable refactoring patterns. Each refactoring pattern is a code change pattern which is "almost simplistic". The point being that these are safe steps. For example, we move the body of a case statement into a function, and call the function in the case statement. Or we move code that is part of a method into its own method, or we move code snippets up or down the stack, addjusting isolation of responsibility.

From my perspective there is an extra issue however. My definition of "refactoring" is any alteration of existing code. If a subsystem of any type (be it a complete server or just a library) is working correctly, and it doesn't require alterations, then refactoring is not an issue, it does not require alteration. Making things pretty or even less chaotic for no reason is not the goal of refactoring. But if code has to change, you are refactoring as well as adding code.

Fowler mentions "not altering external behavior"... then why would we alter the internal mechanism? The answer is so that we can subsequently alter behavior. The better design means more flexibility, prepared for future changes which are blocked by the current design. A better design means better prepared for refactoring, better prepared to cooperate with future alterations required not by the code base, but by the users of the code base. So it's best to remember that it is about altering external behavior, ultimately, especially when arguing for application of the principle.

It's quite possible the reason it's hard to argue with the business side of software development for the time to refactor is because we have not made clear the external behavior benefits a software system will display when continually refactored in key areas. Software made with refactored growth gets more powerful as it ages, insteal of becoming lumbersome and decaying.

Change is continuous in software. Any exceptions are not of concern, all the systems we work on are undergoing change. Software developers are by defintion dealing with the systems that are being changed. A reason to "refactor" is that if you do not consciously refactor, the code will naturally deteriorate, from the law of entropy as applied to information, and bit rot will result. Change is unavoidable, but navigating change with refactoring plans allows us to direct that change so it's not just wandering. If you are touching code, you are changing it, and you are affecting systems that rely on it, and you should keep the project refactoring plan in mind. Thus you should always chose to
incrementally improve that code at an opportune moment, which is always the moment you find yourself changing it.

The lean startup concepts of first, building an mvp, and then, working sprints with scrum, doing kanban, or some other agile methodology, is essentially equivalent to thinking of software development as constant refactoring. The waterfall method was based on the attempt to consider all factors up front, prior to implementation, which is impossible. Thus initial design creates the initial factors, which are subsequently refactored.

Even so called "greenfield" work includes refactoring, first there will be some proof of concept code, and a pre-mvp core, and subseqently those will be refactored to meed the needs of the mvp. Fowler's definition including not changing external behavior still applies, it's just important to realize that pure refactoring is followed by making use of the new abilities of the system, and so practically both things will often be thought of as the same issue. In this case, it's better to think of the change as a refactoring that adds behavior, than and issue that requires changes to support. Refactoring means the design is better, but "chage to support" something new is where hacks come from. Refactoring requires thinking about not only the current change, but the next changes after that.

• refactoring as a development philosophy
  • Good design should make it safe to refactor.
  • The purpose of testing is to make it safer to refactor.
  • Don't refactor accidentally, do it intentionally.
  • Over time, even short periods, there are compound time savings.
  • Over time, even medium periods, there are exponential configurability
    and system ability.

Healthy Code

Personally, I don't like the term "code smell", it seems mean hearted to code that must be of value somehow or else it would be thrown out. Also, "smell" implies that you and sense something bad but don't know exactly where it's coming from, or that it's rotten. Really, if you see a bad idiom at play, or a dangerous one, or a misused one, like a case statement a thousand lines long, it's a bad idiom. It's analagous to network or power cables that are tangled and countless. It's not a smell, it's a mess. Instead, I like to say that my favorite type of software is software that works, and this is a problem and an opportunity.

When you have a bunch of professionals complaining about how terrible and chaotic a system is, it can only be because some business is using it for some vital business process, so it's indespensible. It has some mission critical purpose that in itself proves there is something worthwhile in there. Software engineers should respect that as an emperical fact. That aslo means that with refactoring it can be conserved and eventually embeded, its flaws removed, and coexist as a good citizen within a well designed software system.

Target Designs

• refactoring is not a design
  • you need a target design
  • you use "backward compatibility" techniques to create your new design
    such that at any point in time it coexists with existing code
  • coexistence ensures legacy features are not refactored away as
    happens with a "rewrite"
  • often, refactoring patterns are reversible b/c sometimes it's the
    specific target design that defines which direction is "better"

To make progress via refactoring you have to have an idea of where you want to go. Examples of refactoring in Fowler's book often show both directions of a refactoring step, his simplistic steps are often bi-directional code transformation primatives.

Below are less than simplistic steps from my own experience.

Refactoring No Brainers

To talk about refactoring that is specific to the target design of your code base would require a whole reference project, like provided in Fowler's book. For the purposes of this blog I will merely mention some from my own experience that are patterns which immediately call for refactoring. I would like to note that I respect disagreement on any of these just as I disagree with multiple of Fowler's. They are mostly not machine executable transformations however, because they require the goals and motivation of the engineer. To collect some code in a function together, and move it to another function is simple enough to be carefully done by a human, but a machine lacks the semantic power to choose which code needs to be collected, what to call the new function, or where to put it.

You have cut an paste code throughout your code

You need to make the copied code into a shared function.

1. Review the cut and paste code fragment where-ever you know it's used. You can use variable names to grep for examples. Compare the fragments in detail across a variety of examples to get an idea of how conformant they are to each other, to the most general idea in the code. Often pasted code will have been modified or intersperced with other code for reasons that might be subtle. Some versions will have attempted to be general, but proved not general enough and someone made a special version for themselves. This version may be more general, or more specific with the scars of attempted generality. It's always better to make the copied code more general, then used for the new special purpose, because then it might be backported to the previous page.
2. To make your general case function, choose the most general case you find and write the function by modifying it. If it's an extreme mess, and none are general enough, create your own generalization and write it from scratch with the (generalized) characteristics of the complete set of similar code.
3. With your starter code, in a new function, compare to other copies, generalizing to accomodate the second version.
4. Add arguments as needed to pass variable that had been available in scope of the other function. It's tempting to combine arguments when you find the copy has a similar variable. Don't, fix that later. If the starter function had goodDate and the other has niceDate and they seem the same, wait, they are likely not handled exactly the same, and you don't know the dependencies on that difference. Here is where we are thinking, "don't change behavior". Combining the two variables in the same function, each with their own handling will not affect behavior. Normalizing the use might.
5. Repeat with other fragments, and when they are different, support the difference with an additional argument.
6. As you go replace each incorporated copy with the new generalized function. Confirm the software works and is passing tests, but also do targeted manual tests with small scripts to exercise the generalized function and ensure it's answers are the same as the original copy.
7. Look at the generalized argument list. It might be a mess, with overlapping arguments (e.g. "state" and "stateCapitalized" and "stateCode" and "stateId" and so on) and it's not clear which are interchangeable or merely overlap.
8. Find the generalizations for this data. For example, if you have a canonical "stateId", then the generalized function can take that, and look up the rest. If niceDate and goodDate are mostly similar, they can be replaced with displayDate serving the purpose of both.
9. If you want callers of the original copies to not have to change, and use their unique argument lists or names, you can create thin thunk functions that continue to accept those arguments and call the new generalized function. In fact, this is something you can just set up an the begining if you want to make it easy to swap functions around while you work, ensuring it's easy to move backward if the generalization fails.

Using named arguments for functions is very useful in this regard since ordering doesn't matter, allowing parts of the code to ignore changes to the arguments. This means when generalizing by adding new arguments that are needed for generalization won't interfere with the function signature or other code within the function.

You have a 1000 line function

1. Collect the lines of code within the function. All the lines that refer to the same variable should be bunched together. If the language supports it, hopefully, put the variable declaration at the top of these lines.
2. Move that code to a function, decide what it's responsibility is and name it appropriately.
3. Organize the new functions into subsystems of isolated responsibility.
4. As the function is collected this way, you will start to have functions that call the first round of code, and these go into a library of a higher layer relative to the first round of functions.
5. The perfect stopping point is when the function reads like an engineer explaining what the functions job is to another engineer.

You have a 1000 line nested switch statement

1. Create an abstraction for a command
2. Create an abstraction for a command executor
3. Create and abstraction for a command context
4. The end result is all cases encapuslated in command objects that accept command contexts
5. The end result is the switch repaces by a command executor call

The situation a giant case statement is taking advantage of is generall the global context. That should be placed in the context. The function will now get their inputs from the context. The commands will have names/ids that are stored in a command executor configuration. The switch statement is replaced by a command executor call, new commands can easily be added, and the commands can be executed by alternate executors, which is often useful.

Good design patterns and Refactoring

• in general good design is that which is easy to refactor
  • OOD encapsulates so that solutions can be replaced
  • OOD encapsulates so that multiple solutions can be retained and swapped in/out
  • refactoring is enabled by code comprehensibility
  • breaking things into units of responsibility makes things easy to refactor
  • testing makes it easy to refactor, if unintended consequences are detected (regression testing)
• modern methodology, Agile, Lean, Kanban, Scrum... are partly about refactoring or assume it
  • making an MVP is getting a starting point to subsequently refactor
  • Sprints are interpretting all work as time boxed refactoring
  • Kanban is delta-ticket tracking refactor patches
  • All agile sees development as iterative in nature, where you have a "continuous" development methodology, trying to be as granular.
• If you start with something running and end with something running, you refactored.

To really explain refactoring though I want to harken back to before it was a word, when we had alternate words for refactoring. For one, we didn't admit to refactoring or have a word for it. We 'worked' on code and sometimes if it became unworkable, we "rewrote" it, or "started over".

My introduction to the term, but not the concept, of "refactoring" was from the book Refactoring: Improving the Design of Existing Code, by Martin Fowler in 1999. I might have heard the term earlier, I think I had, but the book cemented it all together. I had loved object oriented design, not as a purist but because it allowed me to change code in and out. It was inherently like a plug in system, not a run-time one, but one for developers, to have classes and object instances working together to achieved desired behavior. The part of object oriented design I liked happened to make refactoring easier.

What I and others were calling iterative development, that now we call agile, or closely related, is all about assuming you are refactoring. You achieve a MVP, and after that, you run sprints refactoring. Even new features are interpreted as a refactoring b/c the system needs refactoring to accomodate the new feature, if it really is new.

Refactoring steps are doable chunks of change, it works at the start, if considered non-optimal or even buggy. It also works when your done, even if considered far from finished. Code that is easy to change is easy to test. Code that's easy to change is easy to understand. Granted, these are tautologies in that... if you don't make it easy to understand, it'll be hard to change, if you don't make it easy to test, you'll make it hard to change. The point is these things go together. Learning how to make code easy to change leads to how we make code easy to understand and easy to change, and the points all reinforce each other.

What's interesting about Martin Fowlers book is that it has many examples of code problems you often see, and how to refactor them. I find that in the decades since reading this book my entire  programming approach has been formed into refactoring techniques. In that time, even green field work has to work with old systems or have some sort of backward compatibility. Refactoring is key.

I am always refactoring. Even green-field is refactoring. I start with scripts to prove concepts, I refactor that into a script calling reusable useful libraries, then that into classes using those libraries and then into a command line tool that performs services, first, especially, that are useful in development, and then into servers and client software.

What's neat about refactoring is getting used to turning the kind of thing you run into that might haunt and afflict you, into what you want it to be. You can't do it right away, but just the knowledge reveals how to isolate your new code from the horrors of the code debt surrounding it. It's not a magic wand but it is possible to refactor code as you work so that your job working with it slowly gets better.