There is now a new, much different, edition of this essay which can be found here.

Narrative Engineering

Theory has been a colossal failure.  It’s arguably done worse than nothing; at best toothless against the relentless seismic activity of technological innovation and at worst a cronyistic abuse of the same massive power it’s so often propounded to speak truth to.

Reality is complicated, but there are times where the obvious is in front of us: academic financial theories have bankrupted those who dared to take them seriously in practice, macroeconomic forecasts have zero track record of successful prediction (and how could they, wouldn’t any obvious prediction just get priced in?), and let’s just not bother getting into the history of centralized economic planning.  Also, has psychoanalysis ever made anyone happy?  Asking for a friend.

Even biology doesn’t seem so hot.  Sure, modern medicine has done wonders in a lot of cases, but consider for a second that this may have been on account of moving away from theory.  Models of physiology like the balance of humors were tidy and neat, just like fancy equations about supply and demand or monetary velocity and inflation, but it goes without saying that the results were laughable and clearly wrong.  We might have more formal knowledge, but before you go assuming that textbook theories are responsible for the incremental victories we’ve carved out against pestilence, consider the current state of known metabolic pathways in the human body:

Yes, but computers are powerful, we have math, science!  Can’t all of these variables be untangled?  So glad you asked; consider the famous Three Body Problem posited by Henri Poincare:

Imagine a planet orbiting a star, nothing else in the picture, at all.  This is the whole universe.  Only having to take these two things into account, it’s possible to write up a mathematical equation that reliably describes their trajectory; in time, as they pull on each other, they will eventually settle on a predictable stable pattern.  Now, let’s say that planet has a moon.  Now everything goes to hell: this doesn’t just make the problem harder, it makes it literally impossible.  Poincare proved that there exists no equation to describe this situation, not least because there is no ultimately stable long-run pattern.  Eventually, even the tiniest difference in the initial positions, speeds, and masses of these celestial bodies will dramatically change the outcome, which means that even if you make a computer simulation of this, you can’t ever fully predict it because your inputs will have a finite number of digits and will have to be ever so slightly different than the actual initial conditions.

That’s a bit abstract, so let’s consider a different example.  Edward Lorenz, a meteorologist and mathematician, built a simulation of the weather with, you guessed it, three variables, all of which interacted with one another, which made every run different.  At one point, however, Lorenz decided he wanted to repeat a particular trajectory, so he typed in a set of numbers from the middle of one of his earlier runs only to find that he got a completely different trajectory.  Why?  It turned out that the data that got printed out only had three digits after each decimal point, but the computer was saving numbers with six decimal points.

What was the point of this digression?  Well, if that happens with three variables, what happens with the picture above?  At this point it seems like a miracle that anything works at all, but indeed machines work, in a given year there are tens of millions of flights but only around a dozen crash, diseases get cured, vast amounts of goods are reliably transported across the world at ever increasing volumes.  None of this was ultimately accomplished IKEA-style from textbook theories; it was the result of relentless tinkering and trial and error, the work of engineering.  Engineering is messy, it has no explicit rules and its approximations are sloppy to degrees that make theorists spit out their coffee.

It’s almost as if there’s something magical about the good old combination of intuition and trial-and-error, and by all means, it often can’t be beat.  But why?  Is there something supernatural about gut feelings and lived experience?  It can certainly be very powerful in some cases, such as one retold in Daniel Kahneman’s Thinking Fast and Slow:

The psychologist Gary Klein tells the story of a team of firefighters that entered a house in which the kitchen was on fire. Soon after they started hosing down the kitchen, the commander heard himself shout, “Let’s get out of here!” without realizing why. The floor collapsed almost immediately after the firefighters escaped. Only after the fact did the commander realize that the fire had been unusually quiet and that his ears had been unusually hot. Together, these impressions prompted what he called a “sixth sense of danger.” He had no idea what was wrong, but he knew something was wrong. It turned out that the heart of the fire had not been in the kitchen but in the basement beneath where the men had stood.

How did he do this without even consciously knowing what he saw?  After decades of experience, you see more and more common patterns, and the more familiar you are with them, the fewer cues you need to know what pattern you’re seeing.  The expertise of chess masters works quite similarly--they don’t need to do exhaustive calculations because they have the experience needed to simplify their reasoning.  One might even start to believe this works for something like picking stocks, but this isn’t the case.  Unlike chess, there are few if any truly stable patterns in the movements of stocks (and anyway, if there were, enough people would exploit them that the prices would change until they don’t exist anymore.)

Gut feelings simply have no special predictive power: just because your past experience corroborates a pattern, doesn’t mean that it exists.  You might have had some success at picking stocks, but when you zoom out and look at the numbers, that’s just a result of the fact that if enough chimpanzees bang at enough typewriters, eventually one of them will write a coherent sentence.  Just like the equations and simulations that attempt to predict the weather and the movements of planets, gut feelings used in the service of prediction are models, simplifications of reality that allow you to do more with less.

Models

The fundamental concept of information theory, a field of applied math developed by Claude Shannon to theorize about reliable communication, is compression.  To compress something is to describe it with less data.  Some compression is lossy, such as when you take music from a vinyl record and turn it into an mp3: the basic information is still there, but there is a loss of detail.  In cases where no details are truly lost, the compression is lossless, which we’ll focus on for now.  Consider the following string of binary digits:

111111111111111111

And this one:

1010101000010101

Which one of these would take more space to describe?  With the former, you can just say “repeat the number 1 18 times”, the latter?  Well, there might be some way to make the description shorter but you see what I’m getting at.  A mathematical theory called Kolmogorov complexity deals with exactly this subject by asking “what is the smallest computer program that I can write that would output this?”

Such a computer program constitutes a model.  A more general way of imagining compression without having to think about computers is the concept of a homomorphism: imagine any number of objects that are related to each other in certain ways.  To create a homomorphism, you create a new group consisting of some of (but not necessarily all of) those objects such that the objects in this new group still have the same relationships to one another that they did in the old group.  In more general terms, it consists of creating a smaller version of something that has the exact same structure.

A simple example of a homomorphism is the seconds hand on an analog clock: imagine one that ticks every second and one that makes ticks that are twice as big every two seconds.  The former makes more fine-grained movements, but they still both tell time accurately. More importantly, you can convert the movements of the more "fine-grained" clock to the movements the "coarser" clock without losing its structure. What do I mean by "structure"? In the case of clocks, you have to think about how any two movements of the second hand on a clock can be added together to give you a new movement: in the case of a single tick for every second, adding 10 ticks to 30 ticks is the equivalent of advancing 40 ticks, and adding 30 ticks to 50 ticks is the equivalent of advancing 20 ticks since the hand goes in a cycle of 60 seconds. I can do the same for the other more "coarse" clock, except in this case the hand goes in a cycle of 30 seconds, which changes the rules of addition, but only superficially, because if I take any two movements from the "coarse" clock and add them together before switching to the "fine" clock, it will be exactly the same as converting both movements to the measurements of the "fine" clock and then adding them together there.

This property is also known as congruence, which may be familiar from grade school geometry.  A model that is congruent with its subject is one that’s capable of anticipating its behavior. Why is this? Consider how you can tell how much time will pass with even a relatively "coarse" clock: it won't get out of sync with the more "fine" clock (yes I know technically clocks can fall out of sync anyway due to relativity but that has nothing to do with this!) Similarly, a model that treats a subject in a more "coarse" manner won't fall out of sync with what it's modelling so long as there's congruence, which means that in the diagram below, it doesn't matter which of the two paths you choose to get from the top-left corner to the bottom-right:

Here you can think of the downward arrow on the right as essentially a prediction about how the "actual" thing on the top-left will change. If the prediction were inaccurate, then taking the left-hand arrow (watching the object change) and then applying the bottom arrow (encoding the changed object) would give you a different result than first taking the top arrow (encoding the original object into your model) and then using the model on the encoding (applying the model's prediction.) If, on the other hand, your predictions reliably come true, then there is a congruence between your model and the thing being modeled where, just like the example of the clocks, you can convert the "finer" phenomena into the "coarser" phenomena and then compute and the results will be the same as working within the former and only in the end encoding it into the latter. In category theory, a diagram that achieves this lack of path-dependence is said to commute; and an encoding, such as the one shown above, that creates a diagram that commutes is called a functor.

Of course the problem is that a model can appear to be congruent until it doesn’t; like when Lorenz found out that the version of his simulation that only used three digits after the decimal point wasn’t congruent with the one that used six, or when the gambler suddenly “loses his touch”.

Falsification

So how do we get around this problem?  According to your high school teacher, we just need The Scientific Method™: find a variable to test, keep all other variables exactly the same, see what results you get from your experiments; who gives a shit about the theorizing when you can just keep testing things until you know what does what?  Well, it might be fun for that first variable, but let’s say you can’t get any meaningful information about it.  Time to move on to the second variable, and then the third, and so forth.  And let’s just say for the sake of argument you’re able to go through all the variables.  What if you still find nothing?  Well, better start checking specific combinations of variables, first in pairs, and then in triplets, each round taking exponentially longer than the last.  Let’s just suppose there are a mere ten variables, each with two values: true or false.  That’s literally over a thousand combinations to test; that might not sound so bad, but now consider twenty such variables: the total number of configurations is now over one million; all of this assuming the existence of some finite bucket of well-defined variables lying around that you can just stick your hand in.

So at some point you’re going to have to simplify it by proposing some underlying factor that isn’t superficially observable, and now that brings models back into it with all of their potential pitfalls.  Well, maybe we can’t know which ones are ultimately true, but it only takes one counter-example to prove a rule wrong, and that was Karl Popper’s idea of how progress is achieved.  A theory achieves a basic kind of plausibility when the experiment can be reproduced, but if it’s wrong it will inevitably be culled by some experiment that demonstrates a counterexample.  The idea is very intuitive, but it runs into difficulties in the real world: any number of errant things could have messed up the experiment and given you a false positive; maybe your lens is smudged or someone wrote down the wrong data.  To his credit, Popper accounted for this:

"non-reproducible single occurrences are of no significance to science. Thus a few stray basic statements contradicting a theory will hardly induce us to reject it as falsified. We shall take it as falsified only if we discover a reproducible effect which refutes the theory"

There’s just one issue: if a counter-example requires reproducibility in the same manner as a theory, then there’s no longer any fundamental asymmetry between proving and disproving a rule!  This daming logical flaw comes not so much from a wrong answer as it does from asking the wrong question: trial and error, and for that matter engineering, isn’t about getting predictions right or wrong, and it’s most definitely not about establishing a scientific claim as unequivocally True or False, it’s about doing what works, of which models, predictions, experiments, and falsifications constitute an invaluable part but are not the essence of it.  In scientific theory there are indeed the occasional cases of a theory proving more fundamentally correct than another, such as the discovery of the theory of relativity, which can account for everything Newton’s laws accounted for plus phenomena that Newton’s laws could not account for, but it would be misleading to say that those phenomena invalidated Newtonian mechanics.  Newtonian mechanics still proves to this day to be effectively true for whole lot of things, but it eventually reached the limits of what it could do to facilitate new discoveries.  

At this point it would be fair to suggest I’m playing word games, so let me put it another way: there is a difference between a model seeming like it works but then turning out to be bunk, and a model actually working but just not everywhere.  This isn’t just some offhand practical concern, it has fundamental implications about the very idea of validity: Godel’s Incompleteness Theorem tells us that any mathematical system we construct will inevitably have holes of its own making; questions that are the result of the constructed system but can’t be answered by any chain of logic that obeys the rules of the system.

This doesn’t just mean that there’s no such thing as a universally valid model, it also implies a process in which new models exist because of the need to fill in the holes left by previous models.  In mathematics, this is readily apparent: negative numbers are a direct consequence of being able to subtract numbers--without them, subtracting a larger number from a smaller number pulls the mathematician into a dead end because there’s no cogent result to work with.  This is the same reason imaginary numbers exist: the square root of a negative number might seem logically absurd, but creating a system that allows for them has led to major breakthroughs in physics by expanding the range of mathematical expressions available to the thinker.

Are these models being completed insofar as they bring us closer to some a-priori truth?  I’m not going to debate the nature of objectivity here because I don’t actually need to: adding negative numbers, imaginary numbers, and so forth allows the mathematician to do more.  Even in that supposedly ethereal realm there is no escaping the real world that we live and breathe in; and this is no less the case with science or any other production of knowledge.  Models factor out noise and simplify the process, but they are a middleman, not the engine, and for that matter not the end goal either.  The end goal always has been and always will be efficacy.

Praxis

So if models can narrow down the amount of combinations to test, the question remains of how to pick a model. Again, there's no bucket holding all the models that you can just stick your hand in, you have to construct a model, which means picking relevant attributes.  Of course, attributes themselves are not things that can merely be picked out of some bucket, they are defined relative to other concepts, which themselves are defined relative to other concepts, and so forth.  The positivists claimed that all this eventually settled on an absolute ground of “sense data”, from which everything else could be built up from self-evident foundations, but this too is completely smashed to bits by Godel’s proof that you can never have enough axioms to say everything.

Nonetheless, we somehow manage to narrow things down.  It’s not just science: in everyday life people are easily able to describe things, give instructions, coordinate, make things work.  How?  By tinkering.  Just like science utilizes models that work well enough, people, and for that matter organisms in general, manage to do things that simply work well enough for their purposes. There’s no guarantee of truth or complete reliability, but billions of years later on this planet, things chug along pretty impressively, even if not by any means perfectly.

This kind of filtering doesn’t have any “rigorous” way of defining and controlling for variables; it’s not a matter of prediction and correspondence, it’s about doing, and it’s so fundamental to the way we think that even something as basic as perception is not passive.  To take the example of vision: the ambient arrays of light that hit our eyes don’t on their own have enough information to tell us what we’re looking at, which is why M.C. Escher was able to create paintings that defied any coherent physics.  We need to move both our eyes and our bodies around to narrow down what it could be and find what’s invariant.^[1]

And because it’s about doing, it does not necessarily leave the object or environment in question untouched; there is no strict separation between subject and object.  Spiders have been shown to do just this in the way they spin and modulate their webs, which they often make more taut if they want to be alerted to smaller prey (the tradeoff being one of finding more food vs conserving energy by only making an effort when it’s a large meal).^[2]  What matters is that within the whole ensemble, a tractable pattern (or, since it’s something involving what we do and not just passively perceive, you could say it’s a tractable habit) emerges.  Positivists have attempted to drive this nebulous dance out of the land of knowledge, but to think this is possible is as absurd as thinking that our bodies can survive without microorganisms.

Fields like physics might have precise measurements, well-known natural laws described by powerful models, strict codes of protocol; but these a-priori rules didn’t just descend from the heavens, we were only able to get there by improvisationally seeking out pockets of order within the chaos and gradually refining these practices into increasingly rigid and universal structures.  The Enlightenment wasn’t a bunch of people all of a sudden waking up and saying “superstition bad” (“superstition” often just means “rule of thumb that I don’t like”), it was the culmination of a process that took the entire lifetime of civilization.  

Nor does further progress get made by mechanically extrapolating from existing formalisms: science is inextricably enfleshed in tinkering.  However many abstractions there are, scientists still have to do the experiments by interacting with the material: they operate instruments, pass down and receive folk wisdom, have conversations, and most illustrative of all, they understand where corners can (and must) be cut; which is why Popper was right to understand that an errant observation here or there can’t be considered an inherently damning thing.  All this is why despite all the torrential downpours of data that amount to nothing, a revolutionary theory often comes with just a little bit of data.  More simply isn’t better, data without abduction is as inert as a hammer without a hand.

Storytelling

So how do we get from seemingly random interactions with reality to stuff like high-tech enterprises involving thousands of people?  Simple answer: we try stuff and keep doing it if it works and then try more stuff with the help of the stuff that works, kind of like evolution.  But this on its own is only half the story: after all, just like there’s no magical bucket of models to pick from, there’s also no magical bucket of actions to pick from (besides, if the actions were all in a bucket how would you pick the action of picking from the bucket?)  

When we, or other animals, or any organisms for that matter, engage in this kind of seemingly random activity, what we’re really doing is playing.  We play with objects, we play with words, we play with other people, and yes I admit I walked right into that one.  Play can be something as simple as bouncing a ball against the floor or something as elaborate as a chess championship or a group of physicists studying quantum entanglement.  In some cases there might be a goal like getting a ball through a hoop, but there’s nothing essential to play involving goals, or for that matter direct control; for one thing, a game gets boring when it’s too predictable.  The desire for control of course might motivate a game, but this actually puts the cart before the horse: direct control, when possible, allows us to do more.  When playing, we’re seeking additional degrees of freedom: for example, one doesn’t need to specify goals in a video game because there are clearly some directions that allow you to do more things than others; even if the “goal” of SimCity were to run a city into the ground financially, people would be automatically drawn to the challenge of growing the city because you still have the option of running it into the ground in that scenario but not vice versa.

This might seem like a roundabout argument because I’m still talking about gaining control, but you do not necessarily need perfect or direct control over anything to achieve greater agency.  What is necessary is a conflict, a body^[3] of constraints against which our actions are defined.  A conflict, unlike a goal, doesn’t have a specific criteria: it can be resolved in many different ways, not necessarily always desirable ones.  The only criteria for a conflict to be resolved is that the potential for new avenues of action has been exhausted.

All action, whether out of playfulness or necessity is ultimately centered around conflict: we eat when we’re hungry, we fight or flee when we’re in danger, we do all kinds of things in the name of love, while boredom inexorably sprouts play like weeds growing through pavement cracks.  Conflict foisted upon us gives us a clear sense of what we have to do, while playing involves messing around with things in search of some new conflict; either way, conflict provides relevance for our subsequent actions.  It doesn’t tell us what to do, it creates a sense of why we might choose to do one thing or another; a vector^[2] that gives us a sense of which way is forward.

Our actions, when crystallized around such a fulcrum, become a narrative: a progression of actions related through a shared conflict.  In fact, the relationships between such actions are what give them any meaning, in the most fundamental sense of the term, at all: we can only define ideas in relation to other ideas, any such idea existing in the first place only because it’s useful to us in some way; that is, it helps us resolve some conflict.  

Of course, many stories are speculative interpretations of past events and in many cases intentionally make-believe: how then is it fundamentally a question of action?  The key is that ideas exist only in relation to actions; even an abstraction is just a potential structure that links two ideas when conceptualized by some overarching task.  A story, being a relationship between actions, must therefore be told, whether by a narrator, or a troupe of actors, or even the activity in your brain that simulates the events of a novel as you read it.  Without any kind of enactment, a text, whether a book, a script, a movie, or even an arrangement of neurons with the potential to recall an oral tradition, is just an inert object.  For one to enact a story, however, is to act in relation to that story's conflict and for those periods of time live it.

None of this, however, is to suggest any kind of reductive stimulus-response mechanism.  A story is not an algorithm.  For one thing, physical actions do not happen in discrete ticks.  Unlike a line of code an action does not have a state of either being executed or not executed, nor can it be simply paused or reversed. If I stretch my legs and hit a wall before I fully extend, that doesn't mean I didn't do anything. Any interruption or change in the progression of a narrative act irreversibly changes the story. To try to break up a narrative into units like that of a mechanical clock simply doesn't make any sense: even if one could find a “smallest unit” of action, breaking up the arc of any larger enactment would erase most if not all of the relevant information.  Shakespearean soliloquy doesn’t stay the same if it’s interrupted by a pause; in the physical world where time is continuous and always moving, silence is no less an action than anything else.

Even more importantly, the traversal of a text is never predetermined or even definitively bounded; an actor is always interpreting the text according to their own unique background and therefore carrying out the instructions in a unique way.  Many such differences do not necessarily change the story in any noticeable way but there is always the possibility of an enactment that completely changes its import, whether it's a show-stealing performance by a dramatic actor, a subtle but profound variation on a piece of classical music, a new critical interpretation of a novel that gains widespread attention, or even a new scientific theory that takes its inspiration from a previously overlooked idea or a new technology that has yet to be accounted for. Storytelling is fundamentally an act of invention, and one's enactment of a text an application of technology.

But the essence of story is a shared sense of conflict, whether amongst real life people or fictional characters.  A story can have many different texts; there are multiple versions of the Bible, novels have many translations, and everybody's going to remember an oral tradition differently. Nonetheless, many different enactments can share a single story; the story, or what Todorov referred to as the fabula, is what everybody mutually understands, the invariance across all of the enactments. This is what makes it possible for people to be on the same page in real life: everybody is living a great many different narratives at different times, but they can all when necessary play their respective parts in one that they share.

Narcissism, for the record, is not arrogance; it's the belief that one's private narratives are the only narratives, or at least the only “authentic” ones. But I digress.

Theory Revisited

Unlike a model, a story is not a static reduction of some phenomenon. A story does not inherently describe anything, it provides affordances through which those who enact it can continue moving.  Nor, as I have pointed out before, are these affordances in any way fixed: the places they lead are contingent upon the agency of those who enact the narrative.

This simultaneous openness and schematization allows endeavors such as scientific discovery, for example, to have the common ground necessary for people to have debates and experiments that are falsifiable while still allowing for the possibility of new ideas and events that do not cleanly fit in.  Time, as a concept, was for a very long time assumed to be a static backdrop with no relationship to space, but problems that arose with synchronizing clocks around the world opened the door to a new foundational conceptualization. Without the introduction of such a problem, the idea of time and space being “related” would sound like an absurdity, but the introduction of this novel phenomenon changed the unwritten rules of how one was allowed to read the text. New actions, in this case the widespread adoption and use of clocks to coordinate larger networks, can always alter the course of a story in unforeseen ways.

All this being said, a story, being a context through which actions are related, still places constraints in the sense that some actions will still be outright irrelevant and others will take on a specific significance. This places limits on where a story can go, and of course without such constraints nothing could mean anything. A story remains useful so long as one can continue to move “forward” in some way: that is, to expand their agency.  

Fortunately, stories begin and end: as the space of possibilities continues to get richer, the narrative rises, eventually hitting a point of maximum possibility at the climax.  The climax of a well written movie is thrilling precisely because of that sense of possibility; that even as though it’s all narrowed down to a single pivotal moment, that moment will decide the meaning of all those prior events, and in doing so constitutes an exceptionally rich event.  After this, the narrative slides into denouement, in which possibility rapidly atrophies by virtue of these constraints.

The same thing happens in science, said best by Thomas Kuhn in his masterpiece The Structure of Scientific Revolutions.  Science, in Kuhn’s view, proceeds according to paradigms: an existing theoretical and practical framework that consists not only of theoretical assumptions but all the physical and institutional embodiments that I mentioned earlier.  As long as that paradigm continues to produce promising results, it is in a phase called normal science in which scientists contribute incrementally by essentially solving puzzles inscribed in stable rules that continue to work.  There eventually comes a time, however, where the paradigm starts failing to produce any new results, either failing to predict what it ought to be predicting, or simply finding no new applications or even relevant questions.

Such sluggishness inevitably hurtles into crisis as the paradigm grinds to a halt and the people and institutions involved enter a phase known as extraordinary science: a process of bricolage in which disparate concepts from all kinds of unconventional places are connected in novel ways to create the foundations of a new paradigm.  At every step of the way, the paradigm’s health is a question of action, of what it allows scientists to do.  The questions and challenges motivating scientists and defining their praxis serve as a conflict through which their endeavors find common ground, and it is precisely at the point where a new grand theory must replace the old that such a story reaches a climax: when the old paradigm fails, the future becomes deeply contingent on a small window of time in which the possibilities are profoundly unpredictable.  After such a replacement is found and people and institutions calibrate to the new ideas, the story wraps up and ends in preparation for an entirely new one.

So what of all those other theories and disciplines that have failed to make reliable predictions or introduce new technologies?  On aesthetic and practical grounds alike, there are plenty I’d love to see thrown into the rubbish-bin of history, but imagining that a theory is only worth its immediate successes is not the right way to look at it; plenty of interesting questions have been asked that only much later proved profoundly useful.  This bias towards valuing direct short-term usefulness comes not least from the implicit assumption in many places that “theory” and “model” are synonyms, and to be fair I have been using those terms somewhat interchangeably throughout this essay.  Where a model is a simplification of some phenomena that says something about its structure, a theory is a discursive foundation upon which one talks about such things.  If a paradigm is a story, then a theory is a text, a background upon which the story anchors itself.

But a text is useful insofar that it catalyzes a vibrant story, and a theory insofar that a paradigm keeps unfolding an ever richer basis for action.  A paradigm need not “prove” things in some seemingly absolute way or offer immediate technological applications; if it can continue to ask increasingly interesting questions by putting forth increasingly nuanced concepts then it is in its own right experiencing success.  This might seem like naive relativism, but increasing agency is the only way we can define progress, and even what we call the “hard sciences” could only come to be after millenia of unstable “natural philosophy” and alchemy.  

One might at this point ask how, given this argument, do we address disciplines that seem to only amount to echo-chambers that create increasingly convoluted explanations of why contradicting information doesn’t make them wrong, but the existence of any such paradigm is first and foremost an issue of economics: a paradigm, being a fundamentally material process, what the kids call a “living, breathing thing”, cannot survive, cannot exist, in a vacuum; it needs real resources.  If a paradigm fails to serve any purpose beyond socially or financially validating its constituents, the only way it could be surviving is through rent-seeking.

This is not an argument along the lines of “if you’re so smart why aren’t you rich?”, nor is it an argument against ideas and practices that might not prove immediately instrumental.  Just like a good theory can say a lot with very little data, it can also by analogous logic do just fine without any kind of economy of scale, picking up speed as it makes and discovers ideas, technologies and practices that get adopted by adjacent communities and enterprises.  If an idea proves relevant, if it makes sense of important questions, to enough people in the right ways, the narrative will continue to be enacted.

This is not some call to surrender “truth” to money or popularity.  Everybody has pressing needs, passions, and curiosities that determine how they expend their resources, each of whom has every right to try to find the answers that work for them.  Nor is there some simple democratic consensus on the alleged utility of some intellectual or artistic pursuit: as long as there’s a community of people who see an idea as worth pursuing, it is enough for them to set a narrative in action by committing the time and resources to start a new paradigm.

Just like you wouldn’t invest all of your money in one stock, what’s absolutely needed is a rich ecology of narratives; scientific, philosophical, artistic, or otherwise.  Without this, society chokes off the key property that makes narratives possible in the first place: intertextuality.  When an actor enacts some narrative, their choice of how to do so is shaped by past experience, which itself comes from other narratives previously played out.  In this way, narratives are cobbled together, bricolaged, out of other ones; perhaps disproportionately so during extraordinary phases, but also always to some extent during normal ones.

Some might call this a “marketplace of ideas”, but I find that this metaphor doesn’t capture the institutional threats to a vibrant ecology.  What’s needed is what Paul Feyerabend called epistemological anarchy.  While he defined it as “anything goes”, this doesn’t quite square with the rest of this essay.  While I can’t speak for those who politically label themselves anarchists, anarchy, as a process, as a practice, fundamentally involves dismantling monopolies.  While the stranglehold of monopolized financial resources is undoubtedly a major problem, there are intertwined monopolies on social capital that cause the overall narratives themselves to be disproportionately owned by the few and subject to rent-seeking, with certain narrow institutional paths being the only way offered to the inquisitive for finding any kind of narrative in which to play a part.

It is not primarily more top-down “rigor” that we need: any prescriptive scientific “method” (e.g. p-values) can be easily hacked, and a failing methodology is just a symptom of a lack of efficacy (why fake success unless you don't have it?)  Rigor follows from the maturity of a practice, when ideas and conjectures finally converge on some pocket of stability.  What we need in most cases is more tinkering that leads to interesting places; in other words, there need to be more opportunities and equitably distributed resources for people and communities to engage in the process of engineering new narratives; engaging in seemingly random praxis that leads to increasingly cogent insights.

Such narratives are important not only for the eventual discoveries of new technologies and profound truths, but also for the health of our own social exchanges and subjectivities.  A society of nothing but bread and circuses at the expense of ritual, literature and the arts is one that traps us in increasingly maladaptive affective patterns.  While the neurologists can carp about “mirror neurons” and “monkey see monkey do”, all that needs to be understood in the context of this essay is that one is always travelling through different narratives, and as such they constitute what actions we’ll have available to bring to new ones.  There’s nothing “morally degrading” about enjoying some lightweight entertainment, but popular media, today’s vehicle for propaganda, is itself a reduction of a great many affectations into a much smaller set of interactions that are not sufficient on their own to help us deal with the complexity of other people or even our own inner journeys.  

Some argue that there’s already plenty of classics out there, that we need only rely on the past and get over our self-importance in thinking there’s anything new under the sun.  There are indeed a lot of constants in human nature, but there are also always new actions and by extension new modes of subjectivity that engender qualitatively novel needs.  We are always spinning off new narratives, each bringing with it new challenges that require new strategies for enactment, and with that, a constantly growing need for new ideas and practices;that is, new feats of engineering.

---

Postscript: July 2021

I wanted to qualify the bit about "extraordinary science" and the difference between theories and models, not least because I've had a lot of subtle changes in my ideas about epistemology since the time I wrote this essay.

While I still stand by my assertion that there is a fundamentally contingent (and by extension unpredictable) nature to new theories, which are foundational texts, as opposed to models, which by contrast are guided by an attempt to structurally mirror the phenomena (which themselves are always expressed to an observer according to the vocabulary of some theory) they ostensibly correspond with, I made their construction sound ultimately arbitrary in some way by failing to better explain how this task is carried out. This is something I plan to devote an entire follow-up essay to, but I want to acknowledge this shortcoming here and give a rough summary of what creates the formal terrain upon which a material paradigm crystallizes:

Think back to the earlier exposition on functors and how they're defined by achieving a certain kind of path-independence, a.k.a. invariance. This invariance does not come out of thin air: it's the result of connecting up material affordances until one has the ability to move around in a way where they can always retrace their steps in some sense, and in doing so, construct a new map for what was once uncharted territory. By doing this, one creates what's known in mathematics as operational closure. In the essay I cited things like imaginary numbers as allowing the mathematician to do more, but I didn't articulate the most important part of that: that it allows the mathematician to go in more directions without simply falling off the edge of the map: without negative numbers, subtracting a larger number from a smaller number is something that you can't come back from, whereas the number "-1", even if you assign no "meaning" to it, is a placeholder that allows you continue working with it even if you want a result that ultimately doesn't include negative numbers (for example, you can't have a negative amount of cows, but you can use that as a placeholder to say that you owe someone a cow and that you can balance out the whole ledger at another time.)

So operational closure gives you invariance (path independence), and similarly these closures enrich the fundamental underlying vocabulary you're working with by giving you new invariances to aim for in your models. In short, this means that you've expanded the number of moves you can make through the craft of composing existing moves together, often from disparate places. This means that a new concept is not simply something arbitrary that may or may not help people do things, but is in fact on some level the embodiment of an entirely new affordance; therefore, while one may not have uncovered some new Platonic form, they have created something nonetheless universal relative to its predecessors and in this sense unequivocally real insofar that it offers additional agency that cannot be denied.

The difference between normal and extraordinary science in this regard is actually to an extent one of degree: normal science involves filling in minor gaps in a somewhat routine fashion with limited tinkering, whereas extraordinary science is when one ends up throwing the kitchen sink at an otherwise intractable problem; both are an instance of Godel's Incompleteness Theorems, in which one has to exercise a degree of abduction in order to compensate for not being able to get an answer simply by computing on the axioms, even though the question is written entirely in terms of those same axioms. The qualitative difference between the two is that it's only extraordinary science that synthesizes new concepts, whereas normal science refines them by tinkering around the edges to fit them more cleanly into everyday practice and application; a theory, being a text, is sort of like a system of axioms, but it's a bit more nebulous than that, because it also comes with a way to be seen in context and some other stuff, but I'll leave that for later. In both cases, however, a concept is something very real insofar as it's a new way to move that redefines the very space one is moving in, and that this isn't imaginary or some "hyperstition" (I hate that obfuscating word) but something coextensive with the material rituals and affordances of a given paradigm; a new way to play out a narrative. Again, it will take an entirely new essay to explain this in a satisfactory way, but this chapter in my in-progress hypertext book might do a bit more justice to what I'm trying to say until that sequel essay is written.

Also, if you haven't read this essay recently, today I just changed the section on models to be more clear and include a description of functors, which ultimately describe certain things I'm talking about much better than homomorphisms.

---