Category: Technology & Society

Antifragile

The core thesis of “Antifragile”, a book from Nicholas Taleb, is that some things suffer from volatility – but some other things gain from volatility. We should avoid making things fragile, but instead antifragile.

Antifragile goes beyond robustness or resilience in the sense that things that are antifragile not only tolerate volatility, but benefit from it. Taleb claims there is no word for this, hence the neologism “antifragile”.

One recurring and intuitive example of an antifragile system is nature itself. Through volatility (mutation of species, random natural events, etc.) the system becomes better. Other examples in the book include: entrepreneurship, health, education, city-state, artisanship.

Behind the idea of antifragility is the idea of optionality. To be antifragile, you should have many options that you can use (but are not obliged to), depending on how the situation changes. If options are cheap, you can have many of them and reduce your risk exposure, or better, reverse it to benefit from volatility. With the right options, you can be robust against small volatility, but also rare events (“fat tails” / “Black Swan”).

That’s how Taleb made its fortune, he was an option trader. While many operations don’t turn a profit, some have occasionally large payoffs. The book is an generalisation of the concept of optionality to other areas of life.

Nature is antifragile, because it has many options due to the diversity of species, and the diversity of random mutations. Many natural events are inconsequential, but once in a while, a natural event arises that leads to change in the ecosystem. Entrepreneurship is antifragile, because of the diversity of companies. Many fail, but once in a while a new idea work on (as venture capitalists surely know). Your body is antifragile, because a small occasional stress on some function (muscle, nutrition) makes you stronger.

Antifragility and optionality both relate to diversity. Nature is diverse, hence anti fragile. Entrepreneurship is diverse, hence anti fragile. More generally, trial-and-error is antifragile, since it increases diversity. Being an artisan as opposed to have a very special role in BigCorp makes you more diversely employable, hence antifragile. A small stress on a system with diversity will force new pathways that exercise and reinforce diversity even more.

My main critique of the book is that this link is obvious but not really explored. Diversity is a simple and well established concept. It’s risk management 101. I understand that Taleb brings another perspective to the topic with optionality and antifragility (and rare events), but it’s also not completely different. Just another angle. The same can be said about the study of complex adaptive system. These systems are the class of systems of interest to Taleb. Yet he completely ignores this field of research.

Taleb warns in the book about interventionism. Many interventions fragilise systems by removing diversity. The education system tends for instance to format the way we think, but we need people that think out-of-the-box to progress. Big companies, of course, are all about standardisation and economies of scale. They are fragile to disruption.

Interventionism also focus on what we know, but ignore what what we don’t. This unknown hides events deemed improbable but with disastrous consequences. Only time will reveal them. Vivid examples are instant catastrophes like Chernobyl, but also catastrophe building up with time, like climate change.

Interventions with good intentions but negative effects are easy to find (medecine, foreign policy). There’s an existing word for it: iatrogenics. If the unknown is big and the reward small, sometimes doing nothing is better.

The subtitle of the German version of the book is « a guide for a world we don’t understand ». In a way, I prefer it to the English subtitle « things that gain from disorder ». The German subtitle emphasis this view of the world, where we don’t know things, or can’t understand them. We believe everything is quantifiable (especially risk) and explainable, but it’s not. It’s also not needed to live well in this world. You don’t need to know how your body works to realize that exercising is good (also referred as: Green lumber fallacy). You don’t need to know what bad could happen, just err on the side of precaution with optionality/diversity.

Taleb warns of modernism, too. Stick to the classics, since they have better chance to pass the test of time rather than to jump to latest bandwagon (also referred as: Lindy effect). Interestingly, this goes against the former praise for optionality and trial-and-error. Unfortunately, this tension is not really addressed in the book.

There’s a tension, because the line of reasoning that Taleb proposes works at two levels: the level of the system and the level of the individual actor in the system. For instance, your cells are fragile, but you body as a whole is antifragile. Companies are fragile, but a diverse economic system as a whole is antifragile. While he briefly mentions both levels, most of the book does not really distinguish between the two.

In the latter chapters, Taleb introduces ethics around antifragility. Simply put: you shouldn’t exploit the fragility of others to make yourself more antifragile. Risks should align with rewards (referred as the agency problem). It is ethical for an entrepreneur with “skin in the game” by taking risks himself to be rewarded with large payoffs if he succeeds. A middle manager taking no risk personally, but risking the pension funds of other citizens, doesn’t act ethically. The opposite of this behavior is a hero. A hero is someone taking risk not only in his interests, but in the interests of others.

Taleb sees himself as a philosopher, whose special brand of heroism is enlightening people about fragility, and calling out people that fragilize the system at the expense of others (if you see fraud, you should call fraud). The book is thus mixed with references to philosophy, discussion about fragility, and stories of Fat Tony and Nero Tulip. Both are two kinds of alter ego of Taleb. Fat Tony makes money without “understanding too much about the world” while Nero Tulip is erudite. At the end of the book, Fat Tony dies and leaves Nero with 20 millions. I suspect that Fat Tony represents here the first career of Taleb as a trader, and Nero Tulip his second career as a book writer / philosopher.

Taleb is arrogant as expected, but the book is enjoyable to read and I liked many of the ideas presented.

Data Lock-Out: Understanding the Risks

My computer died a few days ago. Fortunately, I had a backup and could restore my data without problem on another laptop. Still, I’ve been wondering in the meantime: what if the restore hadn’t worked? How easily could I be locked out of my data ?

I have data online and data offline. My online data are mostly stored by google. If say, my account is compromised and due to a misbehavior from the hacker, my account is disabled. Would I ever be able to recover my online data? Not sure.

My data offline are stored on the harddrive, which I regularly backup with time machine. If a ransomware encrypts all my data, the backup shouldn’t be affected. Unless the ransomware encrypts slowly over months, without me noticing, and suddenly activates the lock out. Am I sure ransomeware don’t work like this? Not sure.

My laptop suffered a hardware failure. It hanged during booting, and no safe booting mode made it through. The “target disk” mode seemed still to work, though. It would have been a very bad luck, to not be able to access either the data on the harddisk or the backup. Both should fail simultaneously. But can we rule out this possibility? Not sure.

Harddisks and backup can be encrypted with passwords. I don’t make use of this option because I believe it could make things harder in case I have to recover the data. I could for instance have simply forgotten my password. Or some part could be corrupted. Without encryption I guess the bad segment can be skipped; with encryption I don’t know. Granted, these are speculative considerations. But are they completely irrational? Not sure.

Connecting my old backup to the new computer turned out to be more complicated than I thought. It involved two adapters: one for firewire to thunderbolt 2 adapter and one thunderbolt 2 to thunderbolt 4 adapter. Protocol and hardware evolve. With some more older technology, could it have turned out to be impossible to connect it to the new world? Not sure.

The probability of any of these scenario happening is small. It would be very bad luck and in some case would require multiple things to go wrong at once. But the impact would be very big—20 years of memory not lost, but inaccessible. There’s no need to be paranoid, but it’s worth reflecting on the risks and reduce the exposure.

More:

Stuff Matters

img_3072

Stuff Matters is a very nice little book about the materials that surround us. Organized in ten chapter, each tracing the history of a class of material (metal, paper, glass, plastics, chocolate, gels, graphene, concrete, ceramics, biomaterial), we get to better appreciate how much tinkering and research took place over centuries to discover all these materials and their properties.

Most materials in the book are materials we know and can relate to. We know their basic properties, for instance, that rocks are solid, don’t melt easily, and don’t conduct electricity. Having two young children discovering the world, I marveled at how much they still need to discover. For a large part, we learn from personal experience. But we also learn through school, reading, movies, art, architecture. Materials are everywhere and are an integral part of our society and culture. That’s one theme of the book and it resonated well with me.

Mark Miodownik is a good storyteller. When talking about materials, there’s always a human context that makes the story relatable and engaging. For instance porcelain isn’t just porcelain, it’s also the story of Chinese emperors trying to impress their rivals with refinery and sophistication. At the same time, the book has its good share of technical details. We learn about the structure of many materials, including several modern “high tech” materials such as graphene and aerogel. The book even mentions invisibility cloak, how cool is that?

The whole book is interspersed with personal anecdotes. They don’t feel forced and I enjoyed them. Mark Miodownik commands his subject for sure and is passionate about it. I loved the anecdote about him being profoundly impressed by the Crown Jewels of England as a kid. Not because of the luxury they represent but because of their “primitive” materials, mostly pure gold and gems. We can really feel that his passion for the subject came at an early age. And passion is contagious. This book is a nice example of it.

The Planet Remade

The Planet Remade, a book by Oliver Morton, explores the possibilities and implications of geoengineering – that is, the intentional modifications of the earth’s climate.

The argument in the the book can be summarized as follows:

  1. Climate change represents a risk. Depending on one’s position on climate change, the risk might be bigger or smaller, but there is a risk nevertheless. We ought thus to address this risk.
  2. Reducing carbon emissons, as a way to mitigate climate change, is way harder than most think. There are two challenges with reducing carbon emission. The first is that the benefit is delayed for a long time – decades. This makes it hard to create a momentum. The second is that that we don’t have good a “lever” where to act. Reducing carbon emissions substantially also requires substantial pervasive investments. For these reasons, the following known approaches are hard to pull off:
    1. Direct Air Capture (DAC)
    2. Carbon Capture and Sequestration (CCS)
    3. Ocean pH decrease
    4. Reforestation
  3. A high level of CO2 in itself (that is, if we omit the greenhouse effect and its impact on climate) is not a bad thing. Plants and photosynthesis have evolved billions of year ago when the air had much more CO2. High level of CO2 actually benefits vegation significantly. Models show that such a planet might be greener.
  4. Geoengineering technology to negate the greenhouse effect and cool down the planet should be considered on our way to a sustainable planet. Several geoengineering approaches could be envisonned to achieve such a cool down:
    1. Stratospheric veil
    2. Increased cloud brightness
    3. Increased plant reflection
    4. Increased ocean reflection
  5. The stratospheric veil is the only technology that offers a good “lever”. With a small fleet of airplanes, enough material could be injected in the stratosphere to achieve a significant effect. Also, the effect comes relatively quickly.
  6. We do have some understanding of such veil through the observation of volcanoes. It doesn’t mean that we have enough understanding now to use it without risk, but it means, this approach has potential. It requires more research and if possible, experiments. The best substance for the veil is for instance unclear. Volcanoes emit sulphure, which has some drawback.
  7. Veils have some negative effects. It is still a better option than the negative effects of an uncontrolled global warming. Effects of the veil isn’t homogeneous – some region might benefit but some might be hampered by it. A minimal veil with regionally either a positive effect or at worst zero effect, should be doable.
  8. Reducing carbon emissions and geoengineering a veil are not exclusive. Many people dismiss geoengineering out of fear to lose the focus on carbon reduction. But both are complementary and the best angle to look at, is the research geoengineering to create a “breathing gap” for the transition to zero carbon emissions. The veil isn’t a permantent solution, only a temporary one. With this strategy, we also reduce the risk of the “termination shock” – the risk of forced stop of the veil in the future should we not be able sustain it. Morton warns that researching stratospheric veils as Plan B, in case of emergency, will not lead to solutions that work when needed.
  9. A planet-wide transformation of natural cycles raises moral challenges. As a matter of perspective, we must consider that
    1. we have already transformed the planet at a global scale. For instance, the extensive and systematic use of fertilizer is akin to geoengineering of the nitrogen-cycle.
    2. we actually do already have a veil with a cooling effect because of pollution. As policies to reduce polution stengthen, this veil will disappear, possibly “unmasking” more warming due to greenhouse gases. A geoengineered veil could be seen as a replacement for this polution veil.
  10. Geoenginering belong with nuclear to technologies with powers beyond the human scale. Both could wreak havoc on the planet, or be used for better goods. The justified fear of nuclear technology in the context of nuclear wars doesn’t mean we should put geoengineering in the same basket.

The book doesn’t follow strictly this order – this is my summary after reading. Instead the book is organized in three parts – energies, substances and possibilities – and explores several tangents. The first part expains the basics of climate science and sketches the argument. The second part goes deeper in the various approaches to geoengineering. The third part gives perspectives about the fate of the world and how things could play out. This structure did sometimes feel like meandering, but it worked for me.

Where this summary of the book doesn’t do justice to the book at all is about its coverage of history. I read on wikipedia that Morton has a degree in history and philosophy of science. This penchant for history is clearly visible in the book, which reminded me of books like Energy and Civilization or Stuff Matters. The book retraces the history of many scientific discoveries, which I found entertaining and informative. Moreover, it also addresses the political and moral side of climate mitigation and geoengineering, as well as the shifts in perception of technology over generations. The current “mainstream” perception of technology could change again for topics like geoengineering or nuclear energy.

Books of this kind are usually either written by scientists or journalists. The former might lack storytelling, the latter might lack depth. Morton is a journalist, but also himself active in this “geo clique” since many years. Morton has a great command of the topic but also a good storytelling.

The book left me unsatisfied about two points, though. I would have liked that Morton better explain the view of people against geoengineering and provides a better rebuttal of their points to back up his argument. I also wished he had better explained why stricter policies around carbon emissions are so hard to set up. We successfully did this for CFC and are doing it for sulphuric emissions, so why not carbon? If the answer is that carbon emissions originate from a lot more place, why not start first with strong policies specific to certain industries?

Knowing little about climate science, I certainly learned many things about it reading the book.

  • the difference between stratosphere and troposphere
  • the ozone layer (nice complementary to the ozone story in Limits to Growth)
  • the fact that the stratosphere isn’t homogenous
  • the role of oceans to reflect sunlight but also evaporates water
  • the albedo-effect
  • the Trenberth diagram of energy flows
  • the role of temperature difference as the source for weather
  • the effects of carbon, nitrogen, sulphate, methane, NOx in the air
  • the effect of light diffusion on photosynthesis
  • the geological times
  • the explanation for ice ages
  • levels of CO2 over time
  • the climate is defined by more than a temperature: wetness, dryness, temperature cycles define the climate too

Coming from computer science, “the cloud” means for me internet. Now, the term has for me regained its original meaning with great interest.

More

Limits to Growth

I read this book after reading Factfullness, from Hans Rosling. It was a lucky coincidence. In Factfullness, Rosling argues that our model of the world is often wrong and imprecise. In Limits to Growth (LTG), the authors work out, step by step, such a model.

The model that the authors present is aimed at understanding how growth will continue up to 2100. The authors do not argue that the model captures every possible aspects impacting growth. They are well aware that it’s a model, with some simplification in it. What they tried to do, is to make it usefull for their goal. In this regard, the model is well argumented and realistic. The limits of the model are also disclosed transparently – for instance, the model has no notion of geography or politics.

The situation now in the world is already unsustainable – we are facing overshoot. The formidable challenge up to 2100 is to solve overshoot for the current population, but also to improve the system to accommodate 4 billions people more.

At its core, the model is based on a few key abstractions, which are presented linearly in the book.

  • The first abstraction is the world population. Population growth is controlled by income level and global health measures that reduce death (here you see the link with Factfulness very clearly).
  • The second and third abstractions are sources and sinks. Sources provide natural resources that we can transform into capital to sustain the economy, and sinks absorb the waste or pollution produced by the system. Elements in the system can act as sources and sink – for instance, tree produce wood and do absorb some CO2.
  • The fourth abstraction is that of limits. Sources and sinks have limits, which constrain how much can be provided or absorbed. A limit can be absolute, constraining the finite amount in the world, or a rate, constraining how much can be produced or absorded in a certain time.

You can tune various parameter in the model to compute scenarios about the future. And that’s what they do in the book.

The first scenario is the utopia scenario without limits. In this scenario, growth continues indefinitively and the civilization prospers beyond 2100. The other scenarios are scenario with limits in place and various assumptions about the rate of progress and global policies.

And here is the bad news. In these scenarios, growth tends to stop in the second half of the century which leads to a collapse with population decline. These scenario are no forecast. They only oultine possible trajectories and trends. Unfortunately, no matter how we tune the model, as soon as we re-introduce limits, the collapse trend is clear.

This is also the case for ecomodernism scenarios. There’s a whole chapter in the book explaining why technological progress and market alone, even with ideal parameters, won’t suffice.

Several such scenario lead to collapse due the lag time between reaction and problem. We are dealing with cycles that happen over decades. It takes decades for pollution to disseminate from production site to environment; it takes decades to engineer new technologies; it takes decades to renew or replace existing infrastructure. When the reaction comes, it’s too late. The most optimist scenario (scenario 6) of this category seem to almost achieve sustainability, but ultimately collapse too due to pressure on the system and the cost of technology required for it.

This book is a fascinating analysis, even if the perspective is grim.

In all the scenarios, the same model is used as well as the same limits (except for the scenario without limit). We could of course explore a lot more scenarios, with modifications in the model or of the limits. I’m sure the authors did. But the model and these limits reflect what the authors think approximates best reality. Finding solutions for an alternate reality doesn’t help. But it’s a model after all, so there’s room for critique of the results. And for hope.

What I would have found interesting but isn’t in the book, is a “backward” analysis of how much rate of progress assuming these limits would be needed to achieve true sustainablility. If 2020 told us anything with the pandemic, it’s that if pressure is there, progress can sometimes happen fast.

The story of the ozone hole acts in the book as a case study of a real-world feedback loop with long delay. This story is hopeful, even if unclear how we could replicate such success for other areas. In the last chapter, the authors make a call for a next revolution after the industrial revolution – the sustainability revolution. I like the idea, but wished that the learnings of the ozone story would have been transformed in more concrete inputs in this chapter.

This book left me a better understanding of:

  • Delays in feedback loops
  • The risk of ecomodernism
  • The role of land/food in growth
  • Systems thinking
  • The ozone hole

On top if all that, I’m very impressed that the first version of the book was written in the 70s. 50 years later we only start to believe what they were saying because we start to see it for real (seeing is believing), not because we’ve become more clever.

MORE

“The scientific message of LtGgot lost in the turmoil of the pop-ular debate. Global society is likely to overshoot, said LtG,andthen be forced to decline or collapse – because of significant re-action delays in the global economy. These are the lags in the per -ception and localization of global limits, the significant institu-tional delays involved in (democratic) decision making, and thebiophysical lags between implementation of remedial action andthe improvement of the ecosystem. The real message was appar -ently never picked up by anyone, neither critic or fan.

…

But as con-traction does not occur in the model system until around 2020,historical comparison up to 2010 does not give much guidanceabout the veracity of the contraction part of the LtGbusiness asusual scenario. By 2030, we will have a much clearer answer tothat question.”

Energy and Civilization

The book Energy and Civilization, from Vaclav Smil, looks at the history of civilizations through the lense of energy: how did the methods to store and transform energy change over time?

Almost all aspects of a society can be linked back, directly or indirectly, to energy. At the lowest level we need energy, as food, to sustain our metabolism. For centuries, society has been organized in a relatively straightforward manner around food production. Transformation of society happened out of necessity, to scale to larger population. The role of society is, in a way, to organise the transfer, transformation, or production of energy for our needs. In the case of modern society, this organization can be very complex. Natural resources are transformed using energy into products that are stored, exchanged, and used to transform other resources into products.

I don’t recall Vaclav Smil from every using the word “recursive” in the book, but coming from computer science, the flow of energy through society felt to me this way. We transform energy into heat, light, or motion in a recursive scheme.

The book covers all major energy transitions in great details. From hunter-gatherer to agriculture with animals and water wheels/windmills, from agriculture to stream engines and coal, from steam engines to combustion engines and electricity, and from combustion engine to renewable energy. Nuclear is of course discussed, too. Using this perspective, the industrial revolution spans several energy transitions.

The total cost in energy of a method or process can only be computed if we consider the “recursive” steps too. The horse you use to improve farming also need to be fed. Solar cells must be produced first, before they in turn generate energy. The book covers many analysis of such total costs.

Over the centuries, methods to transform energy – be it with animals like draft horse, or tools and machines like water wheels, steam engine, or coal furnace – have been improved to reduce energy waste. The book makes a great account of the improvements.

To get a sense of the depth of detail, consider that the book contains for instance a comparison of the efficiency of the various water wheels; or a description of the various tools used for farming; or an analysis of the shock on the twin towers on 9/11 in terms of energy. The book is this level deep of details.

The reading isn’t always easy and the book needs some perserverance. But this depth has its reward too, in that it’s a great documentation of human ingenuity.

Going in length to describe the various uses of energy, the book often contrast them with human labor. This made me more appreciative of the incredible power of the many tools and machines we use. Tools and machines at our disposal enable one person to achieve tasks where hundred or more people would have been required before the industrial revolution.

The book is quite short when it comes to the future. It surprised me, but we can’t fault the book for this. After all, the book is titled “Energy and Civilization: a History”, not “Energy and Civilization: an Outlook”.

Like many readers, I bought the book because it was on Bill Gates recommendations (here’s also his review of the book). While I don’t consider the book a must read, I’m thankful of Bill, since the book taught me a good deal about:

  • The various types of energy stores (coal, biomass, nuclear, wood, etc.) and transformations
  • The energy transitions and their timescale (centuries!)
  • How energy flows in society
  • The energy that modern life requires and its contrast with human labor
  • The ingenuity of mankind to transform energy efficiently
  • The challenges ahead of us to transition to renewables

Factfulness

factfullness

If you’ve seen the 20006 TED talk of Hans Rosling, “The best stats you’ve ever seen”, you’re already familiar with the topics of the book Factfulness. There’s of course more material in the book than in the talk.

I took the following take-aways from the book:

Our mental model of the world may be wrong in significant ways

This is the main theme of the book. Rosling shows with several quizz that our knowledge of the world is very imprecise, if not wrong or outdated. As a consequence we have a wrong model of the world, its problem, but also possible solutions. He offers several explanations for this, like our bias for sensational news that misrepresent the state of the world.

Rosling was an expert in global health, so many quizz are about population statistics, e.g. “what is the percentage of vaccinated children worldwide”. He shows that many people underestimate how the quality of life changed in the last decades.

There are various levels of “poor”

When it comes to income levels, here again, our model of the world is too simplistic. Instead of categorizing countries in poor or rich (or developped and developping), Rosling proposes four levels of income. He then shows how life is for these levels – from extreme poverty to “western” rich. I guess I’ve only travelled to places corresponding to level 4 and 3.

The project dollar street that Rosling launched is a website where you can browse families worldwide to see “how people really live”. It’s worth having a look.

It takes many policies to improve the world

Hans worked in global health. The standard of living we have know has only been achieve because many policies have been put into place over decades: vaccination campains, access to clean water, practices in hospital, road safety practices, etc. The average life time is the best metric to figure out the progress. The topic is brought vividly in the book with Hans almost drowning as a kid. Now, such an area would be proctected with fences for small kids.

Focus on individual vs. focus on population are both valid choices

There is an interesting section in the book showing the following dilema: should you put your effort to improve life in a population in average, or should you focus on helping the most the specific patients that come to you? There is no right or wrong position. Bur for Rosling, it was clearly about focusing on the macro perspective.

Put numbers in context

I loved this chapter of the book. Don’t be impressed by numbers alone – small or big. They tell nothing. You must always put them in context.

You need to periodically refresh your world views

The facts you learned 10 years ago aren’t acurate anymore. You need to update once in a while your knowledge.

The domain of Hans was global health. But I can easily find parallel for these take-aways in other domains. For instance, software quality in the software industry has improved thanks to best practices like version control, pair programming, unit testing, etc. This is much like policies in global health. It takes decades to change company cultures and see benefits at the level of the industry.

About outdated world views, I can easy come with examples in the sector of energy, since I have recently read “Energy and civilization“. In this book, I learned about the the fracking revolution and that the USA become major producer of natural gas after 2001. My world view was stuck in the narrative of that time.

One striking example in the book of a poor model of the world was the question related to population growth. How bigger will the population be in 2100? As countries develop, the number of children pro family tend to reach 2, making for a stable population. Many countries have already reached this ratio, more than I thought. Using the projected development of countries with a high ratio, it turns out that the projected population growth from the UN is 11 billion people in 2100 (7.8 billion in 2020). We will have to accomodate 50% more people on earth.

If you think about ist, it’s one of the most important fact to know about the world to understand where the world is heading. If we want to find solutions to climate change and other challenges, we need to have an idea of the population we have to sustain. But I wouldn’t have been able to estimate the population before reading the book.

The book made me more aware that our model of the world may be wrong and that our intuition often fails.

More

Mastering Technology

Things move fast in the IT industry. Half of the technologies I use today didn’t exist ten years ago. There’s a constant push to switch to new technologies.  But when it comes to technologies, I’m kind of conservative. I like proven technologies.

The thing that makes me conservative is that mastering a technology takes a lot longer that we think.

Most advocates of new technologies massively underestimated the learning curve. Sure, you get something working quickly. But truely understanding a new technology takes years.

Take object-oriented programming. On the surface it’s easy to grasp quickly and become productive. But it took the industry something like 20 years to figure out that inheritance isn’t such a great idea. The result is that early object-oriented systems overused inheritance hoping it will favor reuse, whereas it just led to hard mainteance.

The same holds for instance for the idea of distributed objects. It’s easy to grasp and appealing . Yet it took the industry decades to realize that abstracting remote boundaries is a flawed idea. We instead need to explicitly embrace and expose asynchronous API.

Another one of my favorite easy-to-grasp-but-hard-to-master technology is object-relational mappers (e.g. Hibernate). 10 years of experience and I’m still struggling with it as soon as the mapping isn’t entirely trivial.

Want another example? XA Transaction. Updating a database row and sending a message seems to be the poster child for XA Transactions. Well, it turns out that this simple scenario is already problematic. When it didn’t work I learned that I was experiencing the classic “XA 2-PC race-condition“.

There are lots of new technologies being developed right now, like distributed log systems, container schedulers, web framework. I perfectly understand why they are being developed and what problems they supposedly solve.

But don’t try to convince me that they are silver bullets. Every technology choice is a trade-off because you can never fully abstract the underlying complexity away. There’s a price somewhere. Things will break in unexpected way and nobody knows why. Performance will be hard to figure out. Subtle misuse of the technology will only be detected later and be hard to correct. It will take time to figure these things out.

At the end it’s all about risk management. If the technology might provide a strategic advantage, we can talk. The investment might be worth it. But if it’s not even strategic, I would seriously challenge if the risk of using new technologies is worth it.

More

    • How Technology Evolves

    We often take for granted the technology we have and forget that it’s the result of a tedious evolutionary process.

    A Railroad Track is the Width of Two Horses is one of the first stories about the evolution of technology that I remember reading, maybe ten years ago. It rings more like a colorful story than a true historic account, but it nevertheless left an impression on me.

    Later, doing research gave me a bette appreciation how of ideas evolve, cross-polinate and morph over time. True hindsights are rare. It’s a lot about tweaking existing ideas until the right form that works is found.

    Here are some of the most engaging stories about technology history that I’ve read:

    Oh boy, innovation is so a messy process.

    Platforms and Innovation

    I started my career writing flash applications. Then I moved to Java. Both are middleware technologies that abstract the underlying operating system and enable cross-platform interoperability. I’ve actually never wrote a professional application that relied directly on a specific operating system.

    This was fine to me. “Write once, run everywhere” was great for productivity.

    For the kind of applications I was developing, what these middleware stacks provided was enough. Maybe I occasionally wished that drag and drop between the application and its host system was better supported, but that’s it more or less. I didn’t really miss a deeper integration with the rest of the system.

    These technologies were also innovative on their own. Flash enabled developers to create rich web applications back in a time when web sites were mostly static. The same was true of Java and its applets, even if the technology never really took off.

    But middleware technologies also slow down innovation.

    An operating system provider wants developers to adopt its new functionalities as quickly as possible, to innovate and make the platform attractive. Middleware technologies make such adoption harder and slower.

    The official Apple memo “Thoughts on Flash” about not supporting Flash on iOS makes it very clear:

    We know from painful experience that letting a third party layer of software come between the platform and the developer ultimately results in sub-standard apps and hinders the enhancement and progress of the platform.

    The informal post “What really happened with Vista” gives similar arguments against middleware stacks:

    Applications built on [cross-platform] middleware tend to target “lowest common denominator” functionality and are slower to take advantage of new OS capabilities.

    For desktop applications, a good integration with the operating system was a plus, but not a killer. The drag and drop functionality I occasionally missed didn’t impact the whole user experience.

    With mobile devices, everything is different.

    Mobile applications are more focused and need to integrate on the device seamlessly–in terms of user experience, but also connectivity and power consumption. That’s what “Thoughts on Flash” was about.

    Think of notifications. Notifications for desktop applications are nice, but not a killer. For a mobile application, how the application integrates with notifications makes the difference between success and failure. Notifications are becoming the heart of the smartphone experience. You don’t want there to suck.

    Or think of ARKit, Apple’s upcoming augmented reality toolkit. Augmented reality hasn’t yet really hit the mass market and there is lots of potential there. If only, it will make our good old fashion ruler obsolete to measure distances. But such a toolkit relies on specific hardware (sensor, CPU, camera). You don’t want middleware there to slow down adoption.

    Platforms diverge and sometimes converge. They diverge when exclusive capabilities are added and converge when a cross platform standard is adopted.

    With HTML5 we have a good standard for regular applications with desktop-like features. The GMail mobile web application is for instance so well done, that I prefer it to the native iOS version. But you can only go that far with HTML5. If you want to push the envelope, you need to go native and use the full power of the platform.

    For applications in the broader context of the digitalization (social media, artificial intelligence, internet of things) innovation at the platform level will be decisive.

    The platform war will intensify.

    More