The State of Chaos

Perhaps the most succinct way to describe D's development and release process is: Brutally Utilitarian. Roughly once a month D does a "major" release and up to 3 point releases throughout the month. I believe the technical term for D's release model is "rolling release". Whatever is in master at release time is tagged and dumped onto the market. Deprecated code is removed in pretty much whichever major release somebody can get the PR approved. Have a partially implemented feature in master? Oh yea, that absolutely needs to get dumped on the public. Want to change the code-gen, go ahead, just pile it in there with everything else. Oh, you have a legitimate bug fix, yeap, we'll take all of those and shotgun them into the release along with everything else.

The net result is that no release is guaranteed to be "stable" relative to any prior release. In practice, most releases do not break backward compatibility significantly, at least with the more recent releases, but trying to build a library that hasn't been updated in a year can be a challenge. This inevitably leads to a situation where two dependencies require mutually exclusive versions of the compiler to build and development halts. Not a good look in a corporate setting.

LTS is Not the Answer

The chaos above has led to an outcry in the community to bring some sort of order to the chaos. Thus far the community appears to be coalescing around some sort of LTS model as its preferred solution, however, the pushback from the maintainers has primarily been around the complexity of maintaining an LTS compiler release. On this point the maintainers are correct. An LTS release model implies that there are multiple supported historical releases, similar to the Ubuntu model from which most engineers derive their experience with the LTS model.

The reality is that D simply lacks the headcount to achieve this lofty goal. Backporting bug fixes to multiple supported releases is a significant amount of work that is exponentially more costly the further back in time the fix has to go. Furthermore, this level of maintenance complexity is unnecessary and likely counter-productive to the long-term success of D. In the future such a model may be beneficial but under the present limitations, it is simply unworkable.

A Survey of Peers

However, if LTS is a bad idea, what might we do instead? Perhaps looking at what other similar language peers to D might provide a suitable answer. In the name of preventing this article from turning into a book, we'll limit this survey to three popular D competitor languages: C++, Java, and C#/.NET and the survey will not be exhaustive, if it's not important enough to be listed in the Wikipedia on the language it won't included here.

C++

C++ uses a fairly straightforward release model. The language and standard library are updated every 3 years and no guarantee is provided that new library features will compile on older compilers. However, all language and library features from previous versions are guaranteed to work in the newest compiler version, thus ensuring that all prior written code compiles in with the latest version of the compiler.

C++ Release Model:

• Language and Library features are released every 3 years.

• Breaking changes are typically resolved by introducing a new feature.

• Non-breaking bug fixes are at the vendor's discretion.

• Deprecated language features are rare, and are not removed until at least the following major release, if not later.

• While surveying I could not find any examples of non-security-related bug fixes being backported into prior releases from any of the major vendors (GCC, LLVM, MSVC).

The C++ release model is fairly sedate and is frequently criticized for its slow pace, however, that same slow pace allows for code to be rewritten well in advance of the feature being removed. C++ places a high premium on stability and backward compatibility so there is little need for the concept of LTS in C++, there are just different versions. Furthermore, the relative stability of the language means that in nearly all cases, code written 10 or more years ago compiles and executes on the latest compilers with no additional effort.

Java

Java is the only surveyed example of a language that fully employs an LTS model. Throughout its history, Java the language has been highly coupled to Java the library, and as a result, it is possible to tightly couple the language and library in a single LTS release. Language feature deprecations are incredibly rare in Java. The first major language deprecation (finalizers) occurred in March of 2022, and as such, code written in Java 1 is virtually certain to compile in Java 21.

Java Release Model:

• Language and Library features are released every 6 months.

• Breaking changes are typically resolved by introducing a new library feature and obsoleting the old library feature or deprecating and removing the language feature.

• Non-breaking bug fixes are at the vendor's discretion.

• Deprecated language features are rare, and are not removed until some number of releases in the future.

• As Java uses an LTS model, security-related bug fixes are backported to the prior releases.

Java utilizes a fairly rapid release cadence that is combined with high language stability. This release model practically demands an LTS model as the library is relatively fast moving and the stability of the language makes pushing out new features relatively easy. However, it is critically important to note that Java is backed by a massive corporation that provides all the necessary resources to maintain and improve the entire ecosystem. There are teams of people who are paid to do nothing but backport fixes into older but still supported releases.

C#/.NET

C#/.NET employs a hybrid LTS model. The language itself does not use LTS, but the library does. Both the C# (and VB.NET) language and the .NET library are released one year cadence and new C# language features are utilized by the attached .NET library. The .NET Library receives an LTS release every other year, and only the most recent LTS is supported, the practical effect is that only two versions of .NET are supported at any given time.

C#/.NET Release Model:

• Language and Library features are released once per year.

• Breaking changes are typically resolved by introducing a new library feature and obsoleting the old library feature.

• Compiler bugs are resolved in the following version.

• There are no deprecated or removed from the language. Code written in C# 1.0 will compile with C# 12 (upcoming release).

• Library fixes are not backported per se, but the current version of the standard library also supports the LTS version and thus fixes are naturally available to the LTS library.

Perhaps the most interesting characteristic of this release model, is that because nothing is ever removed from the language itself, it is possible to compile code that was built on the very first version of the compiler. However, it is important to note that because the compiler is tightly coupled with the standard library, it is not guaranteed that the latest compiler will generate code capable of running on older versions of the library.

C#/.NET utilizes a yearly release model which attempts to strike a balance between the glacial pace of C++ and the rapid cadence of Java. This cadence allows for ample time to test the new compiler and libraries, the beta campaigns will usually begin about 6 months before the scheduled release date. However, the C# and .NET teams routinely introduce new features in the first 2-3 months of the testing cycle, and while the justification for this is not obvious it does allow them to bring in larger features that may have taken more time to develop As with Java, C# is also backed by a massive corporation, but even so, they have adopted a less intensive release model that requires fewer resources from both the company and the community.

A Path Forward

Having analyzed a few different release models, we can begin to conceptualize a new release model for D that strikes a balance between the resource limitations and the community's need for improved stability. To that end, I believe that a combination of the C# and C++ models would appear to be the most likely fit.

One of the major complaints is that the language itself is an ever-moving target. To that end, the first Feature Release must be used to clean up the definition of language. If the maintainers want to remove something from the language (not the library) they would mark it as deprecated for the next normal monthly release. That release becomes the last monthly release. After that release, the yearly cycle would begin. This gives the community time to prepare for the removal of those features. Once the first yearly release is cut, the language is considered "stable". The process of adding new language features begins with the next feature release cycle. After the first yearly is cut, the removal of language features would involve a significant multi-year deprecation process.

Another idea here is that D may not be utilizing the capabilities of Git to its maximum potential. The model would work like this:

• The 'master' branch is the current feature development branch.

• New features are developed in their entirety in separate feature branches that are branched from 'master'.

• Feature branches are not merged into 'master' until they are complete. The branch is left open until the yearly feature release is cut to facilitate feature-level bug fixes. Additionally, this can be used to facilitate using feature flags for experimental features included in the feature.

• When a new yearly feature release is cut, the release is branched from 'master'.

• Bug fixes are made against the 'master' branch in separate branches. Non-breaking bug fixes can also be merged into the latest release branch from the bug branch.

D Release Model:

• Yearly Language+Library Feature release cycle.

• New features are only made available in the current

• Bug fixes are always rolled into 'master'.

• Non-breaking bug fixes are rolled into the latest release branch, and optionally further back.

• Bug fix releases on the latest release branch are made monthly unless no bugs are fixed that month.

• 6/3/3 Development Model.

  • 6 months for feature development.

  • 3 months of initial testing while still allowing features to be merged.

  • 3 months of final testing, no features are added.

One of the benefits of this model is that it will simplify the process of integrating D with GCC and LLVM as those projects have a slower release cycle. Another benefit is that ample time is provided to the community to better test features before they are released. There is more flexibility for the community here as well, if the engineer wishes to use the latest prerelease version they are free to do so, they may choose to keep updated versions of the latest release branch version, or they may simply standardize on the latest release version and wait for the next release.

The end result is a pseudo-LTS model that keeps the maintance overhead reasonable for the maintainers while addressing the primary complaints of the community.

You have a community of meta-programming-crazed iconoclasts, who can and will make their entire toolkits look like Adr. -Monkyyy

One of my greatest discomfitures with the D community, and a large reason why I have kept myself relegated to the quiet seriousness of the Slack channel, is the relatively high number of religious ideologues cosplaying as iconoclasts on the forums.

By way of example, I am reminded of a conversation I had during one of the DConfs (2017 IIRC) with a well-known D contributor who shall remain nameless and Andrei Alexandrescu on the topic of Garbage Collection versus Manual Memory Management. The core of the argument revolved around whether or not you can know the execution delay you are incurring when calling a destructor and why that was superior to the indeterminate delay of a Garbage collection cycle. The nameless debater took the standpoint that only a system where total control of when and where memory was collected was a valid system due to the unknowable costs of Garbage Collection. After a while, Andrei said something to the effect of "You have no way to know how many other destructors are going to be triggered when you call even one destructor." which is, in typical Andrei fashion, distilled truth. The person responded that they absolutely could know and shortly thereafter Andrei and I gave up and moved on to different conversations.

This is a religious ideology and one that is still highly prevalent in the D community, but most of the arguments I see defending it when confronted with the flaws in the argument (ex: from examples in C++), can usually be decomposed into some form of "No True Scotsman" logical fallacy where the offending language suffered from a lack of sufficiently rigorous adherence to the ideologues One True Path. Religion masquerading as iconoclasm.

One of the more fascinating side-effects of the lack of "corporation" in the D community is that it has become something of a haven for the ideological cast-offs of other, more corporate, languages who are looking for a language that is both small enough that they stand a chance of influencing enough people to convert to their Ideology and yet large enough that they can prove the correctness of their Ideology in a statistically relevant sample size. The result of this has been that D occasionally suffers from bitter internecine ideological warfare with a depressing regularity that has set D back years in terms of actual code shipped.

This has resulted in what I've termed "Ideology Driven Development" or IDD. Which I define as: "A development process whereby one or more groups of ideologically possessed users and their maintainer allies attempt to impose their chosen ideology on the rest of the community by forcing their implementation into the language."

There are two critical components of IDD, the first of which is the omnipresence of the ideologues in the community offering up their ideology as The Solution to whatever it is that has prevented the language from gaining market share. All the community has to do is utterly embrace The Solution and the floodgates of success will open. A common corollary of this is the argument that Large Company "XYZ" (usually the one they work for) will be entirely unable to use the language until the solution has been fully fleshed out and there is no way they could consider it beforehand.

A second and equally critical component of this process are the strident and persistent calls by the Faithful for the removal of all other competing ideologies for the sole purpose of enforcing The One True Path to Enlightenment on the rest of the community. The most common reason given for this demand is that once The Solution has been implemented there is simply no reason to retain any alternatives as The One True Path perfectly covers all possible situations and retaining the alternatives will result in unnecessary code that will consume scarce maintenance resources for no additional benefit as everybody finds Enlightenment and moves their code to The Solution.

I have had many conversations over the years with the Allocator Adherents in the D community where they have made more or less the exact argument that was made above. The Allocator Adherents made a concerted effort to have the GC removed from D on the theory that because we now had Allocators, there was no need to keep the GC as everything would be handled with allocators and keeping the GC just encouraged lazy practices and would unnecessarily delay the removal of GC code in the standard library. Thankfully cooler heads prevailed and Walter kept the GC.

The point of all of this is not to promise that making D more corporate will somehow remove all ideology from the community. That would be impossible. However, the priorities of a corporation mean that they must carefully ringfence their contingent of noisy ideologues to prevent them from running roughshod over their bottom lines.

Another drawback of IDD is that when large corporate teams are evaluating a language for a project beyond "can it do the job" (topic for another article), the primary things that they look for are flexibility and consistency in the language. Ideologies are inherently inflexible ("Why would you go any other way but The One True Path?!") and routinely present stumbling blocks to the team as they leave no way to work around the inherent lack of capability that an inflexible system requires.

Large teams need flexible languages that can approach a problem from a multitude of paths while maintaining broadly consistent idioms. There is a reason that Java and C# are so omnipresent in corporate software toolkits. They are ugly beasts that everyone loves to hate but they just can't seem to quit them because they keep getting the job done with a minimum of fuss. Getting the job done with a minimum of fuss needs to be a top priority for D, and to achieve that goal, D needs to jettison the ideologues that keep fussing.

Mike Parker of The D Foundation Leadership recently posted on the DLang Forum (here) about the IVY Leadership Training that the Foundation leaders completed over the past few months and I'll admit that my initial reaction didn't amount to much more than a mental shrug. I've seen so many of these programs come and go over my career, with so little impact, that I mostly consider them to be an expensive waste of time. However, there were a few responses to the post that I found interesting. Specifically, two posts by a user named "monkyyy". (Internet handles are fun to type, don't ask how many times I had to reread that to get it right.)

The first post is mostly just a link to a blog post they wrote (here) in which they discuss their views on the future of D. And while there is much in that blog post that I disagree with, and I may write a future post about, there is one segment that practically leapt off the screen, and that I completely and wholeheartedly agree with:

You are simply not Go where you can have strict idiomatic language and the std is funded with google money. Or C++ where Microsoft with Enterprize-Grade mediocrity implements whatever API you put out. -Monkyyy

This is correct, D does not have a single corporate benefactor that provides all of the necessary financial and engineering support that is typically required to build out a comprehensive standard library similar to Java (Oracle), .NET (Microsoft), or even the more limited standard libraries of C++ (Microsoft/Apple/etc.) and Go (Google). It does have a few corporate users who contribute opportunistically but there is no steady source of resources for the boring work.

In my opinion, this has been a net benefit to D as it allowed the language to mature and grow in the direction that was most beneficial to the language itself without interference from the corporate need to show a profit on its investments. D has made several pivotal design choices over the years that have made the language incredibly useful and allowed it to earn broader industry adoption as time goes on.

The Benefits of "Going Corporate"

However, along the way D has turned down a few blind alleys that were allowed to grow out of control. And this brings us to the first benefit of making D more "corporate".

One of the major benefits of corporations is that at some point, they must ship something. And that something had better be useful to a maximum number of people because they need to bring in as much money as possible to continue to survive. This need forces corporates to make the painful decision to cut features that are either too niche to be broadly useful or are soaking up limited resources that could be better spent on other projects. It is always better to cut and pivot sooner rather than later. That said, "cutting" is often a euphemism for pausing a project until more resources become available or a more efficient solution is found, and frequently, a more efficient solution is found that ends up getting the project officially cut in favor of the new solution.

Many in the D community find this abhorrent to the point of outright blasphemy. The most frequent counter-argument I hear to something getting cut is "Without this feature the language is completely useless to [myself/my company] and [I/we] will be forced to stop using D." Setting aside the obvious hyperbole, the vast majority of the time what the author means is "I'll have to work harder to accomplish the same task and I don't want to, so I'll just move to [insert language here] instead."

This brings us to another advantage of corporations. Corporations are always balancing the need to ship something that works with the user demand to ship something that does everything they need. Without this balancing, it is far too easy to fall into the tarpit of Ideology Driven Development, which is a subject broad enough that it requires a separate article so I'll leave further discussion until then.

This brings me to the final significant benefit of going corporate. The limitations imposed on corporations require leadership to articulate a clear vision and path forward for D. Then once that vision has been articulated and the path laid out it becomes the responsibility of leadership to make sure that the whole community understands that this is the direction we are going. However, there is one critical difference, there is no requirement that the entire community agrees with said direction. This is not to say that the vision should be developed without community involvement, indeed there should be as much involvement as possible, but it needs to be stated and accepted up front that not everyone is going to be happy with that direction, and that is OK.

An example of how this might look in practice is that Pull Requests that directly work toward the desired direction receive higher priority than PRs that may add useful but otherwise unrelated work. There is no need to prevent people from contributing, but they need to understand that their work is a lower priority, and they need to know that before they even start.

Going Corporate

My view of the D Language Foundations' introduction of IVY is that IVY is nothing but a tool, no different than EMACS, and just as it's not EMACS' fault you wrote a bug, the potential utility of IVY comes not from its mere rote application, but how intelligently it is applied to achieve the goals of the Maintainers and the Community. If IVY can help the DLF and the Maintainers pull us out of the tarpits of Ideological Warfare, improve the clarity of vision or direction, or even encourage the Maintainers to start cutting sooner, then IVY will be a benefit to D and the community. IVY is just a tool, how it's applied is the most relevant factor in measuring its future success.

Furthermore, the very introduction of IVY appears to have caused the stir it did simply because IVY itself is "corporate" and anything perceived as corporate is going to cause significant heartburn among certain segments of the D community, particularly the more Ideologically driven segments. They are right to be worried. Becoming more corporate means that there is less room for Ideological Purity, it's why they left those prior languages and came to D in the first place. We must accept that there will be a noticeable number of folks who are unhappy with any new direction and may choose to leave. In my opinion, that is an acceptable loss, because while these folks can be prodigious contributors, they are also very willing to obstruct anything that does not support their Ideology, even to the detriment of the broader community.

When Mike Parker gave his talk on the history of D at DConf 2022, he wisely ended the talk at the "D is a Town" phase of community development. One of the major differences between a Town and a City is that the Town does not provide all the services the average human consumes, your basic needs are met, but you are likely to need to travel to another (bigger) Town or even the City to gain access to the most advanced/complicated/expensive services you might utilize.

D is not that City. Yet. But it absolutely can be. To make that leap to a City D must focus on direction over ideology, consistency over appeasement, and expanding services over expanding the language. Successfully executing on that is going to require that D begin to take on corporate practices, of which IVY is only one, and arguably not even the most important, but if it moves us towards becoming the City we need to be, then I am all for it. Finally, adopting a more corporate tone will make it easier for corporations to adopt D, and that adoption will be critical to expanding the services that D provides.

In a future post I will talk about what expanding those services may look like but for now, let's ship some code!