Avoiding copies for customizations

by Jeff Vroom

Question: Have you ever had to copy lots of code to customize it, then struggled long-term to maintain the copies?

I've seen this strategy bury more than one company. They copied, or branched in source control and changed large amounts of code to quickly make changes for the next new customer. When you don't have time to make a nice plugin, refactor using object oriented principles to share the code, you can generate technical debt quickly. Pretty soon, your small engineering team can no longer maintain feature demands for existing customers while making customized copies for new customers. Or some platform evolution is required and you have to re-exert much of the same effort to migrate each your customers. If you follow this "slash and burn" style development, as you add customers at some spoint you will lose the ability to maintain the core product and all of the customizations in way that's competitive with other platforms. You better hope your customers feel locked in and can pay for increasing levels of services. The more competitive the situation, the easier the migration, and the faster the product evolution, the more likely you'll lose in the long run.

Most smart engineers take the simpler approach and use flags, or ideally the feature toggle pattern. Even if you engineer your feature toggles to allow them to be configured externally, without modifying the core application code, this approach runs into limits as you add more customers with disparate needs.

To support more customization without layers you'd use some mixture of plugin design patterns, object-oriented development, and inversion-of-control containers (or IOC). Let's look and the strengths and weaknesses of those patterns.

Plugin frameworks

Wordpress is a great example of a plugin framework which offers built-in customization points (aka hinge points): specific API contracts used you can follow to extend or modify the behavior of the UI from separate code. They are carefully designed to solve use cases customers need and typically give the plugin the ability to add new fields, menu items, document types, or more. Plugins can customize page layout and style sheets in many cases.

But it takes a lot of work to design, build and maintain a great plugin framework. It's like a mini-application framework by itself with install, upgrade, compatibility constraints, ordering, uninstall, etc. Plugin-frameworks are so successful it's worth the effort for some application areas where there's a lot of custom requirements (e.g. content management and e-commerce). Still it is very common to run into the limits of what you can customize in a plugin, or have version problems upgrading. It takes a lot of planning and coding on both sides to make it work when situations get complicated.

Object oriented

When you can't find a plugin platform that matches your needs, your next option is to use object oriented programming APIs with inheritance and essentially build your own plugins using code. Instead of copying code, refactor so the customizations are separated in a subclass. But object oriented programming by itself is inadequate to make easily customizable systems. Most frameworks are build using a factory pattern to separate the use of an instance from it's concrete type to allow the type to be customized. But this pattern breaks as soon as the second plugin you install into one application needs to customize the same class.

Inversion of control

In the Java world, the most common and powerful factory pattern is IOC - inversion of control - as implemented by Guice and Spring. This pattern makes it easy to configure the instance class used for any given named reference. Through the configuration (either annotations, properties, XML files or markup) the IOC container builds a graph of component instances, wiring everything up.

To customize with an IOC container, only the configuration is copied and updated for each new type of use. As long as you maintain the compatibility of the published APIs (which includes any APIs used in your configuration) you won't break applications that use your code as you go. When you get a new version of the configuration you copied, you need to merge in any changes which are now required by the contract.

That's a lot to think about on each code change so it's no wonder that rigorous O/O design with IOC component configuration does not work well when your major corporate goal is "time to market". It takes careful planning for customization hooks. You need to refactor as new requirements emerge in the planning process which involves a lot of code rewriting. Features move from code to configuration and back again. The developers of those components must take care with each change to maintain compatibility for old configuration files or upgrades are not seamless. It's in particular very difficult to seamlessly add new features - usually, the developer must modify their copy to add configuration required by the new feature.

There's another problem with inversion of control when used in plugin-frameworks. When you replace a class with your own customized implementation, you've just introduced a conflict with other plugins that have made the same choice. Whichever one of you deploys last will continue working and the other breaks badly without warning. What's needed is a way for each new plugin to layer itself into the previous model, but in a statically typed way to ensure compatibility of the type system at least.

StrataCode layers

StrataCode provides a new option that lets you code rapidly - essentially like you do with Java, Spring, JPA, etc. today but less verbose, easier to read code. You use simple classes and objects for your application code. Just like Java classes, objects can be nested to form hierarchical components. StrataCode's code-generation will convert this all to a nice readable, debuggable Java file via incremental code-transformations.

For simplicity, you can configure fields by directly initializing them. At some point, you might split that initialization out into a separate layer or simply override those values in customized layers. This works for object instance or subclasses. To define a plugin interface, you just choose which types, properties and methods to expose. Layers complement the standard object-oriented design patterns that can be used with your code. When clients are given the ability to modify a type, they can redefine the behavior of that component - including inserting new children or even replacing a child component. Plugin developers see a static-typed set of APIs that can use, override and extend with much less work on your part.

There's less planning for future customizations during design and fewer design decisions about what's in code versus what's in configuration. All code and these configuration layers are traceable in the IDE for easy find and replace. After each code change, you can quickly validate all layers that use that code to detect structural integrity problems without running the code.

With layers, knowing you can make customizations later, you write code that's easier to read.

Assembling component graphs

One of the reasons developers use an IOC framework is to allow configuration of arbitrary graphs of components. Sometimes, as you make references between objects, you need to create a cyclic or upstream reference - i.e. reference a component that has a reference pointing back to the original component. You can't express that with Java's field initialization and there's no way to fix it when the problem arises. With StrataCode, you can add the @Component annotation to the class (or a sub-class) to indicate that it may have cyclic references. It modifies the source code for this class so it supports cyclic references. Lots of code will "just work" but sometimes you'll need to move code from the constructor into the init method if it depends on a field which cannot be initialized because of the cyclic reference.

Runtime or compile time configuration?

Most IOC systems have some runtime overhead - parsing and applying the configuration every time you start the application. With StrataCode, you can use dynamic layers when you want configuration applied at runtime, and compiled layers when you want to avoid that overhead (or are unable to include the StrataCode dynamic runtime library in your application). This organization provides tunable runtime-efficiency and coding-time agility. You have less code which is easier to navigate.

Arbitrary customization

Let's quickly return to the scenario where your customers need rapidly customized applications and fast time to market. With StrataCode layers, you can build simple domain models and wire them together with statically typed references. As your customers need custom changes, you can augment any feature without copying code. Each customer can have their own layers which are unique to their instance of the application. As new customers have overlapping requirements, these layers can be reused or further customized. You refactor the code as you go to minimize the total code footprint and keep track of how your system has been customized. You never have to say "no" to new custom features and never have to copy code unnecessarily to support two variants of the same system.