Layered build and packaging

Layers are a great way to manage builds, packages, versions, tests, deployment configurations, etc. Like Gradle, StrataCode lets you define build configuration using an object oriented, interpreted language. More concise, readable and flexible build configurations than XML. But StrataCode uses it's extended version of Java as the build configuration language so you do not have to learn Groovy and can leverage all of the built-in Java libraries you already know.

Just keep in mind, the code in the layer definition file runs at build/run time and so can't use the code in the layer itself. The layered, component framework you can use in build files reduces copying while improving customization potential for your builds. Layers make it easy to define simple modules - organized by file type or dependency rather than using a lot of project scaffolding. Modules are usually already organized by type so why not make them simpler? You'll end up with more modules that can be managed by more people.

Complete control over your code

StrataCode is more than just a build tool though. It's a complete code-processing environment with the ability to read and make incremental changes to your code. Unlike tools which manipulate your application's byte-code on-the-fly, you can annotate your code to enable structured code transformations, then see and debug the delta in the generated code. You have all of the runtime safety benefits of compiled, statically typed Java code and avoid "untraceable code" side-effects created by dynamic languages, aspect-oriented programming, mixins, and other approaches.

Full IDE for editing build files

StrataCode build files are written in StrataCode and are edited like normal files in IDE with code-hinting and edit-time validation.

Maven integration

Layers can include maven libraries in one of two ways. Your projects can use maven itself for builds and/or installs of dependencies, then just configure the layer to use the resulting .jar in the classpath. A lighter weight approach uses StrataCode's built-in support for maven repositories and POM files, eliminating the need to install maven and make better use of layers for customization. One interesting advantage of the layered design is the ability to switch back and forth between source layers and compiled layers on a module-by-module basis where we use fine-grained modules. This will help optimize the development experience on large projects where you can't have source for everything.

StrataCode project file organization

To start a new project with most frameworks, you use a "create project" wizard which generates the default project files from the configuration you specify. It's your job to figure out what files were generated, where the info you provided is now stored, and how to manage those configured values. Perhaps they are not the same from one developer to the other, or one deployment to the next. Maybe your framework stores these external properties in a separate file, or maybe not. If you're lucky there's some way to externalize them - via environment variables but it's always ad-hoc and varies from one framework to the next.

And the next time you upgrade your tool you may be required to manually update those files. The intent with which you made those changes is gone so there's no clear path for the upgrade. The way you want to organize your source files is defined by your framework, not by design of the developers for this particular type of project.

StrataCode preserves customization intent. With StrataCode, when a layer extends a base-layer, it picks up all of the previous layer's project files, which includes the definitions of file formats and a default directory organization for this layer. You only need to specify files or settings you need to change, and you can change any modifiable apis exposed by the downstream layer.

It's best practice to stick with 'additive changes', that do not remove exposed features of the published contract for upstream layers but there's currently no strict enforcement of that principle. You can replace one class with another one that implements a different contract, as long as the end result can satisfy all statically typed references caught by StrataCode and compile as Java.

With framework features, you can layer your configuration - so each layer contains only the properties you want to set in that project, inheriting the rest. StrataCode combines the layers and generates one or more standard project directories, usually one for each runtime or process but they can share one or more directories like 'web' for a browser based app. So your deployment configuration and tooling from that point on can be the same. This pattern both supports today's project structures, and let's you use StrataCode to make changes one-by-one. New projects use way less configuration and over time, you can use layers to reduce copies to simplify your build/runtime process. The goal is to reduce project configuration, and improve the relevance the configuration that's there.

Unlike many project and build configurations, StrataCode supports an incremental refresh which quickly finds changed files and processes them, or copies them to the build directory. This becomes particularly useful in large, multi-process environments when the manual restart time is large.

Git

StrataCode has a simple command-line integration with git for managing layers. Using layer definition files, you can change how a layer of code is processed - provided with source or compiled against the compiled binary form. You can move layers from one git repository to another, only by updating layer definition files. Because these files are statically typed and support IDE based tooling, everything is easier to manage.

Some branches in git can be avoided by creating layers which patch or replace code, then merged when they no longer are needed. You might copy some files or just add fields and override methods, create new types, etc. It can be a nice way to build a new feature because your changes are kept separate and together for easier navigation. You reduce conflicts with others changing different aspects of the same types - which would mean the same files if you did not use layers.

Layers and contracts

Each layer can be thought of as supplying a set of API contracts - classes, properties, and types. They can be delivered in source or binary form, as long as these types are available to upstream code. When you can isolate how dependencies affect developers or other members of the team, you can optimize workflows - minimizing merge conflicts and making it easier to maintain compatibility over the lifetime of the project. Layers let you separate contracts from classes, processes, and git-repositories. While the need to override or replace a type in this set is rare, without it you are missing an important low level operation. When you can manage the 'modify' operation, you can manage contracts between code, and your overall dependencies in a more efficient way. Mix and match, move and reassemble, plug and play. Some simple use cases:

Options for module packaging

In most project and build configurations, each module or project contains a lot of directories, many of which you might not need for your module. Project directories are one-sized fits all and so include tests, configuration, source, documents, documentation, etc. When a specific task requires editing lots of different files of many different file types, it might make sense to separate them by type. But most projects start out small and focused and derive dfrom some previous project that's pretty specialized. By using layers with the 'replace' operator in both your coding style and tools, you can build module types that meet the needs of your project. Then have small, custom modules instances that have the perfect directory organization.

When you organize modules by their role like tests, documentation, versus configuration you typically pick a structure that aligns better with the needs of the person who manages that information. When you combine files with different lifecycles and different roles in the same module you run into conflicts between users: developer, admin, operations etc . Files are managed by different people, released at different times, and included in different packages, organized in different source control systems, etc.

For these simpler modules, layers eliminate the scaffolding. Simply extend a layer which defines the context for your source files. It will make sure the files are deployed correctly - i.e. only run in test mode, includes in the test.jar file etc.

Layers support more flexible ways to organize files by their lifecycle, and their dependencies. In this case, you invert the directory structure - so you have test, doc, configuration, etc. groups of layers, each of which contains a simple directory of files. Instead of one module which is tightly coupled, you have the option to split off modules which are loosely coupled in a compatible way.

When you have simpler and more flexible module structures, you can create projects that are easier to manage, not just for developers but throughout the organization. Everyone contributes their "slices" of the feature set, where layers can become more feature oriented and less module or project oriented. Framework code controls how those assets are deployed. Static typing and tools keep things structured in the same way a complex object-oriented application is structured. That helps detect conflicts and gives tools for fixing them. This lets developers more flexibly organize source code amongst different git repositories and move back and forth maintaining compatibility with upstream layers.

Layered dependencies

Anyone who has worked in a big Java application is aware of how difficult it can be to manage dependencies. You have to consider not only your dependencies but the "transitive dependencies" - dependencies of your dependencies. How do you ensure compatible versions of the dependencies but also pick up the latest security fixes when they are available? How do you understand what's happening and resolve conflicts - to select the right version, even in the common case when you are inheriting conflicting rules for overlapping packages? When a critical security patch comes out, who do you trust to delegate your transitive dependencies so you pick up that patch as soon as necessary?

Maven provides a flexible but intricate system to specify, inherit, exclude and resolve conflicts between the versions you inherit from transitive dependencies. The order in which modules are sorted is determined by the order in which the dependency graph is traversed. That's based on the order of dependencies in each pom.xml file, and the number of levels of indirection before you reach a given module. So dependencies inherited that are further away are overridden by those that are closer to the first build component.

Layers work in a similar way, but simplify the specification of dependencies, and allow a more intuitive way to override them. With layers, it's the stacking order that determines precedence in all things. That order is based on dependencies primarily but there are more ways to refine the stacking order if the rare case the default is not what you want. Layers are organized into framework, application, and configuration stacks which are independently sorted. The order of the layers, and dependencies can be traced using the IDE, and quickly overridden in a way that you can carefully ensure that override only shows up in the right situations.

It's also easy to separate the transitive dependencies into a new layer - so you can include a package, but not include it's dependencies. Or switch transitive dependencies by switching a layer. That's a better alternative than excluding dependencies, since who knows when those will actually be required rules on your dependencies. This way, a required dependency is always included but these packages of dependencies can be included optionally. Dependencies can be represented as separate layers, features by themselves by with a simple layer definition file that has no code.

Build tagging

Most types of software projects require some way to tag builds with a version number, build date, commit hash, rc #, etc. To achieve the most diagnostic accuracy, developers prefer to tag the executable itself. If you store the version number in a separate file, it's not guaranteed to match what's actually running.

StrataCode offers the BuildInit annotation to provide a simple, declarative way to turn an expression that runs at build time into the initialization value for a property at run time. See the BuildTag example.

Read more and see examples in the documentation.

Generation of deployment files

The ops team typically employs tools like chef and puppet to generate configuration files that need to change at deployment time. Machine names, ports, passwords etc need to be available to the application at runtime.

The layers and code-processing are well designed to tackle this problem as well, using the same toolset as developers. Configuration moves fluidly: compiled into the application or stored on the file system, from a development git repo to an operations repo where either can copy from or layer on the other. The list of layers is all either needs to communicate the precise configuration to the other and it's all traceable in the IDE. All easily configurable with auto-configuring management UIs.