Layered build and packaging
Layers are a great way to manage builds, packages, versions, tests, deployment configurations, etc. Like Gradle, StrataCode lets you define build configuration using an object oriented, interpreted language. More concise, readable and flexible build configurations than XML. But StrataCode uses it's extended version of Java as the build configuration language so you do not have to learn Groovy and can leverage all of the built-in Java libraries you already know.
Just keep in mind, the code in the layer definition file runs at build/run time and so can't use the code in the layer itself. The layered, component framework you can use in build files reduces copying while improving customization potential for your builds. Layers make it easy to define simple modules - organized by file type or dependency rather than using a lot of project scaffolding. Modules are usually already organized by type so why not make them simpler? You'll end up with more modules that can be managed by more people.
Complete control over your code
StrataCode is more than just a build tool though. It's a complete code-processing environment with the ability to read and make incremental changes to your code. Unlike tools which manipulate your application's byte-code on-the-fly, you can annotate your code to enable structured code transformations, then see and debug the delta in the generated code. You have all of the runtime safety benefits of compiled, statically typed Java code and avoid "untraceable code" side-effects created by dynamic languages, aspect-oriented programming, mixins, and other approaches.
Full IDE for editing build files
StrataCode build files are written in StrataCode and are edited like normal files in IDE with code-hinting and edit-time validation.
Layers can include maven libraries in one of two ways. Your projects can use maven itself for builds and/or installs of dependencies, then just configure the layer to use the resulting .jar in the classpath. A lighter weight approach uses StrataCode's built-in support for maven repositories and POM files, eliminating the need to install maven and make better use of layers for customization. One interesting advantage of the layered design is the ability to switch back and forth between source layers and compiled layers on a module-by-module basis where we use fine-grained modules. This will help optimize the development experience on large projects where you can't have source for everything.
StrataCode project file organization
To start a new project with most frameworks, you use a "create project" wizard which generates the default project files from the configuration you specify. It's your job to figure out what files were generated, where the info you provided is now stored, and how to manage those configured values. Perhaps they are not the same from one developer to the other, or one deployment to the next. Maybe your framework stores these external properties in a separate file, or maybe not. If you're lucky there's some way to externalize them - via environment variables but it's always ad-hoc and varies from one framework to the next.
And the next time you upgrade your tool you may be required to manually update those files. The intent with which you made those changes is gone so there's no clear path for the upgrade. The way you want to organize your source files is defined by your framework, not by design of the developers for this particular type of project.
StrataCode preserves customization intent. With StrataCode, when a layer extends a base-layer, it picks up all of the previous layer's project files, which includes the definitions of file formats and a default directory organization for this layer. You only need to specify files or settings you need to change, and you can change any modifiable apis exposed by the downstream layer.
It's best practice to stick with 'additive changes', that do not remove exposed features of the published contract for upstream layers but there's currently no strict enforcement of that principle. You can replace one class with another one that implements a different contract, as long as the end result can satisfy all statically typed references caught by StrataCode and compile as Java.
With framework features, you can layer your configuration - so each layer contains only the properties you want to set in that project, inheriting the rest. StrataCode combines the layers and generates one or more standard project directories, usually one for each runtime or process but they can share one or more directories like 'web' for a browser based app. So your deployment configuration and tooling from that point on can be the same. This pattern both supports today's project structures, and let's you use StrataCode to make changes one-by-one. New projects use way less configuration and over time, you can use layers to reduce copies to simplify your build/runtime process. The goal is to reduce project configuration, and improve the relevance the configuration that's there.
Unlike many project and build configurations, StrataCode supports an incremental refresh which quickly finds changed files and processes them, or copies them to the build directory. This becomes particularly useful in large, multi-process environments when the manual restart time is large.
StrataCode has a simple command-line integration with git for managing layers. Using layer definition files, you can change how a layer of code is processed - provided with source or compiled against the compiled binary form. You can move layers from one git repository to another, only by updating layer definition files. Because these files are statically typed and support IDE based tooling, everything is easier to manage.
Some branches in git can be avoided by creating layers which patch or replace code, then merged when they no longer are needed. You might copy some files or just add fields and override methods, create new types, etc. It can be a nice way to build a new feature because your changes are kept separate and together for easier navigation. You reduce conflicts with others changing different aspects of the same types - which would mean the same files if you did not use layers.
Layers and contracts
Each layer can be thought of as supplying a set of API contracts - classes, properties, and types. They can be delivered in source or binary form, as long as these types are available to upstream code. When you can isolate how dependencies affect developers or other members of the team, you can optimize workflows - minimizing merge conflicts and making it easier to maintain compatibility over the lifetime of the project. Layers let you separate contracts from classes, processes, and git-repositories. While the need to override or replace a type in this set is rare, without it you are missing an important low level operation. When you can manage the 'modify' operation, you can manage contracts between code, and your overall dependencies in a more efficient way. Mix and match, move and reassemble, plug and play. Some simple use cases:
- have one version of a layer retrieve a package as source, the other via a maven artifact - switch back and forth based on the need to debug the artifact, versus need for faster compiles
- layers which do not modify API contracts (either add or modify types, methods, or properties) act just like filters - options, you can turn on/off. Even when a particular option affects several files in different parts of the product, you keep that option's code separate. And using static typing and the generated code, you can quickly navigate back and forth.
- use layers to build code quickly, then refactor into shared, client and server layers - where the overlap defines the synchronization layer. Develop as a monolith, deploy as separate services as by adding annotation layers which create new processes, add process dependencies that repartition the layer stack into separate overlapping stacks. Synchronous apis that cross process boundaries can be transparently converted in the generated code. Asynchronous calls can be run either way if they are in data binding expressions.
Options for module packaging
In most project and build configurations, each module or project contains a lot of directories, many of which you might not need for your module. Project directories are "one size fits all" for a given framework. They include places to store tests, configuration, source, documents, metadata, documentation, etc. When a specific task requires editing lots of different files of many different file types, it might make sense to create a new project to manage this organization but it's actually rare you start a giant new project. And if you organize things this way, it tends to make projects monolithic.
Most projects start out small and focused and built from some context: previous a project, or the assembly of a bunch of frameworks you know you need to include. Layers can start out with that context and inherit defaults for all the stuff and layer in files as needed. You just need to manage the additional complexity of the merge and replace. It's a coding style supported by tools in which you can build module types that meet the needs of your developers and content contributors.
Use small, less complicated custom layers with a good directory organization for each role in your project.
When you organize modules by their role like tests, documentation, versus configuration you typically pick a structure that aligns better with the needs of the person who manages that information. When you combine files with different lifecycles and different roles in the same module you run into conflicts between users: developer, admin, operations etc . Files are managed by different people, released at different times, and included in different packages, organized in different source control systems, etc.
For these simpler modules, layers eliminate the scaffolding. Simply extend a layer which defines the context for your source files. It will make sure the files are deployed correctly - i.e. only run in test mode, includes in the test.jar file etc.
Layers support more flexible ways to organize files by their lifecycle, and their dependencies. You can invert the directory structure - so you have test, doc, configuration, etc. groups of layers, each of which contains a simple directory of files. Each layer has dependencies to make them easy to manage. Instead of one module which is tightly coupled, you have the option to split off modules which are loosely coupled in a compatible way, supported by the static typing.
When you have simpler and more flexible module structures, you can create projects that are easier to manage, not just for developers but throughout the organization. Everyone contributes their "slices" of the feature set, where layers can become more feature oriented and less module or project oriented. Framework code controls how those assets are deployed. Static typing and tools keep things structured in the same way a complex object-oriented application is structured. That helps detect conflicts and gives tools for fixing them. This lets developers more flexibly organize source code amongst different git repositories and move back and forth maintaining compatibility with upstream layers.
Anyone who has worked in a big Java application is aware of how difficult it can be to manage dependencies. You have to consider not only your dependencies but the "transitive dependencies" - dependencies of your dependencies. How do you ensure compatible versions of the dependencies but also pick up the latest security fixes when they are available? How do you understand what's happening and resolve conflicts - to select the right version, even in the common case when you are inheriting conflicting rules for overlapping packages? When a critical security patch comes out, who do you trust to delegate your transitive dependencies so you pick up that patch as soon as necessary?
Maven provides a flexible but intricate system to specify, inherit, exclude and resolve conflicts between the versions you inherit from transitive dependencies. The order in which modules are sorted is determined by the order in which the dependency graph is traversed. That's based on the order of dependencies in each pom.xml file, and the number of levels of indirection before you reach a given module. So dependencies inherited that are further away are overridden by those that are closer to the first build component.
Layers work in a similar way, but simplify the specification of dependencies, and allow a more intuitive way to override them. With layers, it's the stacking order that determines precedence in all things. That order is based on dependencies primarily but there are more ways frameworks can be designed to refine the stacking order. In the rare case the default is not what you want, it's easier to configure things so the default becomes what you want making a centralized configuration change that does not affect applications in your environment.
Layers are organized into framework, application, and configuration stacks which are independently sorted. The order of the layers, and dependencies can be traced using the IDE, and quickly overridden in a way that you can carefully ensure that override only shows up in the right situations.
It's also easy to separate the transitive dependencies into a new layer - so you can include a package, but not include it's dependencies. Or switch transitive dependencies by switching a layer. That's a better alternative than excluding dependencies, since who knows when those will actually be required rules on your dependencies. This way, a required dependency is always included but these packages of dependencies can be included optionally. Dependencies can be represented as separate layers, features by themselves by with a simple layer definition file that has no code.
Most types of software projects require some way to tag builds with a version number, build date, commit hash, rc #, etc. To achieve the most diagnostic accuracy, developers prefer to tag the executable itself. If you store the version number in a separate file, it's not guaranteed to match what's actually running.
StrataCode offers the BuildInit annotation to provide a simple, declarative way to turn an expression that runs at build time into the initialization value for a property at run time. See the BuildTag example.
This is intentionally the only way build configuration is available to the compiled code. The layer definition files can define classes, objects, methods, fields, etc. but those are not visible to code compiled for the runtime.
TODO: it would not be too difficult to turn layer definition files into 'build time constants' - evaluated and replaced in the code with the build time value. I'm not sure it's worth the coding complexity it would create and the use case so far has been pretty minimal hence the BuildInit annotation.
Read more and see examples in the documentation.
Generation of deployment files
The ops team typically employs tools like chef and puppet to generate configuration files that need to change at deployment time. Machine names, ports, passwords etc need to be available to the application at runtime.
The layers and code-processing are well designed to tackle this problem as well, using the same toolset as developers. Configuration moves fluidly: compiled into the application or stored on the file system, from a development git repo to an operations repo where either can copy from or layer on the other. The list of layers is all either needs to communicate the precise configuration to the other and it's all traceable in the IDE. All easily configurable with auto-configuring management UIs.