Sync documentation

For motivation and an overview of why to use synchronization, read this article.

This section discusses how to use the sync framework and some details on how it works under the hood.

Sync runtimes

You typically run StrataCode with a single list of layers, some of which may be limited to run only on the client, some only on the server and the rest run on both. The layer definition file specifies it's process and runtime requirements and the system automatically partitions the stack of layers into a set of stacks, one for each process or runtime. Framework layers add additional processes (see IProcessDefinition) - e.g. jetty.lib adds javaServer and swing.core adds javaDesktop. They can also add additional runtimes e.g. js.lib adds the "js" runtime (see IRuntimeProcessor). At this time, layers also set up synchronization rules between processes, and configure the system to start all processes or generate scripts to be run to deploy code and/or run the code.

At build time, the type systems all processes are available to detect remote methods, either because the remote method only exists in another runtime or an annotation indicates we should make a remote method call to another runtime for this call.

The definition of a process indicates if it supports synchronous remote calls or not. When asynchronous calls are not available, an attempt to make a synchronous call generates an error. It's possible though to run async remote methods in data binding expressions so there's a natural conversion from synchronous to asynchronous calls that's still readable.

SyncMode

The synchronization framework implements client/server communication in an automatic or directed way, based on how you configure your frameworks. The @Sync(syncMode) annotation enables or disables synchronization for a given layer, type or property.

You can set the sync mode explicitly using the @Sync annotation, inherit it from the current layer, or have it computed automatically based on the which types and properties exist in more than one runtime. This gives you complete control over which objects and properties are synchronized without having to specify @Sync on every class just by how you structure your code in layers.

When you do need to configure the Sync mode manually use @Sync:

  // Turn off synchronization 
  @Sync(syncMode=SyncMode.Disabled)
  public class MyClass ...

The syncMode may be one of:

To override the type-level syncMode for any given property, just set the @Sync annotation on that property.

A property inherits the first @Sync annotation of the first type which sets @Sync mode after it's defined in a type hierarchy. This lets you change the syncMode in the type hierarchy, e.g. turn it off for a base class and all of the properties defined in that base class, then turn it on in a sub-class so any new properties added will be synchronized. If your base classes do not set @Sync, even the base class properties will inherit the @Sync on the subclass.

If there is no @Sync tag on a type, it uses the @Sync attribute defined for the layer. If there's no @Sync tag for the layer or the type, the type inherits the @Sync annotation defined on a base-class.

By default, setting @Sync on a type will not affect inherited properties from a base class. You can change that by setting @Sync(includeSuper=true), or by adding a property assignment or just 'override propertyName' in the subclass. Whenever a type the layer of a type refers to a property, it's @Sync attribute will apply, even if the property is not defined in that type.

If no @Sync annotation is found for a property, type, or layer that property is not synchronized.

Destinations

By default you are synchronizing to the default destination - the server if you are on the client and the client if you are on the server. To sync between servers, there is a way to configure destinations. In this case, the server will generate different sync profiles for each destination and manage different sync states for each destination.

Scopes: managing object lifecycle

Scopes are a feature of the StrataCode code generation system that let you manage an object's lifecycle - i.e. how it is created, how references are resolved, and when it is stopped and recycled. They let you use declarative objects, with a single name space that are created and deleted at different times (e.g. per-request, per-user, per-session, etc), and perhaps driven by additional runtime parameters (e.g. per-merchant, per-store, per entry in a list). StrataCode manages the object lifecycle using code generation, so your code is consistently run in a context that makes sense with minimal awareness of when objects are being created and destroyed. Application programmers generally include framework layers that define their desired default policies.

Scopes make StrataCode's synchronization system flexible and powerful because the scopes let you share data in a flexible way, that adheres precisely to security and business policies enforced by the chosen framework.

The default scope is called "global" and is equivalent to using static variables in Java - one per process. A web server runs more than one application so it supports appGlobal to offer a way to store data that's local to that application. It also supports a session scope that manages objects which are per client browser and appSession which are per-app per session, window which are per-browser window and request scope which disposes of instances after each request for stateless applications.

You can use scopes for even more dynamic lifecycles such as per-list item. In that case, you can inject additional variables for the list index and current list item.

Scopes are a flexible hinge point so frameworks can add scopes like per-product catalog, per-inventory source etc. Even templates which are written for a single catalog can then be used in a multi-catalog scenario as long as they are rendered in the context of a given product.

Scopes are a safe name-space sandbox for the current context to access all of the types and properties it needs, declaratively in a strongly-typed compile time model without worrying about lifecycle or dynamic behavior. At the same time they can implement rich dynamic behavior: organizing cells into a grid, or line-items in an order. When you change the scope of an object in a subsequent layer, they let you reuse logic in new contexts - e.g. adding multi-tenant or multiple shipping addresses as a feature by flipping a switch.

To implement a scope, framework code has the necessary hooks available so they do not require much code at all. A new scope can use thread-local state to pass parameters to code placed into the generated getX methods to lookup the right instance. A scope can add variables to the object's creation. In this case, you frequently generate a method to create a new instance passing those variables (e.g. the current list index, the current item). The framework manages creating new items, updating those properties, etc. behind the scenes so your code is simple and declarative and always reflecting the current context.

Scopes on inner objects

When you attach a scope attribute to an inner object, it basically allows that scope a chance to generate the getX method defined as part of the parent class. If it's a static inner class, it works just like a scope on a top-level class.

When it's a scope defined on an inner instance class, it must first validate that it's allowed in that situation. It has the option to use the parent instance as part of it's creation, get the id from the parent and do a global lookup into a different system to retrieve the instance, or whatever you can do as part of the getX method. It can store a variable in the parent instance to cache it or not. This is a flexible hinge point for framework developers to make sure the returned instance is the right one and valid in the current context.

Scope APIs

At runtime, framework code or code generated from framework code can use the scope apis to control the objects in each scope. These APIs include the registry for which scope exist. There's a method to resolve an object instance given it's type name (for a global object reference). You also can find all of the child objects of a particular parent that have a given scope.

Scopes can force the use of static inner objects so the system generates static getX methods. That lets you nest objects with scopes inside each other without fear of dependencies or data leakage - e.g. if you embed a per-user instance as a child of a global instance, as long as the global instance is only accessed in the context of a user's session, this works properly without data being stored in the global instance itself that belongs to the user.

Objects and classes can be assigned a scope via the @Scope annotation or the scope<scopeName> operator. Some scopes will add additional fields to the current context which you can use in your object definitions. Just as with other objects, the framework manages the create and destroy calls for you. Framework developers can attach default scopes for classes with a specific base type, or defined in a specific layer. In this way, a framework can isolate a sandbox for a specific type of application code, and use that code in very flexible ways, all using simple declarative code.

SyncContext

The SyncContext stores all of the synchronized state for another process, or a scope shared by more than one process. When it is shadowing another process, it keeps a cache of its view of the other side's view of the state. When it is shadowing a scope, it keeps a cache of the state up to the last change event it's propagated.

On the client, session, window, and global scopes are collapsed into one SyncContext because each browser is only managing one window and one user's session. On the server however, each scope may have its own SyncContext or use it's parent's syncContext.

The SyncContext organizes changes in SyncLayers. These record changes for a given batch for each sync group. When an instance is synchronized, it is added to the SyncContext. The SyncContext adds property change listeners on all synchronized properties and registers the initial state. Any property values which implement IChangeable have a different listener added to them that listens for change events on the current property value as well. In this case, we can queue up a change even when the properties setX call has not been made. Instead, maybe an element of a list was changed.

When property changes come in for a given instance, they are applied to the SyncContext according to the current syncState. This is a global, thread-local status flag that's set by the framework. It can have one of five values:

You can save and restore the SyncState via the method SyncManager.setSyncState though typically this should only be done by framework code.

The SyncLayer records property change events as well as object creation events. It keeps all of the changes in order and detects when a change has been overridden and omits any changes which have been superceeded.

IChangeable

The StrataCode data binding system supports the IChangeable interface for objects to issue events generated programmatically. You use IChangeable both when you define an object like sc.util.ArrayList which updates by value or an object which updates all of its properties at once. When StrataCode detects an IChangeable on object "a" in an "a.b" reference chain, a listener is added to update that expression when object a calls Bind.sendChangedEvent(a,null). If the value of "b" is IChangeable the binding will also update when it's value is fired.

The synchronization system similarly listens on IChangeable in both ways. When the default event fires, all properties of that object are "refreshed". If their value has changed, they are synchronized. If the default event fires on a property value, the owner property is refreshed.

Cloneable

Property types that are synchronized should be cloneable using Java's normal clone method. An initial copy is made and stored so the sync system can revert changes and also reconize the deltas in what has changed to send diffs in a more concise format.

SyncManager

Each SyncManager manages the sync state for a given destination. The client only has one SyncManager, the server will have one for all clients, and one for each server to server connection.

The SyncManager stores the list of sync types. Each sync type has a list of sync properties.

The SyncManager also manages adding of synchronized instances. As instances are added to the sync manager, it is either given or finds the right scope for that instance, and places the instance under control of the proper SyncContext.

Synchronization format

The SyncLayer serializes itself using a SerializationFormat which enables serialization of the sync layer, and the "applySyncLayer" process which deserializes the sync layer and applies it in a SyncContext.

Currently two SerializationFormats are supported: stratacode and json.

In both formats, the client sends down a layer of changes, the server responds with its own layer of changes.

This format uses a package declaration, modify definitions, property assignments, field definitions for new objects when there are constructor args, otherwise object statements. It's basically a layer of changes that are applied on the remote side so it can support simple state serialization as well as code updates. It's also designed to be readable for easy debugging.

Frameworks can customize the handling of a particular instance type by registering a SyncHandler for an instance. The SyncHandler can substitute a different instance or override how the value is turned into a string in the language format. The SyncHandler can generate code to add or remove an element from a list or use the built-in features provided in the SerializationFormat.

Using the stratacode format makes it easier to read the changes going over the wire because the serialization format is readable (though generated) StrataCode.
When you enable trace and verbose on the SyncManager you can easily detect properties getting synchronized that should not or vice versa. You also have diagnostics up front as which types and properties are synchronized and with what options.

When the server has changes to send to the client it can either use JSON, or it convert the StrataCode language into Javascript where you can just eval the JS. In general, the JSON format will a lot faster for the server but perhaps the javascript version will in some cases be faster for the client.

When your updates include code changes, as detected by the StrataCode refresh process, those are processed with the same system. In this way, your client application can stay in sync, even as you are making updates to the code on both the client and server (as long as the frameworks support updating the classes which are modified).

Sync groups

By default all properties on all types are updated via the same sync operation. In rare cases, you might want to update some properties without updating others. In these cases, you use:

  @Sync(groupName="highPriority")
  class HighPriorityType {

  }

At the API level, you call SyncManager.sendSync("highPriority") explicitly to sync just the properties in that group.

Traceable security

StrataCode only allows your client code to manipulate objects and properties which were declared to be synchronized in the code. This is typically code that's in client-server shared layers so it's pretty straightforward for developers to keep it partitioned from the server-only code. All client/server communication is traceable on compile-time, typed interfaces. You can quickly audit and incrementally update the exposed interfaces from client to server.

Initial layer

The SyncContext maintains an initial layer on the server which represents the initial state of the objects created on the client. Any changes made are stored both in the initial layer and in the current new changes to be sent on the next sync. That way, on an ongoing basis, each sync only contains new changes. But when the client refreshes the page, the initial sync contains all changes made so far in that session.

This lets the synchronization mechanism retain all application state - even transient user inteface state that is not stored in the database. That state is refreshed as part of the initial state of the application automatically.

Failover

When the server fails, the sync command won't find a session which matches this client. It returns an error code which causes the client to sync its state back to the server, restoring the session, current navigation, in-progress forms, etc. By keeping the state for the application on both the client and the server at the same time, and in sync, the application gets fault tolerance without requiring a stateless design backed by a database or distributed cache. If the browser refreshes, even fine-grained application state is restored. If the server session goes away, you pay a small one time cost to restore the session from the client. So not much overhead for the programmer, runtime system, or user experience to robustly deal with failures. Because the client and server stay in sync, they can exchange minimal information, decreasing latency and server processing overhead.

Debugging synchronization

If you are using automatic mode and things are not working, first run with -vsa to turn on all synchronized logging (or -vs for a summary). This adds additional messages to the browser's console log and the server's debug log. The dialog between the client or server is designed to be readable whether you are using 'json' or 'scn'.

Look for calls to addSyncType and addSyncInst in your generated class. It's helpful run a search through the entire generated source to be sure everything you want synchronized is and vice versa. This is particularly useful for layers using @Sync(syncMode = sc.obj.SyncMode.Automatic). You can adjust what gets synchronized by adding @Sync annotations to override the defaults.

Enable -vl or -v to see the initial layers that are in each runtime. This helps determine which layers are overlapping and displays which layers are marked with 'automatic' sync mode.

The code which records a synchronized change will typically run from SyncManager.SyncChangeListener. You can set a breakpoint in the valueInvalidated method and understand why a change is or is not being recorded.

It's helpful to set breakpoints in setX and getX methods. You can walk up and down the stack, even into data binding calls because bindings turn into helpful 'toStrings' in the debugger.