Monday, August 19, 2013

Responsible Entities - Tracking and reporting changes

In the introduction episode, I wrote about an Idea for an architectural model to create data entities that exhibit data related behaviors particularly Population, Validation, and Tracking and Reporting Changes without using JPA. In this episode we take a closer look at the model and the implementation of "tracking and reporting changes".

Basics

In this model, entities are divided into two categories; those that are not going to be persisted back to the database and are usually used to transfer data to higher layers of the application stack (e.g.: display a list); and those that will potentially be persisted back to the database, in other words, "Read-only" and "Editable" ("Persistable") entities. Being "immutable" is one of the key characteristics of the Read-only entities. A "search result" is a good example of a read-only entity. When an item from within the search result is selected to be modified (and saved), an editable entity is used. Although, it isn't very clear from this description that how an item in a read-only entity (search result) can be edited and persisted. This brings me to other aspects of this model:
  1. In the search result example, the result is a "collection" of items each of which is a read-only entity, as well. That's essentially another way of categorizing entities; "Collection" and "Single". Table 1 summarizes the categories.
  2. In order to persist information (using entities), one should always use editable entities. In the search result example, when an item is selected to be modified, an editable entity can be created via either making a copy of the selected read-only entity (in the presentation tier) or reloading the selected item's data from the database (using some sort of unique identifier) onto an editable object. The approach depends on the design and frequency of the updates. We are either "optimistic" that the data hasn't changed since the search result was created or, well, we don't. Either way, the point I'm trying to make is that entities' persistence characteristics don't change. A read-only entity will never become editable. Or editable entities can't be aggregated into read-only collection entities.

Table 1: Categorising Data Entities
Read-only SinglesEditable Singles
Read-only CollectionsComprised of / Aggregate intoNot allowed
Editable CollectionsNot allowedComprised of / Aggregate into

Putting it all in a class diagram, we have the following two figures.

Fig.1: Base classes for Editable entities
Fig.1: Base classes for Editable entities
Fig.1 depicts the two base abstract classes for individual (single) and collection editable entities implementing (realizing) "Editable Entity" interface using which we mandate the basic necessities of an editable class; tracking changes and validation.

Fig.2: Base classes for Read-only entities
Fig.2: Base classes for Read-only entities

Fig.2 shows the read-only counterparts.

Upon first glance, you might notices the following aspects of the model ("WHATs"):

  1. These 4 classes are good candidates for being abstract (which they are).
  2. Single editable entities track changes differently than the collection editable entities. Single entities have a built-in attribute called "changes" in which changes to other attributes are recorded (will explain the "HOW" shortly). Whereas collection entities track adding and removing items.
  3. Collection classes are generics and members are BaseEditableEntity objects (single editable entities) for editable collections and BaseEntity objects (single read-only entities) for read-only collections.

Tracking and Reporting Changes

As mentioned in previous part, tracking changes in data entities has been proven useful in implementing UI interactions as well as addressing concurrency issues. For instance, in the search result example, if user chooses to edit and save an item, you might want to know if something was actually changed. Or you might want to persist only those fields that are modified in order to mitigate the risk of concurrency issues (if you are using an optimistic concurrency control method). Of course, it goes without saying that tracking changes is only applicable to editable entities.

Single Editable Entities

Single editable entities are either created from scratch and persisted (new objects) or loaded (from a read-only entity or database), edited and then persisted (existing objects).
A new object doesn't need much tracking except for the fact that it should be marked "new". Hence BaseEditableEntity has an "isNew" attribute which is false by default and is set to true in its protected constructor. BaseEditableEntity's constructor is then called within single editable entities' constructors.

package ca.amir.entities;

import java.io.Serializable;

public class BaseEditableEntity implements Serializable, EditableEntity { 

 protected boolean isNew = false;

 protected BaseEditableEntity() {
  isNew = true;
 }
}
Snippet 1: The isNew attribute is used to flag new object when built from scratch

As for existing objects, single entities inherit an internal Map<String, Object> collection called "changes" in which the "key" is the name of the changed attribute and "value" is its new value. To populate the collection whilst the values of entity's attributes change, the first method that comes to mind is to have a function to call to which we pass the field name and the new value. Although working, this method requires calling the function everywhere an attribute is being set. I prefer using Aspect Oriented Programming, instead. That is, seamlessly intercepting the calls that set the value of an attribute and if necessary, registering that change with the "changes" Map. Also, we may want the flexibility of tracking changes in certain attributes of an entity. Hence, we could demarcate the attributes that we would like to track with a custom annotation and have our "change-tracking Aspect" to only intercept the calls that set the annotated attributes. Following code snippets summarize the usage of annotation and AOP (utilizing AspectJ) together for this purpose.
package ca.amir.aspects;

import org.aspectj.lang.Signature;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.ProceedingJoinPoint;

import ca.amir.entities.BaseEditableEntity;
import ca.amir.entities.BaseEntity;
import java.lang.reflect.Field;

@Aspect
public class ChangeTrackingAspect {

 @Around("set(@ca.amir.entities.ChangesTraced * *) && target(t) && args(newVal)")
 public void aroundSetField(ProceedingJoinPoint jp, Object t, Object newVal)
   throws Throwable {
  Signature signature = jp.getSignature();
  String fieldName = signature.getName();
  Field field = t.getClass().getDeclaredField(fieldName);
  field.setAccessible(true);
  Object oldVal = field.get(t);

  if ((oldVal == null ^ newVal == null)
    || (oldVal != null && !oldVal.equals(newVal))) {
   ((BaseEditableEntity) t).objectChanged(fieldName, newVal);
  }

  jp.proceed();
 }
}
Snippet 2: The aspect that intercepts calls

Line 15 marks the "aroundSetField" function of the aspect class to run when the value of an attribute with any name and of any type annotated with "ChangesTraced" annotation (Snippet 3) is set ( set(@ca.amir.entities.ChangesTraced * *) ). The function compares attribute's old and new values (lines 24 and 25) and if they are different the aspect calls the "objectChanged" method of BaseEditableEntity (Snippet 4) which registers the change.

package ca.amir.entities;

import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;

/**
 * Custom annotation used to demarcate all attributes
 * of an editable entity for which changes are traced
 */
@Retention(value=RetentionPolicy.RUNTIME)
public @interface ChangesTraced {
}
Snippet 3: "ChangesTraced" custom annotation

package ca.amir.entities;

import java.io.Serializable;
import java.util.Map;
import java.util.HashMap;

public class BaseEditableEntity implements Serializable, EditableEntity { 

 protected Map<String, Object> changes;
 protected boolean isNew = false;

 protected BaseEditableEntity() {
  isNew = true;
  changes = new HashMap<String, Object>();
 }

 public final void objectChanged(String fieldName, Object newValue) {
  changes.put(fieldName, newValue);
 }
}
Snippet 4: "objectChanged" function registers the change


There is a bit of room for a few improvements here which I'll point out and leave the implementation to readers:
  1. This data structure doesn't take undo function into consideration. For instance, in a multistage wizard-like form, it's possible for user to go back in steps and undo a particular change before submitting the entire form. In that case we'd want to revert its corresponded recording.
  2. "objectChanged" function, which is invoked by the Aspect when a change occurs, is generic and isn't aware of the context. But if your tracking requirement extends to editable entities themselves, this method won't suffice. That is, if in addition to recording a change a context specific action is required, this centralized function won't be the correct method to implement it. Hint: Observer pattern.
  3. Different layers in the application stack would most likely want to know about registered changes in an entity. Presentation tier for interaction purposes or data tier for persistence purposes. Usually a generic way of reporting changes will suffice all layers (e.g.: an iterator method). However, there are times that different layers require different views of those changes (as if an entity maintains different contract with different layers). For instance, there might be an optional write-only data attribute in the entity which obviously shouldn't be reported as part of the changes made to the entity when requested by the presentation tier. On the contrary, entity should report it when changes are being persisted into the database. This level of sophistication is only required when you don't control all the layers.

A word on reuse

You might encounter single editable entities with very similar read-only counterparts. For example, a read-only entity to display profile information (fetched from database) and an editable one to edit and update the profile (back to database). It's very tempting to invent different ways to share the data attributes of the two entities; for instance, aggregating a third class containing shared attributes in both entities. Although this enables us to reuse the attributes it couples the read-only and editable entities. Coupling isn't a problem as long as it doesn't affect the fundamentals of the model. Using a third class however, does. Attributes of single editable entities that are tracked for their changes should be annotated with the custom annotation. That isn't the case in read-only entities. Even though read-only entities will ignore the annotation and this contradiction will work for the time being, it should be avoided since this is an indication of an extensibility issue in the model.

Collection Editable Entities

Here I've made the assumption that collection editable entities are containers with no data attributes that need tracking. An example should clarify this.

Fig.3: "OrderLines" collection is merely a container
Fig.3: "OrderLines" collection is merely a container

In this example, "OrderLines" is merely a container with no data attributes of its own which we'd want to track. However, if you have collection entities with data attributes for which you'd want to track changes, then in addition to the change tracking method I'll explain in this section you need to incorporate the method explained in previous section (using AOP and annotation) into your collection entities, as well.

Collections entities with such characteristics only need to track "removed" items. That's because collection members track their own status when they are "new" or "modified". Hence, in addition to the inherited "rows" collection in which members are stored each collection editable entity has a "removed" collection. Removed items are simply moved from "rows" to "removed" ("remove" function) and vice versa when remove is undone (similar to single entities, undo function is currently missing).
package ca.amir.entities;

import java.io.Serializable;
import java.util.List;

public class CollectionBaseEditableEntity<T extends BaseEditableEntity> implements Serializable, EditableEntity {

    protected List rows;
    protected List removed;
 
 public final T remove(int index) {
  T obj = rows.remove(index);
  removed.add(obj);
  return obj;
 }
 
 public final boolean remove(T itemToRemove) {
  if(rows.remove(itemToRemove))
   return removed.add(itemToRemove);
  
  return false;
 }
}
Snippet 5: Tracking removed items

XML Serialization

All entities are serializable. However, if XML serialization is one of your requirements (e.g.: JAX-WS) then you need to demarcate the entities with proper annotations. Please see @XmlType, @XmlAccessorType and @XmlElement for more information.
Also, since collection entities' internal collection attributes ("rows" and "removed") are generics, their actual type will be lost during serialization (type erasure). To get around this issue, CollectionBaseEditableEntity has "transient" getter methods that have to be overridden by entities and annotated with the actual type.
package ca.amir.entities;

import java.io.Serializable;
import java.util.List;
import javax.xml.bind.annotation.XmlType;
import javax.xml.bind.annotation.XmlTransient;

@XmlType(name = "CollectionBaseEditableEntity", namespace="http://entities.amir.ca")
public class CollectionBaseEditableEntity<T extends BaseEditableEntity> implements Serializable, EditableEntity {

    protected List<t> rows;
    protected List<t> removed;
 
    @XmlTransient
    public List<t> getElements() {
     return rows;
    }
}
Snippet 6: Transient getter method for "rows" to be overridden by entities

package ca.amir.entities;

import java.util.List;
import javax.xml.bind.annotation.XmlType;
import javax.xml.bind.annotation.XmlElement;

@XmlType(name = "OrderLines", namespace="http://entities.amir.ca")
public class OrderLines extends CollectionBaseEditableEntity<orderline> {

    @Override
    @XmlElement(type=OrderLine.class)
    public List<OrderLine> getElements() {
     return rows;
    }
}
Snippet 7: Overridden getter method for "rows"

Snippets 5 & 6 depict how "getElements" is overridden and annotated with the actual type of the members ("OrderLine") in a child collection ("OrderLines").

Mapping the Map

java.util.Map doesn't naturally map to a XML representation. That is, annotating it as an XmlElement isn't going to produce a XML form using which you can build the same object back (unmarshall or deserialize). One way to get around it is custom marshalling by the means of a custom XmlAdapter. However, since I use quite a few different types of Map I decided to create a generic XmlAdapter using which I could minimize the amount of code I write for each one. The next 3 snippets show the generic Adapter, its MapType class, and the Member (Entry) class of the MapType class.

public class GenericMapAdapter<K, V> extends
        XmlAdapter<MapType<K, V>, Map<K, V>> {

    @Override
    public final MapType<K, V> marshal(final Map<K, V> v) throws Exception {
        MapType<K, V> obj = new MapType<K, V>();
        for (Entry<K, V> entry : v.entrySet())
            obj.entry.add(MapTypeEntry.createInstance(entry.getKey(),
                    entry.getValue()));
        return obj;
    }

    @Override
    public final Map<K, V> unmarshal(final MapType<K, V> v) throws Exception {
        Map<K, V> obj = new HashMap<K, V>();
        for (MapTypeEntry<K, V> typeEntry : v.entry)
            obj.put(typeEntry.key, typeEntry.value);
        return obj;
    }
}
Snippet 8: Generic XmlAdapter


public final class MapType<K, V> {

    @XmlElement
    final List<MapTypeEntry<K, V>> entry = new ArrayList<MapTypeEntry<K, V>>();
}
Snippet 9: Generic XmlAdapter's MapType class


public final class MapTypeEntry<K, V> {
    @XmlElement
    public K key;

    @XmlElement
    public V value;

    private MapTypeEntry() {
    }

    public static final <K, V> MapTypeEntry<K, V> createInstance(final K k, final V v) {
        MapTypeEntry<K, V> obj = new MapTypeEntry<K, V>();
        obj.key = k;
        obj.value = v;

        return obj;
    }
}
Snippet 10: Member (Entry) class of the MapType class


What's left to do is to use the custom adapter in conjunction with XmlJavaTypeAdapter to annotate a Map type. Obviously, you wouldn't be able to use the Generic XmlAdapter simply because you can't get a class literal from a generic type.

illegal start of expression
@XmlJavaTypeAdapter(GenericMapAdapter<String,Object>.class)
Map<String, Object> changes;
                                                    ^

The solution is, you guessed it, a subclass for each Map type.

public final class StringObjectMapAdapter extends

        GenericMapAdapter<String,Object> {
}


...................


@XmlJavaTypeAdapter(StringObjectMapAdapter.class)
Map<String, Object> changes;
Snippet 11: StringObjectMapAdapter - An XmlAdapter for a Map<String,Object>


Using Maven

If you use Maven to build and package, then you'll need to use AspectJ's Maven plugin in order to compile the aspects. However, the plugin will by default add all .java and .aj files in the project source directories. In order to avoid this problem and also not compiling aspects with maven's compiler plugin, we need to configure the plugins to pick and compile appropriate resources (AspectJ plugin and compiler plugin to compile aspects and non-aspect classes respectively).

I do these sorts of configuration in a company / project wide super POM. Following code shows the configuration.


<pluginManagement>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>2.4</version>
            <configuration>
                <source>1.6</source>
                <target>1.6</target>
                <encoding>UTF-8</encoding>
                <excludes>
                    <exclude>**/aspects/*</exclude>
                </excludes>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>aspectj-maven-plugin</artifactId>
            <version>1.3</version>
            <configuration>
                <verbose>true</verbose>
                <complianceLevel>1.6</complianceLevel>
                <showWeaveInfo>true</showWeaveInfo>
                <aspectDirectory>**/aspects/*</aspectDirectory>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>compile</goal>
                        <goal>test-compile</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjrt</artifactId>
                    <version>1.6.10</version>
                </dependency>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjtools</artifactId>
                    <version>1.6.10</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</pluginManagement>
Snippet 12: Configuration of maven plugins

This configuration instructs the compiler plugin to ignore any java file under the "aspects" directory and AspectJ's plugin to attach itself to "compile" and "test-compile" goals and only build the "aspects" directory.

In the next part we'll discuss validation and population.

No comments: