Less known Solaris Features: Remote Mirror with AVS - Part 2: The fundamentals of Remote mirroring with AVS

Some words about Sun StorageTek Availability Suite at first: We´ve opensourced the product quite a while ago. While it´s a commercial product for Solaris, we´ve integrated it into Solaris Express Community Edition and Developer Edition.

Implementation of the replication

The integration of the replication into solaris is relativly simple. The replication is done by a filter driver in front of the storage device drivers. You can think of it as: Data destined to the harddisk is intercepted by this driver before it goes to the disk to handle the data accordingly to the configuration. This driver can write the data directly to an equivalent driver on a different system or to a queue for later transmission. Furthermore the data is directed to the normal path to the local harddisk.


Some definitions are important to understand the follwoing text:

Synchronous Replication

AVS supports two modes of replication. The first mode is the synchronous replication. From the standpoint to ensure that master and replica are identical volume, this is the best method. A system call writing to a replicated volumes doesn´t complete until the replica has confirmed, that the data was written to the replicated volume. By this mode of operation it´s ensured that all data commited do disk are on both volumes. This is important for databases for example. But it has an disadvantage as well: A single write takes much longer. You have to factor in the round trip time on the network and the amount of time needed to write the data on the replicated disk.

Asynchronous Replication

Out of this reason, asynchronous replication was introduced. You can use it in enviroments with less strict requirements. In this mode the write system calls return when the data has written to the replicated volume and to a per-volume queue. From this queue data is sent to the secondary host. When the secondary host writes the data to the disk, it acknowledges this and the primary host deletes the the data from the queue. This method has the advantage of introducing much less latency to the writes of the system. Especially for long-range replications this is a desired behaviour. This comes at a price: In the case of a failure, a commited read from an application may reside in a replication queue but isn´t transmitted to the secondary host.

Choosing the correct mode

Choosing the correct mode is difficult. You have to do an trade-off between performance and integrity. You can speed up synchronous replication by faster networks with a few hops between both hosts, but a soon as you leave your campus this can be a costly endeavour. You should think about one point: Is the need for “up to the last commited” transaction a real or an assumed need. My experience from the last seven years Sun: As soon as you show people the price tag of “99.999%” availability, the requirements get more differenciated. Or to bring it in a context with remote mirroring: Before ordering this extremly fast and extremly expensive leased line you should talk with the stakeholders with they really need synchronous replication.


Okay, the replication takes care of keeping the replicated volume and the replica identical, when the software runs. But how to sync both volumes when starting replication or later on, when the replication was interrupted? The process to solve this is called synchronization. AVS Remote Mirror knows four modes of replication:

Okay, but what mode of replication should you choose? For the first replication it´s easy … full replication. After this, there is a simple rule of thumb: Whenever in doubt of the integrity of the target volume, use the full replication.


Okay, there is another important term in this technology: Logging. This has nothing to do with writing log messages about the daemons of AVS. Logging is a special mode of operation. This mode is entered when the replication is interrupted. In this case the changes to the primary and secondary will be recorded in the bitmap volume. It´s important that Logging don´t record the change itself. It stores only the information, that a part of the volume has changed. Logging makes the resynchronization of volumes after a disaster more efficient, as you only have to resync the changed parts of a volume as i´ve explained for the mechanism of update replication before.