The first installment in a new blog series, “Best Practices and Procedures in Information Archiving.”
For starters, when talking about approaching active archiving, it is good to keep a few fundamentals in mind.
- Active archiving provides the ability for content and data to be archived on a periodic basis from applications that are still being actively used within the enterprise.
- Data may be viewed and searched, and files can be easily located and retrieved.
- Active archiving can reduce storage, backup and infrastructure costs and improve application performance.
- Archived information is managed in accordance with current compliance regulations and remains accessible to users of any business application.
According to Laura DuBois, Program Vice President for IDC, “Any company that regularly manages high-volume digital assets or faces exponential data growth should strongly consider expanding the use of active archiving solutions as part of their storage strategy.”
When migrating data and content to an active archive, there three main approaches that can be taken based on the accessibility requirements of the data.
These approaches are outlined below:
The Migration Pattern utilizes single direction migration to move content from a source application into an archive solution. At that time, the source application can remove those records since they are now available in the archive. The Migration Pattern allows developers to build automated migration services that move non-active records on a daily basis. Integration team members set the configuration parameters so that the migration can dynamically migrate scoped source records out of a source system either on demand or on a schedule.
Migrations are appropriate for many use cases, including:
- Migrating data from a legacy system that is no longer used.
- Backing up a specific records.
- Making the data available to a larger set of users.
- Scaling the user interface access system to hundreds or thousands of users.
- Optimizing the source application by removing old data.
Once migrated, archived records are available to all corporate users and customers independent of the source system. Without migration, data would be lost any time tools were changed, deeply affecting productivity and compliance.
The Aggregation Pattern takes or receives records from multiple systems and copies or moves those records into just one system. Aggregation removes the need to run multiple migrations on a regular basis, eliminating concerns about data accuracy and synchronization. The Aggregation Pattern is the simplest way to extract and process data from multiple systems into a single application, reporting tool, and user interface.
The aggregation process may optionally include custom logic to standardize the information model. For example, source information may come from SAP, Siebel and JD Edwards, but the push into the archive solution may map attribute names into a standardized ERP model.
Some use cases for the Aggregation Pattern include:
- Updating the archive with source data from multiple data systems.
- Creating a dashboard that pulls data from multiple instances.
- Reporting across multiple systems.
- Building APIs that return data from multiple systems.
- Specifying master data that would be used in multiple locations.
- Applying retention and compliance attributes to multiple systems in a harmonized fashion.
The Aggregation Pattern enables the migration of data from multiple systems so it can be merged into one storage location (that also meets compliance requirements). This ensures that data is always up to date, does not get replicated, and can be available to all end users. This avoids the necessity of having a separate database for merged content and makes reports available in any format or within any repository.
The Distribution Pattern moves data from a single source system to multiple destination systems for end user access. This is generally a one-way synchronization from one archive repository to many.
This pattern may:
- Move all records from the source system to each target
- Move country specific records to each country’s repository.
- Move records based on document status. For example, active records may be moved to a local repository, but the official archive is based in a central repository.
In contrast to the Migration Pattern, the Distribution Pattern is transactional and is optimized for transferring records as quickly as possible. The Distribution Pattern keeps data synchronized between multiple systems on a minute-by-minute basis. The Distribution Pattern allows for the immediate transfer of customer data between systems. As an example, the InfoArchive distribution pattern can enable an action in a customer ordering system to immediately migrate into order fulfillment processing repository.
Some common use cases for the distribution pattern include:
- Moving content closer to users for performance advantages.
- After scanning a document at a central site, the document is automatically routed to the local processing site.
- Distributing Personally Identifiable Information in the European Union.