PAT.File Storage

From OIAr Archive 2013
Jump to navigation Jump to search
Page maturity
This page has maturity level 4 (mature)

This is a Pattern document

Document icon PAT File Storage Version: 0.4 OIAr logo
Document type: Pattern Type Owner:

J.A.H. Schoonderbeek

Create commentary


This Pattern Type belongs to "Infrastructure Sector Core". This facility offers data preservation and data retrieval functionality with the following characteristics:

  • operates on data at the "loosely structured data" level, e.g. files or documents,
  • located at one logical location,
  • may offer the ability to perform storage related "intelligent" tasks, such as replication, migration, encryption, compression and data deduplication. This intelligence may be characterized as
    • "back-end facing" intelligence; functionality that the facility's users don't get (direct) access to, only the administrators and back-end systems;
    • "front-end facing" intelligence; functionality that the facility's users can access and make use of.

Note that File Storage is always implemented on top of some form of Raw Storage. As a consequence, a Pattern Variant dealing with file storage is often based on both the File Storage Pattern Type and the Raw Storage Pattern Type.

As examples: realizations of File Storage (and Raw Storage under it) can include:

  • Content Management Systems (CMS) and alike solutions,
  • (the front-end of) NAS appliances,
  • file share servers.

Note: it's possible that the solution is to make use of Business Rules (e.g. for data deduplication or automatic migration of data to nearline/offline storage). If this is the case, then this pattern must be paired with facilities that deliver this functionality.

Graphical Overview

This is the graphical representation of the infrastructure functions in this Pattern Type, plus their main relations:

Pattern Type File Storage
Pattern Type File Storage

(The source file of this picture can be downloaded here).

Pattern Type Composition

This pattern has the following mandatory and optional subfunctions, expressed in Building Block Types:

Icon Function WA Inclusion Rationale
BT.File Engine icon File Engine SE mandatory This facility handles the actual storing/retrieving of loosely structured data such as (but not restricted to) files or documents. It serves end-users and systems (amongst which application engines and structured data stores). To that end, this facility offers a storage structure that fits the data to be stored, such as a file system for files, and an interface over which data can be offered or retrieved.

The File Engine provides "intelligence" to the file storage solution with respect to its handling of the stored data, with the aim of meeting the quality requirements, improving infrastructure management, and/or providing sufficient performance. It may handle migration/replication to other instances of itself, encryption, compression, de-duplication and other "standard" storage-related processing. It also can handle granular, file-based permissions.
Note that it requires Raw Storage to physically store the bits and bytes that make up the pieces of loosely structured data.

BT.Caching icon Caching MW optional An intelligent storage facility may use caching to increase its capacity to receive and/or return data on request.
BT.Control Interface icon Control Interface SE optional If implemented, this function can be used for several Management Tasks, among others:
  • Restoring (partial) back-ups;
  • Capacity & Performance Management;
  • Provisioning;
  • Rules Management, with which the stored data can be manipulated using metadata, usage, usage patterns or other information that the facility may obtain.

Care must be taken to limit access to these management tasks to authorized systems and users.

BT.Restore icon Restore ST optional Implementing a facility of this type may serve different goals:
  • It can serve a business need for data protection,
  • It can support a technical need related to business continuity, technical exploitation or maintenance.

To support these goals, Restore can support the following activities:

  • restoring backups, either in entirety or partially;
  • storage migration, where data that has been migrated off-line to backup media can be restored to active storage upon demand and/or rules.

Note that the Restore function may or may not be under control of end users.
In realization, Restore can be a sub-function of a versatile File Engine, e.g. the realized facility may be able to restore data from the file storage space under control of that File Engine, or it can be paired with a separate Pattern Variant of Raw Storage or File Storage/Raw Storage explicitly meant for backup & restore.

BT.Backup icon Backup ST optional Implementing a facility of this type always serves to support the Restore function, and thus virtually always appears together with one or more Variants of the Restore function. Furthermore, the criteria that govern the Backup functionality must always be derived from the goals and needs of all of the Restore Variants that it supports.

Note that the Backup function usually is under control of IT Operations, but may occasionally be offered to end users directly.
In its implementation, just as with Restore, Backup can be a sub-function of a versatile File Engine, so that the backups are created on the file storage space under control of that File Engine, or it can be paired with a separate Pattern Variant of Raw Storage or File Storage/Raw Storage explicitly meant for backup & restore.

BT.Archiving icon Archiving ST optional Implementing a facility of this type in a File Storage Pattern Variant may serve different goals
  • to have data immutably preserved for a certain period of time;
  • to free up resources in the File Storage realization.

To support these goals, Archiving offers to perform the following activities:

  • to copy or move data to a secondary location, which is another (logical) location in this same File Storage realization, or another File Storage realization; and
  • to do so according to specific rules and/or schedules; and
  • in case of resource-saving archiving: to copy or move data back from the secondary location to the primary location according to demand and/or schedules;

When the data has been created in the secondary location ("the archive"), it is accessible via a File Engine instance, often one that's either explicitly implemented for the archive.

BT.Scheduling icon Scheduling MW optional If the Pattern is to have automated backup and/or archiving functionality, then this function will govern the respective schedules and initiate separate backup and/or archiving jobs.

Pattern Type Neighbors

This pattern has the following mandatory and optional relations with adjacent (sub)functions, expressed in Pattern Types (PAT). Note: if the table below is empty, then there are no architecturally prescribed relations with adjacent subfunctions:

Function Adjacency Description
Raw Storage mandatory All "loosely structured data" eventually will have to be stored on some physical device, where it's "raw bits". Putting this Pattern under the File Storage pattern delivers this ability.
Authentication & Authorization optional A&A will be required for one or more of the following uses:
  • access to the control interface, that may control aspects ranging from facility management to self-service provisioning of storage space;
  • permission validation for read/write/modification/deletion actions on the loosely structured data being stored;
  • permission validation for actions such as backup, restore and archiving.
Application Hosting optional One of the facilities that require use of file storage is the Application Hosting facility. An Application Hosting facility requires File Storage to store the applications themselves, and possibly the data with which the applications interact.

Pattern Variants based on this type

Pattern VariantBrief descriptionOwnermaturity
PAV.File Storage.LocalFile Storage.LocalJ.A.H. Schoonderbeek1