GP.Managed File Transfer

From OIAr
Revision as of 06:01, 21 October 2014 by Jan Schoonderbeek (talk | contribs) (added Messaging components)
Jump to navigation Jump to search


This is a Generic Pattern document GP Managed File Transfer Version: 0.1 OIAr logo
Document type: Generic Pattern Owner:

J.A.H. Schoonderbeek



Description

This Generic Pattern belongs to "Business Support". Managed File Transfer (abbreviated MFT) indicates the automated transfer of data in a loosely structured form ("files") between IT facilities. These facilities can be owned by the organization, but an MFT instance may also be used to transfer files into or out of the organization, from or to external IT systems. Furthermore, human users may also make use of the MFT pattern to exchange data with other users and/or IT systems. A realization of this MFT Pattern handles the reception and/or collection of files from a source, and their delivery to and/or collection by one or more targets. It may also additionally modify the files by transcoding, encrypting/decrypting, (de)compressing, and/or it may provide means to modify the files in transit in other respects. Incoming and outgoing traffic may be regulated using predescribed rules and/or by schedules that act on times, periods or order. The Pattern requires use of an instance of Data Transport (and optionally Message Handling), since the data to be transported comes in and goes out over these services. Most data transfers will require the Managed File Transfer Pattern to authenticate itself to the source and/or target. Furthermore functionality can be present to provide logging of each file transfer for operational and security purposes.

Services realized

This Pattern realizes the following service(s):

  • Managed File Transfer (This service provides a means to transport loosely structured data from a source to one or more targets, either ad hoc or structurally.)

Functional and Integration view

This is the graphic representation of the functional model of this Generic Pattern:

Generic Pattern Managed File Transfer
Generic Pattern Managed File Transfer


Generic Pattern Composition

This pattern is an aggregation of the following (mandatory and optional) functions, expressed in Generic Functions:

Icon Function Inclusion Rationale
GF.File Engine icon File Engine recommended This function represents the core intelligence of the Managed File Transfer service. This function performs the collection or accepting of files from the source, and the transmission or offering of files to one or more targets.
GF.Presentation Engine icon Presentation Engine recommended This function represents the transcoding that is required when a source offers a file in a different format and/or using a different transfer protocol from the format/protocol used by the file transfer's target(s). Furthermore, if the Pattern offers ad hoc file transfers to human users, then this function shapes the users' experience when using the Managed File Transfer service (e.g. a web environment for uploading and downloading the files).
GF.Caching icon Caching recommended Any file in transit will be buffered in the Managed File Transfer facility while it is processing file transfer tasks such as transcoding. This transient storage of files inside the MFT facility needs attention because of three aspects:
  • Security: the storing of files needs to be secured to the level that's appropriate for the confidentiality required by the business and security department;
  • Storage space: enough space needs to be present to store the maximum volume of files that may reside in transient storage at the facility at any given time; an overflow to the adjacent service File Storage may be in order;
  • Business continuity: when a technical problem or disaster impedes the MFT facility, measures must be taken to account for files that were residing only in transient storage at the time of the problem.
GF.Rules Engine icon Rules Engine optional This function represents the additional intelligence that can drive extra manipulation of files in transit, advanced exception handling, logging and/or routing by the Pattern. Active collection of information on the files in transit, source and/or target(s) may lead to execution of (implicit or explicit) rules.
GF.Logging icon Logging optional This function is often included, as logs of file transfers are required not only by Operations, but also Security/Compliance. Furthermore, a business entity responsible for sending files, or one that expects incoming files, may require access to the logged information to verify data transfers.
GF.Scheduling icon Scheduling optional Scheduling can serve to initiate file tranfers and/or internal maintenance tasks, on regular or predetermined times.
GF.Controlling icon Controlling recommended This function models the manner in which the Pattern's administrators, and possibly authorized clients, can administer file transfers. Among the administrative tasks can be the following:
  • Administering file transfer end points;
  • Administeringfile transfer tasks (which in turn make use of transfer end points);
  • Administering schedules for automated file transfers.

Since the files transferred by the Pattern may be business critical and/or have a certain level of confidentiality, care must be taken to limit access to this function to authorized users.

GF.Encryption icon Encryption optional This function can be used to model the encryption/decryption that is associated with secure file transfer protocols, as well as the manipulation of files in transit, when the MFT is required to encrypt/decrypt files or file archives.
GF.Reduction icon Reduction optional This function can be used to model the compression that is often associated with file transfer protocols, as well as the manipulation of files in transit, when the MFT is required to compress/decompress files or file archives, or manipulate files that sit inside a file archive.
GF.Distribution icon Distribution optional When the MFT facility is required to deliver files to multiple targets, then this function models the handling of addressing and routing.
GF.Message Client icon Message Formatting optional When the MFT facility makes use of a Message Handling services, then the Message Client functionality provides the interface to such a service.

Services connected with this Generic Pattern

This Generic Pattern has the following mandatory and optional relations with adjacent Generic Services.

Service Adjacency Summary Rationale
Data Transport recommended This service transports data between automated systems. All files that an MFT facility handles come in and go out over a Data Transport service. Especially the transfer of files over one or more security delineations require attention, so the explicit modelling of Data Transport is recommended.
Authentication & Authorization optional This service can validate an identity claim, and it can validate the permissions required for an action, as part of an Authentication & Authorization process. The Pattern will likely be required to authenticate itself when collecting and/or delivering a file. Furthermore, the Pattern probably will require systems and users to authenticate themselves when they wish to transfer or collect files. Finally, access to the services provided by the Managed File Transfer Pattern must likely be limited to authorized personnel.
Facilities Monitoring optional This service allows its users to monitor IT facilities with the aim of guarding operational continuity or security. The logs of file transfers are of interest to security and operational personnel, as well as the business units that make use of Managed File Transfer. Connections with Facilities Monitoring services provide these monitoring services with the data with which they can provide their respective users the required insights and overview.
File Storage recommended This service offers clients the ability to store, retrieve and modify data in loosely structured form. A file in transit will be stored in the Managed File Transfer facility when the target is required to actively fetch the file ("pull-out") instead of having the file delivered as soon as it enters the MFT from the source. This storage of files inside the MFT facility needs attention because of three aspects:
  • Security: the storing of files needs to be secured to the level that's appropriate for the confidentiality required by the business and security department;
  • Storage space: enough space needs to be present to store the maximum volume of files that may reside at the facility at any given time (including exceptional times such as when one or more sources cannot accept files for any given reason, e.g. connectivity issues);
  • Business continuity: when a technical problem or disaster impedes the MFT facility, files already accepted from the source but not yet delivered to (all) targets may be at risk of loss or corruption. The connection between the MFT facility and the File Storage service should correctly handle such events.
Message Handling optional This service handles transport, storage and delivery of messages between senders and recipients. This service may be adjacent because of either of the following reasons:
  • files are transported in and/or out of the MFT facility via a messaging protocol;
  • the MFT facility uses Message Handling services to notify security, operational and business users about events in the facility, such as failure/success of file transfers.

Applied Patterns based on this Generic Pattern

The following Applied Patterns are based wholly or in part on this Generic Pattern: