Backing up data and other files is an essential practice to insure against the loss of valuable information. The purpose of back-ups is to restore a system to a current state (as of the date of the most recent back-up) in case of system failure. Back-up media is not intended to serve as short or long term storage of information. Retrieval or archive storage is a separate process used to remove or duplicate files from an active system to another system or digital media for short or long-term storage. The backup and archive may not be running for certain periods of time, this is for operational issues.
The purpose of this policy is to establish a limit on the length of time backups are maintained and to encourage researchers to distinguish between the purposes and practices of backing-up data vs. retrieval or archive storage of data.
The solution we have is built on IBM Tivoli Storage Manager with Hierarchical Storage management (HSM). HSM adds to archiving and file protection for disaster recovery the capability to manage storage devices efficiently, this is a policy based solution.
Snapshots
The Research Data Store also has a facility where ‘snapshots’ of data are held for 4 days, which allows the researcher to recover previous versions or retrieve deleted files on a Windows or Linux device, as long as it is within 4 days of the deletion. Please note that we do not guarantee the availability of snapshots but it may provide a quicker way for you to access accidentally deleted data than requesting access to the backup files. Instructions on how to recover snapshots are linked from here for Windows and Linux.
Backup
The backup will occur overnight and any files that have been created that day will be backed up. Each file will be kept for 90 days and have a maximum of 3 versions. However to economise on the amount of spinning disk required, as files age and become stale we will implement HSM.
HSM rules
HSM will apply when files have not been accessed or modified, but these files may be pre-migrated to tape:
- If the size of the file is greater than 1Gb and has not been modified for more than 10 days then the file is migrated onto tape.
- If the size of the file is greater than 10Mb and has not been modified for more than 90 days then the file is migrated onto tape.
Recovery of data once it has been migrated onto tape
If data which has been migrated onto tape is required, the process to recover will be automatic, however it may take a few minutes for the data to be recovered. Requests for restoring user files will be acted upon within 10 working days, users should request restores as soon as possible.
Definitions
Back-up
A copy of data from an original electronic source transferred to a separate medium (CD, tape, disk). The purpose of back-ups is to restore information lost as a result of purposeful or inadvertent user action or system failure. Example: Daily and weekly backups of an entire system.
Retrieval Storage (Archive)
The movement of data from its original source to a separate medium (CD, tape, diskette). The purpose of retrieval storage is to move data no longer needed on a day-to-day basis to another location from which it can be retrieved at a later date, if needed. Example: Movement of dated transaction files to CD or all data from a completed research trial to zip-disk.
Local disk
Data storage on a personal PC managed by the user will not be backed up, Data on the BlueBEAR and RDS systems will only be backed up.
Deleted files
When a user deletes files, the files are moved from the active area of either the local or remote disk to "trash". Any files that exist in trash continued to exist until the user empties the trash. Any backups that take place while files remain in trash will be copied to the back-up tape along with all other active files. So while the user believes the file has been deleted, it will continue to exist until the trash is emptied and all backup tapes have been rewritten or destroyed.
Business files
Official files associated with the business processes of the research unit. Examples: purchase requisitions, personnel notices, etc should not be stored on BlueBEAR or RDS.
Personal files
Non-official files created and/or used by an individual as part of their work. Examples: email, spreadsheets, informal memos, etc should not be stored on BlueBEAR or RDS.
Lifetime of files
The lifetime of files on local, remote, or retrieval storage is subject to the practices of the individual and/or the administrator of the remote server. Unless deleted, files will exist for 90 days.
File versions
The solution will allow for up to 3 versions of the same file.