Organize your data for long-term storage

Description

Once your data has been transferred to the RESEARCH NAS server into your “default” folder, you have two options:

1- You can keep them as hot data for frequent access and ask to funding agencies to cover the current storage costs.

2- You can store them free of charge on the long-term storage (LTS) platform as cold data for rare access and a period of about 10 years.

In both cases, your “hot” and “cold” data will have to be organized in the form of projects. Each project must correspond either to a funding source or to a specific theme.

This phase is divided into several stages

Stage A – Read the requirements to anticipate Long Term Storage (LTS)

Who Duration
PI 15 minutes

We recommend that you carefully read the following conditions in order to anticipate the required steps which will allow you to preserve your data on the long-term storage platform.

Stage B – Organize your data by project

Who Duration
PI / Groupe Variable

Your data must be sorted by projects associated with a research fund, a theme, etc.  For each project, you must request a project directory on the RESEARCH NAS server using the NAS DCSR dashboard. Once the requested project folder has been opened, move your organized data there.

Step C – Identify “hot” and “cold” data

Who Duration
PI variable

When your data has been sorted by project, identify your current research projects (“hot” data) and your completed research projects (“cold” data).

Step D – Manage access to your hot data (projects in progress)

Who Duration
PI 5 minutes

Once project folders have been created, you can manage the access of your collaborators to your project data / directories. For this you need to use the NAS DCSR dashboard.

Step E – Long-term storage of “cold” data (semi-completed or completed projects)

Who Duration
PI / Uniris / DCSR 3-4 weeks

Long Term Storage (LTS) caracteristics

  • Free of charge (with security and backup)
  • – Time limit for storage duration (at minimum 10 years for published data) with possible extension (1x).
  • Restricted access (limited number of accesses to data on LTS and under request only).
  • Effective organization of your data under a project form (matched with a funding, a specific theme, a publication etc.).
  • Production of a Readme file (document describing the dataset content) for each individual futur TAR subdirectory
  • Naming rules for (TAR) subdirectories in the LTS directory

1-PI contact C. Lebrand (UNIRIS) via the NAS DCSR dashboard before creating a new project directory on the Long-Term Storage Platform (LTS).

Via the interface on the homepage, you will be able to request the LTS tape transfer for part of your data by clicking on the button “Request for long-term storage (LTS)” in the list of actions listed on the interface homepage.

  • – Connect and sign in to your NAS DCSR dashboard via your SWITCH UNIL account using this link 

  • – Select «making a long term storage request» via your DCRS dashboard interface to ask for Long Term Storage (LTS)

  • – To send your request for long term storage of part of your research data on magnetic tape, click on the submit button. Mention if possible which data you would like to transfert for long term storage.

  • – An e-mail will be send automatically to Cécile Lebrand via the generic email address researchdata@unil.ch. C. lebrand will contact you to make an appointment within 15 days and will provide personalized assistance.

 

2- FBM Data steewards  help for organizing your data for long-term storage

2.1- C. Lebrand will informe and guide you through the process of reorganizing and describing your research data in the  form of explanatory documents called a readme files called readme file. You will be will helped to  determine the best organizational strategy for your data.

During this process,  you will be able to  give a temporary access to your data stored on the DCSR NAS to Cécile Lebrand via the Ci interface on the homepage and using the following username: clebrand

2.2. You will need to reorganize your data according to the projects  you will have defined and clean up/eliminate obsolete data.

  • Research data must be organized  into the LTS directory around a given project (matched with a funding, a specific theme, a publication etc.).
  • TAR archive file: size of the data volume in each individual subdirectory to be archived  is free up to 5 TB per subdirectory in the LTS directory. No need to compress your data since this step will be done during the creation of the TAR subdirectory.
  • The TAR files to be archived must remain at the root of the LTS directory.
  • Each TAR subdirectory has to comply to naming rules (see below) and should be accompanied by an independent readme file. We suggest to create a general Readme Template that can be shortly adapted to each individual TAR subdirectory if possible.
  • The naming rule only applies to the first TAR sub-folders in the LTS directory. The TAR archive files created from these directories will have the same name. Within these directories the names of the files and data directories are free.
  • Naming rules for the readme file in each TAR directory: The name and file format of the readme file must not be changed, “LongTermStorage_Data_Description_EN.docx”.
  • Naming rules for subdirectories in the LTS directory (TAR archive file)  The length of the folder name must not exceed a maximum of 40 characters according to the rules below
    • Numbers from 0-9
    • Letters a-z
    • Letters A-Z
    • Hyphen ( – ) OK but not at the beginning or end of the directory name
    • Underline ( _ ) OK but not at the beginning or end of the directory name
    • No white spaces
    • No accented characters or symbols

2.3. Complete the readme file for each distinct subdirectory to be archived in individual TAR archive file following the established guidelines and send it for final revision (max 15 days) to Cécile Lebrand. In the event your readme file are not considered complete enough as to understand the nature of the data set, C. Lebrand will send you an add-in request.

2.4 If your data comes from the fbm_mov folder create new project(s) containing the subdirectories to be migrated to the LTS (see step B – E). Use the NAS DCSR dashboard via your SWITCH UNIL account using this link 

3- Once your readme file have been approved by UNIRIS, they should be included at the root of each individual moved within data subdirectory and the future TAR file should be moved from the D2C (or D1C) directory to the LTS directory. The DCSR  will create the TAR archive files  and transfer your data on magnetic tape with all individual readme file included at the root of  each individual TAR archive file.

After completing the magnetic tape transfer process for the TAR directory, the LTS directory will include the following elements:

  • The TAR file will be renamed by adding the prefix “ARCHIVED*,” and the underlying information will only be accessible in read-only mode.
  • A copy of the readme file “LongTermStorage_Data_Description_EN.docx” will be accessible in read-only mode.
  • A file named “INVENTORY_OF_ARCHIVED_FILES.txt” within the TAR directory will list the archived files with their full paths and provide information on the recovery procedure from the tapes.

This procedure ensures that the archived data remains accessible and easily identifiable. The addition of the “ARCHIVED*” prefix allows UNIRIS and the researcher to quickly confirm that archiving has been completed by simply examining the TAR directory names at a given moment.

Here is the description of the systematic name of the TAR file (self described) on the magnetic tape once they have been archived .

Step F –Data recovery of your data on the LTS platform

Who Duration
PI / Uniris / DCSR  Variable
  • 1. Request at least 7 days in advance.
  • 2. Limited number of accesses with the possibility of retrieving data stored in individual LTS  TAR subdirectory (only once a month and the data should be stored for at least a minimum period of one month).
  • 3. No direct access but official request to the DCRS via  the Ci interface on the homepage (see below).
  • 4. Once the request is made to DCSR, the time required to process the demand will depend on the complexity of the data to be processed.

Procedure for Data recovery of TAR subdirectory from the LTS platform to the NAS DCSR

  • Connect and sign in to your NAS DCSR dashboard via your SWITCH UNIL account using this link 

  • Select «Retrieving data from Long term Storage» via your DCRS dashboard interface to ask for recovery of your data on the Long Term Storage (LTS)

  • To validate your request to retrieve some or all of your research data stored on magnetic tape (LTS), please click on the “Submit” button below. If you already have a specific request or question, you may, if you wish, leave your comment(s) below.

  • An e-mail will be send automatically to The DCSR that will process your demand and you will be able to access and to work with your data on your DCSR-NAS space.
  • Limited number of accesses with the possibility of retrieving data stored on the LTS after a minimum LTS period of one month and once a month only.
  • Once the request has been made to the DCSR, the time taken to process the request will depend on the complexity of the data to be processed.

Links

List of useful links related to the migration of research data to long term storage

-Long-term storage of cold data: cecile.lebrand@unil.ch 

-Usage of the RESEARCH NAS server: helpdesk@unil.ch

– Readme : LongTermStorage_Data_Description_EN

dir4-5-donnees-rech Traitement et gestion des données de recherche Art 8. Stockage des données de recherche et Art. 9 Facturation pour le stockage des données de recherche

Updated: 13.12.2023