Terminology

Under Australia’s FAIR Access Policy Statement, all publicly funded research outputs must follow the FAIR principles.

The FAIR Principles have been developed to make research more visible and to allow researchers to more easily collaborate and maximise the return on investment in research and innovation. The acronym stands for:

Findable

Data can be more findable by: properly describing what the data is; putting it in a permanent and easily searchable place; and making it easy for humans and computers to search for it.

Accessible

Data can be more accessible by: using non-proprietary, standardised and automated methods to supply the data to those who want or need it; letting others know how they can get the data; and letting others know if the data is no longer available.

Interoperable

Data can be more interoperable by: storing and providing the data in widely-used and accessible file formats; describing the data using standard terms (vocabularies) that are relevant and widely known; and describing if it relates to other data and what exactly that relationship is.

Reusable

Data can be more reusable by: making it clear how the data was collected or if there are validity concerns; making any conditions of reuse clear in license readable to humans and machines; and meeting the standards used within the relevant research community.

A file format is the structure of a file that tells a program how to display its contents e.g., Microsoft Word documents are saved in the .docx format.

Researchers may need to use different file formats at different stages in the Research (Data and Management) Asset Lifecycle but for long-term preservation, will need to be stored in a durable format. This ensures files can be opened by future users, perhaps long after the research project has concluded.Where possible, it is recommended to:

USE

• Formats endorsed by standards agencies such as Standards Australia, ISO
• Open formats developed and maintained by communities of interest such as OpenDocument Format
• Lossless formats
• Formats widely used within a given discipline

AVOID

• Proprietary formats
• File format and software obsolescence

Researchers may need to use software that does not save data in a durable format due to discipline-specific or other requirements e.g. specialised programs to capture or generate data. In these circumstances, data needs to be exported to a more durable format such as plain text (if this can be done without losing data integrity) and include it alongside the original files when archiving e.g. export .csv files from SPSS (with value labels) and archive them alongside the .sav files.

Some examples of preferred formats for data archiving are:

• CSV OR Excel spreadsheet (.xlsx) AND OpenDocument Spreadsheet (.ods)
• Plain text (.txt) OR Word document (.docx) AND Rich text (.rtf), PDF/A or OpenDocument Text (.odt)
• Geospatial data: ESRI shapefile (.shp, .shx, .dbf), Geo-referenced TIFF (.tif) and ESRI ASCII Grid (.asc)
• Image files: lossless formats (.tif or .raw) preferred
• Video: MPEG-4 (.mp4)
• Audio: Free Lossless Audio Codec (.flac)

It is also important to document 'data capture' and 'storage formats' as well as 'software' used and their versions – refer to Metadata for further information.

The UK Data Service maintains a list of recommended and acceptable formats for agencies, researchers and others depositing social, economic and population data in their collection.

Packaged files can be used for archiving large collections of heterogenous datasets with some provisos:

• Use archives with extensions .zip or .tar
• Zip the data without any data compression
• If possible, avoid encrypting the files
• Be aware that very large packages may be difficult to open from a browser - ETH-Bibliothek recommends packages of less than 2GB
• Avoid long path lengths in your folder structure. Long file names combined with a detailed folder hierarchy may lead to path lengths exceeding 256 characters. This hampers further processing in Windows and WinZip cannot unpack such containers.

File names are often taken for granted, but when working on complex research projects it’s important to be able to retrieve files quickly and effectively. It’s good practice to adopt a file naming convention which is consistent, logical and descriptive. Abbreviations and codes can be used as long as they are clear and uniformly applied.

File names could include information such as:

• Project or experiment name or acronym
• Researcher name/initials
• Year or date of experiment
• Location/spatial coordinates
• Data type
• File version number

It's also a good idea to include a readme.txt file in the directory that explains the naming format and any abbreviations or code used.

Avoid really long file names and special characters like ~ ! @ # $ % ^ & * ( ) ` ; < > ? , [ ] { } ' ‘| in file names, directory paths and field names. Spaces in file names can also cause problems for some software or web applications, so underscores ( _ ), dashes ( - ), or camel case (e.g. FileName) could be used instead.

Re-naming multiple files is onerous but there are bulk renaming utilities that can help, such as:

• For Windows Bulk Rename Utility is free and is simple to use. Recommended.
• For Macs try Renamer 4
• Maketecheasier has a few more suggestions for bulk rename utilities for Windows.

Choose a folder structure for your research project that is uniform and logical in its organisation. If you are working on a collaborative project, it becomes all the more important to use a structure that is well-organised and clear to all parties involved. The kind of folder and file directory structure may ultimately depend on the nature of your research project, the disciplinary area you are working within, and the technical complexity involved. The UK Data Service provides some general advice on folder structures and notes that it helps to restrict the level of folders to three or four deep, and not to have more than ten items in each list.

You may also like to take a look at this tweet from @micahgallen for an effective directory structure (for research projects) and notice how many researchers this resonated with!