Before starting this article, I need to make a disclaimer: I am not speaking on behalf of my employer, and what I am writing here represents only my personal opinions.
In this article, I assume that you already know what the FAIR principles mean and that you work with, or at least have some experience with, Research Data Management (RDM), for example, as a research software developer, PhD student, postdoc, master's student, data steward, or researcher.
As a software developer who has worked with Research Data Management (RDM) for almost 9 years, I have learned some lessons, especially while developing solutions for it.
My idea here is to present a simple solution architecture to help make research data FAIR, discussing the pros and cons of this solution with supporting arguments. It's important to understand that FAIR is a set of guiding principles and best practices, not a specific framework, architecture, or technology. These principles provide guidance for making research data Findable, Accessible, Interoperable, and Reusable, without prescribing how they should be implemented.
Every architectural decision has trade-offs, and this is far from being the best or only solution, but it is an approach that I think could help improve FAIR practices and Research Data Management.
The Complaints
One of the main complaints I've heard from researchers and students about making their data FAIR is that it adds more work to their already busy schedules, and they don't have enough time to do it.
But the good news is that part of the responsibility for implementing the FAIR principles lies with the university or research institute you are affiliated with. However, you, as a researcher or student, still need to do your part, as you can see in the picture below:
The favorable scenario
FAIR best practices should also be part of an institution's research culture, with IT providing the tools and infrastructure needed to support their implementation.
What tools would I propose?
Now comes the fun part, at least for me as a software developer. And sorry, it involves AI.
I know you may be skeptical about using AI with your data, particularly when dealing with confidential, sensitive, unpublished, or personally identifiable information, but I'll explain how I think it should be used.
The proposed workflow is simple:
During your research, prepare your data and its metadata, using open formats as much as possible, such as CSV files, or widely supported formats like JPEG. If proprietary software was used, mention its name and version to help others understand how the data was created or processed.
-
Important files for your dataset include:
- README.txt: A file explaining the dataset, its contents, how it was created, and how to use it.
- LICENSE.txt: A file describing the license under which the dataset can be used and shared, including any applicable restrictions.
- metadata.csv: A data dictionary explaining the meaning of the variables, vocabulary, data types, and units used. Keep in mind that a data dictionary is only part of the dataset's metadata. Other information, such as authors, dates, identifiers, and related publications, may also be needed.
- provenance.txt: The origin story of your data: who, what, where, when, and how. It should describe where the data came from, how it was collected, and any transformations or processing steps applied to it.
These files are practical recommendations, not strict FAIR requirements. The FAIR principles don't prescribe specific filenames or formats. However, providing clear documentation, provenance, and licensing information helps others understand and reuse your research data.
Keep in mind that having these files doesn't automatically make a dataset FAIR. Other aspects, such as persistent identifiers, machine-readable metadata, findability, accessibility, and appropriate repository infrastructure, are also important.
Now the diagram with how everything will be connected and the descriptions of this workflow below the image:
The proposed workflow is simple:
Prepare and upload the dataset: The researcher or student places their dataset files (
README.txt,LICENSE.txt,metadata.csv,provenance.txt) and a/datafolder containing data files from instruments, sensors, etc., into a predefined folder structure. Each file or folder can also have its own metadata, if applicable.Harvest and organize the data: The application automatically harvests the data and metadata files. Based on the provided metadata, it creates the corresponding entries, assigns unique identifiers, and archives the data files, linking each file to its metadata. Ideally, the application should also support persistent identifiers, such as DOIs, when applicable.
Find and discover data: Researchers and other users can now search for and discover datasets and files based on the metadata provided.
Manage access and metadata: The tool should also allow users to define who can access or download the data, whether the data is open, restricted, or subject to an embargo period. Users should also be able to edit, create, or delete metadata (using soft deletion where appropriate) and upload files through an interface. Access to restricted data should be controlled according to institutional policies and user permissions.
Link data to research projects: The application should be able to link harvested data or manually entered data and metadata to specific research projects, allowing researchers to organize and manage their datasets within the context of their research.
Automate as much as possible: For example, data generated by lab instruments or sensors could be automatically harvested and linked to existing metadata previously entered into the system by researchers and students.
What if AI is enabled?
AI should be optional, allowing users or institutions to enable or disable it.
An AI agent could read the data and metadata, suggest metadata fields, identify relevant ontologies and controlled vocabularies, and propose relationships between datasets, files, and metadata. The user should be able to review, approve, modify, or reject the schema proposed by the AI agent before it is applied.
The AI could also help assess how FAIR a dataset is by evaluating its metadata and other available information against established FAIR assessment criteria. The tool could present an indicative FAIR score or percentage based on a clearly defined assessment method, showing what is already good, what needs improvement, and recommendations based on the FAIR principles.
However, the score should be treated as guidance, not as an absolute measure of FAIRness, since some aspects cannot be reliably assessed automatically.
Ideally, the research institute or university would provide a locally hosted AI model so that research data and metadata could remain within the institution's controlled infrastructure, with appropriate security measures, access controls, and safeguards. This would be especially important for confidential, sensitive, or unpublished research data.
Metadata standards
The application should support established metadata standards, such as Dublin Core, with a minimum set of required fields defined by the institution or research community. These could include:
- Title: The title of the dataset or file.
- Description or Abstract: A description of the data and its purpose.
- Creators: The people or organizations responsible for creating the data.
- Date of Collection: When the data was collected.
- Geographic Coverage: The geographic area covered by the data, such as a bounding box, when applicable.
- Keywords: Terms describing the dataset, preferably using controlled vocabularies.
- Identifier: A unique identifier for the dataset or file.
- License and Access Rights: Information about how the data can be accessed and reused.
Users should also be able to select other metadata standards, such as Darwin Core for biodiversity data, depending on their research domain and requirements.
Final considerations
This is a very high-level overview of how an IT solution could serve as a tool to help researchers make their data FAIR.
Technical details such as data storage, archiving, backup strategies, security implementation, and infrastructure design are outside the scope of this article.
The goal is not to propose the perfect architecture, but to show how a relatively simple workflow, supported by institutional IT infrastructure and optionally AI, could reduce the manual work required from researchers while improving the findability, accessibility, interoperability, and reusability of their research data.
What Are the Pros and Cons?
The pros: The main advantage of this process is having a practical and standardized way to collect research data and help make it FAIR, with minimal additional work for researchers and students. We only ask them to prepare their data and metadata and upload their files into a predefined folder structure on the platform or through a user interface, such as a web browser or an installed application.
This approach could reduce repetitive tasks, especially metadata entry and dataset registration, while allowing researchers to focus more on their research. However, researchers would still need to provide meaningful documentation and verify that their data and metadata are accurate.
The cons: We may lose some granularity in the metadata and predefined relationships between entities. A simplified metadata model might not capture all the details required by specific research communities. However, nothing prevents us from adding more detailed metadata and defining these relationships based on the specific needs of each research project.
Conclusion
I have seen several applications with very cool features, but from my experience, I've noticed that many researchers and students want to make their data FAIR. However, we, as IT professionals, should provide them with feasible and practical solutions rather than tools that create more work for them.
Of course, not all researchers have the same understanding of FAIR principles or access to the same resources, which is why institutions should provide the necessary support, infrastructure, and guidance.
The goal should be to make FAIR practices a natural part of the research workflow, not an additional burden.
Top comments (0)