DEV Community: Flavio Francisco

Let's be FAIR with Open Science

Flavio Francisco — Sun, 25 Oct 2026 09:03:09 +0000

Before starting this article, I need to make a disclaimer: I am not speaking on behalf of my employer, and what I am writing here represents only my personal opinions.

In this article, I assume that you already know what the FAIR principles mean and that you work with, or at least have some experience with, Research Data Management (RDM), for example, as a research software developer, PhD student, postdoc, master's student, data steward, or researcher.

As a software developer who has worked with Research Data Management (RDM) for almost 9 years, I have learned some lessons, especially while developing solutions for it.

My idea here is to present a simple solution architecture to help make research data FAIR, discussing the pros and cons of this solution with supporting arguments. It's important to understand that FAIR is a set of guiding principles and best practices, not a specific framework, architecture, or technology. These principles provide guidance for making research data Findable, Accessible, Interoperable, and Reusable, without prescribing how they should be implemented.

Every architectural decision has trade-offs, and this is far from being the best or only solution, but it is an approach that I think could help improve FAIR practices and Research Data Management.

The Complaints

One of the main complaints I've heard from researchers and students about making their data FAIR is that it adds more work to their already busy schedules, and they don't have enough time to do it.

But the good news is that part of the responsibility for implementing the FAIR principles lies with the university or research institute you are affiliated with. However, you, as a researcher or student, still need to do your part, as you can see in the picture below:

The favorable scenario

FAIR best practices should also be part of an institution's research culture, with IT providing the tools and infrastructure needed to support their implementation.

What tools would I propose?

Now comes the fun part, at least for me as a software developer. And sorry, it involves AI.

I know you may be skeptical about using AI with your data, particularly when dealing with confidential, sensitive, unpublished, or personally identifiable information, but I'll explain how I think it should be used.

The proposed workflow is simple:

During your research, prepare your data and its metadata, using open formats as much as possible, such as CSV files, or widely supported formats like JPEG. If proprietary software was used, mention its name and version to help others understand how the data was created or processed.
Important files for your dataset include:
- README.txt: A file explaining the dataset, its contents, how it was created, and how to use it.
- LICENSE.txt: A file describing the license under which the dataset can be used and shared, including any applicable restrictions.
- metadata.csv: A data dictionary explaining the meaning of the variables, vocabulary, data types, and units used. Keep in mind that a data dictionary is only part of the dataset's metadata. Other information, such as authors, dates, identifiers, and related publications, may also be needed.
- provenance.txt: The origin story of your data: who, what, where, when, and how. It should describe where the data came from, how it was collected, and any transformations or processing steps applied to it.

These files are practical recommendations, not strict FAIR requirements. The FAIR principles don't prescribe specific filenames or formats. However, providing clear documentation, provenance, and licensing information helps others understand and reuse your research data.

Keep in mind that having these files doesn't automatically make a dataset FAIR. Other aspects, such as persistent identifiers, machine-readable metadata, findability, accessibility, and appropriate repository infrastructure, are also important.

Now the diagram with how everything will be connected and the descriptions of this workflow below the image:

The proposed workflow is simple:

Prepare and upload the dataset: The researcher or student places their dataset files (README.txt, LICENSE.txt, metadata.csv, provenance.txt) and a /data folder containing data files from instruments, sensors, etc., into a predefined folder structure. Each file or folder can also have its own metadata, if applicable.
Harvest and organize the data: The application automatically harvests the data and metadata files. Based on the provided metadata, it creates the corresponding entries, assigns unique identifiers, and archives the data files, linking each file to its metadata. Ideally, the application should also support persistent identifiers, such as DOIs, when applicable.
Find and discover data: Researchers and other users can now search for and discover datasets and files based on the metadata provided.
Manage access and metadata: The tool should also allow users to define who can access or download the data, whether the data is open, restricted, or subject to an embargo period. Users should also be able to edit, create, or delete metadata (using soft deletion where appropriate) and upload files through an interface. Access to restricted data should be controlled according to institutional policies and user permissions.
Link data to research projects: The application should be able to link harvested data or manually entered data and metadata to specific research projects, allowing researchers to organize and manage their datasets within the context of their research.
Automate as much as possible: For example, data generated by lab instruments or sensors could be automatically harvested and linked to existing metadata previously entered into the system by researchers and students.

What if AI is enabled?

AI should be optional, allowing users or institutions to enable or disable it.

An AI agent could read the data and metadata, suggest metadata fields, identify relevant ontologies and controlled vocabularies, and propose relationships between datasets, files, and metadata. The user should be able to review, approve, modify, or reject the schema proposed by the AI agent before it is applied.

The AI could also help assess how FAIR a dataset is by evaluating its metadata and other available information against established FAIR assessment criteria. The tool could present an indicative FAIR score or percentage based on a clearly defined assessment method, showing what is already good, what needs improvement, and recommendations based on the FAIR principles.

However, the score should be treated as guidance, not as an absolute measure of FAIRness, since some aspects cannot be reliably assessed automatically.

Ideally, the research institute or university would provide a locally hosted AI model so that research data and metadata could remain within the institution's controlled infrastructure, with appropriate security measures, access controls, and safeguards. This would be especially important for confidential, sensitive, or unpublished research data.

Metadata standards

The application should support established metadata standards, such as Dublin Core, with a minimum set of required fields defined by the institution or research community. These could include:

Title: The title of the dataset or file.
Description or Abstract: A description of the data and its purpose.
Creators: The people or organizations responsible for creating the data.
Date of Collection: When the data was collected.
Geographic Coverage: The geographic area covered by the data, such as a bounding box, when applicable.
Keywords: Terms describing the dataset, preferably using controlled vocabularies.
Identifier: A unique identifier for the dataset or file.
License and Access Rights: Information about how the data can be accessed and reused.

Users should also be able to select other metadata standards, such as Darwin Core for biodiversity data, depending on their research domain and requirements.

Final considerations

This is a very high-level overview of how an IT solution could serve as a tool to help researchers make their data FAIR.

Technical details such as data storage, archiving, backup strategies, security implementation, and infrastructure design are outside the scope of this article.

The goal is not to propose the perfect architecture, but to show how a relatively simple workflow, supported by institutional IT infrastructure and optionally AI, could reduce the manual work required from researchers while improving the findability, accessibility, interoperability, and reusability of their research data.

What Are the Pros and Cons?

The pros: The main advantage of this process is having a practical and standardized way to collect research data and help make it FAIR, with minimal additional work for researchers and students. We only ask them to prepare their data and metadata and upload their files into a predefined folder structure on the platform or through a user interface, such as a web browser or an installed application.

This approach could reduce repetitive tasks, especially metadata entry and dataset registration, while allowing researchers to focus more on their research. However, researchers would still need to provide meaningful documentation and verify that their data and metadata are accurate.

The cons: We may lose some granularity in the metadata and predefined relationships between entities. A simplified metadata model might not capture all the details required by specific research communities. However, nothing prevents us from adding more detailed metadata and defining these relationships based on the specific needs of each research project.

Conclusion

I have seen several applications with very cool features, but from my experience, I've noticed that many researchers and students want to make their data FAIR. However, we, as IT professionals, should provide them with feasible and practical solutions rather than tools that create more work for them.

Of course, not all researchers have the same understanding of FAIR principles or access to the same resources, which is why institutions should provide the necessary support, infrastructure, and guidance.

The goal should be to make FAIR practices a natural part of the research workflow, not an additional burden.

Also, Data governance is one of the pillars of FAIR and Open Science practices. It establishes the responsibilities, policies, and processes needed to manage research data effectively, while institutional infrastructure and tools help support their implementation.

This proposed framework is just one tool to help facilitate these practices, not a replacement for proper data governance.

Sources:
GO FAIR - the FAIR principles explained

The AI Iceberg

Flavio Francisco — Sat, 24 Oct 2026 12:01:39 +0000

We see a lot of news about AI everywhere, and much of it can bring anxiety and uncertainty about the future. While nobody can predict exactly how far AI will go, think back to when the internet was first created—who could have imagined it would one day replace our home phones and completely change how we watch TV?

But what is just hype, and what should we actually take seriously? That is exactly what I will explore in this article.

First, let's look at the "AI iceberg" represented in the picture below, where I will try to break down each layer in a friendly, easy-to-understand way.

On the surface, it represents all the most well-known, user-friendly tools available on the market to use the power of LLMs (Large Language Models), Agents, and image generation through prompts— but wait: What are prompts?

Prompts are instructions that you can give to the AI to do a task, create an Agent (I will explain later what it is), create a skill, generate an image, summarize a text, or just correct spelling and grammar errors.

For example, this is the prompt that I often use when I am writing emails or other texts, like this article:

From now on, for any text I paste here in English, French, Dutch, or Brazilian Portuguese (just for this session), please fix any grammar and spelling mistakes. Keep in mind that my native language is Brazilian Portuguese. With as few changes as possible, adapt the text so it sounds natural in those specific cultures: US/Canadian for English, Metropolitan French for French, and Netherlandic Dutch for Dutch. Keep the original meaning and wording as close to mine as possible.

To keep this text from getting too long, I'll stop diving into this ocean of an iceberg at the Agent level.

AI Agents

Let's now dive a little bit below the surface, where some of those popular Generative AI tools use AI agents to perform more complex tasks. With the following image, I will explain in a nutshell what is what, without too much technical jargon or detail.

LLM

Why are they called Large Language Models? The word "large" refers specifically to the sheer number of parameters inside the AI's network—typically ranging from billions to trillions—which makes them massive compared to older language models.

When we talk about "parameters," think of them like the billions of connections in a human brain. They allow the AI to learn how language works, using statistics to predict the next most likely word.

MCP (Tools)

Model Context Protocol (MCP) is a little technical, but to keep things simple and understandable to everyone, it provides a standardized way for AI applications to discover and interact with predefined functions (from services on the internet or locally), external tools, and data sources, making them accessible to LLMs.

Memory

Memory can be classified as short-term or long-term and can use databases, files, or other storage mechanisms. Memory helps retain information that can be retrieved later by the runtime orchestration system.

Run Time Orchestration System

Software that manages the execution loop, state management, tool execution, routing and handoffs, error handling, and safety limits. It could execute the following process, for example, as shown in the image below:

What are the impacts of AI now?

This is a very good question. We see a lot of hype and news aimed at driving engagement, which can cause anxiety for many of us, especially if we don't understand what AI is and what it's capable of.

Think of AI as an iceberg. Deep at the bottom, its foundation is mathematics and statistics. Then, as we move up a level, we have machine learning, followed by deep learning, until we reach the surface with generative AI (ChatGPT, Copilot, Claude, Gemini).

AI can perform tasks based on what it has learned during training. Depending on the AI system, it can do things like generate text and images, drive a car, pilot a drone, or describe images and videos.

A good metaphor I heard from a colleague is: "Using AI is like giving your brain a car."

However, there's an interesting nuance. A car doesn't automatically make someone a better driver, and AI doesn't automatically make someone a better thinker.

AI can improve your productivity if you use it as a tool for many tasks. For example, as a software developer, today I can create new features for an application in minutes or hours, whereas before, it could take days, weeks, or even a month. But AI doesn't replace me. Quite the opposite: it takes my knowledge and expertise to determine whether the generated results are safe, maintainable, and correct before they can be used.

But what could happen as AI evolves?

I believe many jobs will be transformed rather than completely replaced, although the impact will vary significantly across professions.

A team could become more productive with the same number of people.

Some fields are already feeling the effects of AI, such as translators, voice actors, and visual artists, including illustrators. Other fields are also being affected by the use of AI-powered robots in automation processes.

I cannot say what the future impact of AI will be as it continues to develop, but we can already see changes in our work environments. For some professions, the way we work today is not the same as it was three years ago.

Some companies are also reducing the number of junior professionals they hire in AI-exposed occupations as AI advances.

The only recommendation I can give you is to learn how to use and get the most out of AI tools like ChatGPT, Claude, Copilot, Gemini... you name it. These tools could become as essential to our work as Excel, Word, and PowerPoint became 10 or 20 years ago.

Another thing to consider is how we should use AI without harming our careers and reputations. For example, some people use AI to read documents, generate summaries, and respond to colleagues. I don't think this is always a good idea. If your colleague is asking for your opinion, are you using Copilot to help you understand the document, or are you letting Copilot form your opinion for you?

AI should help us think, not do all our thinking for us.

I'll leave this question open for you to think about: How are you using AI today?

Database normalization.

Flavio Francisco — Mon, 20 Mar 2023 12:07:35 +0000

Some facts

The words normalization and normal from refer to the structure of a database.
Normalization was developed by IBM researcher E.F. Codd in the 1970s.
Normalization increases clarity in organizing data in databases.

Scenario

Let's say that you are hired to develop a new eCommerce website for some company, than your client shows you the following table:

As you can see the table above has duplicated data and to better organize and optmize our data to be consumed by our new application we do the data modelling called: database normalization.

The 3 most used database normalization are:

First Normal Form (1NF);
Second Normal Form (2NF) and
Third Normal Form (3NF).

Also we have:

Boyce-Codd Normal Form (BCNF);
Fourth Normal Form (4NF) and
Fifth Normal Form (5NF).

I will explain only the 1NF, 2NF and the 3NF because they are the most common used.
The other forms will be explained in different articles.

First Normal Form (1NF)

To be considered a 1NF table:

Each table must have unique columns;
Using row order to convey information is not permitted;
Mixing data types within a primary key is not permitted;
Having a table without a primary key is not permitted and
Repeating groups are not permitted.

As you can see the 'Product Types' columns now is one column, but we can improve our data using the 2NF and that I will explain below.

Second Normal Form (2NF)

To be considered normalized in the 2NF our table must be already normalized as 1NF and each field of a table that is not the primary key is determined by that primary key.
But what is a primary key (PK)? Basically is a code the identifies a unique row in a table.

Take a look at the table above now. We have: Id, Client Name, Phone Number, Product Name, Quantity, Value and Product Type. Since the Id is the PK of the client we can apply the 2NF in the following way:

Does the phone number can be unique for a client? (for simplicity let's say: yes)
Does a product can be unique for a client? (no)
Does the quantity of an ordered product can be unique by a client? (no)
Does the value belongs to the client? (no)
Does the product type belongs to the client? (no).

Based on the answers above we can group the information in a such way that each data will be represented by an unique primary key.

NOTE: a primary key (PK) is a unique value for each row in a table. The PK is used to identify a unique record (row) in a database.
This value cannot be null and can be either a unique column or a set of column that represents an unique combination in a table.
We may have also the foreign Key (FK) is a column or group of columns in a relational database that represents the data in a linked table. A foreign key is just the reference from an table in another table.

Client

Product

But what does happened with the information of the orders? Now we will apply the third normal form (3NF).

Third Normal Form (3NF)

Every non-key attribute in a table should depend on the key, the whole key, and nothing but the key.

An entity is said to be in the third normal form when, it satisfies the criteria to be in the second normal form (2NF) and there is no transitive functional dependency. What this mean? If the table A dependent on table B and the table B is dependent in on the table C then C is transitively dependent on A and B.

So we need to identify the tables were the non-primary-keys field depend on something other than the primary key.

Our candidate for that is the Product table and the field: Product Type. Why? Because a product can only belong to a specific product type and product type can belong to multiple products and also the relations between clients and orders.
We will than have the following entities to have our database in a 3NF.

Client

Product

Product Type

Order

NOTE: I just decided to keep this database model simple. Of course we can include more columns as: creation time; total value; price before taxes and etc, but for explanation I would like to keep it simple.

To conclude our database model will look like the picture below:

Translating our data model we can say that:

1 client can have either 0 or many orders (0 to N);
1 product can be either in 0 or many orders (0 to N);
1 product can have 1 product type (1 to 1) and
1 product type can be assigned to many products (1 to N).

Normalization of a database offers several advantages, including the elimination of redundant information and the maintenance of data consistency. By avoiding the repetition of data, you can optimize storage space and reduce the likelihood of errors. Additionally, by using a unique identifier like the product's ID (primary key) to refer to data, changes to one piece of information, such as a product name, will not affect other related data. This can help ensure the accuracy and reliability of your database over time.

Well this is my first article on the internet and I hope that you enjoyed it.