When you’re working with any data or files in your research, you’re likely using some form of system, naming scheme or rules to store and easily find it. However, even if your data is neatly stored in well-named folders, it can sometimes be challenging to recall how it is all interconnected, especially if your data is accumulated, filtered, and refined over multiple steps throughout your research. For example, you can use raw data, clean it, combine it, merge it with other data sources, and filter it based on specific parameters, creating a network of dependencies that is difficult to remember. Once you update your raw data or change a parameter in your pipeline, you need to remember the entire process to rerun it. In this article, I want to show you a simple way of organising and documenting your data in Obsidian using the canvas.

My research deals with forests. I have tables describing in which forest plots which plants grow and at which abundances. Other tables describe which traits these species have or what climate was measured at these plots. It is a vast network of tables that are all connected in some way or another. My goal is to remember which data is used in which part of my research and how it is all connected. The current state looks something like this:

This documentation makes only sense to my research, but the method can be adapted for you as well. It is the result of three months of full-time work and is intended primarily as a means to document where data is stored and which scripts operate on which data. You can see in the video that mainly there are tables and blue boxes that are somehow connected. It allows me to:

1. Find the right data to answer my research questions
2. Find the scripts I used to process my data easily
3. Understand dependencies between various data sources
4. Visualise my data collection
5. Saves me a lot of time in the day-to-day work with this data

Your research is likely very different from mine, so you will find your own use cases and benefits. Instead of CSV tables, you might be organising papers for a systematic literature review or paintings and painters. However, the overarching benefit for everyone is that you document not only what you have but also how it connects and relates. This saves you a lot of time. Give it a try, it only takes half an hour to document in this way and may save you a lot of headaches down the road.

Let’s break down my example into details so you can adapt it to your needs.

What is the Obsidian Canvas?

The Obsidian Canvas is just a space that you can arrange notes, images and textboxes on. As simple as it is, the possibilities of using it are endless. If you have never used Obsidian, consider the top 10 reasons to switch to Obsidian. Here is how to create a canvas:

The canvas has many use cases, here are my two main uses for it:

Using the Canvas to organize academic projects

Obsidian Canvas for organizing academic papers

Defining the Elements of Your Documentation Canvas

In my research, I use four elements on the canvas:

• CSV tables (and other data formats) that store my data
• Scripts that transform, aggregate, compute something new or mix this data.
• Parameters that influence the scripts.
• Sometimes I document “Problems” as well, which helps me remember decisions and assumptions I made.

Each of them gets a colour – It is a good idea to put a legend somewhere for your future self:

Combining Canvas Elements

In this next step, I document how my raw data from different sources (in different formats) is being aggregated into two tables. The script responsible for this is called: dataParsingPipeline.py. If I ever need to re-run this step I immediately know where to find the script:

Notice that the blue boxes for scripts also describe a bit what is happening inside the script.

Storing Tables

While you can story any file type in Obsidian, you can’t preview most of them. To circumvent this, I simply take a screenshot of the table and drag into the canvas. Most importantly I name the screenshot identical to the name of the file, so I know where to find this table. In this example, my CSV file is called “AllNormed” and is stored in the folder “Changes”, so I name the screenshot of this CSV table: “Changes.AllNormed.jpg” making it apparent where the file resides:

Taking screenshots of tables helps me immensely to find the right data. With about 50 different tables in all different formats, knowing what the format of the table is helps me code and also understand logically how to generate new data.

Documenting problems

Quite often, data, especially in life sciences, is not precise. Ecologists in particular deal with so much “dirty” data and have to constantly make decisions on how to clean it, combine it and so on. I like to document these decisions and point out possible problems while creating my data canvas:

You can use emojis or colour schemes to highlight “problem boxes”.

Visualise important data

Tables are great for computers but terrible for humans. For important tables, I tend to add a small graph visualising the data in the tables:

If you don’t want to waste any time visualising data, just use a tool like Julius AI, which can visualise any data in any way you like.

Keep Track of Parameters

If you are using scripts to work on your data, you will likely have some parameters. For example, I have data from four different sources, but sources 3 and 4 are almost identical, so I decided to only use sources 1-3. This is a setting that I document and possibly change at a later point:

You can use more formal ways of documenting here and also point out where the parameters are stored.

Keep Track of Old and Unused Data

Research is not linear, so chances are you will end up with data that you don’t use anymore. But possibly you might revisit this step in the future. So don’t throw things out, document what you have and push it off to the fringes of your canvas.

Here, for example, I replaced climate data about my forest research with a different source that has better spatial resolution but worse temporal resolution. I might go back to this old data, so it is still there.

Summary

Use Obsidian Canvas for documenting what type of data you have and how it connects.

1. Create a legend of boxes for scripts, data tables, parameters, raw data sources etc.
2. Connect them into a logical flow showing what depends on what and describe these connections.
3. Paste preview screenshots of your data into the canvas for better memory.
4. Document problems and visualise your data where appropriate.

Most importantly, think outside the box. While this example uses CSV files and Python scripts, you might have a very different domain/problem. The key is to be systematic and to develop a “visual language” for your documentation canvas.