Pick documents

An annotation project rarely covers a whole database: you first select the documents worth annotating, such as the notes of one department over a year. In this tutorial, you turn a query on the sample database into an annotation file holding an editable copy of the selected notes, without writing the query yourself.

Filter the notes

In the Metanno sidebar, open the notes table of the clinic source. Type oncologie in the box under department, then click Apply source filters: the tab now shows the oncology notes only.

Copy the code

Click Copy code in the tab toolbar, then Prepare annotation, and paste the copied code in a notebook cell. It connects to the database, queries the oncology notes, saves them to an annotation file and opens it. Its comments describe each step, and the paths are the ones of your own machine.

Before running it, adapt two lines to your project. Add the radiology notes to the cohort, so that breast imaging reports are reviewed with the oncology notes, and give the file a meaningful name:

-    .filter(_['department'] == 'oncologie')
+    .filter(_['department'].isin(['oncologie', 'radiologie']))
...
-destination = Path('/home/you/metanno-home') / 'annotations' / 'annotation.pret'
+destination = Path('/home/you/metanno-home') / 'annotations' / 'oncology.pret'

With these changes, the code reads as below, with your own paths instead of /home/you. Paste your version in the empty cell under it:

from pathlib import Path

import ibis
from ibis import _

from pret import DataFrame, create_store, load_store, save_store
from metanno import view_opener


# Connections of metanno.yaml, rerun this cell if a connection is lost
clinic = ibis.connect('duckdb:///home/you/clinic.duckdb')

# Queries select source records, change the filters and limits for your cohort
notes = (
    clinic.table('notes')
    .filter(_['department'].isin(['oncologie', 'radiologie']))
    .limit(500)
)

# Tables maps the prepared data to the Store keys expected by the view recipe
tables = {
    'notes': DataFrame(notes),
}

# Parameters are keyword arguments for the view recipe, they do not change the source queries
parameters = {}

# Save editable copies of the query results and added fields, without changing the source database
destination = Path('/home/you/metanno-home') / 'annotations' / 'oncology.pret'
# Annotation views copy every table into the page so filters and edits apply at once, up to 50 MB.
# The options are saved with the Store and used again when it is reopened
options = {'dataframe_sync': 'full', 'dataframe_max_bytes': 50 * 1024 * 1024}
save_store(create_store(tables, sync=True), destination, metadata={
    'view': 'table', 'parameters': parameters, **options,
})
store = load_store(destination, **options)
view_opener(store)

Run it, then click Open with Metanno: the four selected notes open in an editable tab. oncology.pret also appears under Annotations in the sidebar, where you or a colleague can reopen it later.

Next

The notes are ready, but nothing is annotated yet. Pre-annotate them with EDS-NLP, so that annotators check entities instead of finding them all by hand.