Query & search registries

This guide walks through different ways of querying & searching LaminDB registries.

Let’s start by creating a few exemplary datasets and saving them into a LaminDB instance (hidden cell).

Show code cell content Hide code cell content

# !pip install 'lamindb[bionty]'
!lamin init --storage ./test-registries --modules bionty

# python
import lamindb as ln
import bionty as bt
from lamindb.core import datasets

ln.track("pd7UR7Z8hoTq0000")

# Create non-curated datasets
ln.Artifact(datasets.file_jpg_paradisi05(), key="images/my_image.jpg").save()
ln.Artifact(datasets.file_fastq(), key="raw/my_fastq.fastq").save()
ln.Artifact.from_df(datasets.df_iris(), key="iris/iris_collection.parquet").save()

# Create a more complex case
# observation-level metadata
ln.Feature(name="cell_medium", dtype="cat[ULabel]").save()
ln.Feature(name="sample_note", dtype="str").save()
ln.Feature(name="cell_type_by_expert", dtype="cat[bionty.CellType]").save()
ln.Feature(name="cell_type_by_model", dtype="cat[bionty.CellType]").save()
# dataset-level metadata
ln.Feature(name="temperature", dtype="float").save()
ln.Feature(name="study", dtype="cat[ULabel]").save()
ln.Feature(name="date_of_study", dtype="date").save()
ln.Feature(name="study_note", dtype="str").save()

## Permissible values for categoricals
ln.ULabel.from_values(["DMSO", "IFNG"], create=True).save()
ln.ULabel.from_values(
    ["Candidate marker study 1", "Candidate marker study 2"], create=True
).save()
bt.CellType.from_values(["B cell", "T cell"], create=True).save()

# Ingest dataset1
adata = datasets.small_dataset1(format="anndata")
curator = ln.Curator.from_anndata(
    adata,
    var_index=bt.Gene.symbol,
    categoricals={
        "cell_medium": ln.ULabel.name,
        "cell_type_by_expert": bt.CellType.name,
        "cell_type_by_model": bt.CellType.name,
    },
    organism="human",
)
artifact = curator.save_artifact(key="example_datasets/dataset1.h5ad")
artifact.features.add_values(adata.uns)

# Ingest dataset2
adata2 = datasets.small_dataset2(format="anndata")
curator = ln.Curator.from_anndata(
    adata2,
    var_index=bt.Gene.symbol,
    categoricals={
        "cell_medium": ln.ULabel.name,
        "cell_type_by_model": bt.CellType.name,
    },
    organism="human",
)
artifact2 = curator.save_artifact(key="example_datasets/dataset2.h5ad")
artifact2.features.add_values(adata2.uns)

→ initialized lamindb: testuser1/test-registries

→ connected lamindb: testuser1/test-registries

→ created Transform('pd7UR7Z8hoTq0000'), started new Run('2ZeiNvhz...') at 2025-01-12 14:05:47 UTC

! indexing datasets with gene symbols can be problematic: https://docs.lamin.ai/faq/symbol-mapping

• saving validated records of 'var_index'

✓ added 3 records from public with Gene.symbol for "var_index": 'CD8A', 'CD4', 'CD14'

✓ "var_index" is validated against Gene.symbol

✓ "cell_medium" is validated against ULabel.name

✓ "cell_type_by_expert" is validated against CellType.name

✓ "cell_type_by_model" is validated against CellType.name

! indexing datasets with gene symbols can be problematic: https://docs.lamin.ai/faq/symbol-mapping

• saving validated records of 'var_index'

✓ added 1 record from public with Gene.symbol for "var_index": 'CD38'

✓ "var_index" is validated against Gene.symbol

✓ "cell_medium" is validated against ULabel.name

✓ "cell_type_by_model" is validated against CellType.name

Get an overview

The easiest way to get an overview over all artifacts is by typing df(), which returns the 100 latest artifacts in the Artifact registry.

import lamindb as ln

ln.Artifact.df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1

You can include fields from other registries.

ln.Artifact.df(
    include=[
        "created_by__name",
        "ulabels__name",
        "cell_types__name",
        "feature_sets__registry",
        "suffix",
    ]
)

Show code cell output Hide code cell output

	uid	key	description	created_by__name	ulabels__name	cell_types__name	feature_sets__registry	suffix
id
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	Test User1	{Candidate marker study 2, DMSO, IFNG}	{B cell, T cell}	{bionty.Gene, Feature}	.h5ad
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	Test User1	{Candidate marker study 1, DMSO, IFNG}	{B cell, T cell}	{bionty.Gene, Feature}	.h5ad
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	Test User1	{None}	{None}	{None}	.parquet
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	Test User1	{None}	{None}	{None}	.fastq.gz
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	Test User1	{None}	{None}	{None}	.jpg

You can include information about which artifact measures which feature.

df = ln.Artifact.df(features=True)
ln.view(df)  # for clarity, leverage ln.view() to display dtype annotations

Show code cell output Hide code cell output

	uid	key	description	cell_type_by_expert	cell_type_by_model	study	cell_medium	temperature	study_note	date_of_study
id	str	str	str	cat[bionty.CellType]	cat[bionty.CellType]	cat[ULabel]	cat[ULabel]	float	str	date
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	nan	{'B cell', 'T cell'}	{'Candidate marker study 2'}	{'DMSO', 'IFNG'}	{21.6}	{'We had a great time performing this study and the results look compelling.'}	{'2024-12-01'}
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	{'B cell', 'T cell'}	{'B cell', 'T cell'}	{'Candidate marker study 1'}	{'DMSO', 'IFNG'}	nan	nan	nan
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	nan	nan	nan	nan	nan	nan	nan
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	nan	nan	nan	nan	nan	nan	nan
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	nan	nan	nan	nan	nan	nan	nan

The flattened table that includes information from all relevant registries is easier to understand than the normalized data. For comparison, here is how to see the later.

ln.view()

Show code cell output Hide code cell output

****************

* module: core *

****************

Artifact

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1

Feature

	uid	name	dtype	type	unit	description	synonyms	_expect_many	space_id	run_id	created_at	created_by_id	aux	_branch_code
id
8	wA9qL3Ay1T3M	study_note	str	None	None	None	None	True	1	1	2025-01-12 14:05:49.256491+00:00	1	None	1
7	SAPr0CrVMJvN	date_of_study	date	None	None	None	None	True	1	1	2025-01-12 14:05:49.249167+00:00	1	None	1
6	Cm9EHvLI3WHO	study	cat[ULabel]	None	None	None	None	True	1	1	2025-01-12 14:05:49.244174+00:00	1	None	1
5	1dEHbuLRsQ4Y	temperature	float	None	None	None	None	True	1	1	2025-01-12 14:05:49.238410+00:00	1	None	1
4	eZcA0YgjeNYj	cell_type_by_model	cat[bionty.CellType]	None	None	None	None	True	1	1	2025-01-12 14:05:49.233377+00:00	1	None	1
3	szGdbfLmXyQB	cell_type_by_expert	cat[bionty.CellType]	None	None	None	None	True	1	1	2025-01-12 14:05:49.227583+00:00	1	None	1
2	9X7moPuv5U8J	sample_note	str	None	None	None	None	True	1	1	2025-01-12 14:05:49.221805+00:00	1	None	1

FeatureSet

	uid	name	n	dtype	registry	hash	space_id	run_id	created_at	created_by_id	aux	_branch_code
id
1	jg7FOv89LyFNUtutQEOW	None	3	int	bionty.Gene	f2UVeHefaZxXFjmUwo9Ozw	1	1	2025-01-12 14:05:53.698926+00:00	1	None	1
2	ihUHDpWlndvoFR7TsBJx	None	4	None	Feature	6hO4Oh2CkJyZ8zHhzjfDpA	1	1	2025-01-12 14:05:53.705177+00:00	1	None	1
3	avHdeMiRM47hhF8lYbEB	None	3	int	bionty.Gene	QW2rHuIo5-eGNZbRxHMDCw	1	1	2025-01-12 14:05:56.509148+00:00	1	None	1
4	sbJJBZfcEdEvCm7UUAoV	None	2	None	Feature	FKSCl0fzfjZ889Nywzgzig	1	1	2025-01-12 14:05:56.515412+00:00	1	None	1

FeatureValue

	value	hash	space_id	feature_id	run_id	created_at	created_by_id	aux	_branch_code
id
1	21.6	None	1	5	1	2025-01-12 14:05:53.788999+00:00	1	None	1
2	2024-12-01	None	1	7	1	2025-01-12 14:05:53.789084+00:00	1	None	1
3	We had a great time performing this study and ...	None	1	8	1	2025-01-12 14:05:53.789151+00:00	1	None	1
4	22.6	None	1	5	1	2025-01-12 14:05:56.581727+00:00	1	None	1
5	2025-02-13	None	1	7	1	2025-01-12 14:05:56.581829+00:00	1	None	1

Run

	uid	name	started_at	finished_at	reference	reference_type	_is_consecutive	_status_code	space_id	transform_id	report_id	_logfile_id	environment_id	initiated_by_run_id	created_at	created_by_id	aux	_branch_code
id
1	2ZeiNvhz2MX9VnDSfw03	None	2025-01-12 14:05:47.124727+00:00	None	None	None	None	0	1	1	None	None	None	None	2025-01-12 14:05:47.124814+00:00	1	None	1

Storage

	uid	root	description	type	region	instance_uid	space_id	run_id	created_at	created_by_id	aux	_branch_code
id
1	6KXtsBrTt8qi	/home/runner/work/lamindb/lamindb/docs/test-re...	None	local	None	hlGq1WkbeSSf	1	None	2025-01-12 14:05:39.936706+00:00	1	None	1

Transform

	uid	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	created_at	created_by_id	aux	_branch_code
id
1	pd7UR7Z8hoTq0000	registries.ipynb	Query & search registries	notebook	None	None	None	None	1	None	None	True	2025-01-12 14:05:47.117746+00:00	1	None	1

ULabel

	uid	name	is_concept	description	reference	reference_type	space_id	run_id	created_at	created_by_id	aux	_branch_code
id
5	lCqmsMnE	cell_medium	False	None	None	None	1	1	2025-01-12 14:05:53.575327+00:00	1	None	1
4	ADvxchlb	Candidate marker study 2	False	None	None	None	1	1	2025-01-12 14:05:49.287032+00:00	1	None	1
3	C5PU6WG7	Candidate marker study 1	False	None	None	None	1	1	2025-01-12 14:05:49.286862+00:00	1	None	1
2	1NXaQvx0	IFNG	False	None	None	None	1	1	2025-01-12 14:05:49.271680+00:00	1	None	1
1	niUJwTRg	DMSO	False	None	None	None	1	1	2025-01-12 14:05:49.271465+00:00	1	None	1

******************

* module: bionty *

******************

CellType

	uid	name	ontology_id	abbr	synonyms	description	space_id	source_id	run_id	created_at	created_by_id	aux	_branch_code
id
2	7gRvACvc	T cell	None	None	None	None	1	None	1	2025-01-12 14:05:49.637378+00:00	1	None	1
1	1m3SGd1l	B cell	None	None	None	None	1	None	1	2025-01-12 14:05:49.637231+00:00	1	None	1

Gene

	uid	symbol	stable_id	ensembl_gene_id	ncbi_gene_ids	biotype	synonyms	description	space_id	source_id	organism_id	run_id	created_at	created_by_id	aux	_branch_code
id
4	iFxDa8hoEWuW	CD38	None	ENSG00000004468	952	protein_coding	CADPR1	CD38 molecule	1	11	1	1	2025-01-12 14:05:56.405952+00:00	1	None	1
3	3bhNYquOnA4s	CD14	None	ENSG00000170458	929	protein_coding		CD14 molecule	1	11	1	1	2025-01-12 14:05:53.558594+00:00	1	None	1
2	1j4At3x7akJU	CD4	None	ENSG00000010610	920	protein_coding	T4|LEU-3	CD4 molecule	1	11	1	1	2025-01-12 14:05:53.558499+00:00	1	None	1
1	6Aqvc8ckDYeN	CD8A	None	ENSG00000153563	925	protein_coding	P32|CD8|CD8ALPHA	CD8 subunit alpha	1	11	1	1	2025-01-12 14:05:53.558359+00:00	1	None	1

Organism

	uid	name	ontology_id	scientific_name	synonyms	description	space_id	source_id	run_id	created_at	created_by_id	aux	_branch_code
id
1	1dpCL6Td	human	NCBITaxon:9606	homo_sapiens	None	None	1	1	1	2025-01-12 14:05:50.573469+00:00	1	None	1

Source

	uid	entity	organism	name	in_db	currently_used	description	url	md5	source_website	space_id	dataframe_artifact_id	version	run_id	created_at	created_by_id	aux	_branch_code
id
103	5JnV	BioSample	all	ncbi	False	True	NCBI BioSample attributes	s3://bionty-assets/df_all__ncbi__2023-09__BioS...	918db9bd1734b97c596c67d9654a4126	https://www.ncbi.nlm.nih.gov/biosample/docs/at...	1	None	2023-09	None	2025-01-12 14:05:40.178621+00:00	1	None	1
102	MJRq	bionty.Ethnicity	human	hancestro	False	True	Human Ancestry Ontology	https://github.com/EBISPOT/hancestro/raw/3.0/h...	76dd9efda9c2abd4bc32fc57c0b755dd	https://github.com/EBISPOT/hancestro	1	None	3.0	None	2025-01-12 14:05:40.178536+00:00	1	None	1
101	6vJm	bionty.DevelopmentalStage	mouse	mmusdv	False	False	Mouse Developmental Stages	http://aber-owl.net/media/ontologies/MMUSDV/9/...	5bef72395d853c7f65450e6c2a1fc653	https://github.com/obophenotype/developmental-...	1	None	2020-03-10	None	2025-01-12 14:05:40.178450+00:00	1	None	1
100	10va	bionty.DevelopmentalStage	mouse	mmusdv	False	True	Mouse Developmental Stages	https://github.com/obophenotype/developmental-...		https://github.com/obophenotype/developmental-...	1	None	2024-05-28	None	2025-01-12 14:05:40.178364+00:00	1	None	1
99	7Zm9	bionty.DevelopmentalStage	human	hsapdv	False	False	Human Developmental Stages	http://aber-owl.net/media/ontologies/HSAPDV/11...	52181d59df84578ed69214a5cb614036	https://github.com/obophenotype/developmental-...	1	None	2020-03-10	None	2025-01-12 14:05:40.178279+00:00	1	None	1
98	1GbF	bionty.DevelopmentalStage	human	hsapdv	False	True	Human Developmental Stages	https://github.com/obophenotype/developmental-...		https://github.com/obophenotype/developmental-...	1	None	2024-05-28	None	2025-01-12 14:05:40.178194+00:00	1	None	1
97	1atB	Drug	all	chebi	False	False	Chemical Entities of Biological Interest	s3://bionty-assets/df_all__chebi__2024-07-27__...		https://www.ebi.ac.uk/chebi/	1	None	2024-07-27	None	2025-01-12 14:05:40.178108+00:00	1	None	1

Auto-complete records

For registries with less than 100k records, auto-completing a Lookup object is the most convenient way of finding a record.

import bionty as bt

# query the database for all ulabels or all cell types
ulabels = ln.ULabel.lookup()
cell_types = bt.CellType.lookup()

Show me a screenshot

With auto-complete, we find a ulabel:

study1 = ulabels.candidate_marker_study_1
study1

Show code cell output Hide code cell output

ULabel(uid='C5PU6WG7', name='Candidate marker study 1', is_concept=False, created_by_id=1, run_id=1, space_id=1, created_at=2025-01-12 14:05:49 UTC)

Get one record

get errors if more than one matching records are found.

print(study1.uid)

# by uid
ln.ULabel.get(study1.uid)

# by field
ln.ULabel.get(name="Candidate marker study 1")

Show code cell output Hide code cell output

C5PU6WG7

ULabel(uid='C5PU6WG7', name='Candidate marker study 1', is_concept=False, created_by_id=1, run_id=1, space_id=1, created_at=2025-01-12 14:05:49 UTC)

Query multiple records

Filter for all artifacts annotated by a ulabel:

ln.Artifact.filter(ulabels=study1).df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1

To access the results encoded in a filter statement, execute its return value with one of:

• df(): A pandas DataFrame with each record in a row.

• all(): A QuerySet.

• one(): Exactly one record. Will raise an error if there is none. Is equivalent to the .get() method shown above.

• one_or_none(): Either one record or None if there is no query result.

Note

filter() returns a QuerySet.

The registries in LaminDB are Django Models and any Django query works.

LaminDB re-interprets Django’s API for data scientists.

What does this have to do with SQL?

Under the hood, any .filter() call translates into a SQL select statement.

LaminDB’s registries are object relational mappers (ORMs) that rely on Django for all the heavy lifting.

Of note, .one() and .one_or_none() are the two parts of LaminDB’s API that are borrowed from SQLAlchemy. In its first year, LaminDB built on SQLAlchemy.

Search for records

You can search every registry via search(). For example, the Artifact registry.

ln.Artifact.search("iris").df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	None	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1

Here is more background on search and examples for searching the entire cell type ontology: How does search work?

Query related registries

Django has a double-under-score syntax to filter based on related tables.

This syntax enables you to traverse several layers of relations and leverage different comparators.

ln.Artifact.filter(created_by__handle__startswith="testuse").df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1

The filter selects all artifacts based on the users who ran the generating notebook. Under the hood, in the SQL database, it’s joining the artifact table with the user table.

Another typical example is querying all datasets that measure a particular feature. For instance, which datasets measure "CD8A". Here is how to do it:

cd8a = bt.Gene.get(symbol="CD8A")
# query for all feature sets that contain CD8A
feature_sets_with_cd8a = ln.FeatureSet.filter(genes=cd8a).all()
# get all artifacts
ln.Artifact.filter(feature_sets__in=feature_sets_with_cd8a).df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1

Instead of splitting this across three queries, the double-underscore syntax allows you to define a path for one query.

ln.Artifact.filter(feature_sets__genes__symbol="CD8A").df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1

Filter operators

You can qualify the type of comparison in a query by using a comparator.

Below follows a list of the most import, but Django supports about two dozen field comparators field__comparator=value.

and

ln.Artifact.filter(suffix=".h5ad", ulabels=study1).df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1

less than/ greater than

Or subset to artifacts greater than 10kB. Here, we can’t use keyword arguments, but need an explicit where statement.

ln.Artifact.filter(ulabels=study1, size__gt=1e4).df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1

in

ln.Artifact.filter(suffix__in=[".jpg", ".fastq.gz"]).df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	None	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	None	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1

order by

ln.Artifact.filter().order_by("created_at").df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1

# reverse ordering
ln.Artifact.filter().order_by("-created_at").df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1

ln.Artifact.filter().order_by("key").df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1

# reverse ordering
ln.Artifact.filter().order_by("-key").df()

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1

contains

ln.Transform.filter(name__contains="search").df().head(5)

Show code cell output Hide code cell output

	uid	id	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	created_at	created_by_id	aux	_branch_code

And case-insensitive:

ln.Transform.filter(name__icontains="Search").df().head(5)

Show code cell output Hide code cell output

	uid	id	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	created_at	created_by_id	aux	_branch_code

startswith

ln.Transform.filter(name__startswith="Research").df()

Show code cell output Hide code cell output

	uid	id	key	description	type	source_code	hash	reference	reference_type	space_id	_template_id	version	is_latest	created_at	created_by_id	aux	_branch_code

or

ln.Artifact.filter(ln.Q(suffix=".jpg") | ln.Q(suffix=".fastq.gz")).df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
1	7zehFvNzAqbgyKke0000	images/my_image.jpg	None	.jpg	None	None	29358	r4tnqmKI_SjrkdLzpuWp4g	None	None	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.924560+00:00	1	None	1
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	None	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1

negate/ unequal

ln.Artifact.filter(~ln.Q(suffix=".jpg")).df()

Show code cell output Hide code cell output

	uid	key	description	suffix	kind	otype	size	hash	n_files	n_observations	_hash_type	_key_is_virtual	_curator	_overwrite_versions	space_id	storage_id	version	is_latest	run_id	created_at	created_by_id	aux	_branch_code
id
2	QatK4Aha5pFaG8sF0000	raw/my_fastq.fastq	None	.fastq.gz	None	None	20	hi7ZmAzz8sfMd3vIQr-57Q	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:48.935725+00:00	1	None	1
3	aoAsGajTPK9DUant0000	iris/iris_collection.parquet	None	.parquet	dataset	DataFrame	5088	9_QyZIRSh4ExiWhliEBYyw	None	NaN	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:49.205031+00:00	1	None	1
4	jGN3p3eaCksf7tPQ0000	example_datasets/dataset1.h5ad	None	.h5ad	dataset	AnnData	25088	YMNwVfQZ78zwkB4shAQMfQ	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:53.665297+00:00	1	None	1
5	KCYOQeHTEq2jxdMN0000	example_datasets/dataset2.h5ad	None	.h5ad	dataset	AnnData	22384	yI0uyeBcL20WSAClKeREVA	None	3.0	md5	True	None	False	1	1	None	True	1	2025-01-12 14:05:56.476583+00:00	1	None	1