Advanced: Databricks notebooks
Place notebook analysis in the data pipeline without promising self-service access.
openJII uses Databricks in its ingestion and processing architecture. Researchers with advanced work may also collaborate through notebooks, but the current web interface has no self-service Open notebook or Request Databricks access control.
Start with a normal dataset export when the work fits in local R, Python, Julia, or a statistics package. That route is portable and keeps the analysis environment under your control.
When a notebook collaboration may help
- the dataset is too large for a practical local export;
- analysis needs raw or intermediate pipeline data not exposed as a convenient table;
- a team needs a shared, reproducible compute environment;
- platform engineers must add or validate a reusable transformation.
Before asking for help, prepare the experiment ID, dataset/table name, scientific question, required columns, expected output, collaborators, and any data-access constraints. Send that context through Getting help.
The medallion terms
- Bronze preserves source records close to their ingested form.
- Silver validates and standardizes data for reliable reuse.
- Gold presents analysis-oriented tables or aggregates.
These names describe pipeline responsibilities, not a guarantee that every researcher can browse each internal layer. For the technical design, see Architecture & Pipeline and Medallion layers.
Never place credentials, private device keys, or data outside your approved access scope in a notebook or support request.