Skip to content

Add Gemini-based Dataset Explorer #2157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 170 additions & 0 deletions docs/notebooks/151_dataset_explorer.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a href=\"https://colab.research.google.com/github/gee-community/geemap/blob/master/docs/notebooks/151_dataset_explorer.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"/></a>\n",
"\n",
"**Earth Engine Dataset Explorer**\n",
"\n",
"The source code is adapted from the scienceai_ee_dataset_explorer_v0.ipynb notebook at <https://bit.ly/48cE24D>.\n",
"Credit to the original author Renee Johnston (<https://github.com/raj02006>).\n",
"\n",
"**Please note that this feature experimental. Using the Gemini API may incur charges.** Please checkout the pricing [here](https://ai.google.dev/pricing).\n",
"\n",
"\n",
"## Overview\n",
"\n",
"This notebook, adapted from the source code by the Science AI team in Google Research, implements the Earth Engine Dataset Explorer, designed to help users discover datasets within the Earth Engine catalog that are relevant to their geospatial analysis tasks. The core functionality includes:\n",
"\n",
"1. Loading and processing Earth Engine dataset metadata and pre-computed embeddings.\n",
"2. Implementing a dataset search function that uses vector similarity to find relevant datasets based on user queries.\n",
"3. Creating an interactive user interface that displays search results, LLM-generated dataset details, code samples, and map visualizations.\n",
"\n",
"To run the dataset explorer, run all cells, and then check out the \"Earth Engine Dataset Explorer\" section at the end of the notebook.\n",
"\n",
"\n",
"## Setup Details and Billing\n",
"\n",
"You will need:\n",
"\n",
"- A Google cloud project with the Earth Engine API enabled. ([Details](https://developers.google.com/earth-engine/cloud/earthengine_cloud_project_setup)).\n",
"- A Gemini API key. ([Details](https://ai.google.dev/gemini-api/docs/api-key)).\n",
"\n",
"\n",
"Each of the above can be stored in the [colab \"Secrets\" panel](https://medium.com/@parthdasawant/how-to-use-secrets-in-google-colab-450c38e3ec75). Add the following strings as secrets:\n",
"\n",
" - Use `GOOGLE_PROJECT_ID` for the Cloud project id.\n",
" - Use `GOOGLE_API_KEY` for the Gemini API key\n",
"\n",
"## Caveats\n",
"\n",
" - THIS TOOL IS UNSAFE, AS IT AUTOMATICALLY RUNS LLM-GENERATED\n",
"PYTHON CODE! USE AT YOUR OWN RISK.\n",
"\n",
" - This is an early prototype, bugs and unexpected behavior are likely. Code improvements and refactors to follow.\n",
"\n",
" - Currently no spatial or temporal filtering of the datasets occurs as part of the dataset search functionality. Filtering only happens based on semantic relevance. We hope to incorporate spatiotemporal filtering soon in a future version. Stay tuned.\n",
"\n",
" - The very lightweight use of the TextEmbedding API from VertexAI requires billing to be enabled in your Cloud project. It should be an extremely minimal expense. ([Details](https://cloud.google.com/vertex-ai/generative-ai/pricing)).\n",
"\n",
" - For assistance, please email [email protected].\n",
"\n",
"## Installation\n",
"\n",
"Uncomment the following line to uninstall the existing geemap package. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# %pip uninstall geemap -y"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Uncomment the second line to install the latest version of geemap. Restart the kernel after installing geemap."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# %pip install -U \"geemap[ai]\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Import library"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from geemap.ai import DatasetExplorer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create the Earth Engine Dataset Explorer\n",
"\n",
"Run the following cell to create the Earth Engine Dataset Explorer. It may take a few seconds to load datasets and embeddings. Please be patient."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"explorer = DatasetExplorer()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Display the Earth Engine Dataset Explorer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"explorer.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Type a query in the search bar and press Enter to search for relevant datasets. The search results will be displayed in the table below. Click on a dataset to view more details, code samples, and a map visualization.\n",
"\n",
"![image](https://github.com/user-attachments/assets/682c393c-3221-4ed9-9943-6fa25a196b3b)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
1 change: 1 addition & 0 deletions docs/tutorials.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,3 +160,4 @@ More video tutorials for geemap and Earth Engine are available on my [YouTube ch
148. DataTable Charts ([notebook](https://geemap.org/notebooks/148_chart_data_table))
149. Using Gemini and EE Genie for automatic code generation ([notebook](https://geemap.org/notebooks/149_gemini))
150. Visualizing Earth Engine data in 3D with MapLibre ([notebook](https://geemap.org/notebooks/150_maplibre))
151. Earth Engine Dataset Explorer ([notebook](https://geemap.org/notebooks/151_dataset_explorer))
1 change: 1 addition & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ More video tutorials for geemap and Earth Engine are available on my [YouTube ch
148. DataTable Charts ([notebook](https://geemap.org/notebooks/148_chart_data_table))
149. Using Gemini and EE Genie for automatic code generation ([notebook](https://geemap.org/notebooks/149_gemini))
150. Visualizing Earth Engine data in 3D with MapLibre ([notebook](https://geemap.org/notebooks/150_maplibre))
151. Earth Engine Dataset Explorer ([notebook](https://geemap.org/notebooks/151_dataset_explorer))

### 1. Introducing the geemap Python package for interactive mapping with Google Earth Engine

Expand Down
Loading