ENA Database

Overview

The European Nucleotide Archive (ENA) is a comprehensive public repository for nucleotide sequence data and associated metadata. Access and query DNA/RNA sequences, raw reads, genome assemblies, and functional annotations through REST APIs and FTP for genomics and bioinformatics pipelines.

When to Use This Skill

This skill should be used when:

• Retrieving nucleotide sequences or raw sequencing reads by accession

• Searching for samples, studies, or assemblies by metadata criteria

• Downloading FASTQ files or genome assemblies for analysis

• Querying taxonomic information for organisms

• Accessing sequence annotations and functional data

• Integrating ENA data into bioinformatics pipelines

• Performing cross-reference searches to related databases

• Bulk downloading datasets via FTP or Aspera

Core Capabilities

1. Data Types and Structure

ENA organizes data into hierarchical object types:

Studies/Projects - Group related data and control release dates. Studies are the primary unit for citing archived data.

Samples - Represent units of biomaterial from which sequencing libraries were produced. Samples must be registered before submitting most data types.

Raw Reads - Consist of:

• Experiments: Metadata about sequencing methods, library preparation, and instrument details

• Runs: References to data files containing raw sequencing reads from a single sequencing run

Assemblies - Genome, transcriptome, metagenome, or metatranscriptome assemblies at various completion levels.

Sequences - Assembled and annotated sequences stored in the EMBL Nucleotide Sequence Database, including coding/non-coding regions and functional annotations.

Analyses - Results from computational analyses of sequence data.

Taxonomy Records - Taxonomic information including lineage and rank.

2. Programmatic Access

ENA provides multiple REST APIs for data access. Consult references/api_reference.md for detailed endpoint documentation.

Key APIs:

ENA Portal API - Advanced search functionality across all ENA data types

• Documentation: https://www.ebi.ac.uk/ena/portal/api/doc

• Use for complex queries and metadata searches

ENA Browser API - Direct retrieval of records and metadata

• Documentation: https://www.ebi.ac.uk/ena/browser/api/doc

• Use for downloading specific records by accession

• Returns data in XML format

ENA Taxonomy REST API - Query taxonomic information

• Access lineage, rank, and related taxonomic data

ENA Cross Reference Service - Access related records from external databases

• Endpoint: https://www.ebi.ac.uk/ena/xref/rest/

CRAM Reference Registry - Retrieve reference sequences

• Endpoint: https://www.ebi.ac.uk/ena/cram/

• Query by MD5 or SHA1 checksums

Rate Limiting: All APIs have a rate limit of 50 requests per second. Exceeding this returns HTTP 429 (Too Many Requests).

3. Searching and Retrieving Data

Browser-Based Search:

• Free text search across all fields

• Sequence similarity search (BLAST integration)

• Cross-reference search to find related records

• Advanced search with Rulespace query builder

Programmatic Queries:

• Use Portal API for advanced searches at scale

• Filter by data type, date range, taxonomy, or metadata fields

• Download results as tabulated metadata summaries or XML records

Example API Query Pattern:

import requests

Search for samples from a specific study

base_url = "https://www.ebi.ac.uk/ena/portal/api/search" params = { "result": "sample", "query": "study_accession=PRJEB1234", "format": "json", "limit": 100 }

response = requests.get(base_url, params=params) samples = response.json()

4. Data Retrieval Formats

Metadata Formats:

• XML (native ENA format)

• JSON (via Portal API)

• TSV/CSV (tabulated summaries)

Sequence Data:

• FASTQ (raw reads)

• BAM/CRAM (aligned reads)

• FASTA (assembled sequences)

• EMBL flat file format (annotated sequences)

Download Methods:

• Direct API download (small files)

• FTP for bulk data transfer

• Aspera for high-speed transfer of large datasets

• enaBrowserTools command-line utility for bulk downloads

5. Common Use Cases

Retrieve raw sequencing reads by accession:

Download run files using Browser API

accession = "ERR123456" url = f"https://www.ebi.ac.uk/ena/browser/api/xml/{accession}"

Search for all samples in a study:

Use Portal API to list samples

study_id = "PRJNA123456" url = f"https://www.ebi.ac.uk/ena/portal/api/search?result=sample&query=study_accession={study_id}&format=tsv"

Find assemblies for a specific organism:

Search assemblies by taxonomy

organism = "Escherichia coli" url = f"https://www.ebi.ac.uk/ena/portal/api/search?result=assembly&query=tax_tree({organism})&format=json"

Get taxonomic lineage:

Query taxonomy API

taxon_id = "562" # E. coli url = f"https://www.ebi.ac.uk/ena/taxonomy/rest/tax-id/{taxon_id}"

6. Integration with Analysis Pipelines

Bulk Download Pattern:

• Search for accessions matching criteria using Portal API

• Extract file URLs from search results

• Download files via FTP or using enaBrowserTools

• Process downloaded data in pipeline

BLAST Integration: Integrate with EBI's NCBI BLAST service (REST/SOAP API) for sequence similarity searches against ENA sequences.

7. Best Practices

Rate Limiting:

• Implement exponential backoff when receiving HTTP 429 responses

• Batch requests when possible to stay within 50 req/sec limit

• Use bulk download tools for large datasets instead of iterating API calls

Data Citation:

• Always cite using Study/Project accessions when publishing

• Include accession numbers for specific samples, runs, or assemblies used

API Response Handling:

• Check HTTP status codes before processing responses

• Parse XML responses using proper XML libraries (not regex)

• Handle pagination for large result sets

Performance:

• Use FTP/Aspera for downloading large files (>100MB)

• Prefer TSV/JSON formats over XML when only metadata is needed

• Cache taxonomy lookups locally when processing many records

Resources

This skill includes detailed reference documentation for working with ENA:

references/

api_reference.md - Comprehensive API endpoint documentation including:

• Detailed parameters for Portal API and Browser API

• Response format specifications

• Advanced query syntax and operators

• Field names for filtering and searching

• Common API patterns and examples

Load this reference when constructing complex API queries, debugging API responses, or needing specific parameter details.