ARCHS⁴ Help

First Steps

About ARCHS⁴

ARCHS⁴ provides access to gene counts from HiSeq 2000 and HiSeq 2500 platforms for human and mouse experiments. The website allows download of the data in H5 format for programmatic access as well as a 3-dimensional view of the sample space. Search features allow browsing of the data by meta data annotation, signature similarity and functional enrichment. Selected sample sets can be downloaded into a tab separated through auto-generated R scripts for further analysis. Reads are aligned with Kallisto using a custom cloud computing platform. Human samples are aligned against GRCh38 cdna reference and mouse samples against GRCm38 cdna. All processable files from the GEO/SRA database since June 2018 are processed and available for download.

Download files

Raw read counts can be downloaded by browsing to the top of the ARCHS⁴ website and selecting the Download option. The raw counts are separated by human and mouse samples and are provided in H5 format. H5 provides efficient compression of the gene counts data. Tab separated files can be extracted with the provided code and auto-generated scripts provided by the web interface.

Selected sample and gene sets are displayed in the Search Results section. Under downloads scripts can be downloaded for the given gene sets. In case of gene sets a text file containing the gene symbols is provided.

Gene sets can be exported to Enrichr for further analysis.

Chrome Extension

The ARCHS⁴ chrome extension is a browser extension that adds content to the landing pages of RNA-seq datasets available on the Gene Expression Omnibus (GEO) when samples have been processed by ARCHS⁴. The extension adds links to download files that contain the aligned reads mapped to genes with counts, as well as a heatmap visualization summary of the expression data from the processed samples using Clustergrammer. The ARCHS4 Chrome extension installed from the Chrome web store. An example of an enhanced GEO landing page for series GSE77243 is shown.

Gene landing pages

Gene landing pages are accessible through the search fields on the top right of the ARCHS4 interface. Genes can be searched by Entrez gene symbol. The gene search returns the expression distribution for major tissues and cell lines and predictions of biological function and regulatory properties of the target gene. If a gene is previously known to be part of a predicted gene set the terms are marked in green. If a gene has sufficient number of prior knowledge annotations a ROC curve shows how well the prior knowledge about the gene can be recovered.

Data visualization

WebGL data viewer

The data view port displays samples or genes relative to the other samples and genes in the dataset. Samples/Genes with similar expression are clustered. The layout is computed with t-SNE.

The visualization is dynamic and allows rotation and zooming. The two icons on the top right are the manual selection tool and a toggle button that moves the view port into a smaller window to the left of the webpage.

The colors of samples and gene sets can be modified in the result section.

Viewer options

The data view can display 4 precalculated data visualizations: human samples, human genes, mouse samples and mouse genes. To toggle between species, select either the human or mouse button on the left. To change from sample view to gene view select one of the yellow buttons. A new view will automatically be loaded on selection. Prior selections are saved and will be loaded again if the former mode is selected.

The colors of sample and gene sets can be modified in the result section.

Viewer user interaction

The visualization is dynamic an allows rotation and zoom. The two icons on the top right are the manual selection tool and a toggle button that moves the view port into a smaller window to the left of the webpage.

Data handling

About the H5 file format

Hierarchical Data Format (HDF) is an open source file format for large data storage. It allows programmatic accessibility of matrix entries based on column and row indices while allowing for efficient data compression. The H5 files provided by ARCHS⁴ contain raw read counts as well as detailed meta data information extracted from GEO.

Parsing H5 file

Scripts to extract tab separated gene expression files can be created through the graphical user interface of ARCHS⁴. The script has to be executed as an R-script. A free version of R can be downloaded from: www.rstudio.com. Upon execution the script should install all required dependencies, and then download the full gene expression file before extracting the selected samples.

# R script to download selected samples
# Copy code and run on a local machine to initiate download
# Check for dependencies and install if missing
library("rhdf5")    # can be installed using Bioconductor

destination_file = "human_matrix_v9.h5"
extracted_expression_file = "GSM2679484_expression_matrix.tsv"
url = "https://s3.amazonaws.com/mssm-seq-matrix/human_matrix_v9.h5"

# Check if gene expression file was already downloaded, if not in current directory download file form repository
if(!file.exists(destination_file)){
    print("Downloading compressed gene expression matrix.")
    download.file(url, destination_file, quiet = FALSE, mode = 'wb')
}

# Selected samples to be extracted
samp = c("GSM2679452","GSM2679453","GSM2679454","GSM2679455","GSM2679456","GSM2679457","GSM2679458","GSM2679459","GSM2679460","GSM2679461","GSM2679462","GSM2679463","GSM2679464","GSM2679465","GSM2679466","GSM2679467","GSM2679468","GSM2679469","GSM2679470","GSM2679471","GSM2679472","GSM2679473","GSM2679474","GSM2679475","GSM2679476","GSM2679477","GSM2679478","GSM2679479","GSM2679480","GSM2679481","GSM2679482",
"GSM2679483","GSM2679484","GSM2679485","GSM2679486","GSM2679487","GSM2679488","GSM2679489","GSM2679490","GSM2679491","GSM2679492","GSM2679493","GSM2679494","GSM2679495","GSM2679496","GSM2679497","GSM2679498","GSM2679499","GSM2679500","GSM2679501","GSM2679502","GSM2679503","GSM2679504","GSM2679505","GSM2679506","GSM2679507","GSM2679508","GSM2679509","GSM2679510","GSM2679511","")

# Retrieve information from compressed data
samples = h5read(destination_file, "meta/samples/geo_accession")
genes = h5read(destination_file, "meta/genes/genes")

# Identify columns to be extracted
sample_locations = which(samples %in% samp)

# extract gene expression from compressed data
expression = t(h5read(destination_file, "data/expression", index=list(sample_locations, 1:length(genes))))
H5close()
rownames(expression) = genes
colnames(expression) = samples[sample_locations]

# Print file
write.table(expression, file=extracted_expression_file, sep="\t", quote=FALSE, col.names=NA)
print(paste0("Expression file was created at ", getwd(), "/", extracted_expression_file))

Batch effect correction

Extracted samples from a specified tissue can originate from multiple series with slightly different experimental conditions. If desired batch effects from gene expression can be removed with the Combat library.

# R script to download selected samples
# Copy code and run on a local machine to initiate download
# Check for dependencies and install if missing

library("rhdf5")    # can be installed using Bioconductor
library("preprocessCore")
library("sva")

destination_file = "human_matrix_v9.h5"
extracted_expression_file = "GSM2679484_expression_matrix.tsv"
url = "https://s3.amazonaws.com/mssm-seq-matrix/human_matrix_v9.h5"

# Check if gene expression file was already downloaded, if not in current directory download file form repository
if(!file.exists(destination_file)){
    print("Downloading compressed gene expression matrix.")
    download.file(url, destination_file, quiet = FALSE, mode = 'wb')
}

# Selected samples to be extracted
samp = c("GSM2679452","GSM2679453","GSM2679454","GSM2679455","GSM2679456","GSM2679457","GSM2679458","GSM2679459","GSM2679460","GSM2679461","GSM2679462","GSM2679463","GSM2679464","GSM2679465","GSM2679466","GSM2679467","GSM2679468","GSM2679469","GSM2679470","GSM2679471","GSM2679472","GSM2679473","GSM2679474","GSM2679475","GSM2679476","GSM2679477","GSM2679478","GSM2679479","GSM2679480","GSM2679481","GSM2679482",
"GSM2679483","GSM2679484","GSM2679485","GSM2679486","GSM2679487","GSM2679488","GSM2679489","GSM2679490","GSM2679491","GSM2679492","GSM2679493","GSM2679494","GSM2679495","GSM2679496","GSM2679497","GSM2679498","GSM2679499","GSM2679500","GSM2679501","GSM2679502","GSM2679503","GSM2679504","GSM2679505","GSM2679506","GSM2679507","GSM2679508","GSM2679509","GSM2679510","GSM2679511","")

# Retrieve information from compressed data
samples = h5read(destination_file, "meta/samples/geo_accession")
genes = h5read(destination_file, "meta/genes/genes")

# Identify columns to be extracted
sample_locations = which(samples %in% samp)

# extract gene expression from compressed data
expression = t(h5read(destination_file, "data/expression", index=list(sample_locations, 1:length(genes))))
H5close()
rownames(expression) = genes
colnames(expression) = samples[sample_locations]

# normalize samples and correct for differences in gene count distribution
expression = log2(expression+1)
expression = normalize.quantiles(expression)

rownames(expression) = genes
colnames(expression) = samples[sample_locations]

# correct batch effects in gene expression
batchid = match(series, unique(series))
correctedExpression <- ComBat(dat=expression, batch=batchid, par.prior=TRUE, prior.plots=FALSE)

Search tools

Meta data search

Metadata search parses the tissue description field from GEO to find matches with the entered search term. The search ignores spaces and is case insensitive. Results are highliged in the data viewer and a result is added to the result list.

We preselected a series of cellular tissues based by cellular system. This allows simple browsing of the data for tissues of interest. Some tissue selections can return empty for either mouse or human samples.

Signature search

Signature search uses a list of high expressed genes and low expressed genes and identifies samples that match the given input. The gene expression is z-score normalized across samples to identify the relative gene expression.

Enrichment search

Enrichment search highlights samples that are enriched in gene sets from 8 gene set libraries (CHEA 2016, KEA 2016, Encode TF ChIP-seq 2016, KEGG 2016, MGI mammalian phenotype, Human phenotype, GO Biological Process, GO cellular component, GO molecular function)

Gene search

Open gene landing page when searching by gene symbol. A set of genes can be highlighted by selecting a gene set library and a corresponding gene set.

API

import json
import requests

URL = 'https://maayanlab.cloud/custom/rooky'

data = {
    "type": "full_signature",
    "signatureName": "example_query",
    "species": "human",
    "siggenes": ["A1BG", "AADACL3", "AASS", "ABCD4", "ABTB2", "TNS1", "TPTEP1"], 
    "signature": [1000, 21300, 1231, 0, 19923, 4000, 20000]
}

r = requests.post(URL, json=data)

r.status_code
r.json()

{
    'name': ['example_query'],
    'samples': [1462973, 2270363, 1897293, ...],
    'similarity': [[-0.9423], [1.1858], [-0.6544], ...]
}