--- title: "GPU-Accelerated Computing" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{GPU-Accelerated Computing} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( dpi = 72, fig.retina = 1, # keep the built vignettes small (CRAN size limit) collapse = TRUE, comment = "#>", fig.width = 7, fig.height = 5, message = FALSE, warning = FALSE ) # CRAN allows at most two cores options(RcppML.threads = 2) has_suggests <- requireNamespace("ggplot2", quietly = TRUE) ``` ## Motivation At scale, dimensionality reduction is bottlenecked by dense linear algebra. Each NMF iteration computes Gram matrices ($H H^T$, $W^T W$) and right-hand-side products ($A H^T$, $W^T A$) — operations that map directly onto GPU hardware where thousands of cores execute matrix multiplies in parallel. RcppML's GPU backend uses NVIDIA CUDA with cuBLAS and cuSPARSE to accelerate these operations, achieving **10–40× speedups** on real single-cell datasets even against a fully-utilized 56-thread CPU. The GPU advantage is not uniform across operations. Standard NMF already benefits significantly, but **cross-validation and SVD randomized show the largest gains** because these modes perform dense BLAS work (Gram matrix corrections, block matrix multiplies) that maps directly onto GPU hardware. The GPU is optional — all code falls back to CPU transparently when CUDA is unavailable. ## Quick Start ```{r eval = FALSE} library(RcppML) # Check GPU availability gpu_available() #> [1] TRUE gpu_info() #> device name total_mb free_mb #> 1 0 NVIDIA H100 NVL 95830 93500 # NMF with GPU — same API, add resource = "gpu" model <- nmf(A, k = 20, resource = "gpu", seed = 42) ``` The `resource` parameter accepts `"gpu"`, `"cpu"`, or `"auto"` (default). With `"auto"`, RcppML uses GPU when available and falls back to CPU otherwise. ## Benchmarks on Real Data How much faster is GPU in practice? We benchmarked 10 operations spanning standard NMF, cross-validation, IRLS (KL divergence), sparse-mask NMF, and SVD on two real single-cell RNA-seq datasets. ### Data Preparation ```{r eval = FALSE} library(RcppML) library(Matrix) library(SeuratData) # GitHub only: remotes::install_github("satijalab/seurat-data") library(Seurat) # ── hcabm40k: 40,000 bone marrow cells ── data("hcabm40k") hca <- UpdateSeuratObject(hcabm40k) counts <- hca[["RNA"]]$counts # 17,369 genes × 40,000 cells # Select top 5,000 variable genes, log-normalize gene_vars <- Matrix::rowMeans(counts^2) - Matrix::rowMeans(counts)^2 top5k <- head(order(gene_vars, decreasing = TRUE), 5000) hca_5k <- counts[top5k, ] lib_sizes <- Matrix::colSums(hca_5k) hca_norm <- log1p(hca_5k %*% Diagonal(x = 1e4 / lib_sizes)) hca_norm <- as(hca_norm, "dgCMatrix") # 5,000 × 40,000, nnz ≈ 33M, density ≈ 16.5% # 10K-cell subset for expensive k = 64 operations set.seed(42) hca_10k <- hca_norm[, sample(ncol(hca_norm), 10000)] # 5,000 × 10,000, nnz ≈ 8.3M # ── pbmc3k: 500 PBMCs (bundled with RcppML) ── data(pbmc3k, package = "RcppML") tmp <- tempfile(fileext = ".spz") writeBin(pbmc3k, tmp) pbmc <- st_read(tmp) pbmc_norm <- log1p(pbmc %*% Diagonal(x = 1e4 / Matrix::colSums(pbmc))) pbmc_norm <- as(pbmc_norm, "dgCMatrix") # 8,000 × 500, nnz ≈ 412K ``` ### Results All benchmarks ran on an **NVIDIA H100 NVL** (96 GB HBM3) against a **56-thread Intel Xeon Gold 6238R** (2 × 28 cores, all threads active). NMF iterations were forced with `tol = 0` and `maxit = 20`. ```{r benchmark-table, echo = FALSE} bench_df <- data.frame( Operation = c( "NMF (k=20, 40K cells)", "NMF (k=64, 10K cells)", "Cross-validation NMF (k=64, 10K cells)", "Cross-validation NMF (k=16, pbmc3k)", "KL-divergence NMF (k=16, pbmc3k)", "Sparse-mask NMF (k=20, 10K cells)", "SVD Lanczos (k=10, 40K cells)", "SVD randomized (k=10, 40K cells)", "SVD IRLBA (k=10, 40K cells)", "NMF (k=20, pbmc3k)" ), Dataset = c( "hcabm40k", "hcabm40k (10K)", "hcabm40k (10K)", "pbmc3k", "pbmc3k", "hcabm40k (10K)", "hcabm40k", "hcabm40k", "hcabm40k", "pbmc3k" ), CPU_s = c(38.45, 29.23, 75.31, 4.04, 23.37, 10.50, 4.78, 17.77, 5.30, 2.18), GPU_s = c(2.78, 0.88, 2.39, 0.20, 1.98, 0.75, 0.44, 0.41, 0.38, 0.21), stringsAsFactors = FALSE, check.names = FALSE ) bench_df$Speedup <- bench_df$CPU_s / bench_df$GPU_s bench_df <- bench_df[order(bench_df$Speedup, decreasing = TRUE), ] knitr::kable( data.frame( Operation = bench_df$Operation, `CPU (s)` = sprintf("%.1f", bench_df$CPU_s), `GPU (s)` = sprintf("%.2f", bench_df$GPU_s), Speedup = sprintf("%.0f×", bench_df$Speedup), check.names = FALSE ), align = "lrrr", caption = "CPU (56-thread Xeon) vs. GPU (H100 NVL) benchmarks" ) ``` ```{r benchmark-plot, echo = FALSE, fig.width = 7, fig.height = 4.2, eval = has_suggests} library(ggplot2) bench_df$Type <- ifelse(grepl("Cross-validation", bench_df$Operation), "Cross-validation", ifelse(grepl("KL", bench_df$Operation), "IRLS (KL loss)", ifelse(grepl("SVD", bench_df$Operation), "SVD", ifelse(grepl("Sparse-mask", bench_df$Operation), "Sparse-mask NMF", "Standard NMF")))) # Shorter labels for plot readability bench_df$Short <- gsub(" \\(.*", "", bench_df$Operation) bench_df$Label <- sprintf("%s (%.0f\u00d7)", bench_df$Short, bench_df$Speedup) bench_df$Label <- factor(bench_df$Label, levels = rev(bench_df$Label)) ggplot(bench_df, aes(x = Speedup, y = Label, fill = Type)) + geom_col(width = 0.65) + scale_fill_manual(values = c( "Cross-validation" = "#e63946", "IRLS (KL loss)" = "#f4a261", "Standard NMF" = "#457b9d", "Sparse-mask NMF" = "#2a9d8f", "SVD" = "#a8dadc" )) + labs(x = "GPU speedup (\u00d7 faster than CPU)", y = NULL, fill = NULL) + theme_minimal(base_size = 10) + theme( legend.position = "bottom", legend.key.size = unit(0.4, "cm"), panel.grid.major.y = element_blank(), axis.text.y = element_text(size = 8.5), plot.title = element_text(face = "bold", size = 11) ) + ggtitle("GPU speedup by operation (H100 NVL vs. 56-thread Xeon)") ``` ### Key Observations Three patterns stand out: 1. **SVD randomized shows the largest speedup** (43×). Randomized SVD performs full matrix–dense block multiplies ($A Q$, $A^T B$) that translate into a handful of cuSPARSE/cuBLAS calls on GPU. Cross-validation NMF at high rank is close behind (32×) because CV corrects the Gram matrix for every held-out entry each iteration — a $O(k^2)$ update per test-set column that batches naturally on GPU. 2. **All operations achieve ≥ 10× speedup**, even against a fully-utilized 56-thread Xeon. NMF at high rank ($k = 64$, 33×), IRLS KL-divergence (12×), and sparse-mask NMF (14×) all benefit from GPU BLAS acceleration. 3. **Speedups scale with problem size and rank**. The 40K-cell NMF at $k = 20$ (14×) benefits less than $k = 64$ on 10K cells (33×) because higher rank increases the Gram matrix and NNLS work that the GPU absorbs efficiently. With fewer CPU threads (e.g., 8), speedups increase proportionally. ## Why GPU Gains Are Larger for CV and IRLS Standard NMF spends most of each iteration on two operations: $$H H^T \quad (k \times k), \qquad A H^T \quad (m \times k)$$ Cross-validation adds a correction step: for each held-out column $j$ in the test set, the Gram matrix is adjusted by subtracting the contribution of the missing entries: $$G_j = H H^T - \sum_{i \in \text{mask}(j)} h_i h_i^T$$ On CPU, this per-column correction loop is sequential. On GPU, the corrections for all test columns are batched into a single sparse outer-product kernel. The result: CV overhead that dominates CPU time becomes negligible on GPU, pushing CV speedups well above standard NMF speedups. IRLS losses (KL divergence, Gamma–Poisson, etc.) similarly increase per-iteration work by solving weighted NNLS subproblems that require additional Gram matrix updates with observation-specific weights — again, heavy BLAS work that the GPU absorbs with minimal overhead. ## API Reference ### Resource Override Priority The compute backend is determined by (highest priority first): 1. `resource` parameter in `nmf()` or `svd()` 2. `RCPPML_RESOURCE` environment variable 3. `options(RcppML.gpu = TRUE/FALSE)` 4. Auto-detection (GPU if available, else CPU) ```{r eval = FALSE} # Force CPU even when GPU is available model <- nmf(A, k = 20, resource = "cpu", seed = 42) # Session-wide override via environment variable Sys.setenv(RCPPML_RESOURCE = "cpu") # Or via R option options(RcppML.gpu = FALSE) ``` ### GPU-Supported Features In this release the GPU runs only the configurations that have been checked against the same references as the CPU code. Every other request runs on the CPU and says so once: a message under `resource = "auto"`, a warning when `resource = "gpu"` was asked for. No option is dropped. | Request | Where it runs | |:--------|:--------------| | `nmf()` on sparse or dense input, Gaussian loss (`loss = "mse"`), with `nonneg`, `norm`, a user or SVD-based start | GPU | | ... with `L1` or `L2` on sparse input | GPU | | `svd()` / `pca()`: unconstrained `lanczos`, `krylov`, `randomized`, `deflation` | GPU | | Bipartition / `dclust()` | GPU | | `L1` or `L2` on dense input | CPU | | Cross-validation (`test_fraction`, `k = "auto"`) | CPU | | Non-Gaussian losses, `robust`, zero inflation | CPU | | `mask`, `target_H`, `upper_bound` | CPU | | `L21`, `angular`, `graph_W` / `graph_H` | CPU | | `projective`, `symmetric` | CPU | | `svd()` with constraints, penalties, `mask`, CV, `scale = TRUE`, or `method = "irlba"` | CPU | | Reading `.spz` files onto the GPU (`st_read_gpu()`) | not available | On the GPU, `solver = "cholesky"` is served by coordinate descent, which gives the exact non-negative least squares solution. The rows marked CPU are planned GPU work; each will move once its kernel matches the CPU result. ### GPU-Direct StreamPress Reading For very large datasets stored as `.spz` files, `st_read_gpu()` transfers the matrix directly into GPU memory, avoiding a full CPU-side `dgCMatrix` copy: ```{r eval = FALSE} gpu_handle <- st_read_gpu("large_dataset.spz") model <- nmf(gpu_handle, k = 30, seed = 42) st_free_gpu(gpu_handle) # optional — GC handles cleanup ``` See the [StreamPress](streampress.html) vignette for `.spz` format details. ## Building the GPU Library A normal installation (`install.packages("RcppML")` or `remotes::install_github("zdebruine/RcppML")`) builds the CPU package only, even when the CUDA Toolkit is on the `PATH`. In that state `gpu_available()` returns `FALSE` and every function runs on the CPU. The GPU shared library is a separate build with `nvcc`, done once per machine after installing or updating RcppML: ```{r eval = FALSE} library(RcppML) install_gpu() # compiles RcppML_gpu.so with nvcc (a few minutes) gpu_status() # shows what was searched, what loaded, and why not gpu_available() #> [1] TRUE ``` `gpu_status()` is the first thing to run when `gpu_available()` is `FALSE`: it reports whether `nvcc` was found, where the library was looked for, and the reason it was not loaded. ### Requirements - **NVIDIA GPU** with compute capability $\geq$ 7.0 (Volta or newer) - **CUDA Toolkit** $\geq$ 11.0 with cuBLAS and cuSPARSE (12.x recommended) ### Building from a source checkout Developers working in a clone of the repository can use the project Makefile instead: ```{bash eval = FALSE} # 1. Load CUDA on your system module load cuda/12.8.1 # HPC example # 2. Build the GPU shared library (use the project Makefile) cd RcppML/src/ make -f Makefile.gpu install # Or manually: nvcc -O3 \ -gencode arch=compute_80,code=sm_80 \ -gencode arch=compute_90,code=sm_90 \ -std=c++17 -Xcompiler -fPIC --shared \ -I../inst/include \ -o ../inst/lib/RcppML_gpu.so \ gpu_bridge_nmf.cu gpu_bridge_svd.cu gpu_bridge_cluster.cu \ gpu_bridge_utils.cu gpu_bridge_assess.cu sp_gpu_bridge.cu \ -lcublas -lcusparse -lcuda # 3. Verify Rscript -e 'library(RcppML); cat("GPU:", gpu_available(), "\n")' #> GPU: TRUE ``` Adapt the `-gencode` flags to your GPU architecture: ```{r arch-table, echo = FALSE} arch_df <- data.frame( `GPU Family` = c("V100", "A100", "H100", "RTX 3090", "RTX 4090"), `Compute Capability` = c("7.0", "8.0", "9.0", "8.6", "8.9"), Flag = c("arch=compute_70,code=sm_70", "arch=compute_80,code=sm_80", "arch=compute_90,code=sm_90", "arch=compute_86,code=sm_86", "arch=compute_89,code=sm_89"), check.names = FALSE, stringsAsFactors = FALSE ) knitr::kable(arch_df, align = "llr", caption = "CUDA architecture flags by GPU family") ``` ### Library Search Path `gpu_status()` lists the locations RcppML searches for `RcppML_gpu.so`, in order, and which one it loaded. ### GPU Memory NMF GPU memory usage is approximately: $$\text{Memory} \approx \text{nnz} \times 12 + (m + n) \times k \times 8 \text{ bytes}$$ where nnz is the number of non-zeros, $m$ and $n$ are matrix dimensions, and $k$ is rank. For a 30,000 × 20,000 matrix with 5% density at $k = 30$, this is roughly 3.6 GB — well within a 32 GB GPU. ### HPC / SLURM Usage ```{bash eval = FALSE} #!/bin/bash #SBATCH --partition=gpu #SBATCH --gres=gpu:1 #SBATCH --cpus-per-task=8 #SBATCH --mem=32G #SBATCH --time=04:00:00 module load r/4.5.2 cuda/12.8.1 Rscript my_analysis.R ``` ## Troubleshooting **GPU not detected?** ```{r eval = FALSE} gpu_available(force_recheck = TRUE) #> [1] FALSE gpu_status() ``` Common causes: the GPU library has not been built (run `install_gpu()`), CUDA drivers missing or outdated (`nvidia-smi` to check), GPU compute capability < 7.0, `RcppML_gpu.so` not on the search path, or architecture mismatch (library compiled for `sm_90` won't load on a V100). **Precision**: GPU uses FP32 for core operations and FP64 selectively for numerically sensitive accumulations. Results are equivalent to CPU (FP64) within floating-point tolerance. ## Key Takeaways 1. **Same API, different backend**: `resource = "gpu"` is the only change needed. 2. **10–40× speedups** on real single-cell data against a fully-utilized 56-thread CPU, with SVD randomized and cross-validation showing the largest gains. With fewer CPU threads, speedups scale proportionally. 3. **Automatic fallback**: If the GPU is unavailable or a request is outside the supported set above, RcppML runs it on the CPU and tells you. 4. **Direct disk-to-GPU** via `st_read_gpu()` for very large `.spz` datasets. *See the [NMF Fundamentals](nmf-fundamentals.html) vignette for comprehensive parameter documentation and the [StreamPress](streampress.html) vignette for `.spz` file format details.*