Spark-Redis

Judgement

What it is

Integration

What it does

A connector that lets Apache Spark read and write Redis strings, hashes, lists, sets and sorted sets as RDDs or DataFrames, including on Redis Cluster and in streaming jobs.

Adopt it when

Your Spark jobs need to load from or write to Redis, such as feeding a cache or feature store.

Watch out

The main branch targets Spark 3.2 with Scala 2.12, so it lags current Spark releases.

Ready for

Usable

Runs as

Library

Built with

Scala, Apache Spark and Redis

How established

Established: started 11 years ago, latest release v3.1.0.

What the license allows

You can build it into a product you sell, as long as you keep its copyright notice.

Judged on 2026-11-06 from the project's README and GitHub data. The license and age lines are worked out directly.

Built from

Follow it down: Spark-Redis → Scala3 → JDK → LLVM project → runs on a microchip

How compilers were built, back to punched paper tape

Open-source alternatives to Spark-Redis

1. 1
   pandas pandas-dev

   Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

   Building block Production-ready You analyse spreadsheets, CSVs or database extracts in Python.

   ★ 50k BSD-3-Clause Python Active, pushed today
   
2. 2
   DuckDB duckdb

   DuckDB is a high-performance analytical database system

   Building block Production-ready You analyse data that fits on one machine and want fast SQL without running a server.

   ★ 43k MIT C++ Active, pushed 6 days ago
3. 3
   Polars pola-rs

   Extremely fast Query Engine for DataFrames, written in Rust

   Building block Production-ready Your pandas workloads are slow or run out of memory on a single machine and you can rewrite them in Polars' expression API.

   ★ 40k MIT Rust Active, pushed today
   
4. 4
   marimo marimo-team

   A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

   Developer tool Usable Notebooks need to live in git, rerun reproducibly and double as small internal data apps without moving to Streamlit.

   ★ 23k Apache-2.0 Python Active, pushed today
   
5. 5
   PyGWalker Kanaries

   PyGWalker: Turn your dataframe into an interactive UI for visual analysis

   Developer tool Usable Data analysts want to explore a DataFrame visually without writing plotting code.

   ★ 16k Apache-2.0 Python Active, pushed 2 months ago
6. 6
   JupyterLab jupyterlab

   JupyterLab computational environment.

   Developer tool Production-ready Data teams work in notebooks all day and want several notebooks, consoles and files open side by side, with extensions installed from PyPI.

   ★ 15k BSD-3-Clause TypeScript Active, pushed today
7. 7
   Jupyter Notebook jupyter

   Jupyter Interactive Notebook

   Developer tool Production-ready Analysts or researchers want the familiar single-document notebook interface for exploring data and sharing worked examples.

   ★ 13k BSD-3-Clause Jupyter Notebook Active, pushed 6 days ago
   
8. 8
   cuDF NVIDIA

   cuDF - GPU DataFrame Library

   Building block Production-ready You process large tabular data and have NVIDIA GPUs available, especially to speed up existing pandas or Polars code.

   ★ 9.8k Apache-2.0 C++ Active, pushed today