Scala

Apache Kafka Streamsapache/kafka

A distributed event streaming platform for building high-performance data pipelines, streaming analytics, and data integration.

Deeplearning4jdeeplearning4j/deeplearning4j

A comprehensive JVM-based deep learning ecosystem for building, training, and deploying models with support for model import and distributed training.

bfg-repo-cleanerrtyley/bfg-repo-cleaner

A faster, simpler alternative to git-filter-branch for removing large files and sensitive data from Git repositories.

kafka-manageryahoo/kafka-manager

A web-based tool for managing Apache Kafka clusters, enabling cluster inspection, topic management, and partition operations.

Delta Lakedelta-io/delta

An open-source storage framework that enables building a Lakehouse architecture with ACID transactions and scalable metadata handling.

h2oh2oai/h2o-3

An open-source, in-memory platform for distributed and scalable machine learning with support for a wide range of algorithms and big data technologies.

gatlinggatling/gatling

An open-source load testing platform for modern performance engineering with code-first test definition and asynchronous architecture.

Smilehaifengl/smile

A fast and comprehensive machine learning framework for Java, Scala, and Kotlin with state-of-the-art algorithms and data visualization.

Apache Hudiapache/hudi

An open data lakehouse platform for incremental data processing with upserts, deletes, and time-travel queries.

Catstypelevel/cats

A lightweight, modular, and extensible library for functional programming in Scala.

Microsoft ML for Apache SparkAzure/mmlspark

An open-source library for building massively scalable machine learning pipelines on Apache Spark.

aerosolveairbnb/aerosolve

A machine learning library designed for human interpretability, featuring debuggable models and a feature transform language.

scalazscalaz/scalaz

A Scala library providing principled functional programming abstractions, including type classes and purely functional data structures.

Furyalipay/fury

A high-performance multi-language serialization framework using JIT compilation and zero-copy techniques for fast data exchange.

spark-nlpJohnSnowLabs/spark-nlp

A state-of-the-art Natural Language Processing library built on Apache Spark, offering 100,000+ pretrained models and pipelines in 200+ languages.

deequawslabs/deequ

A library built on Apache Spark for defining unit tests to measure data quality in large datasets.

Scaldingtwitter/scalding

A Scala API for Cascading that simplifies writing Hadoop MapReduce jobs with Scala integration.

Breezescalanlp/breeze

A numerical processing library for Scala, providing generic, clean, and powerful linear algebra and scientific computing capabilities.

Shapelessmilessabin/shapeless

A type class and dependent type based generic programming library for Scala, enabling boilerplate reduction and arity abstraction.

shogunshogun-toolbox/shogun

A unified and efficient machine learning toolbox with C++ core and multi-language interfaces, developed since 1999.

ElasticMQ (.8k)softwaremill/elasticmq

An in-memory message queue system with an Amazon SQS-compatible interface, designed for testing and local development.

spark-jobserverspark-jobserver/spark-jobserver

A RESTful job server for Apache Spark that provides a service interface for submitting and managing Spark jobs, jars, and contexts.

Twitter Utiltwitter/util

A collection of idiomatic, small, general-purpose Scala utilities for time, futures, pooling, and hashing.

driver-kotlin-coroutinemongodb/mongo-java-driver

Official MongoDB drivers for Java, Kotlin, and Scala applications to interact with MongoDB databases.

Ammonite-Shelllihaoyi/ammonite

A deprecated Scala REPL and scripting environment with improved terminal interaction and script runner capabilities.

Sciospotify/scio

A Scala API for Apache Beam and Google Cloud Dataflow, enabling unified batch and streaming data processing.

circecirce/circe

A JSON library for Scala and Scala.js with a focus on functional programming and type safety.

FS2(prev. 'Scalaz-Stream')functional-streams-for-scala/fs2

A purely functional, effectful, and polymorphic stream processing library for Scala built on Cats and Cats-Effect.

Apache Sedonaapache/incubator-sedona

A cluster computing framework for processing large-scale geospatial data within Apache Spark, Flink, and other big data systems.

Algebirdtwitter/algebird

A Scala library providing abstract algebra types and structures for building aggregation systems and analytics.

Coursiercoursier/coursier

A pure Scala library for fetching and managing artifacts and applications, serving as a dependency resolver and launcher.

summingbirdtwitter/summingbird

A library for writing MapReduce programs that execute on distributed platforms like Storm and Scalding using Scala/Java collection-like syntax.

Scaloidpocorall/scaloid

A Scala library that simplifies Android development with concise DSLs, eliminating boilerplate while maintaining type safety.

Mocky (k)MockyAbstract/Mocky

A simple tool to generate custom HTTP responses for testing web services and mocking backend APIs.

Elasticsearch Hadoopelastic/elasticsearch-hadoop

Native integration library for using Elasticsearch with Hadoop, Spark, and Hive for real-time search and analytics on big data.

sangriasangria-graphql/sangria

A Scala library for building spec-compliant GraphQL servers and clients.