PigPen
A Clojure library for writing map-reduce queries that compile to Apache Pig or Cascading, enabling distributed data processing with Clojure syntax.
What is PigPen?
PigPen is a Clojure library that enables developers to write distributed map-reduce queries using Clojure syntax, which then compile to Apache Pig or Cascading for execution on big data clusters. It solves the problem of bridging Clojure's expressive power with scalable data processing frameworks, allowing functional data transformations without switching languages.
Clojure developers working with large-scale data processing who want to leverage Apache Pig or Cascading without learning Pig Latin or Java APIs, and data engineers building ETL pipelines in a functional style.
Developers choose PigPen because it allows them to write distributed data processing logic in pure Clojure, with seamless local testing and debugging, while still benefiting from the scalability of proven big data frameworks like Pig and Cascading.
Overview
Map-Reduce for Clojure
Use Cases
Best For
- Writing ETL pipelines in Clojure for Hadoop clusters
- Processing large datasets with functional transformations
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
