TensorFrames
TensorFlow binding for Apache Spark DataFrames, enabling TensorFlow program execution on Spark data.
What is TensorFrames?
TensorFrames is an experimental library that integrates TensorFlow with Apache Spark DataFrames, enabling distributed TensorFlow computations directly on Spark clusters. It allows users to manipulate Spark DataFrames using TensorFlow programs, bridging distributed data processing and machine learning operations. However, note that the project is deprecated, with pandas UDFs recommended as an alternative.
Data engineers and data scientists working with Apache Spark who need to run TensorFlow computations on distributed datasets within Spark workflows. It is particularly relevant for those using Scala or Python (via PySpark) in Spark 2.4+ environments.
Developers choose TensorFrames for its ability to execute TensorFlow graphs on Spark DataFrames without extensive infrastructure changes, offering automatic shape inference and block-wise operations for efficient processing. It provides a seamless integration between Spark's distributed data processing and TensorFlow's machine learning capabilities, though it is now deprecated in favor of pandas UDFs.
Overview
[DEPRECATED] Tensorflow wrapper for DataFrames on Apache Spark
Use Cases
Best For
- Running TensorFlow computations on distributed Spark DataFrames for scalable machine learning tasks.
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
