A fast parallel directory traversal library for Go that significantly outperforms filepath.WalkDir and godirwalk.
Fastwalk is a Go library that provides fast parallel directory traversal, offering a high-performance alternative to the standard library's filepath.WalkDir function. It solves the problem of slow directory scanning by using concurrent goroutines to process filesystem entries, resulting in 2.5-6x speed improvements depending on the operating system.
Go developers who need to efficiently scan large directory trees, such as those building file indexing tools, backup systems, or any application that requires fast filesystem traversal.
Developers choose Fastwalk because it provides dramatic performance improvements over standard Go directory walking functions while maintaining API compatibility, requiring minimal code changes to integrate, and offering better memory efficiency than existing alternatives like godirwalk.
Fast directory traversal for Golang
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Uses multiple goroutines for parallel traversal, with benchmarks showing 2.5x to 6x speed improvements over filepath.WalkDir across macOS, Linux, and Windows.
Maintains the same behavior and callback signature as filepath.WalkDir, making integration easy with minimal code changes, as stated in the README's features.
Allocates 50% less memory and requires 25% fewer allocations on macOS compared to the standard library, based on provided benchmark data.
Offers a configurable option to safely follow symbolic links during traversal, addressing a common filesystem edge case.
Requires callback functions to be safe for concurrent use, adding development overhead compared to the sequential filepath.WalkDir, as noted in the usage section.
On Windows, benchmarks show a 19% increase in memory allocation and 29% more allocations compared to filepath.WalkDir, which could be a concern for memory-constrained applications.
Focuses solely on traversal; lacks built-in support for real-time monitoring, complex filtering, or other advanced filesystem operations that might be needed in comprehensive tools.