A microservice for collecting and delivering npm package download counts via a simple API.
npm/download-counts is a microservice that provides historical download counts for npm packages through a minimal web API. It processes raw npm log data from AWS S3 and makes aggregated statistics available for querying, enabling developers and tools to track package popularity and usage trends over time.
Developers, data analysts, and tool creators who need programmatic access to npm package download statistics for monitoring, analysis, or integration into their applications.
It offers a straightforward JSON API with point, range, and bulk queries for essential download metrics without unnecessary complexity, and it automatically processes daily npm logs to keep data updated.
Background jobs and a minimal service for collecting and delivering download counts
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
The API uses minimal endpoints like /downloads/point/{period} and returns clean JSON, making integration straightforward without complex parsing.
Supports comma-separated lists for multiple packages in single requests, reducing API calls for fetching data in bulk as shown in the examples.
Offers both point and range queries for custom periods, enabling detailed trend analysis over specific date ranges.
Processes npm logs daily via a map-reduce cluster after UTC midnight, ensuring data is refreshed without manual intervention.
The README states this codebase isn't used by npm anymore, and documentation drifts, posing risks for long-term reliance.
Requires setting up VirtualBox, Vagrant, and MySQL with a VM, adding significant overhead compared to using the official API directly.
Data is updated only after UTC midnight with timezone-related lags, making it unsuitable for real-time applications as noted in the data source section.
The backfill script for importing from S3 requires AWS credentials and is intended for npm, Inc. use, limiting external deployment flexibility.