How to integrate jscpd into a CI/CD pipeline like GitHub Actions?

Add a step in your workflow YAML that runs 'npx jscpd' with flags for paths and thresholds. jscpd is used by GitHub Super Linter, so you can reference their documentation for examples.

jscpd vs SonarQube for duplicate code detection?

jscpd is focused solely on duplication detection with wide language support and modular APIs, while SonarQube offers broader code quality analysis but may have licensing costs. jscpd is better for lightweight, customizable integration.

Does jscpd work with TypeScript or other transpiled languages?

Yes, jscpd supports TypeScript and many other languages, but for accuracy, run it on the source files before transpilation, as tokenization might be affected by generated code.

How to customize the minimum token length for duplication detection?

Use the '--min-tokens' flag in the CLI or set it in the programming API options to adjust sensitivity, with details available in the configuration documentation.

Can jscpd detect duplicates across different file formats, like JSON and YAML?

Yes, jscpd can detect duplicates across supported formats, but ensure all relevant file types are included in the scan pattern for comprehensive results.

Open-Awesome

jscpd

MITTypeScriptv5.0.12

A copy/paste detector for source code that supports over 150 programming languages and formats.

Visit Website GitHub

5.9k stars249 forks0 contributors

What is jscpd?

jscpd is a copy/paste detector for programming source code that identifies duplicated blocks across files. It helps developers reduce technical debt by finding code clones in over 150 programming languages and formats using the Rabin-Karp algorithm.

Target Audience

Developers, teams, and tool maintainers who need to enforce code quality, reduce duplication, and integrate duplication detection into CI/CD pipelines or code analysis services.

Value Proposition

Developers choose jscpd for its extensive language support, multiple integration options (CLI, server, API), and modular architecture that allows deep customization of the detection process.

Overview

Copy/paste detector for programming source code, supports 223 formats. AI-ready with token-efficient reporter, skill and MCP server.

Use Cases

Best For

Identifying copy/paste technical debt in large codebases
Integrating duplicate detection into CI/CD workflows
Enforcing code quality standards in team projects
Building custom code analysis tools with duplication detection
Analyzing multi-language repositories for code clones
Generating duplication reports for code review processes

Not Ideal For

Real-time code editing environments needing instant duplication feedback
Projects requiring automated refactoring or code fixes based on detection
Teams with very large repositories prioritizing fastest scan times, as LevelDB store is slower

Pros & Cons

Pros

Broad Format Support

Detects duplicates in over 150 programming languages and document formats, making it versatile for multi-language codebases as highlighted in the README.

Flexible Integration Methods

Offers CLI, server API, and programming API, enabling seamless use in CI/CD pipelines and custom tools, with examples provided in the usage sections.

Modular and Customizable

Built as separate packages like core, finder, and tokenizer, allowing deep customization for specific detection needs, as described in the packages table.

Efficient Detection Algorithm

Implements the Rabin-Karp algorithm for efficient clone finding, balancing speed and accuracy in large codebases.

Cons

Performance Overhead for Large Repos

The README admits the LevelDB store, used for big repositories, is slower than the default in-memory store, which can impact scan performance.

No Built-in Refactoring

jscpd only identifies duplicates without providing automated fixes or suggestions, requiring manual intervention for code improvements.

Complex Setup for Advanced Use

The modular architecture and multiple APIs, while powerful, can be complex to configure for custom integrations compared to all-in-one tools.

Frequently Asked Questions

Related Projects

ESLint

Find and fix problems in your JavaScript code.

expose yourself

Last commit10 months ago

http-server

A simple, zero-configuration, command-line http server

Stars14,200

Forks1,559

Last commit3 months ago

gtop

System monitoring dashboard for terminal

Stars9,922

Forks337

Last commit8 months ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub