Question 1

How do I exclude stop words in WordsCounted?

Accepted Answer

Use the exclude option with a string, regexp, lambda, symbol, or array. For example, pass a string like 'the and is' to filter common stop words, or use a lambda for custom logic, as detailed in the README.

Question 2

Does WordsCounted handle numbers in text analysis?

Accepted Answer

By default, it excludes numbers, but you can override this by passing a custom regexp pattern, such as /[\p{Alnum}\-']+/, to include alphanumeric characters in tokens for more flexible analysis.

Question 3

WordsCounted vs Treat for Ruby NLP?

Accepted Answer

WordsCounted excels at tokenization and detailed text statistics with flexible filtering, while Treat offers a broader NLP toolkit including stemming and classification. Choose WordsCounted for lightweight analysis and Treat for comprehensive NLP workflows.

Question 4

How to process large text files efficiently with WordsCounted?

Accepted Answer

Load files directly using from_file, but note that it reads the entire content into memory. For very large files, consider chunking manually, as the gem doesn't mention streaming or memory optimization features.

Question 5

Can WordsCounted analyze multilingual text with special characters?

Accepted Answer

Yes, it robustly handles UTF/unicode characters, treating diacritics as part of tokens. For example, Arabic words are processed correctly, and you can customize tokenization with regexp for specific language needs.

Question 6

What's the difference between token frequency and density in WordsCounted?

Accepted Answer

Token frequency shows how often each token appears, while density represents each token's proportion relative to the total token count, calculated as a percentage with adjustable precision, as explained in the API section.

words_counted

What is words_counted?

Overview

Use Cases

Best For

Related Projects

Found a gem we're missing?

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions