Question 1

How to set up incremental data sync with Elasticsearch JDBC importer?

Accepted Answer

Use the 'statefile' parameter and a SQL query with a timestamp condition, like 'select * from products where mytimestamp > ?', binding to '$metrics.lastexecutionstart'. This saves state between runs to fetch only new records, as shown in the README example.

Question 2

Elasticsearch JDBC importer vs Logstash JDBC input: which should I use?

Accepted Answer

The JDBC importer is a standalone Java tool optimized for batch imports with fine-grained scheduling and SQL control, while Logstash is part of the ELK stack with better real-time streaming and plugin ecosystem. Choose based on your need for lightweight batch jobs versus integrated data pipelines.

Question 3

Can it handle nested JSON from database joins?

Accepted Answer

Yes, but with caveats. It uses column labels with dots as path separators (e.g., 'contact.employee') to create structured JSON, and bracket notation for arrays. However, the README notes limitations with deeply nested objects, so complex joins may require workarounds.

Question 4

Is Elasticsearch JDBC importer still maintained?

Accepted Answer

The last release listed is from 2016 for Elasticsearch 2.3.4, suggesting low activity. Check the GitHub repository for recent updates, but for newer Elasticsearch versions, consider alternatives or community forks to avoid compatibility issues.

Question 5

How to import CSV files using this tool?

Accepted Answer

Use the bundled CSV JDBC driver by setting 'driver' to 'org.xbib.jdbc.csv.CsvDriver' and the URL to point to your CSV directory, then run SQL like 'select * from mycsvfile'. The README provides an example with parameters for column types and separators.

Question 6

What databases are supported?

Accepted Answer

It works with any JDBC-compatible database, including MySQL, PostgreSQL, and SQL Server, as shown in examples. However, you must manually provide the JDBC driver jar for your database, as noted in the 'Bundled drivers' section.

elasticsearch-jdbc

What is elasticsearch-jdbc?

Overview

Use Cases

Best For

Related Projects

Found a gem we're missing?

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions