More SQL Parsing!
A Python SQL parser that converts SQL queries into JSON-izable parse trees for translation to non-SQL datastores.
What is More SQL Parsing!?
mo-sql-parsing is a Python SQL parser that converts SQL queries into JSON-izable parse trees, enabling translation of SQL for use with non-SQL datastores. It solves the problem of providing a unified SQL interface across different database engines by parsing queries into a structured, manipulable format. The library supports multiple SQL dialects and allows programmatic analysis and transformation of SQL syntax.
Developers and data engineers building tools that need to translate or analyze SQL across different database systems, such as query translators, database abstraction layers, or SQL-based data pipelines.
Developers choose mo-sql-parsing for its robust multi-dialect support, clean JSON output that simplifies programmatic handling, and bidirectional SQL-JSON conversion, which is rare in open-source SQL parsers.
Overview
Let's make a SQL parser so we can provide a familiar interface to non-sql datastores!
Use Cases
Best For
- Translating SQL queries for execution on NoSQL or non-relational datastores
- Building database abstraction layers that support multiple SQL dialects
- Analyzing or rewriting SQL queries programmatically via JSON manipulation
- Creating SQL linting or formatting tools that need parse tree access
- Implementing custom SQL-based query engines or interpreters
- Generating SQL from structured data representations for dynamic query building
Not Ideal For
- Production systems requiring a stable, unchanging parse tree format due to frequent breaking version changes
- Applications needing full support for all SQL DML operations (beyond basic INSERT/UPDATE/DELETE) and stored procedures across all dialects
- Real-time SQL formatting or linting tools that depend on up-to-date, feature-complete JSON-to-SQL conversion
- High-performance environments where parsing speed is critical, as JSON conversion and dialect handling add overhead
Pros & Cons
Pros
Parses SQL from MySQL, SQLServer, BigQuery, Snowflake, and others with specific flavor modes, as detailed in the SQL Flavours section.
Converts SQL clauses into property-mapped JSON objects, making parse trees easy to manipulate programmatically, shown in parsing examples like SELECT clauses as arrays.
Includes a formatter to generate SQL from JSON documents, enabling SQL reconstruction and translation for non-relational datastores.
Offers an option to output functions in a consistent {op, args, kwargs} structure via calls=normal_op, simplifying traversal and analysis.
Cons
Project status admits only basic DML and limited UDF support, with CREATE PROCEDURE/FUNCTION restricted to MySQL, reducing utility for complex database workflows.
Version updates (e.g., 10 and 9) introduced significant output format changes, such as COUNT(DISTINCT) representation, which can break existing code without careful migration.
The formatter for generating SQL from JSON is noted to usually lag behind the parser, potentially causing inconsistencies or missing features in bidirectional use.
Version history describes error reporting as 'still terrible' (version 7), making debugging difficult when parsing fails or encounters unsupported SQL.
Frequently Asked Questions
Related Projects
SQLGlotPython SQL Parser and Transpiler
jOOQjOOQ is the best way to write SQL in Java
JSqlParserJSqlParser parses an SQL statement and translate it into a hierarchy of Java classes. The generated hierarchy can be navigated using the Visitor Pattern
sqlparseA non-validating SQL parser module for Python
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.