When dealing with LLM, if you know some tips on prompt engineering, you should know that we often need to  output the model in a structured format  . Structured output has many advantages over plain text. We can parse and use the data naturally through programs. Among the many structured formats, JSON is a very ideal choice. It brings many benefits, such as:

• Unified format: Each return is the expected structure, which greatly facilitates subsequent data processing and consumption
• Reduce hallucinations: Requiring the model to generate data in a fixed format can constrain the model to a certain extent and reduce the possibility of arbitrarily "fabricating" irrelevant content
• Clear relationships: JSON's nested structure naturally reflects the hierarchy and relationship between data
• The type is clear: It is clear at a glance whether the field is a string, number, or Boolean value, which is convenient for direct use in the application

When I was developing AI-related services in Python recently, I deeply felt the convenience of having LLM return JSON. However, in practice, I also encountered a headache: LLM is essentially a probabilistic model. Even if you repeatedly emphasize and clearly require it in the prompt (for example, telling it that the generated JSON must be json.loads Successfully parsed), it sometimes still outputs malformed JSON. Common failures are:

• Missing closing brace } or square brackets ]
• Double quotes in string values ​​are not escaped properly (\”), causing the parsing to be interrupted
• Too many or too few necessary commas

These problems not only affect program parsing, but also often cause the entire process to be interrupted. At first, I thought that the only way to save the situation was to constantly adjust the prompt and add a retry mechanism. But later, when I read the "Prompt Engineering White Paper" published by Google, I noticed that it mentioned a tool - json-repair, which is used to automatically fix illegal JSON output by LLM! Its appearance greatly reduces the mental burden and code complexity of handling such bad cases.

So I conducted an in-depth study of this library and compiled this article, hoping to help you who are facing the same problem.

What is json-repair

json-repair Is a Python library for fixing invalid JSON.

It is worth mentioning that this is a relatively new library. The author clearly mentioned in the Readme that the original intention of developing it was to solve the problem that the JSON format returned by LLM is not standardized - which coincides with our needs.

The library's functions are as follows:

1. Fix syntax errors in JSON

• Deal with missing quotes, misplaced commas, unescaped characters, incomplete key-value pairs, etc.
• Fix missing quotes, non-standard values ​​(such as true, false, null), and broken key-value structures.
• Fix malformed JSON arrays and objects
• Complete an incomplete or broken array/object by adding necessary elements (such as commas, brackets) or default values ​​(such as null, empty string "").
• Supports processing JSON that contains extra non-JSON content (such as comments or erroneous characters), and cleans it up to ensure the structure is valid.
• Autocomplete missing JSON values
• Automatically fill in missing fields with reasonable default values ​​(such as empty string "" or null) to ensure that legal JSON is finally generated.

To repair broken JSON, json-repair implements a simple BNF-based parser internally. It follows the following grammar rules to understand and repair JSON:

<json> ::= <primitive> | <container>

<primitive> ::= <number> | <string> | <boolean>
; Where:
; <number> is a valid real number expressed in one of a number of given formats
; <string> is a string of valid characters enclosed in quotes
; <boolean> is one of the literal strings 'true', 'false', or 'null' (unquoted)

<container> ::= <object> | <array>
<array> ::= '[' [ <json> *(', ' <json>) ] ']' ; A sequence of JSON values ​​separated by commas
<object> ::= '{' [ <member> *(', ' <member>) ] '}' ; A sequence of 'members'
<member> ::= <string> ': ' <json> ; A pair consisting of a name, and a JSON value

Get started json-repair

It is very simple to use, just a function:repair_jsonYou can experiment with the official Playground https://mangiucugna.github.io/json_repair/.

Here we take JSON without escaping correctly escaped double quotes as an example (LLM can easily return this JSON structure):

As you can see,repair_json The function successfully fixes the unescaped double quotes to \", so that the subsequent json.loads The call was able to proceed successfully. Please note:repair_json  The return value is still a string, you need to use it again json.loads or similar library to convert it to a Python object.

Note that although json-repair Very powerful, but not omnipotent. If the JSON string is severely damaged, such as completely missing the parentheses of the root object or the structure is so confusing that the original intention cannot be inferred, it cannot be repaired.json-repair is a post-processing tool to salvage problematic output. It does not address the root cause of why LLM produces malformed JSON. Best practice is still to combine good prompt engineering and possibly model fine-tuning to try to improve the quality of the original output.

json-repair The advantage of is that it provides a general, robust and fix-focused solution. It does not rely on a specific prompt or model, but directly processes the output string and tries to restore it to a standard format that can be parsed by standard libraries such as json.loads. It serves as a supplement to the existing process (prompt optimization, model selection) and provides a layer of reliable protection.

Benchmark

To verify json-repair  I conducted a simple Benchmark test to test the efficiency of real applications. Test setup: Input 1000 slightly damaged JSON strings to simulate common LLM output errors, and then measure json-repair Time consumption for different types of JSON fixes.

From the test results,repair_json The performance is excellent:

• The overall performance is excellent : it can process nearly 20,000 broken JSONs per second, with an average processing time of only 0.088 milliseconds per piece, which shows that this library is very efficient in large-scale data processing scenarios.
• Good stability : The standard deviation is 0.16 milliseconds, which is relatively small, indicating that the processing time does not fluctuate much and the performance is stable and predictable.
• Strong scalability :
• As the processing length increases (from short to long), the processing time increases from 0.049 ms to 0.259 ms
• As the processing complexity increases (from simple to complex), the processing time increases from 0.049 milliseconds to 0.159 milliseconds (although the time increases with the complexity, the increase is controllable, and the millisecond processing speed is maintained even in the most complex cases)
• Balanced error type processing : The processing time for different types of JSON errors (quote errors, mismatched brackets, extra commas, unescaped quotes, etc.) is similar, all around 0.08-0.09 milliseconds, with no obvious performance bottlenecks.

repair_json It is well optimized and maintains high performance even in the face of complex JSON breakages, which is a solid performance improvement over many solutions that require regular expressions or complex parsing.

Summary and suggestions

When developing applications involving LLM, it is strongly recommended that you make the model output structured data as much as possible, especially in JSON format. This can not only effectively reduce the risk of model hallucinations, but also significantly improve the reliability of returned data and the operability of downstream programs.

However, due to the inherent mechanism of LLM, even if the prompt is carefully designed, the output JSON still has a certain probability of defects. repair_json Such a library, which automates the robustness processing of LLM output, is a very pragmatic and efficient approach.

From the actual test and use,repair_json  It has the advantages of high repair success rate and fast processing speed. It is worth considering integration in various systems that need to process LLM output JSON as a reliable insurance.