# Basic Syntax of jq_schema
"TLDR: This article introduces the jq_schema syntax encountered when using LangChain to perform RAG processing on JSON file data. jq_schema is the query syntax of the jq tool, used to parse, filter, and transform JSON data. The article documents common jq_schema syntax, including selecting the entire JSON data, selecting specific keys, selecting keys within nested objects, selecting array elements, and selecting specific elements within arrays. Additionally, the article provides advanced syntax, such as filtering and conditions, the pipe operator `|`, string processing, calculations, and mathematical operations. Finally, through case analysis, it demonstrates how to extract multiple fields from JSON objects and extract array elements into a specific format."
Today, while using LangChain to perform RAG processing on a JSON file's data, I encountered an unfamiliar parameter called jq_schema when using jsonLoader. jq_schema is the query syntax of the jq tool, and jq is a lightweight command-line tool and query language specifically designed for parsing, filtering, and transforming JSON data. Here, I will briefly document the common jq_schema syntax.
Basic Syntax
-
.(Root node): Selects the entire JSON data.{ "name": "Alice", "age": 25 }Expression:
.Result:{ "name": "Alice", "age": 25 } -
.key: Selects a specific key in an object. Expression:.nameResult:"Alice" -
.key1.key2: Selects a key in a nested object.{ "user": { "name": "Alice", "details": { "age": 25 } } }Expression:
.user.details.ageResult:25 -
.[]: Selects each element in an array.[1, 2, 3, 4]Expression:
.[]Result:1 2 3 4 -
.[index]: Selects a specific element in an array.[10, 20, 30, 40]Expression:
.[1]Result:20
Advanced Syntax
-
Filtering and Conditions:
-
Select elements in an array that meet a condition: Expression:
.[] | select(.id > 1)Result:[ {"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}, {"id": 3, "name": "Charlie"} ]{"id": 2, "name": "Bob"} {"id": 3, "name": "Charlie"}
-
-
Pipe Operator
|:- Passes the output of one expression as the input to the next expression.
{ "users": [ {"name": "Alice", "age": 25}, {"name": "Bob", "age": 30} ] }Expression:
.users[] | .nameResult:"Alice" "Bob" -
String Processing:
{ "message": "Hello, world!" }Expression:
.message | ascii_upcaseResult:"HELLO, WORLD!" -
Computation and Mathematical Operations:
[1, 2, 3, 4]Expression:
map(. * 2)Result:[2, 4, 6, 8]
Case Analysis
-
Extracting multiple fields from a JSON object:
{ "user": { "name": "Alice", "age": 25, "location": "Wonderland" } }Expression:
{name: .user.name, age: .user.age}Result:{ "name": "Alice", "age": 25 } -
Extracting array elements into a specific format:
[ {"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"} ]Expression:
.[] | "\(.id): \(.name)"Result:"1: Alice" "2: Bob"
Useful Functions
-
length: Returns the length of an array or string.[1, 2, 3, 4, 5]Expression:
lengthResult:5 -
map: Applies an expression to each element in an array.[1, 2, 3]Expression:
map(. * 2)Result:[2, 4, 6] -
has(key): Checks whether a key exists in an object.{"name": "Alice", "age": 25}Expression:
has("age")Result: