Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#7176] json.h update #7181

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 44 additions & 16 deletions src/support/json.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@

#include "support/istring.h"
#include "support/safe_integer.h"
#include "support/utilities.h"

namespace json {

Expand Down Expand Up @@ -257,12 +258,22 @@ struct Value {
while (*curr && is_json_space(*curr)) \
curr++; \
}
#define skip_escaped_characters(ptr) \
while (*ptr && *ptr != '"') { \
if (*ptr == '\\' && *(ptr + 1)) { \
ptr++; \
} \
ptr++; \
}
skip();
if (*curr == '"') {
// String
curr++;
char* close = strchr(curr, '"');
assert(close);
char* close = curr;
skip_escaped_characters(close);
if (!(*close == '"')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!(*close == '"')) {
if (*close != '"') {

wasm::Fatal() << "Assertion failed (close == '\"') ";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
wasm::Fatal() << "Assertion failed (close == '\"') ";
wasm::Fatal() << "missing end quote";

(because "assertion" suggests it might be a debug-only assertion)

}
*close = 0; // end this string, and reuse it straight from the input
setString(curr);
curr = close + 1;
Expand Down Expand Up @@ -305,20 +316,37 @@ struct Value {
skip();
setObject();
while (*curr != '}') {
assert(*curr == '"');
curr++;
char* close = strchr(curr, '"');
assert(close);
*close = 0; // end this string, and reuse it straight from the input
IString key(curr);
curr = close + 1;
skip();
assert(*curr == ':');
curr++;
skip();
Ref value = Ref(new Value());
curr = value->parse(curr);
(*obj)[key] = value;
if (*curr == '"') {
curr++;
char* close = curr;
skip_escaped_characters(close);
if (!(*close == '"')) {
wasm::Fatal() << "Assertion failed (close == '\"') ";
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern of skipping escaped characters appears more than once. How about adding a helper function findEndOfQuoted()?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While doing so, the asserts at the end (next line) could be error checks with Fatal() << "error message" (so they also work in non-debug builds - asserts are debug-only).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added 2 helpers using macros : (used macros to stay consistent with the style of this file)

  • pattern of skipping chars
  • runtime assert (to take in account release builds), also I am guessing that we need to change all assert(..) occurrences in parse function at least, since all of them are required ?

Now I have a doubt about the "quoted unquoted" keys in json, while testing I used the file referenced here. the said file have keys unquoted, this is allowed by most of the tools out there (browser included) but this doesn't seem to be allowed by JSON RFC.

I added 2 tests to cover the PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the JSON spec, but it makes sense for us (as just another tool) to support patterns that are commonly supported by tools, even if the spec isn't clear there.

*close = 0; // end this string, and reuse it straight from the input
IString key(curr);
curr = close + 1;
skip();
assert(*curr == ':');
curr++;
skip();
Ref value = Ref(new Value());
curr = value->parse(curr);
(*obj)[key] = value;
} else {
// Unquoted key
char* start = curr;
while (*curr && *curr != ':' && !is_json_space(*curr)) {
curr++;
}
assert(*curr == ':');
IString key(std::string(start, curr - start).c_str());
curr++;
skip();
Ref value = Ref(new Value());
curr = value->parse(curr);
(*obj)[key] = value;
}
skip();
if (*curr == '}') {
break;
Expand Down
8 changes: 8 additions & 0 deletions test/lit/metadce/functions_with_names_need_escape.wat
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
;; NOTE: Assertions have been generated by update_lit_checks.py --all-items and should not be edited.
;; RUN: wasm-metadce %s --graph-file %s.json -S -o -

(module
(func $f (export "f")
(nop)
)
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading the full test now, I see that this does check we don't error during parse, but it doesn't check that we use the string. We do have more specific testing for JSON,

https://github.com/WebAssembly/binaryen/blob/main/test/gtest/json.cpp

Perhaps we can add a test there? Such a test could check that the output of parsing is correct too. Let me know if you want help writing such a test.

13 changes: 13 additions & 0 deletions test/lit/metadce/functions_with_names_need_escape.wat.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"name": "root",
"reaches": [
"f","f~!@#$%^&*()_+`-={}|[]\\:\";'<>?,./"
],
"root": true
},
{
"name": "f",
"export": "f"
}
]
8 changes: 8 additions & 0 deletions test/lit/metadce/keys_quoted_unquoted.wat
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
;; NOTE: Assertions have been generated by update_lit_checks.py --all-items and should not be edited.
;; RUN: wasm-metadce %s --graph-file %s.json -S -o -

(module
(func $f (export "f")
(nop)
)
)
13 changes: 13 additions & 0 deletions test/lit/metadce/keys_quoted_unquoted.wat.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
name: "root",
reaches: [
"f"
],
root: true
},
{
"name": "f",
"export": "f"
}
]
Loading