New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ONNX] Improve error checking for large model export #37798
Conversation
BowenBao
commented
May 4, 2020
- Add error message when onnx model file path is not a string.
- Add error message when model size exceed 2GB when large model export is not turned on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks.
@@ -957,6 +964,11 @@ std::tuple<std::string, RawDataExportMap> export_onnx( | |||
add_node_names, | |||
use_external_data_format, | |||
onnx_file_path); | |||
const size_t proto_size = graph_encoder.get_model_proto().ByteSizeLong(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering whether model_proto will be larger than 2GB, or just directly truncated when dumping to string.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nvm, not a valid question
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@houseroad merged this pull request in 122587d. |