[ONNX] Fix for constant folding: Slice, Added ReduceL1 and ReduceL2 #35280

neginraoof · 2020-03-24T06:40:00Z

1- Added support for constant folding onnx::ReduceL1 and onnx::ReduceL2
2- Fixed constant folding for slice as onnx::Slice opset 11 supports negative axes and indices
3- Updated export of select opset 11
4- Separated test environment for test_utility_functions as environment variables could be overwritten by caffe2 quantization tests on CI

…oof/constFoldNorm

dr-ci · 2020-03-24T07:04:17Z

💊 CircleCI build failures summary and remediations

As of commit 586f50f (more details on the Dr. CI page):

1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following build failures do not appear to be due to upstream breakages (reran 1 job to discount flakiness):

pytorch_windows_vs2019_py36_cuda10.1_test1 (1/1)

Step: "Test" (full log | pattern match details) <confirmed not flaky by 2 failures>

pip._vendor.urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

  File "C:\Jenkins\Miniconda3\lib\site-packages\pip\_internal\network\utils.py", line 39, in response_chunks 
    decode_content=False, 
  File "C:\Jenkins\Miniconda3\lib\site-packages\pip\_vendor\urllib3\response.py", line 564, in stream 
    data = self.read(amt=amt, decode_content=decode_content) 
  File "C:\Jenkins\Miniconda3\lib\site-packages\pip\_vendor\urllib3\response.py", line 529, in read 
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining) 
  File "C:\Jenkins\Miniconda3\lib\contextlib.py", line 99, in __exit__ 
    self.gen.throw(type, value, traceback) 
  File "C:\Jenkins\Miniconda3\lib\site-packages\pip\_vendor\urllib3\response.py", line 443, in _error_catcher 
    raise ProtocolError("Connection broken: %r" % e, e) 
pip._vendor.urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)) 
 
(base) circleci@PACKER-5E70C982 C:\Users\circleci\project>if "10" == "9" goto cuda_build_9  
 
(base) circleci@PACKER-5E70C982 C:\Users\circleci\project>if "10" == "10" goto cuda_build_10  
 
(base) circleci@PACKER-5E70C982 C:\Users\circleci\project>pushd .  
 
(base) circleci@PACKER-5E70C982 C:\Users\circleci\project>if "" == "" (call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64 )  else (call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver= )  
********************************************************************** 
** Visual Studio 2019 Developer Command Prompt v16.5.0

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 38 times.

facebook-github-bot

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

torch/csrc/jit/passes/onnx/constant_fold.cpp

torch/onnx/symbolic_opset11.py

lara-hdr · 2020-03-24T21:49:14Z

torch/csrc/jit/passes/onnx/constant_fold.cpp

+    // It needs to be adjusted (+= dim value) for aten op
+    auto less_mask = at::lt(indices, 0);
+    auto indices_corr = at::add(indices, inputTensorValues[0].sizes()[axis]);
+    auto indices_masked = at::where(less_mask, indices_corr, indices);


I am not sure to understand what is happening here.
My understanding is that :

indices contains positive and negative indices

less mask is a mask to identify the negative indices

indices_corr is the "corrected" indices; inputTensorValues[0].sizes()[axis] is added for all indices even if they are positive ?

indices_masked is a mask on the corrected negative indices ?

The "where" op, takes values from "indices_corr" where condition (indices < 0) is true, and takes values from "indices" where condition is false. So in cases where indices < 0, we have indices += dim value.

do we have a test covering this pass?

Yes, this is in test_utility_functions line 157. Index is < 0

…oof/constFoldNorm

… into neraoof/constFoldNorm

…oof/constFoldNorm

… into neraoof/constFoldNorm # Conflicts: # torch/csrc/jit/passes/onnx/constant_fold.cpp

facebook-github-bot

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

neginraoof · 2020-04-01T00:29:15Z

@houseroad Can we close this? Thanks

houseroad

Thanks

facebook-github-bot · 2020-04-01T14:24:39Z

@houseroad merged this pull request in 60a3e82.

neginraoof added 6 commits March 23, 2020 21:11

Fix for slice

06773c9

Merge branch 'master' of https://github.com/pytorch/pytorch into nera…

de1a4b6

…oof/constFoldNorm

Fix for negative indices

7ff415f

typo

662c870

Separate utils tests

82308bb

separate utils tests

48ecf18

neginraoof requested a review from apaszke as a code owner March 24, 2020 06:40

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Mar 24, 2020

pytorchbot added the open source label Mar 24, 2020

flake8 fix

ecd7505

Fix for gather opset 10/11

4bfeac5

albanD requested a review from houseroad March 24, 2020 17:59

albanD added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Mar 24, 2020

facebook-github-bot reviewed Mar 24, 2020

View reviewed changes

Fix for negative axis value in gather constant folding

b340749

houseroad requested review from BowenBao, lara-hdr and KsenijaS March 24, 2020 21:17

lara-hdr reviewed Mar 24, 2020

View reviewed changes

neginraoof added 2 commits March 25, 2020 13:16

Update constant_fold.cpp

3e0dd69

Update constant_fold.cpp

8ac82f1

lara-hdr approved these changes Mar 26, 2020

View reviewed changes

neginraoof added 7 commits March 26, 2020 17:43

Enabling the test since we're separating CI jobs

733be10

Merge branch 'master' of https://github.com/pytorch/pytorch into nera…

de4c005

…oof/constFoldNorm

Merge branch 'neraoof/constFoldNorm' of github.com:neginraoof/pytorch…

8a8a7da

… into neraoof/constFoldNorm

Merge branch 'master' of https://github.com/pytorch/pytorch into nera…

66a8dfa

…oof/constFoldNorm

Merge branch 'master' of https://github.com/pytorch/pytorch into nera…

333c354

…oof/constFoldNorm

Merge branch 'master' of https://github.com/pytorch/pytorch into nera…

092f9c2

…oof/constFoldNorm

Merge branch 'master' into neraoof/constFoldNorm

3ad53a4

neginraoof added 2 commits March 30, 2020 15:38

clang formatting fix

a1f50db

Merge branch 'neraoof/constFoldNorm' of github.com:neginraoof/pytorch…

586f50f

… into neraoof/constFoldNorm # Conflicts: # torch/csrc/jit/passes/onnx/constant_fold.cpp

facebook-github-bot reviewed Mar 31, 2020

View reviewed changes

houseroad approved these changes Apr 1, 2020

View reviewed changes

facebook-github-bot closed this in 60a3e82 Apr 1, 2020

facebook-github-bot added the merged label Apr 1, 2020

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ONNX] Fix for constant folding: Slice, Added ReduceL1 and ReduceL2 #35280

[ONNX] Fix for constant folding: Slice, Added ReduceL1 and ReduceL2 #35280

neginraoof commented Mar 24, 2020 •

edited

dr-ci bot commented Mar 24, 2020 •

edited

facebook-github-bot left a comment

lara-hdr Mar 24, 2020

neginraoof Mar 25, 2020

lara-hdr Mar 26, 2020

neginraoof Mar 27, 2020

facebook-github-bot left a comment

neginraoof commented Apr 1, 2020

houseroad left a comment

facebook-github-bot commented Apr 1, 2020

[ONNX] Fix for constant folding: Slice, Added ReduceL1 and ReduceL2 #35280

[ONNX] Fix for constant folding: Slice, Added ReduceL1 and ReduceL2 #35280

Conversation

neginraoof commented Mar 24, 2020 • edited

dr-ci bot commented Mar 24, 2020 • edited

💊 CircleCI build failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_windows_vs2019_py36_cuda10.1_test1 (1/1)

facebook-github-bot left a comment

Choose a reason for hiding this comment

lara-hdr Mar 24, 2020

Choose a reason for hiding this comment

neginraoof Mar 25, 2020

Choose a reason for hiding this comment

lara-hdr Mar 26, 2020

Choose a reason for hiding this comment

neginraoof Mar 27, 2020

Choose a reason for hiding this comment

facebook-github-bot left a comment

Choose a reason for hiding this comment

neginraoof commented Apr 1, 2020

houseroad left a comment

Choose a reason for hiding this comment

facebook-github-bot commented Apr 1, 2020

neginraoof commented Mar 24, 2020 •

edited

dr-ci bot commented Mar 24, 2020 •

edited