onnx-1.7.0/ 0000775 0000000 0000000 00000000000 13655345213 011210 5 ustar root root onnx-1.7.0/.gitignore 0000664 0000000 0000000 00000002132 13655345213 013176 0 ustar root root ## General
# Compiled Object files
*.slo
*.lo
*.o
*.cuo
# Compiled Dynamic libraries
*.so
*.dylib
*.pyd
# Compiled Static libraries
*.lai
*.la
*.a
# Compiled protocol buffers
*.pb.h
*.pb.cc
onnx/*_pb2.py
onnx/*_pb.py
onnx/*_pb2.pyi
onnx/*_pb.pyi
# Compiled python
*.pyc
# Compiled MATLAB
*.mex*
# IPython notebook checkpoints
.ipynb_checkpoints
# Editor temporaries
*.swn
*.swo
*.swp
*~
# Sublime Text settings
*.sublime-workspace
*.sublime-project
# Eclipse Project settings
*.*project
.settings
# QtCreator files
*.user
# PyCharm files
.idea
# Visual Studio Code files
.vscode
# OSX dir files
.DS_Store
## ONNX
# build, distribute, and bins (+ python proto bindings)
build
build_*
.build_debug/*
.build_release/*
.setuptools-cmake-build/*
# setup.py intermediates
.eggs
dist
onnx.egg-info
*.ninja
.ninja_deps
.ninja_log
compile_commands.json
# generated files
onnx/version.py
compile_commands.json
# test generated files
.cache
.coverage
onnx/examples/.coverage.nbval
.pytest_cache
test_report
# autocomplete
.ycm_extra_conf.py
# test coverage data files
*.gcov
.mypy_cache
virtualenv
venv
onnx-1.7.0/README.md 0000664 0000000 0000000 00000015513 13655345213 012474 0 ustar root root 
[](https://travis-ci.org/onnx/onnx)
[](https://dev.azure.com/onnx-pipelines/onnx/_build/latest?definitionId=5&branchName=master)
[](http://powerci.osuosl.org/job/onnx-ppc64le-nightly-build/)
[Open Neural Network Exchange (ONNX)](https://onnx.ai) is an open ecosystem that empowers AI developers
to choose the right tools as their project evolves. ONNX provides an open source format for AI models, both deep learning and traditional ML. It defines an extensible computation graph model, as well as definitions of built-in operators and standard
data types. Currently we focus on the capabilities needed for inferencing (scoring).
ONNX is [widely supported](http://onnx.ai/supported-tools) and can be found in many frameworks, tools, and hardware. Enabling interoperability between different frameworks and streamlining the path from research to production helps increase the speed of innovation in the AI community. We invite the community to join us and further evolve ONNX.
# Use ONNX
* [Tutorials for creating ONNX models](https://github.com/onnx/tutorials).
* [Pre-trained ONNX models](https://github.com/onnx/models)
# Learn about the ONNX spec
* [Overview](docs/Overview.md)
* [ONNX intermediate representation spec](docs/IR.md)
* [Versioning principles of the spec](docs/Versioning.md)
* [Operators documentation](docs/Operators.md)
* [Python API Overview](docs/PythonAPIOverview.md)
# Programming utilities for working with ONNX Graphs
* [Shape and Type Inference](docs/ShapeInference.md)
* [Graph Optimization](docs/Optimizer.md)
* [Opset Version Conversion](docs/VersionConverter.md)
# Contribute
ONNX is a [community project](community). We encourage you to join the effort and contribute feedback, ideas, and code. You can participate in the [SIGs](community/sigs.md) and [Working Groups](community/working-groups.md) to shape the future of ONNX.
Check out our [contribution guide](https://github.com/onnx/onnx/blob/master/docs/CONTRIBUTING.md) to get started.
If you think some operator should be added to ONNX specification, please read
[this document](docs/AddNewOp.md).
# Discuss
We encourage you to open [Issues](https://github.com/onnx/onnx/issues), or use Gitter for more real-time discussion:
[](https://gitter.im/onnx/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# Follow Us
Stay up to date with the latest ONNX news. [[Facebook](https://www.facebook.com/onnxai/)] [[Twitter](https://twitter.com/onnxai)]
# Installation
## Binaries
A binary build of ONNX is available from [Conda](https://conda.io), in [conda-forge](https://conda-forge.org/):
```
conda install -c conda-forge onnx
```
## Source
### Linux and MacOS
You will need an install of protobuf and numpy to build ONNX. One easy
way to get these dependencies is via
[Anaconda](https://www.anaconda.com/download/):
```
# Use conda-forge protobuf, as default doesn't come with protoc
conda install -c conda-forge protobuf numpy
```
You can then install ONNX from PyPi (Note: Set environment variable `ONNX_ML=1` for onnx-ml):
```
pip install onnx
```
You can also build and install ONNX locally from source code:
```
git clone https://github.com/onnx/onnx.git
cd onnx
git submodule update --init --recursive
python setup.py install
```
Note: When installing in a non-Anaconda environment, make sure to install the Protobuf compiler before running the pip installation of onnx. For example, on Ubuntu:
```
sudo apt-get install protobuf-compiler libprotoc-dev
pip install onnx
```
### Windows
When building on Windows it is highly recommended that you also build protobuf locally as a static library. The version distributed with conda-forge is a DLL and this is a conflict as ONNX expects it to be a static lib.
#### Instructions to build protobuf and ONNX on windows
Step 1 : Build protobuf locally
```
git clone https://github.com/protocolbuffers/protobuf.git
cd protobuf
git checkout 3.9.x
cd cmake
# Explicitly set -Dprotobuf_MSVC_STATIC_RUNTIME=OFF to make sure protobuf does not statically link to runtime library
cmake -G "Visual Studio 15 2017 Win64" -Dprotobuf_MSVC_STATIC_RUNTIME=OFF -Dprotobuf_BUILD_TESTS=OFF -Dprotobuf_BUILD_EXAMPLES=OFF -DCMAKE_INSTALL_PREFIX=
msbuild protobuf.sln /m /p:Configuration=Release
msbuild INSTALL.vcxproj /p:Configuration=Release
```
Step 2: Build ONNX
```
# Get ONNX
git clone https://github.com/onnx/onnx.git
cd onnx
git submodule update --init --recursive
# Set environment variables to find protobuf and turn off static linking of ONNX to runtime library.
# Even better option is to add it to user\system PATH so this step can be performed only once.
# For more details check https://docs.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-library?view=vs-2017
set PATH=\bin;%PATH%
set USE_MSVC_STATIC_RUNTIME=0
# Optional : Set environment variable `ONNX_ML=1` for onnx-ml
# Build ONNX
python setup.py install
```
If you do not want to build protobuf and instead want to use protobuf from conda forge then follow these instructions.
However please note : This method is just added as a convenience for users and there is very limited support from ONNX team when using this method.
#### Instructions to build ONNX on windows in anaconda environment
```
# Use conda-forge protobuf
conda install -c conda-forge protobuf=3.9.2 numpy
# Get ONNX
git clone https://github.com/onnx/onnx.git
cd onnx
git submodule update --init --recursive
# Set environment variable for ONNX to use protobuf shared lib
set CMAKE_ARGS="-DONNX_USE_PROTOBUF_SHARED_LIBS=ON"
# Build ONNX
# Optional : Set environment variable `ONNX_ML=1` for onnx-ml
python setup.py install
```
## Verify Installation
After installation, run
```
python -c "import onnx"
```
to verify it works. Note that this command does not work from
a source checkout directory; in this case you'll see:
```
ModuleNotFoundError: No module named 'onnx.onnx_cpp2py_export'
```
Change into another directory to fix this error.
# Testing
ONNX uses [pytest](https://docs.pytest.org) as test driver. In order to run tests, first you need to install pytest:
```
pip install pytest nbval
```
After installing pytest, do
```
pytest
```
to run tests.
# Development
Check out [contributor guide](https://github.com/onnx/onnx/blob/master/docs/CONTRIBUTING.md) for instructions.
# License
[MIT License](LICENSE)
# Code of Conduct
[ONNX Open Source Code of Conduct](https://onnx.ai/codeofconduct.html)
onnx-1.7.0/.gitmodules 0000664 0000000 0000000 00000000360 13655345213 013364 0 ustar root root [submodule "third_party/pybind11"]
path = third_party/pybind11
url = https://github.com/pybind/pybind11.git
branch = master
[submodule "third_party/benchmark"]
path = third_party/benchmark
url = https://github.com/google/benchmark.git
onnx-1.7.0/cmake/ 0000775 0000000 0000000 00000000000 13655345213 012270 5 ustar root root onnx-1.7.0/cmake/ONNXConfig.cmake.in 0000664 0000000 0000000 00000001633 13655345213 015612 0 ustar root root # - Config file for the ONNX package
# It defines the following variable(s)
# ONNX_INCLUDE_DIRS - include directories for FooBar
# as well as ONNX targets for other cmake libraries to use.
# library version information
set(ONNX_VERSION "@ONNX_VERSION@")
# import targets
include ("${CMAKE_CURRENT_LIST_DIR}/ONNXTargets.cmake")
# include directory.
#
# Newer versions of CMake set the INTERFACE_INCLUDE_DIRECTORIES property
# of the imported targets. It is hence not necessary to add this path
# manually to the include search path for targets which link to gflags.
# The following lines are here for backward compatibility, in case one
# would like to use the old-style include path.
get_filename_component(
CMAKE_CURRENT_LIST_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
get_filename_component(
_INSTALL_PREFIX "${CMAKE_CURRENT_LIST_DIR}/../../../" ABSOLUTE)
set(ONNX_INCLUDE_DIRS "${_INSTALL_PREFIX}/include")
onnx-1.7.0/cmake/summary.cmake 0000664 0000000 0000000 00000003736 13655345213 015000 0 ustar root root # Prints accumulated ONNX configuration summary
function (onnx_print_configuration_summary)
message(STATUS "")
message(STATUS "******** Summary ********")
message(STATUS " CMake version : ${CMAKE_VERSION}")
message(STATUS " CMake command : ${CMAKE_COMMAND}")
message(STATUS " System : ${CMAKE_SYSTEM_NAME}")
message(STATUS " C++ compiler : ${CMAKE_CXX_COMPILER}")
message(STATUS " C++ compiler version : ${CMAKE_CXX_COMPILER_VERSION}")
message(STATUS " CXX flags : ${CMAKE_CXX_FLAGS}")
message(STATUS " Build type : ${CMAKE_BUILD_TYPE}")
get_directory_property(tmp DIRECTORY ${PROJECT_SOURCE_DIR} COMPILE_DEFINITIONS)
message(STATUS " Compile definitions : ${tmp}")
message(STATUS " CMAKE_PREFIX_PATH : ${CMAKE_PREFIX_PATH}")
message(STATUS " CMAKE_INSTALL_PREFIX : ${CMAKE_INSTALL_PREFIX}")
message(STATUS " CMAKE_MODULE_PATH : ${CMAKE_MODULE_PATH}")
message(STATUS "")
message(STATUS " ONNX version : ${ONNX_VERSION}")
message(STATUS " ONNX NAMESPACE : ${ONNX_NAMESPACE}")
message(STATUS " ONNX_BUILD_TESTS : ${ONNX_BUILD_TESTS}")
message(STATUS " ONNX_BUILD_BENCHMARKS : ${ONNX_BUILD_BENCHMARKS}")
message(STATUS " ONNX_USE_LITE_PROTO : ${ONNX_USE_LITE_PROTO}")
message(STATUS " ONNXIFI_DUMMY_BACKEND : ${ONNXIFI_DUMMY_BACKEND}")
message(STATUS " ONNXIFI_ENABLE_EXT : ${ONNXIFI_ENABLE_EXT}")
message(STATUS "")
message(STATUS " Protobuf compiler : ${PROTOBUF_PROTOC_EXECUTABLE}")
message(STATUS " Protobuf includes : ${PROTOBUF_INCLUDE_DIRS}")
message(STATUS " Protobuf libraries : ${PROTOBUF_LIBRARIES}")
message(STATUS " BUILD_ONNX_PYTHON : ${BUILD_ONNX_PYTHON}")
if (${BUILD_ONNX_PYTHON})
message(STATUS " Python version : ${PY_VERSION}")
message(STATUS " Python executable : ${PYTHON_EXECUTABLE}")
message(STATUS " Python includes : ${PYTHON_INCLUDE_DIR}")
endif()
endfunction()
onnx-1.7.0/cmake/ONNXConfigVersion.cmake.in 0000664 0000000 0000000 00000000570 13655345213 017157 0 ustar root root set(PACKAGE_VERSION "@ONNX_VERSION@")
# Check whether the requested PACKAGE_FIND_VERSION is compatible
if("${PACKAGE_VERSION}" VERSION_LESS "${PACKAGE_FIND_VERSION}")
set(PACKAGE_VERSION_COMPATIBLE FALSE)
else()
set(PACKAGE_VERSION_COMPATIBLE TRUE)
if ("${PACKAGE_VERSION}" VERSION_EQUAL "${PACKAGE_FIND_VERSION}")
set(PACKAGE_VERSION_EXACT TRUE)
endif()
endif()
onnx-1.7.0/cmake/unittest.cmake 0000664 0000000 0000000 00000006744 13655345213 015164 0 ustar root root set(UT_NAME ${PROJECT_NAME}_gtests)
set(ONNX_ROOT ${PROJECT_SOURCE_DIR})
set(ONNXIFI_TEST_DRIVER onnxifi_test_driver_gtests)
include(${ONNX_ROOT}/cmake/Utils.cmake)
find_package(Threads)
set(${UT_NAME}_libs ${googletest_STATIC_LIBRARIES})
set(${ONNXIFI_TEST_DRIVER}_libs ${googletest_STATIC_LIBRARIES})
list(APPEND ${UT_NAME}_libs onnx)
list(APPEND ${UT_NAME}_libs onnx_proto)
list(APPEND ${UT_NAME}_libs onnxifi_loader)
list(APPEND ${UT_NAME}_libs onnxifi)
list(APPEND ${UT_NAME}_libs ${PROTOBUF_LIBRARIES})
list(APPEND ${ONNXIFI_TEST_DRIVER}_libs onnx)
list(APPEND ${ONNXIFI_TEST_DRIVER}_libs onnx_proto)
list(APPEND ${ONNXIFI_TEST_DRIVER}_libs onnxifi_loader)
list(APPEND ${ONNXIFI_TEST_DRIVER}_libs ${PROTOBUF_LIBRARIES})
list(APPEND ${ONNXIFI_TEST_DRIVER}_libs onnxifi)
file(GLOB_RECURSE ${UT_NAME}_src "${ONNX_ROOT}/onnx/test/cpp/*.cc")
file(GLOB_RECURSE ${ONNXIFI_TEST_DRIVER}_src "${ONNX_ROOT}/onnx/backend/test/cpp/*.h" "${ONNX_ROOT}/onnx/backend/test/cpp/*.cc")
function(AddTest)
cmake_parse_arguments(_UT "" "TARGET" "LIBS;SOURCES" ${ARGN})
list(REMOVE_DUPLICATES _UT_LIBS)
list(REMOVE_DUPLICATES _UT_SOURCES)
add_executable(${_UT_TARGET} ${_UT_SOURCES})
add_dependencies(${_UT_TARGET} onnx onnx_proto googletest)
target_include_directories(${_UT_TARGET}
PUBLIC ${googletest_INCLUDE_DIRS}
${ONNX_INCLUDE_DIRS}
${PROTOBUF_INCLUDE_DIRS}
${ONNX_ROOT}
${CMAKE_CURRENT_BINARY_DIR})
target_link_libraries(${_UT_TARGET} ${_UT_LIBS} ${CMAKE_THREAD_LIBS_INIT})
if(TARGET protobuf::libprotobuf)
target_link_libraries(${_UT_TARGET} protobuf::libprotobuf)
else()
target_link_libraries(${_UT_TARGET} ${PROTOBUF_LIBRARIES})
endif()
if(WIN32)
target_compile_options(${_UT_TARGET}
PRIVATE /EHsc # exception handling - C++ may throw,
# extern "C" will not
)
add_msvc_runtime_flag(${_UT_TARGET})
endif()
if(MSVC)
target_compile_options(${_UT_TARGET}
PRIVATE /wd4146 # unary minus operator applied to
# unsigned type, result still
# unsigned from include\google\protob
# uf\wire_format_lite.h
/wd4244 # 'argument': conversion from 'google::
# protobuf::uint64' to 'int', possible
# loss of data
/wd4267 # Conversion from 'size_t' to 'int',
# possible loss of data
/wd4996 # The second parameter is ignored.
)
endif()
set(TEST_ARGS)
if(ONNX_GENERATE_TEST_REPORTS)
# generate a report file next to the test program
list(
APPEND
TEST_ARGS
"--gtest_output=xml:$.$.results.xml>"
)
endif()
add_test(NAME ${_UT_TARGET}
COMMAND ${_UT_TARGET} ${TEST_ARGS}
WORKING_DIRECTORY $)
endfunction(AddTest)
addtest(TARGET ${UT_NAME} SOURCES ${${UT_NAME}_src} LIBS ${${UT_NAME}_libs})
addtest(TARGET ${ONNXIFI_TEST_DRIVER} SOURCES ${${ONNXIFI_TEST_DRIVER}_src} LIBS ${${ONNXIFI_TEST_DRIVER}_libs})
onnx-1.7.0/cmake/external/ 0000775 0000000 0000000 00000000000 13655345213 014112 5 ustar root root onnx-1.7.0/cmake/external/googletest.cmake 0000664 0000000 0000000 00000002444 13655345213 017274 0 ustar root root include (ExternalProject)
set(googletest_INCLUDE_DIRS ${CMAKE_CURRENT_BINARY_DIR}/googletest/src/googletest/googletest/include)
set(googletest_URL https://github.com/google/googletest.git)
set(googletest_BUILD ${CMAKE_CURRENT_BINARY_DIR}/googletest/)
set(googletest_TAG e93da23920e5b6887d6a6a291c3a59f83f5b579e)
#0fe96607d85cf3a25ac40da369db62bbee2939a5
if(WIN32)
set(googletest_STATIC_LIBRARIES
${CMAKE_CURRENT_BINARY_DIR}/googletest/src/googletest/googletest/Release/gtest.lib)
else()
set(googletest_STATIC_LIBRARIES
${CMAKE_CURRENT_BINARY_DIR}/googletest/src/googletest/googletest/libgtest.a)
endif()
if(ONNX_USE_MSVC_STATIC_RUNTIME)
set(ONNX_USE_MSVC_SHARED_RUNTIME OFF)
else()
set(ONNX_USE_MSVC_SHARED_RUNTIME ON)
endif()
ExternalProject_Add(googletest
PREFIX googletest
GIT_REPOSITORY ${googletest_URL}
GIT_TAG ${googletest_TAG}
DOWNLOAD_DIR "${DOWNLOAD_LOCATION}"
BUILD_IN_SOURCE 1
BUILD_COMMAND ${CMAKE_COMMAND} --build . --config Release --target gtest
INSTALL_COMMAND ""
CMAKE_CACHE_ARGS
-DCMAKE_BUILD_TYPE:STRING=Release
-DBUILD_GMOCK:BOOL=OFF
-DBUILD_GTEST:BOOL=ON
-Dgtest_force_shared_crt:BOOL=${ONNX_USE_MSVC_SHARED_RUNTIME}
BUILD_BYPRODUCTS ${googletest_STATIC_LIBRARIES}
)
onnx-1.7.0/cmake/Utils.cmake 0000664 0000000 0000000 00000002250 13655345213 014371 0 ustar root root #
# Add MSVC RunTime Flag
function(add_msvc_runtime_flag lib)
if(${ONNX_USE_MSVC_STATIC_RUNTIME})
target_compile_options(${lib} PRIVATE $<$>:/MT> $<$:/MTd>)
else()
target_compile_options(${lib} PRIVATE $<$>:/MD> $<$:/MDd>)
endif()
endfunction()
function(add_onnx_global_defines target)
target_compile_definitions(${target} PUBLIC "ONNX_NAMESPACE=${ONNX_NAMESPACE}")
if(ONNX_ML)
target_compile_definitions(${target} PUBLIC "ONNX_ML=1")
endif()
if(ONNX_USE_LITE_PROTO)
target_compile_definitions(${target} PUBLIC "ONNX_USE_LITE_PROTO=1")
endif()
endfunction()
function(add_whole_archive_flag lib output_var)
if("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
set(${output_var} -Wl,-force_load,$ PARENT_SCOPE)
elseif(MSVC)
# In MSVC, we will add whole archive in default.
set(${output_var} -WHOLEARCHIVE:$>
PARENT_SCOPE)
else()
# Assume everything else is like gcc
set(${output_var}
"-Wl,--whole-archive $ -Wl,--no-whole-archive"
PARENT_SCOPE)
endif()
endfunction()
onnx-1.7.0/setup.py 0000664 0000000 0000000 00000027051 13655345213 012727 0 ustar root root from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from distutils.spawn import find_executable
from distutils import sysconfig, log
import setuptools
import setuptools.command.build_py
import setuptools.command.develop
import setuptools.command.build_ext
from collections import namedtuple
from contextlib import contextmanager
import glob
import os
import shlex
import subprocess
import sys
import platform
from textwrap import dedent
import multiprocessing
TOP_DIR = os.path.realpath(os.path.dirname(__file__))
SRC_DIR = os.path.join(TOP_DIR, 'onnx')
TP_DIR = os.path.join(TOP_DIR, 'third_party')
CMAKE_BUILD_DIR = os.path.join(TOP_DIR, '.setuptools-cmake-build')
WINDOWS = (os.name == 'nt')
CMAKE = find_executable('cmake3') or find_executable('cmake')
MAKE = find_executable('make')
install_requires = []
setup_requires = []
tests_require = []
extras_require = {}
################################################################################
# Global variables for controlling the build variant
################################################################################
# Default value is set to TRUE\1 to keep the settings same as the current ones.
# However going forward the recomemded way to is to set this to False\0
USE_MSVC_STATIC_RUNTIME = bool(os.getenv('USE_MSVC_STATIC_RUNTIME', '1') == '1')
ONNX_ML = not bool(os.getenv('ONNX_ML') == '0')
ONNX_VERIFY_PROTO3 = bool(os.getenv('ONNX_VERIFY_PROTO3') == '1')
ONNX_NAMESPACE = os.getenv('ONNX_NAMESPACE', 'onnx')
ONNX_BUILD_TESTS = bool(os.getenv('ONNX_BUILD_TESTS') == '1')
DEBUG = bool(os.getenv('DEBUG'))
COVERAGE = bool(os.getenv('COVERAGE'))
################################################################################
# Version
################################################################################
try:
git_version = subprocess.check_output(['git', 'rev-parse', 'HEAD'],
cwd=TOP_DIR).decode('ascii').strip()
except (OSError, subprocess.CalledProcessError):
git_version = None
with open(os.path.join(TOP_DIR, 'VERSION_NUMBER')) as version_file:
VersionInfo = namedtuple('VersionInfo', ['version', 'git_version'])(
version=version_file.read().strip(),
git_version=git_version
)
################################################################################
# Pre Check
################################################################################
assert CMAKE, 'Could not find "cmake" executable!'
################################################################################
# Utilities
################################################################################
@contextmanager
def cd(path):
if not os.path.isabs(path):
raise RuntimeError('Can only cd to absolute path, got: {}'.format(path))
orig_path = os.getcwd()
os.chdir(path)
try:
yield
finally:
os.chdir(orig_path)
################################################################################
# Customized commands
################################################################################
class ONNXCommand(setuptools.Command):
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
class create_version(ONNXCommand):
def run(self):
with open(os.path.join(SRC_DIR, 'version.py'), 'w') as f:
f.write(dedent('''\
# This file is generated by setup.py. DO NOT EDIT!
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
version = '{version}'
git_version = '{git_version}'
'''.format(**dict(VersionInfo._asdict()))))
class cmake_build(setuptools.Command):
"""
Compiles everything when `python setupmnm.py build` is run using cmake.
Custom args can be passed to cmake by specifying the `CMAKE_ARGS`
environment variable.
The number of CPUs used by `make` can be specified by passing `-j`
to `setup.py build`. By default all CPUs are used.
"""
user_options = [
(str('jobs='), str('j'), str('Specifies the number of jobs to use with make'))
]
built = False
def initialize_options(self):
self.jobs = None
def finalize_options(self):
if sys.version_info[0] >= 3:
self.set_undefined_options('build', ('parallel', 'jobs'))
if self.jobs is None and os.getenv("MAX_JOBS") is not None:
self.jobs = os.getenv("MAX_JOBS")
self.jobs = multiprocessing.cpu_count() if self.jobs is None else int(self.jobs)
def run(self):
if cmake_build.built:
return
cmake_build.built = True
if not os.path.exists(CMAKE_BUILD_DIR):
os.makedirs(CMAKE_BUILD_DIR)
with cd(CMAKE_BUILD_DIR):
build_type = 'Release'
# configure
cmake_args = [
CMAKE,
'-DPYTHON_INCLUDE_DIR={}'.format(sysconfig.get_python_inc()),
'-DPYTHON_EXECUTABLE={}'.format(sys.executable),
'-DBUILD_ONNX_PYTHON=ON',
'-DCMAKE_EXPORT_COMPILE_COMMANDS=ON',
'-DONNX_NAMESPACE={}'.format(ONNX_NAMESPACE),
'-DPY_EXT_SUFFIX={}'.format(sysconfig.get_config_var('EXT_SUFFIX') or ''),
]
if COVERAGE:
cmake_args.append('-DONNX_COVERAGE=ON')
if COVERAGE or DEBUG:
# in order to get accurate coverage information, the
# build needs to turn off optimizations
build_type = 'Debug'
cmake_args.append('-DCMAKE_BUILD_TYPE=%s' % build_type)
if WINDOWS:
cmake_args.extend([
# we need to link with libpython on windows, so
# passing python version to window in order to
# find python in cmake
'-DPY_VERSION={}'.format('{0}.{1}'.format(*sys.version_info[:2])),
])
if USE_MSVC_STATIC_RUNTIME:
cmake_args.append('-DONNX_USE_MSVC_STATIC_RUNTIME=ON')
if platform.architecture()[0] == '64bit':
cmake_args.extend(['-A', 'x64', '-T', 'host=x64'])
else:
cmake_args.extend(['-A', 'Win32', '-T', 'host=x86'])
if ONNX_ML:
cmake_args.append('-DONNX_ML=1')
if ONNX_VERIFY_PROTO3:
cmake_args.append('-DONNX_VERIFY_PROTO3=1')
if ONNX_BUILD_TESTS:
cmake_args.append('-DONNX_BUILD_TESTS=ON')
if 'CMAKE_ARGS' in os.environ:
extra_cmake_args = shlex.split(os.environ['CMAKE_ARGS'])
# prevent crossfire with downstream scripts
del os.environ['CMAKE_ARGS']
log.info('Extra cmake args: {}'.format(extra_cmake_args))
cmake_args.extend(extra_cmake_args)
cmake_args.append(TOP_DIR)
subprocess.check_call(cmake_args)
build_args = [CMAKE, '--build', os.curdir]
if WINDOWS:
build_args.extend(['--config', build_type])
build_args.extend(['--', '/maxcpucount:{}'.format(self.jobs)])
else:
build_args.extend(['--', '-j', str(self.jobs)])
subprocess.check_call(build_args)
class build_py(setuptools.command.build_py.build_py):
def run(self):
self.run_command('create_version')
self.run_command('cmake_build')
generated_python_files = \
glob.glob(os.path.join(CMAKE_BUILD_DIR, 'onnx', '*.py')) + \
glob.glob(os.path.join(CMAKE_BUILD_DIR, 'onnx', '*.pyi'))
for src in generated_python_files:
dst = os.path.join(
TOP_DIR, os.path.relpath(src, CMAKE_BUILD_DIR))
self.copy_file(src, dst)
return setuptools.command.build_py.build_py.run(self)
class develop(setuptools.command.develop.develop):
def run(self):
self.run_command('build_py')
setuptools.command.develop.develop.run(self)
class build_ext(setuptools.command.build_ext.build_ext):
def run(self):
self.run_command('cmake_build')
setuptools.command.build_ext.build_ext.run(self)
def build_extensions(self):
for ext in self.extensions:
fullname = self.get_ext_fullname(ext.name)
filename = os.path.basename(self.get_ext_filename(fullname))
lib_path = CMAKE_BUILD_DIR
if os.name == 'nt':
debug_lib_dir = os.path.join(lib_path, "Debug")
release_lib_dir = os.path.join(lib_path, "Release")
if os.path.exists(debug_lib_dir):
lib_path = debug_lib_dir
elif os.path.exists(release_lib_dir):
lib_path = release_lib_dir
src = os.path.join(lib_path, filename)
dst = os.path.join(os.path.realpath(self.build_lib), "onnx", filename)
self.copy_file(src, dst)
class mypy_type_check(ONNXCommand):
description = 'Run MyPy type checker'
def run(self):
"""Run command."""
onnx_script = os.path.realpath(os.path.join(os.path.dirname(os.path.abspath(__file__)), "tools/mypy-onnx.py"))
returncode = subprocess.call([sys.executable, onnx_script])
sys.exit(returncode)
cmdclass = {
'create_version': create_version,
'cmake_build': cmake_build,
'build_py': build_py,
'develop': develop,
'build_ext': build_ext,
'typecheck': mypy_type_check,
}
################################################################################
# Extensions
################################################################################
ext_modules = [
setuptools.Extension(
name=str('onnx.onnx_cpp2py_export'),
sources=[])
]
################################################################################
# Packages
################################################################################
# no need to do fancy stuff so far
packages = setuptools.find_packages()
install_requires.extend([
'protobuf',
'numpy',
'six',
'typing>=3.6.4; python_version < "3.5"',
'typing-extensions>=3.6.2.1',
])
################################################################################
# Test
################################################################################
setup_requires.append('pytest-runner')
tests_require.append('pytest')
tests_require.append('nbval')
tests_require.append('tabulate')
if sys.version_info[0] == 3:
# Mypy doesn't work with Python 2
extras_require['mypy'] = ['mypy==0.600']
################################################################################
# Final
################################################################################
setuptools.setup(
name="onnx",
version=VersionInfo.version,
description="Open Neural Network Exchange",
ext_modules=ext_modules,
cmdclass=cmdclass,
packages=packages,
include_package_data=True,
install_requires=install_requires,
setup_requires=setup_requires,
tests_require=tests_require,
extras_require=extras_require,
author='bddppq',
author_email='jbai@fb.com',
url='https://github.com/onnx/onnx',
entry_points={
'console_scripts': [
'check-model = onnx.bin.checker:check_model',
'check-node = onnx.bin.checker:check_node',
'backend-test-tools = onnx.backend.test.cmd_tools:main',
]
},
)
onnx-1.7.0/MANIFEST.in 0000664 0000000 0000000 00000000413 13655345213 012744 0 ustar root root recursive-include onnx *.h *.c *.cc *.proto
recursive-include third_party *
include LICENSE
recursive-include onnx/backend/test/data *
recursive-include onnx/examples *
recursive-include cmake *
recursive-include tools *
include VERSION_NUMBER
include CMakeLists.txt
onnx-1.7.0/third_party/ 0000775 0000000 0000000 00000000000 13655345213 013541 5 ustar root root onnx-1.7.0/third_party/pybind11/ 0000775 0000000 0000000 00000000000 13655345213 015170 5 ustar root root onnx-1.7.0/third_party/benchmark/ 0000775 0000000 0000000 00000000000 13655345213 015473 5 ustar root root onnx-1.7.0/onnx/ 0000775 0000000 0000000 00000000000 13655345213 012172 5 ustar root root onnx-1.7.0/onnx/tools/ 0000775 0000000 0000000 00000000000 13655345213 013332 5 ustar root root onnx-1.7.0/onnx/tools/net_drawer.py 0000664 0000000 0000000 00000011755 13655345213 016047 0 ustar root root # A library and utility for drawing ONNX nets. Most of this implementation has
# been borrowed from the caffe2 implementation
# https://github.com/caffe2/caffe2/blob/master/caffe2/python/net_drawer.py
#
# The script takes two required arguments:
# -input: a path to a serialized ModelProto .pb file.
# -output: a path to write a dot file representation of the graph
#
# Given this dot file representation, you can-for example-export this to svg
# with the graphviz `dot` utility, like so:
#
# $ dot -Tsvg my_output.dot -o my_output.svg
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import argparse
from collections import defaultdict
import json
from onnx import ModelProto, GraphProto, NodeProto
import pydot # type: ignore
from typing import Text, Any, Callable, Optional, Dict
OP_STYLE = {
'shape': 'box',
'color': '#0F9D58',
'style': 'filled',
'fontcolor': '#FFFFFF'
}
BLOB_STYLE = {'shape': 'octagon'}
_NodeProducer = Callable[[NodeProto, int], pydot.Node]
def _escape_label(name): # type: (Text) -> Text
# json.dumps is poor man's escaping
return json.dumps(name)
def _form_and_sanitize_docstring(s): # type: (Text) -> Text
url = 'javascript:alert('
url += _escape_label(s).replace('"', '\'').replace('<', '').replace('>', '')
url += ')'
return url
def GetOpNodeProducer(embed_docstring=False, **kwargs): # type: (bool, **Any) -> _NodeProducer
def ReallyGetOpNode(op, op_id): # type: (NodeProto, int) -> pydot.Node
if op.name:
node_name = '%s/%s (op#%d)' % (op.name, op.op_type, op_id)
else:
node_name = '%s (op#%d)' % (op.op_type, op_id)
for i, input in enumerate(op.input):
node_name += '\n input' + str(i) + ' ' + input
for i, output in enumerate(op.output):
node_name += '\n output' + str(i) + ' ' + output
node = pydot.Node(node_name, **kwargs)
if embed_docstring:
url = _form_and_sanitize_docstring(op.doc_string)
node.set_URL(url)
return node
return ReallyGetOpNode
def GetPydotGraph(
graph, # type: GraphProto
name=None, # type: Optional[Text]
rankdir='LR', # type: Text
node_producer=None, # type: Optional[_NodeProducer]
embed_docstring=False, # type: bool
): # type: (...) -> pydot.Dot
if node_producer is None:
node_producer = GetOpNodeProducer(embed_docstring=embed_docstring, **OP_STYLE)
pydot_graph = pydot.Dot(name, rankdir=rankdir)
pydot_nodes = {} # type: Dict[Text, pydot.Node]
pydot_node_counts = defaultdict(int) # type: Dict[Text, int]
for op_id, op in enumerate(graph.node):
op_node = node_producer(op, op_id)
pydot_graph.add_node(op_node)
for input_name in op.input:
if input_name not in pydot_nodes:
input_node = pydot.Node(
_escape_label(
input_name + str(pydot_node_counts[input_name])),
label=_escape_label(input_name),
**BLOB_STYLE
)
pydot_nodes[input_name] = input_node
else:
input_node = pydot_nodes[input_name]
pydot_graph.add_node(input_node)
pydot_graph.add_edge(pydot.Edge(input_node, op_node))
for output_name in op.output:
if output_name in pydot_nodes:
pydot_node_counts[output_name] += 1
output_node = pydot.Node(
_escape_label(
output_name + str(pydot_node_counts[output_name])),
label=_escape_label(output_name),
**BLOB_STYLE
)
pydot_nodes[output_name] = output_node
pydot_graph.add_node(output_node)
pydot_graph.add_edge(pydot.Edge(op_node, output_node))
return pydot_graph
def main(): # type: () -> None
parser = argparse.ArgumentParser(description="ONNX net drawer")
parser.add_argument(
"--input",
type=Text, required=True,
help="The input protobuf file.",
)
parser.add_argument(
"--output",
type=Text, required=True,
help="The output protobuf file.",
)
parser.add_argument(
"--rankdir", type=Text, default='LR',
help="The rank direction of the pydot graph.",
)
parser.add_argument(
"--embed_docstring", action="store_true",
help="Embed docstring as javascript alert. Useful for SVG format.",
)
args = parser.parse_args()
model = ModelProto()
with open(args.input, 'rb') as fid:
content = fid.read()
model.ParseFromString(content)
pydot_graph = GetPydotGraph(
model.graph,
name=model.graph.name,
rankdir=args.rankdir,
node_producer=GetOpNodeProducer(
embed_docstring=args.embed_docstring,
**OP_STYLE
),
)
pydot_graph.write_dot(args.output)
if __name__ == '__main__':
main()
onnx-1.7.0/onnx/tools/update_model_dims.py 0000664 0000000 0000000 00000007076 13655345213 017374 0 ustar root root from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from six import string_types
from typing import Any, List, Text, Dict, Set
from onnx import ModelProto, ValueInfoProto
import onnx.checker
def update_inputs_outputs_dims(model, input_dims, output_dims): # type: (ModelProto, Dict[Text, List[Any]], Dict[Text, List[Any]]) -> ModelProto
"""
This function updates the dimension sizes of the model's inputs and outputs to the values
provided in input_dims and output_dims. if the dim value provided is negative, a unique dim_param
will be set for that dimension.
Example. if we have the following shape for inputs and outputs:
shape(input_1) = ('b', 3, 'w', 'h')
shape(input_2) = ('b', 4)
and shape(output) = ('b', 'd', 5)
The parameters can be provided as:
input_dims = {
"input_1": ['b', 3, 'w', 'h'],
"input_2": ['b', 4],
}
output_dims = {
"output": ['b', -1, 5]
}
Putting it together:
model = onnx.load('model.onnx')
updated_model = update_inputs_outputs_dims(model, input_dims, output_dims)
onnx.save(updated_model, 'model.onnx')
"""
dim_param_set = set() # type: Set[Text]
def init_dim_param_set(dim_param_set, value_infos): # type: (Set[Text], List[ValueInfoProto]) -> None
for info in value_infos:
shape = info.type.tensor_type.shape
for dim in shape.dim:
if dim.HasField('dim_param'):
dim_param_set.add(dim.dim_param) # type: ignore
init_dim_param_set(dim_param_set, model.graph.input) # type: ignore
init_dim_param_set(dim_param_set, model.graph.output) # type: ignore
init_dim_param_set(dim_param_set, model.graph.value_info) # type: ignore
def update_dim(tensor, dim, j, name): # type: (ValueInfoProto, Any, int, Text) -> None
dim_proto = tensor.type.tensor_type.shape.dim[j]
if isinstance(dim, int):
if dim >= 0:
if dim_proto.HasField('dim_value') and dim_proto.dim_value != dim:
raise ValueError('Unable to set dimension value to {} for axis {} of {}. Contradicts existing dimension value {}.'
.format(dim, j, name, dim_proto.dim_value))
dim_proto.dim_value = dim
else:
generated_dim_param = name + '_' + str(j)
if generated_dim_param in dim_param_set:
raise ValueError('Unable to generate unique dim_param for axis {} of {}. Please manually provide a dim_param value.'
.format(j, name))
dim_proto.dim_param = generated_dim_param
elif isinstance(dim, string_types):
dim_proto.dim_param = dim
else:
raise ValueError('Only int or str is accepted as dimension value, incorrect type: {}'.format(type(dim)))
for input in model.graph.input:
input_name = input.name
input_dim_arr = input_dims[input_name]
for j, dim in enumerate(input_dim_arr):
update_dim(input, dim, j, input_name)
for output in model.graph.output:
output_name = output.name
output_dim_arr = output_dims[output_name]
for j, dim in enumerate(output_dim_arr):
update_dim(output, dim, j, output_name)
onnx.checker.check_model(model)
return model
onnx-1.7.0/onnx/tools/__init__.py 0000664 0000000 0000000 00000000000 13655345213 015431 0 ustar root root onnx-1.7.0/onnx/checker.cc 0000664 0000000 0000000 00000064572 13655345213 014123 0 ustar root root #include "onnx/checker.h"
#include "onnx/defs/schema.h"
#include "onnx/defs/tensor_proto_util.h"
#include "onnx/proto_utils.h"
#include "onnx/string_utils.h"
#include
#include
#include
namespace ONNX_NAMESPACE {
namespace checker {
#define enforce_has_field(proto, field) \
do { \
if (!proto.has_##field()) { \
fail_check( \
"Field '", #field, "' of ", #proto, " is required but missing."); \
} \
} while (0)
#define enforce_has_repeated_field(proto, field) \
do { \
if (!proto.field##_size()) { \
fail_check("Repeated Field '", #field, "' is required but missing."); \
} \
} while (0)
#define enforce_non_empty_field(proto, field) \
do { \
if (proto.field().empty()) { \
fail_check( \
"Field '", \
#field, \
"' of ", \
#proto, \
" is required to be non-empty."); \
} \
} while (0)
void check_value_info(
const ValueInfoProto& value_info,
const CheckerContext& ctx) {
enforce_non_empty_field(value_info, name);
// Relax constraint for subgraph input/output.
if (!ctx.is_main_graph())
return;
enforce_has_field(value_info, type);
const auto value_case = value_info.type().value_case();
switch (value_case) {
case TypeProto::kTensorType: {
const auto& type = value_info.type().tensor_type();
enforce_has_field(type, elem_type);
enforce_has_field(type, shape);
} break;
case TypeProto::kSequenceType: {
const auto& type = value_info.type().sequence_type();
enforce_has_field(type, elem_type);
} break;
case TypeProto::kMapType: {
const auto& type = value_info.type().map_type();
enforce_has_field(type, key_type);
enforce_has_field(type, value_type);
} break;
#ifdef ONNX_ML
case TypeProto::kOpaqueType:
break;
case TypeProto::kSparseTensorType: {
const auto& type = value_info.type().sparse_tensor_type();
enforce_has_field(type, elem_type);
enforce_has_field(type, shape);
} break;
#endif
default:
fail_check(
"Unrecognized type value case (value_info name: ",
value_info.name(),
"): ",
value_case);
}
}
void check_tensor(const TensorProto& tensor, const CheckerContext& ctx) {
enforce_has_field(tensor, data_type);
if (tensor.data_type() == TensorProto::UNDEFINED) {
fail_check(
"setting data_type field (tensor name: ",
tensor.name(),
") to UNDEFINED is not allowed");
}
int num_value_fields = 0;
const char* value_field = nullptr;
#define check_data_field(field) \
bool has_##field = tensor.field().size(); \
if (has_##field) { \
++num_value_fields; \
value_field = #field; \
}
check_data_field(float_data);
check_data_field(int32_data);
check_data_field(string_data);
check_data_field(int64_data);
check_data_field(raw_data);
check_data_field(double_data);
check_data_field(uint64_data);
#undef check_data_field
bool stored_externally = tensor.has_data_location() &&
tensor.data_location() == TensorProto::EXTERNAL;
if (stored_externally) {
if (num_value_fields != 0) {
fail_check(
"Data of TensorProto ( tensor name: ",
tensor.name(),
") is stored externally and should not have data field.",
value_field);
}
bool has_location = false;
for (const StringStringEntryProto& entry : tensor.external_data()) {
if (entry.has_key() && entry.has_value() && entry.key() == "location") {
has_location = true;
if (!std::ifstream(ctx.get_model_dir() + entry.value())) {
fail_check(
"Data of TensorProto ( tensor name: ",
tensor.name(),
") should be stored in ",
ctx.get_model_dir() + entry.value(),
", but it doesn't exist or is not accessible.");
}
}
}
if (!has_location) {
fail_check(
"TensorProto ( tensor name: ",
tensor.name(),
") is stored externally but doesn't have a location.");
}
return;
}
int64_t nelem = 1;
for (auto x : tensor.dims()) {
nelem *= x;
}
if (nelem == 0 && num_value_fields != 0) {
fail_check(
"TensorProto (tensor name: ",
tensor.name(),
") is 0-element but contains data!");
}
if (nelem != 0 && num_value_fields != 1) {
fail_check(
"TensorProto (tensor name: ",
tensor.name(),
") should contain one and only one value field.");
}
if (has_raw_data) {
if (tensor.data_type() == TensorProto::STRING) {
fail_check(
"STRING data (tensor name: ",
tensor.name(),
") should not be stored in raw_data field");
}
return;
} else {
#define check_field(field) \
if (nelem != 0 && !has_##field) { \
fail_check( \
"values of data_type '", \
tensor.data_type(), \
"' should be stored in field '", \
#field, \
"' instead of '", \
value_field, \
"'"); \
}
switch (tensor.data_type()) {
case TensorProto::FLOAT:
case TensorProto::COMPLEX64:
check_field(float_data);
break;
case TensorProto::DOUBLE:
case TensorProto::COMPLEX128:
check_field(double_data);
break;
case TensorProto::INT32:
case TensorProto::UINT8:
case TensorProto::INT8:
case TensorProto::UINT16:
case TensorProto::INT16:
case TensorProto::BOOL:
case TensorProto::FLOAT16:
case TensorProto::BFLOAT16:
check_field(int32_data);
break;
case TensorProto::INT64:
check_field(int64_data);
break;
case TensorProto::UINT32:
case TensorProto::UINT64:
check_field(uint64_data);
break;
case TensorProto::STRING:
check_field(string_data);
break;
default:
fail_check(
"Unrecognized data_type (tensor name: ",
tensor.name(),
"): ",
tensor.data_type());
}
}
#undef check_field
}
// Check that the index data stored in a SparseTensorProto is valid.
// indices: a 1-dimensional tensor; indices[i] represents the
// linearized index value for the i-th nonzero value.
void check_sparse_tensor_indices_1(
const TensorProto& indices,
const SparseTensorProto& sparse_tensor_proto,
size_t nnz) {
int dense_rank = sparse_tensor_proto.dims_size();
int64_t dense_size = 1;
for (int i = 0; i < dense_rank; ++i)
dense_size *= sparse_tensor_proto.dims(i);
if (static_cast(indices.dims(0)) != nnz)
fail_check(
"Sparse tensor indices (",
indices.name(),
") has ",
indices.dims(0),
" values, but NNZ is ",
nnz);
// Check if indices appear in ascending order, and if they have valid
// values. The i-th value in index_data is the linear index of the i-th
// non-zero value.
const std::vector index_data = ParseData(&indices);
int64_t prev_index = -1;
for (size_t i = 0; i < nnz; ++i) {
int64_t curr_index = index_data[i]; // linearized index of i-th value
if (curr_index < 0 || curr_index >= dense_size)
fail_check(
"Sparse tensor (",
indices.name(),
") index value at position [",
i,
"] out of range [0, ",
dense_size - 1,
"]");
if (curr_index <= prev_index) {
fail_check(
"Sparse tensor (",
indices.name(),
") index value at position [",
i,
"] not in sorted order.");
}
prev_index = curr_index;
}
}
// Check that the index data stored in a SparseTensorProto is valid.
// indices: a 2-dimensional tensor; indices[i,j] represents the j-th
// index value for the i-th nonzero value.
void check_sparse_tensor_indices_2(
const TensorProto& indices,
const SparseTensorProto& sparse_tensor_proto,
size_t nnz) {
int dense_rank = sparse_tensor_proto.dims_size();
if (static_cast(indices.dims(0)) != nnz)
fail_check(
"Sparse tensor indices (",
indices.name(),
") first dimension size does not equal NNZ.");
if (indices.dims(1) != dense_rank)
fail_check(
"Sparse tensor indices (",
indices.name(),
") second dimension size does not match rank of tensor.");
// Check if indices appear in ascending order, and if they have valid
// values.
const std::vector index_data = ParseData(&indices);
int64_t prev_index = -1;
for (size_t i = 0; i < nnz; ++i) {
int64_t curr_index = 0; // linearized index of i-th value
for (int j = 0; j < dense_rank; ++j) {
auto index_ij = index_data[i * dense_rank + j];
if ((index_ij < 0) || (index_ij >= sparse_tensor_proto.dims(j)))
fail_check(
"Sparse tensor (",
indices.name(),
") index value at position [",
i,
",",
j,
"] out of range.");
curr_index = curr_index * sparse_tensor_proto.dims(j) + index_ij;
}
if (curr_index <= prev_index) {
fail_check(
"Sparse tensor (",
indices.name(),
") index value at position [",
i,
"] not in lexicographic sorted order.");
}
prev_index = curr_index;
}
}
void check_sparse_tensor(
const SparseTensorProto& sparse_tensor_proto,
const CheckerContext& ctx) {
enforce_has_field(sparse_tensor_proto, values);
const TensorProto& values = sparse_tensor_proto.values();
check_tensor(values, ctx);
// values must be a tensor of shape [NNZ]
// Currently we restrict the value associated with a particular index-tuple
// to be a single value. In the future, if there is a requirement,
// we may extend this to permit the value to be a "sub-tensor", in which
// case values will have dimension > 1.
if (values.dims_size() != 1)
fail_check("Sparse tensor values (", values.name(), ") must have rank 1.");
size_t nnz = static_cast(values.dims(0));
int dense_rank = sparse_tensor_proto.dims_size();
if (dense_rank == 0) {
// TODO: Should we add a name field for a sparse-tensor-proto?
// Currently, values has a name, but message may be a bit confusing.
fail_check(
"Sparse tensor (", values.name(), ") must have a dense-rank > 0");
}
for (int i = 0; i < dense_rank; ++i) {
if (sparse_tensor_proto.dims(i) <= 0)
fail_check(
"Sparse tensor (", values.name(), ") dimensions are not positive.");
}
if (sparse_tensor_proto.has_indices()) {
const TensorProto& indices = sparse_tensor_proto.indices();
check_tensor(indices, ctx);
if (indices.data_type() != TensorProto::INT64)
fail_check(
"Sparse tensor indices (", indices.name(), ") must have INT64 type.");
switch (indices.dims().size()) {
case 1:
// Indices in linearized format
check_sparse_tensor_indices_1(indices, sparse_tensor_proto, nnz);
return;
case 2:
// Check COO-style index. E.g., an index for a 3D tensor is a 3-tuple.
check_sparse_tensor_indices_2(indices, sparse_tensor_proto, nnz);
return;
default:
fail_check(
"Sparse tensor indices (",
indices.name(),
") must have rank 1 or 2.");
}
} else if (nnz != 0)
fail_check("Sparse tensor (", values.name(), ") has no index values.");
}
// NB: This is a generic "attribute well-formedness" check, it doesn't
// actually test if an attribute is valid per a schema
void check_attribute(
const AttributeProto& attr,
const CheckerContext& ctx,
const LexicalScopeContext& lex_ctx) {
enforce_non_empty_field(attr, name);
if (ctx.get_ir_version() >= 0x00000002) {
enforce_has_field(attr, type);
}
int used_fields = 0;
#define check_type(expected_type) \
if (attr.has_type() && attr.type() != expected_type) { \
fail_check( \
"type field and data field mismatch in attribute ", attr.name(), "."); \
}
#define check_singular_field(field, type) \
if (attr.has_##field()) { \
++used_fields; \
check_type(type); \
}
#define check_repeated_field(field, type) \
if (attr.field##_size() > 0) { \
++used_fields; \
check_type(type); \
}
check_singular_field(f, AttributeProto::FLOAT);
check_singular_field(i, AttributeProto::INT);
check_singular_field(s, AttributeProto::STRING);
check_singular_field(t, AttributeProto::TENSOR);
check_singular_field(g, AttributeProto::GRAPH);
check_singular_field(sparse_tensor, AttributeProto::SPARSE_TENSOR);
check_repeated_field(floats, AttributeProto::FLOATS);
check_repeated_field(ints, AttributeProto::INTS);
check_repeated_field(strings, AttributeProto::STRINGS);
check_repeated_field(tensors, AttributeProto::TENSORS);
check_repeated_field(graphs, AttributeProto::GRAPHS);
check_repeated_field(sparse_tensors, AttributeProto::SPARSE_TENSORS);
#undef check_type
#undef check_singular_field
#undef check_repeated_field
// Normally, used_fields is expected to be 1.
// In proto3, when the value to be set is type default value (say 0 for
// int), used_fields may be 0.
if (used_fields > 1) {
fail_check(
"Attribute (name: ",
attr.name(),
") should not contain more than one value field.");
}
if (!ctx.is_main_graph()) {
// It's an attribute of a node in function body.
if (attr.has_ref_attr_name() && used_fields != 0) {
// The attribute proto is supposed to refer to data outside and does not
// have its own value field set.
fail_check(
"Attribute (name: ",
attr.name(),
") should refer to attribute in parent node.");
}
}
if (attr.has_t()) {
check_tensor(attr.t(), ctx);
}
if (attr.has_sparse_tensor()) {
check_sparse_tensor(attr.sparse_tensor(), ctx);
}
if (attr.has_g()) {
CheckerContext subgraph_ctx(ctx);
subgraph_ctx.set_is_main_graph(false);
check_graph(attr.g(), subgraph_ctx, lex_ctx);
}
for (const auto& tensor : attr.tensors()) {
check_tensor(tensor, ctx);
}
for (const auto& sparse_tensor : attr.sparse_tensors()) {
check_sparse_tensor(sparse_tensor, ctx);
}
if (attr.graphs().size() > 0) {
CheckerContext subgraph_ctx(ctx);
subgraph_ctx.set_is_main_graph(false);
for (const auto& graph : attr.graphs()) {
check_graph(graph, subgraph_ctx, lex_ctx);
}
}
}
void check_node(
const NodeProto& node,
const CheckerContext& ctx,
const LexicalScopeContext& lex_ctx) {
enforce_non_empty_field(node, op_type);
if (node.input().empty() && node.output().empty()) {
fail_check(
"NodeProto (name: ",
node.name(),
", type: ",
node.op_type(),
") has zero input and zero output.");
}
// Put the removed experimental ops here
static std::set experimental_ops = {"ATen",
"Affine",
"ConstantFill",
"Crop",
"DynamicSlice",
"GRUUnit",
"GivenTensorFill",
"ImageScaler",
"ParametricSoftplus",
"Scale",
"ScaledTanh"};
if (experimental_ops.count(node.op_type())) {
std::cerr << "Warning: " << node.op_type() << " was a removed "
<< "experimental ops. In the future, we may directly "
<< "reject this operator. Please update your model as soon "
<< "as possible." << std::endl;
return;
}
// Resolve domain for node
const auto& opset_imports = ctx.get_opset_imports();
auto dit = opset_imports.find(node.domain());
if (dit == opset_imports.end()) {
fail_check("No opset import for domain '" + node.domain() + "'");
}
auto domain_version = dit->second;
for (const auto& attr : node.attribute()) {
check_attribute(attr, ctx, lex_ctx);
}
const auto* schema = ctx.get_schema_registry()->GetSchema(
node.op_type(), domain_version, node.domain());
if (!schema) {
if (node.domain() == ONNX_DOMAIN || node.domain() == AI_ONNX_ML_DOMAIN ||
node.domain() == "ai.onnx" || node.domain() == AI_ONNX_TRAINING_DOMAIN) {
// fail the checker if op in built-in domains has no schema
fail_check(
"No Op registered for " + node.op_type() +
" with domain_version of " +
ONNX_NAMESPACE::to_string(domain_version));
} else {
// TODO: expose the registration of the op schemas appropriately in
// python, so we can load and register operators in other domains
//
// before we complete the above todo, let's skip the schema check for
// now
}
} else if (schema->Deprecated()) {
fail_check(
"Op registered for " + node.op_type() +
" is deprecated in domain_version of " +
ONNX_NAMESPACE::to_string(domain_version));
} else {
schema->Verify(node);
}
}
void check_graph(
const GraphProto& graph,
const CheckerContext& ctx,
const LexicalScopeContext& parent_lex) {
enforce_non_empty_field(graph, name);
for (const auto& value_info : graph.input()) {
check_value_info(value_info, ctx);
}
for (const auto& value_info : graph.output()) {
check_value_info(value_info, ctx);
}
// Inherit values available in outer scope
// Note that we do not allow shadowing, so the presence of an already-defined
// name is always an error.
LexicalScopeContext lex_ctx{parent_lex};
for (const auto& value_info : graph.input()) {
// TODO: If shadowing isn't allowed, this should maybe use
// this_or_ancestor_graph_has
if (lex_ctx.this_graph_has(value_info.name())) {
fail_check(
"Graph must be in single static assignment (SSA) form, however '",
value_info.name(),
"' has been used as graph input names multiple times.");
}
lex_ctx.add(value_info.name());
}
for (const auto& init : graph.initializer()) {
if (ctx.get_ir_version() <= 0x00000003) {
// Initializers are a subset of graph inputs for IR_VERSION <= 3
if (!lex_ctx.this_graph_has(init.name())) {
fail_check(init.name() + " in initializer but not in graph input");
}
} else {
// An initializer is allowed to have the same name as an input,
// but is not required to (for IR_VERSION >= 4)
lex_ctx.add(init.name());
}
check_tensor(init, ctx);
}
// TODO: Need to check that sparse-initializers names are distinct from
// initializer names. It looks like the existing checker does not check for
// certain duplication of names: e.g., two entries in the initializer list
// with same name. Will add a new integrated check.
for (const auto& sparse_init : graph.sparse_initializer()) {
check_sparse_tensor(sparse_init, ctx);
lex_ctx.add(sparse_init.values().name());
}
for (const auto& node : graph.node()) {
// nodes must be in topologically sorted order
for (const auto& input : node.input()) {
// explicit optional input
if (input.empty()) {
continue;
}
if (!lex_ctx.this_or_ancestor_graph_has(input)) {
fail_check(
"Nodes in a graph must be topologically sorted, however input '",
input,
"' of node: \n",
ProtoDebugString(node),
"\n is not output of any previous nodes.");
}
}
// This needs to happen before SSA check since we don't want to recurse and
// find that outputs from control flow ops are colliding with names in the
// inner block
try {
check_node(node, ctx, lex_ctx);
} catch (ValidationError& ex) {
ex.AppendContext("Bad node spec: " + ProtoDebugString(node));
throw ex;
}
// check for SSA form
for (const auto& output : node.output()) {
// optional output
if (output.empty()) {
continue;
}
if (lex_ctx.this_or_ancestor_graph_has(output)) {
fail_check(
"Graph must be in single static assignment (SSA) form, however '",
output,
"' has been used as output names multiple times.");
}
lex_ctx.add(output);
}
}
}
void check_function(
const FunctionProto& function,
const CheckerContext& ctx,
const LexicalScopeContext& parent_lex) {
enforce_non_empty_field(function, name);
enforce_has_field(function, since_version);
CheckerContext ctx_copy = ctx;
std::unordered_map opsets;
for (auto& relied_opset : function.opset_import()) {
auto& domain = relied_opset.domain();
auto version = relied_opset.version();
opsets[domain] = static_cast(version);
}
ctx_copy.set_opset_imports(opsets);
LexicalScopeContext lex_ctx{parent_lex};
for (const auto& input : function.input()) {
// TODO: If shadowing isn't allowed, this should maybe use
// this_or_ancestor_graph_has
if (lex_ctx.this_graph_has(input)) {
fail_check(
"Graph must be in single static assignment (SSA) form, however '",
input,
"' has been used multiple times.");
}
lex_ctx.add(input);
}
std::unordered_set outputs;
for (const auto& output : function.output()) {
auto result = outputs.insert(output);
if (!result.second) {
fail_check(
"function (",
function.name(),
") should not have duplicate outputs specified.");
}
}
std::unordered_set attrs;
for (const auto& attr : function.attribute()) {
auto result = attrs.insert(attr);
if (!result.second) {
fail_check(
"function (",
function.name(),
") should not have duplicate attributes specified.");
}
}
for (const auto& node : function.node()) {
// nodes must be in topologically sorted order
for (const auto& input : node.input()) {
// explicit optional input
if (input.empty()) {
continue;
}
if (!lex_ctx.this_graph_has(input)) {
fail_check(
"Nodes in a function must be topologically sorted, however input '",
input,
"' of node: \n",
ProtoDebugString(node),
"\n is neither output of any previous nodes nor input of the function.");
}
}
check_node(node, ctx_copy, lex_ctx);
// check for SSA form
for (const auto& output : node.output()) {
// optional output
if (output.empty()) {
continue;
}
if (lex_ctx.this_or_ancestor_graph_has(output)) {
fail_check(
"Function must be in single static assignment (SSA) form, however '",
output,
"' has been used as output names multiple times.");
}
lex_ctx.add(output);
}
}
}
void check_model(const ModelProto& model, CheckerContext& ctx) {
if (!model.ir_version()) {
fail_check("The model does not have an ir_version set properly.");
}
if (model.ir_version() > IR_VERSION) {
fail_check("Your model ir_version is higher than the checker's.");
}
if (model.metadata_props_size() > 1) {
std::unordered_set keys;
for (const StringStringEntryProto& entry : model.metadata_props()) {
auto i = keys.insert(entry.key());
if (!i.second) {
fail_check("Your model has duplicate keys in metadata_props.");
}
}
}
std::unordered_map versions;
ctx.set_ir_version(static_cast(model.ir_version()));
std::unordered_map opset_imports;
for (const auto& opset_import : model.opset_import()) {
opset_imports[opset_import.domain()] =
static_cast(opset_import.version());
}
if (model.ir_version() >= 3) {
if (opset_imports.empty())
fail_check(
"model with IR version >= 3 must specify opset_import for ONNX");
} else {
if (opset_imports.empty())
opset_imports[ONNX_DOMAIN] = 1;
else
fail_check(
"model with IR version < 3 cannot have opset_import specified");
}
ctx.set_opset_imports(opset_imports);
LexicalScopeContext lex_ctx;
check_graph(model.graph(), ctx, lex_ctx);
}
void check_model(const std::string& model_path) {
ModelProto model;
std::fstream model_stream(model_path, std::ios::in | std::ios::binary);
if (!model_stream.good()) {
fail_check(
"Unable to open model file:",
model_path,
". Please check if it is a valid file.");
}
std::string data{std::istreambuf_iterator{model_stream},
std::istreambuf_iterator{}};
if (!ParseProtoFromBytes(&model, data.c_str(), data.size())) {
fail_check(
"Unable to parse model from file:",
model_path,
". Please check if it is a valid protobuf file of model.");
}
CheckerContext ctx;
std::string model_dir;
size_t pos = model_path.find_last_of("\\/");
if (pos != std::string::npos) {
model_dir = model_path.substr(0, pos + 1);
}
ctx.set_model_dir(model_dir);
check_model(model, ctx);
}
void check_model(const ModelProto& model) {
CheckerContext ctx;
check_model(model, ctx);
}
#undef fail_check
#undef enforce_has_field
#undef enforce_has_repeated_field
#undef enforce_non_empty_field
} // namespace checker
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/mapping.py 0000664 0000000 0000000 00000004703 13655345213 014203 0 ustar root root from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from onnx import TensorProto
from typing import Text, Any
import numpy as np # type: ignore
TENSOR_TYPE_TO_NP_TYPE = {
int(TensorProto.FLOAT): np.dtype('float32'),
int(TensorProto.UINT8): np.dtype('uint8'),
int(TensorProto.INT8): np.dtype('int8'),
int(TensorProto.UINT16): np.dtype('uint16'),
int(TensorProto.INT16): np.dtype('int16'),
int(TensorProto.INT32): np.dtype('int32'),
int(TensorProto.INT64): np.dtype('int64'),
int(TensorProto.BOOL): np.dtype('bool'),
int(TensorProto.FLOAT16): np.dtype('float16'),
int(TensorProto.DOUBLE): np.dtype('float64'),
int(TensorProto.COMPLEX64): np.dtype('complex64'),
int(TensorProto.COMPLEX128): np.dtype('complex128'),
int(TensorProto.UINT32): np.dtype('uint32'),
int(TensorProto.UINT64): np.dtype('uint64'),
int(TensorProto.STRING): np.dtype(np.object)
}
NP_TYPE_TO_TENSOR_TYPE = {v: k for k, v in TENSOR_TYPE_TO_NP_TYPE.items()}
TENSOR_TYPE_TO_STORAGE_TENSOR_TYPE = {
int(TensorProto.FLOAT): int(TensorProto.FLOAT),
int(TensorProto.UINT8): int(TensorProto.INT32),
int(TensorProto.INT8): int(TensorProto.INT32),
int(TensorProto.UINT16): int(TensorProto.INT32),
int(TensorProto.INT16): int(TensorProto.INT32),
int(TensorProto.INT32): int(TensorProto.INT32),
int(TensorProto.INT64): int(TensorProto.INT64),
int(TensorProto.BOOL): int(TensorProto.INT32),
int(TensorProto.FLOAT16): int(TensorProto.UINT16),
int(TensorProto.BFLOAT16): int(TensorProto.UINT16),
int(TensorProto.DOUBLE): int(TensorProto.DOUBLE),
int(TensorProto.COMPLEX64): int(TensorProto.FLOAT),
int(TensorProto.COMPLEX128): int(TensorProto.DOUBLE),
int(TensorProto.UINT32): int(TensorProto.UINT32),
int(TensorProto.UINT64): int(TensorProto.UINT64),
int(TensorProto.STRING): int(TensorProto.STRING),
}
STORAGE_TENSOR_TYPE_TO_FIELD = {
int(TensorProto.FLOAT): 'float_data',
int(TensorProto.INT32): 'int32_data',
int(TensorProto.INT64): 'int64_data',
int(TensorProto.UINT16): 'int32_data',
int(TensorProto.DOUBLE): 'double_data',
int(TensorProto.COMPLEX64): 'float_data',
int(TensorProto.COMPLEX128): 'double_data',
int(TensorProto.UINT32): 'uint64_data',
int(TensorProto.UINT64): 'uint64_data',
int(TensorProto.STRING): 'string_data',
int(TensorProto.BOOL): 'int32_data',
}
onnx-1.7.0/onnx/onnx-operators_pb.h 0000664 0000000 0000000 00000000336 13655345213 016024 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#pragma once
#include "onnx/onnx_pb.h"
#ifdef ONNX_ML
#include "onnx/onnx-operators-ml.pb.h"
#else
#include "onnx/onnx-operators.pb.h"
#endif onnx-1.7.0/onnx/onnxifi_wrapper.c 0000664 0000000 0000000 00000070012 13655345213 015550 0 ustar root root /*
* ONNX wrapper api discovers vendor-specific implementations installed in
* the system and exposes them under a single interface.
*
*
*
*/
#include
#include
#include
#include
#include
#ifdef _WIN32
#include
#else
#include
#include
#include
#include
#endif
#include
#include
#if defined(_WIN32)
#define ONNXIFI_FILENAME_WILDCARD L"\\onnxifi-*.dll"
#define ONNXIFI_FILENAME_WILDCARD_LENGTH 14
#elif defined(__APPLE__)
/* Minimum filename: "libonnxifi-?.dylib" */
#define ONNXIFI_FILENAME_MIN 18
#define ONNXIFI_FILENAME_PREFIX "libonnxifi-"
#define ONNXIFI_FILENAME_SUFFIX ".dylib"
#else
/* Minimum filename: "libonnxifi-?.so" */
#define ONNXIFI_FILENAME_MIN 15
#define ONNXIFI_FILENAME_PREFIX "libonnxifi-"
#define ONNXIFI_FILENAME_SUFFIX ".so"
#endif
#define ONNXIFI_BACKEND_ID_MAGIC UINT32_C(0x2EDD3764)
#define ONNXIFI_BACKEND_MAGIC UINT32_C(0x4B9B2902)
#define ONNXIFI_GRAPH_MAGIC UINT32_C(0xD9ACFACD)
#define ONNXIFI_EVENT_MAGIC UINT32_C(0x18C1D735)
struct onnxifi_backend_id_wrapper {
uint32_t magic;
onnxBackendID backend_id;
struct onnxifi_library* library;
};
struct onnxifi_backend_wrapper {
uint32_t magic;
onnxBackend backend;
struct onnxifi_library* library;
};
struct onnxifi_graph_wrapper {
uint32_t magic;
onnxGraph graph;
struct onnxifi_library* library;
};
struct onnxifi_event_wrapper {
uint32_t magic;
onnxEvent event;
struct onnxifi_library* library;
};
static struct onnxifi_library* libraries = NULL;
static uint32_t num_libraries = 0;
#ifdef _WIN32
static INIT_ONCE init_guard = INIT_ONCE_STATIC_INIT;
static BOOL CALLBACK load_all_windows_backends(
PINIT_ONCE init_once,
PVOID parameter,
PVOID* context)
{
WCHAR* onnxifi_library_wildcard = NULL;
HANDLE find_file_handle = INVALID_HANDLE_VALUE;
WIN32_FIND_DATAW find_file_data;
UINT system_directory_path_length = GetSystemDirectoryW(NULL, 0);
if (system_directory_path_length == 0) {
fprintf(stderr, "Error: failed to get system directory path: %u\n",
(unsigned int) GetLastError());
goto cleanup;
}
onnxifi_library_wildcard = malloc(sizeof(WCHAR) *
(system_directory_path_length + ONNXIFI_FILENAME_WILDCARD_LENGTH + 1));
if (onnxifi_library_wildcard == NULL) {
fprintf(stderr,
"Error: failed to allocate %Iu bytes for ONNXIFI path\n",
sizeof(WCHAR) *
(system_directory_path_length + ONNXIFI_FILENAME_WILDCARD_LENGTH + 1));
goto cleanup;
}
if (GetSystemDirectoryW(
onnxifi_library_wildcard, system_directory_path_length) == 0)
{
fprintf(stderr, "Error: failed to get system directory path: %u\n",
(unsigned int) GetLastError());
goto cleanup;
}
memcpy(onnxifi_library_wildcard + system_directory_path_length,
ONNXIFI_FILENAME_WILDCARD,
sizeof(WCHAR) * (ONNXIFI_FILENAME_WILDCARD_LENGTH + 1));
find_file_handle = FindFirstFileW(onnxifi_library_wildcard, &find_file_data);
if (find_file_handle == INVALID_HANDLE_VALUE) {
const DWORD error = GetLastError();
if (error != ERROR_FILE_NOT_FOUND) {
fprintf(stderr,
"Error: failed to list ONNXIFI libraries %S: error %u\n",
onnxifi_library_wildcard, (unsigned int) error);
}
goto cleanup;
}
for (;;) {
struct onnxifi_library library;
if (!onnxifi_load(ONNXIFI_LOADER_FLAG_VERSION_1_0,
find_file_data.cFileName, &library))
{
fprintf(stderr, "Error: failed to load library %S\n",
find_file_data.cFileName);
continue;
}
struct onnxifi_library* new_libraries =
realloc(libraries, (num_libraries + 1) * sizeof(struct onnxifi_library));
if (new_libraries == NULL) {
fprintf(stderr, "Error: failed to allocate space for library %S\n",
find_file_data.cFileName);
onnxifi_unload(&library);
continue;
}
/* All actions for the new library succeeded, commit changes */
libraries = new_libraries;
memcpy(&libraries[num_libraries], &library, sizeof(library));
num_libraries++;
if (FindNextFileW(find_file_handle, &find_file_data) != FALSE) {
const DWORD error = GetLastError();
if (error != ERROR_NO_MORE_FILES) {
fprintf(stderr,
"Error: failed to some of ONNXIFI libraries %S: error %u\n",
onnxifi_library_wildcard, (unsigned int) error);
}
break;
}
}
cleanup:
if (find_file_handle != INVALID_HANDLE_VALUE) {
FindClose(find_file_handle);
}
free(onnxifi_library_wildcard);
return TRUE;
}
#else
static pthread_once_t init_guard = PTHREAD_ONCE_INIT;
#ifndef ONNXIFI_SEARCH_DIR
#ifdef __APPLE__
#define ONNXIFI_SEARCH_DIR "/opt/onnx/lib/"
#else
#define ONNXIFI_SEARCH_DIR "/usr/lib/"
#endif
#endif
/* Finds filename in a null-terminated file path */
static inline const char* find_filename(const char* filepath) {
const char* filename_separator = strrchr(filepath, '/');
if (filename_separator == NULL) {
return filepath;
} else {
return filename_separator + 1;
}
}
static inline bool is_onnxifi_path(const char* filepath) {
const char* filename = find_filename(filepath);
const size_t filename_length = strlen(filename);
if (filename_length < ONNXIFI_FILENAME_MIN) {
/* Filename too short */
return false;
}
const char* filename_end = filename + filename_length;
/* Expected filename structure: */
if (memcmp(filename,
ONNXIFI_FILENAME_PREFIX,
strlen(ONNXIFI_FILENAME_PREFIX)) != 0)
{
/* Prefix mismatch */
return false;
}
const char* suffix = filename_end - strlen(ONNXIFI_FILENAME_SUFFIX);
if (memcmp(suffix,
ONNXIFI_FILENAME_SUFFIX,
strlen(ONNXIFI_FILENAME_SUFFIX)) != 0)
{
/* Suffix mismatch */
return false;
}
return true;
}
static void load_all_posix_backends(void) {
DIR* directory = opendir(ONNXIFI_SEARCH_DIR);
if (directory == NULL) {
fprintf(stderr, "Error: failed to open directory %s: %s\n",
ONNXIFI_SEARCH_DIR, strerror(errno));
return;
}
for (;;) {
/* Required to distinguish between error and end of directory in readdir */
errno = 0;
struct dirent* entry = readdir(directory);
if (entry == NULL) {
if (errno != 0) {
fprintf(stderr, "Error: failed to read directory %s: %s\n",
ONNXIFI_SEARCH_DIR, strerror(errno));
}
goto cleanup;
}
if (!is_onnxifi_path(entry->d_name)) {
continue;
}
struct onnxifi_library library;
if (!onnxifi_load(ONNXIFI_LOADER_FLAG_VERSION_1_0, entry->d_name, &library))
{
fprintf(stderr, "Error: failed to load library %s\n", entry->d_name);
continue;
}
struct onnxifi_library* new_libraries =
realloc(libraries, (num_libraries + 1) * sizeof(struct onnxifi_library));
if (new_libraries == NULL) {
fprintf(stderr, "Error: failed to allocate space for library %s\n",
entry->d_name);
onnxifi_unload(&library);
continue;
}
/* All actions for the new library succeeded, commit changes */
libraries = new_libraries;
memcpy(&libraries[num_libraries], &library, sizeof(library));
num_libraries++;
}
cleanup:
if (directory != NULL) {
if (closedir(directory) != 0) {
/* Error */
fprintf(stderr, "Warning: failed to close directory %s: %s\n",
ONNXIFI_SEARCH_DIR, strerror(errno));
}
}
}
#endif
/* Platform-independent wrapper for init_once */
static void load_all_libraries_once(void) {
#ifdef _WIN32
InitOnceExecuteOnce(&init_guard, load_all_windows_backends, NULL, NULL);
#else
pthread_once(&init_guard, load_all_posix_backends);
#endif
}
static onnxStatus wrap_backend_ids(
struct onnxifi_library* library,
size_t num_backends,
onnxBackendID* backend_ids)
{
size_t num_wrapped_backends = 0;
for (; num_wrapped_backends < num_backends; num_wrapped_backends++) {
struct onnxifi_backend_id_wrapper* backend_id_wrapper =
(struct onnxifi_backend_id_wrapper*)
malloc(sizeof(struct onnxifi_backend_id_wrapper));
if (backend_id_wrapper == NULL) {
goto cleanup;
}
backend_id_wrapper->magic = ONNXIFI_BACKEND_ID_MAGIC;
backend_id_wrapper->backend_id = backend_ids[num_wrapped_backends];
backend_id_wrapper->library = library;
/* Replace backend ID with its wrapper */
backend_ids[num_wrapped_backends] = (onnxBackendID) backend_id_wrapper;
}
return ONNXIFI_STATUS_SUCCESS;
cleanup:
/* Unwrap all the backends */
for (size_t i = 0; i < num_wrapped_backends; i++) {
struct onnxifi_backend_id_wrapper* backend_id_wrapper =
(struct onnxifi_backend_id_wrapper*) backend_ids[i];
assert(backend_id_wrapper->magic == ONNXIFI_BACKEND_ID_MAGIC);
/* Replace wrapper with the wrapped backend ID */
backend_ids[i] = backend_id_wrapper->backend_id;
/* Safety precaution to avoid use-after-free bugs */
memset(backend_id_wrapper, 0, sizeof(struct onnxifi_backend_id_wrapper));
free(backend_id_wrapper);
}
return ONNXIFI_STATUS_NO_SYSTEM_MEMORY;
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxGetBackendIDs(
onnxBackendID* backendIDs,
size_t* numBackends)
{
load_all_libraries_once();
onnxStatus status;
/* Number of backend IDs requested to be stored by the caller */
const size_t num_expected_ids = (backendIDs == NULL) ? 0 : *numBackends;
/* Number of backend IDs in the system */
size_t num_available_ids = 0;
/* Number of backend IDs wrapped and ready to return */
size_t num_wrapped_ids = 0;
onnxBackendID* backend_ids = NULL;
if (num_expected_ids != 0) {
backend_ids = malloc(num_expected_ids * sizeof(onnxBackendID));
if (backend_ids == NULL) {
status = ONNXIFI_STATUS_NO_SYSTEM_MEMORY;
goto error;
}
}
/* Safety precaution to avoid dangling pointer bugs */
memset(backend_ids, 0, num_expected_ids * sizeof(onnxBackendID));
for (size_t l = 0; l < num_libraries; l++) {
if (num_expected_ids > num_available_ids) {
/* Query and wrap backend IDs from ONNXIFI library */
const size_t max_library_ids = num_expected_ids - num_available_ids;
size_t num_library_ids = max_library_ids;
status = libraries[l].onnxGetBackendIDs(
&backend_ids[num_available_ids],
&num_library_ids);
const size_t num_stored_ids =
(num_library_ids < max_library_ids) ? num_library_ids : max_library_ids;
switch (status) {
case ONNXIFI_STATUS_SUCCESS:
case ONNXIFI_STATUS_FALLBACK:
status = wrap_backend_ids(
&libraries[l],
num_stored_ids,
&backend_ids[num_available_ids]);
if (status != ONNXIFI_STATUS_SUCCESS) {
/* Release unwrapped backends for this library */
for (size_t i = 0; i < num_stored_ids; i++) {
(void) libraries[l].onnxReleaseBackendID(backend_ids[i]);
/* Safety precaution to avoid use-after-free bugs */
backend_ids[i] = NULL;
}
/* Release wrapped backends for other libraries */
goto error;
}
num_wrapped_ids += num_stored_ids;
num_available_ids += num_library_ids;
break;
default:
/* Release wrapped backends for other libraries */
goto error;
}
} else {
/* Not enough space in user-provided buffer: only count the backend IDs */
size_t num_library_ids = 0;
status = libraries[l].onnxGetBackendIDs(NULL, &num_library_ids);
if (status != ONNXIFI_STATUS_FALLBACK) {
/* Release wrapped backends for other libraries */
goto error;
}
num_available_ids += num_library_ids;
}
}
/*
* Successful return:
* - Copy backend IDs to user-provided memory (if applicable)
* - Store number of backend IDs to user-provided variable
* - Return success or fallback codes
*/
if (backendIDs == NULL) {
*numBackends = num_available_ids;
return ONNXIFI_STATUS_FALLBACK;
} else {
memcpy(backendIDs, backend_ids, num_wrapped_ids * sizeof(onnxBackendID));
free(backend_ids);
*numBackends = num_available_ids;
return num_available_ids <= num_expected_ids ?
ONNXIFI_STATUS_SUCCESS : ONNXIFI_STATUS_FALLBACK;
}
error:
/*
* Error return:
* - Release all wrapped backend IDs.
* - Deallocate wrapper structures.
*/
for (size_t i = 0; i < num_wrapped_ids; i++) {
struct onnxifi_backend_id_wrapper* backend_id_wrapper =
(struct onnxifi_backend_id_wrapper*) backend_ids[i];
assert(backend_id_wrapper->magic == ONNXIFI_BACKEND_ID_MAGIC);
(void) backend_id_wrapper->library->onnxReleaseBackendID(
backend_id_wrapper->backend_id);
/* Safety precaution to avoid use-after-free bugs */
memset(backend_id_wrapper, 0, sizeof(struct onnxifi_backend_id_wrapper));
free(backend_id_wrapper);
}
free(backend_ids);
return status;
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxReleaseBackendID(
onnxBackendID backendID)
{
if (backendID == NULL) {
return ONNXIFI_STATUS_INVALID_ID;
}
struct onnxifi_backend_id_wrapper* backend_id_wrapper =
(struct onnxifi_backend_id_wrapper*) backendID;
if (backend_id_wrapper->magic != ONNXIFI_BACKEND_ID_MAGIC) {
return ONNXIFI_STATUS_INVALID_ID;
}
if (backend_id_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_ID;
}
/* Call onnxReleaseBackendID with unwrapped backend ID */
const onnxStatus status =
backend_id_wrapper->library->onnxReleaseBackendID(
backend_id_wrapper->backend_id);
/*
* Note: ReleaseBackendID either succeeded, or failed with internal error.
* Either way, it is not safe to use the backend ID again.
*/
memset(backend_id_wrapper, 0, sizeof(struct onnxifi_backend_id_wrapper));
free(backend_id_wrapper);
return status;
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxGetBackendInfo(
onnxBackendID backendID,
onnxBackendInfo infoType,
void* infoValue,
size_t* infoValueSize)
{
if (backendID == NULL) {
return ONNXIFI_STATUS_INVALID_ID;
}
const struct onnxifi_backend_id_wrapper* backend_id_wrapper =
(const struct onnxifi_backend_id_wrapper*) backendID;
if (backend_id_wrapper->magic != ONNXIFI_BACKEND_ID_MAGIC) {
return ONNXIFI_STATUS_INVALID_ID;
}
if (backend_id_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_ID;
}
/* Call onnxGetBackendInfo with unwrapped backend ID */
return backend_id_wrapper->library->onnxGetBackendInfo(
backend_id_wrapper->backend_id,
infoType,
infoValue,
infoValueSize);
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxGetBackendCompatibility(
onnxBackendID backendID,
size_t onnxModelSize,
const void* onnxModel)
{
if (backendID == NULL) {
return ONNXIFI_STATUS_INVALID_ID;
}
const struct onnxifi_backend_id_wrapper* backend_id_wrapper =
(const struct onnxifi_backend_id_wrapper*) backendID;
if (backend_id_wrapper->magic != ONNXIFI_BACKEND_ID_MAGIC) {
return ONNXIFI_STATUS_INVALID_ID;
}
if (backend_id_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_ID;
}
/* Call onnxGetBackendCompatibility with unwrapped backend ID */
return backend_id_wrapper->library->onnxGetBackendCompatibility(
backend_id_wrapper->backend_id,
onnxModelSize,
onnxModel);
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxInitBackend(
onnxBackendID backendID,
const uint64_t* auxPropertiesList,
onnxBackend* backend)
{
if (backend == NULL) {
return ONNXIFI_STATUS_INVALID_POINTER;
}
*backend = NULL;
if (backendID == NULL) {
return ONNXIFI_STATUS_INVALID_ID;
}
const struct onnxifi_backend_id_wrapper* backend_id_wrapper =
(const struct onnxifi_backend_id_wrapper*) backendID;
if (backend_id_wrapper->magic != ONNXIFI_BACKEND_ID_MAGIC) {
return ONNXIFI_STATUS_INVALID_ID;
}
if (backend_id_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_ID;
}
struct onnxifi_backend_wrapper* backend_wrapper =
(struct onnxifi_backend_wrapper*)
malloc(sizeof(struct onnxifi_backend_wrapper));
if (backend_wrapper == NULL) {
return ONNXIFI_STATUS_NO_SYSTEM_MEMORY;
}
memset(backend_wrapper, 0, sizeof(struct onnxifi_backend_wrapper));
/* Call onnxInitBackend with unwrapped backend ID */
const onnxStatus status = backend_id_wrapper->library->onnxInitBackend(
backend_id_wrapper->backend_id,
auxPropertiesList,
&backend_wrapper->backend);
if (status == ONNXIFI_STATUS_SUCCESS) {
/* Success, return wrapped graph */
backend_wrapper->magic = ONNXIFI_BACKEND_MAGIC;
backend_wrapper->library = backend_id_wrapper->library;
*backend = (onnxBackend) backend_wrapper;
return ONNXIFI_STATUS_SUCCESS;
} else {
/* Failure, release allocated memory */
free(backend_wrapper);
return status;
}
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxReleaseBackend(
onnxBackend backend)
{
if (backend == NULL) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
struct onnxifi_backend_wrapper* backend_wrapper =
(struct onnxifi_backend_wrapper*) backend;
if (backend_wrapper->magic != ONNXIFI_BACKEND_MAGIC) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
if (backend_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
/* Call onnxReleaseBackend with unwrapped backend handle */
const onnxStatus status = backend_wrapper->library->onnxReleaseBackend(
backend_wrapper->backend);
/*
* Note: ReleaseBackend either succeeded, or failed with internal error.
* Either way, it is not safe to use the backend handle again.
*/
memset(backend_wrapper, 0, sizeof(struct onnxifi_backend_wrapper));
free(backend_wrapper);
return status;
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxInitEvent(
onnxBackend backend,
onnxEvent* event)
{
if (event == NULL) {
return ONNXIFI_STATUS_INVALID_POINTER;
}
*event = NULL;
if (backend == NULL) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
const struct onnxifi_backend_wrapper* backend_wrapper =
(const struct onnxifi_backend_wrapper*) backend;
if (backend_wrapper->magic != ONNXIFI_BACKEND_MAGIC) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
if (backend_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
struct onnxifi_event_wrapper* event_wrapper =
(struct onnxifi_event_wrapper*)
malloc(sizeof(struct onnxifi_event_wrapper));
if (event_wrapper == NULL) {
return ONNXIFI_STATUS_NO_SYSTEM_MEMORY;
}
memset(event_wrapper, 0, sizeof(struct onnxifi_event_wrapper));
/* Call onnxInitEvent with unwrapped backend handle */
const onnxStatus status = backend_wrapper->library->onnxInitEvent(
backend_wrapper->backend,
&event_wrapper->event);
if (status == ONNXIFI_STATUS_SUCCESS) {
/* Success, return wrapped graph */
event_wrapper->magic = ONNXIFI_EVENT_MAGIC;
event_wrapper->library = backend_wrapper->library;
*event = (onnxEvent) event_wrapper;
return ONNXIFI_STATUS_SUCCESS;
} else {
/* Failure, release allocated memory */
free(event_wrapper);
return status;
}
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxSignalEvent(
onnxEvent event)
{
if (event == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
const struct onnxifi_event_wrapper* event_wrapper =
(const struct onnxifi_event_wrapper*) event;
if (event_wrapper->magic != ONNXIFI_EVENT_MAGIC) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
if (event_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
/* Call onnxSignalEvent with unwrapped backend handle */
return event_wrapper->library->onnxSignalEvent(
event_wrapper->event);
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxGetEventState(
onnxEvent event,
onnxEventState* state)
{
if (state == NULL) {
return ONNXIFI_STATUS_INVALID_POINTER;
}
*state = ONNXIFI_EVENT_STATE_INVALID;
if (event == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
const struct onnxifi_event_wrapper* event_wrapper =
(const struct onnxifi_event_wrapper*) event;
if (event_wrapper->magic != ONNXIFI_EVENT_MAGIC) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
if (event_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
/* Call onnxGetEventState with unwrapped backend handle */
return event_wrapper->library->onnxGetEventState(
event_wrapper->event, state);
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxWaitEvent(
onnxEvent event)
{
if (event == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
const struct onnxifi_event_wrapper* event_wrapper =
(const struct onnxifi_event_wrapper*) event;
if (event_wrapper->magic != ONNXIFI_EVENT_MAGIC) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
if (event_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
/* Call onnxWaitEvent with unwrapped backend handle */
return event_wrapper->library->onnxWaitEvent(
event_wrapper->event);
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxReleaseEvent(
onnxEvent event)
{
if (event == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
struct onnxifi_event_wrapper* event_wrapper =
(struct onnxifi_event_wrapper*) event;
if (event_wrapper->magic != ONNXIFI_EVENT_MAGIC) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
if (event_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
/* Call onnxReleaseEvent with unwrapped event handle */
const onnxStatus status = event_wrapper->library->onnxReleaseEvent(
event_wrapper->event);
/*
* Note: ReleaseEvent either succeeded, or failed with internal error.
* Either way, it is not safe to use the event handle again.
*/
memset(event_wrapper, 0, sizeof(struct onnxifi_event_wrapper));
free(event_wrapper);
return status;
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI onnxInitGraph(
onnxBackend backend,
const uint64_t* auxPropertiesList,
size_t onnxModelSize,
const void* onnxModel,
uint32_t weightsCount,
const onnxTensorDescriptorV1* weightDescriptors,
onnxGraph* graph)
{
if (graph == NULL) {
return ONNXIFI_STATUS_INVALID_POINTER;
}
*graph = NULL;
if (backend == NULL) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
struct onnxifi_backend_wrapper* backend_wrapper =
(struct onnxifi_backend_wrapper*) backend;
if (backend_wrapper->magic != ONNXIFI_BACKEND_MAGIC) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
if (backend_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_BACKEND;
}
struct onnxifi_graph_wrapper* graph_wrapper =
(struct onnxifi_graph_wrapper*)
malloc(sizeof(struct onnxifi_graph_wrapper));
if (graph_wrapper == NULL) {
return ONNXIFI_STATUS_NO_SYSTEM_MEMORY;
}
memset(graph_wrapper, 0, sizeof(struct onnxifi_graph_wrapper));
/* Call onnxInitGraph with unwrapped backend handle */
const onnxStatus status = backend_wrapper->library->onnxInitGraph(
backend_wrapper->backend,
auxPropertiesList,
onnxModelSize,
onnxModel,
weightsCount,
weightDescriptors,
&graph_wrapper->graph);
switch (status) {
case ONNXIFI_STATUS_SUCCESS:
case ONNXIFI_STATUS_FALLBACK:
/* Success, return wrapped graph */
graph_wrapper->magic = ONNXIFI_GRAPH_MAGIC;
graph_wrapper->library = backend_wrapper->library;
*graph = (onnxGraph) graph_wrapper;
return status;
default:
/* Failure, release allocated memory */
free(graph_wrapper);
return status;
}
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI onnxSetGraphIO(
onnxGraph graph,
uint32_t inputsCount,
const onnxTensorDescriptorV1* inputDescriptors,
uint32_t outputsCount,
const onnxTensorDescriptorV1* outputDescriptors)
{
if (graph == NULL) {
return ONNXIFI_STATUS_INVALID_GRAPH;
}
struct onnxifi_graph_wrapper* graph_wrapper =
(struct onnxifi_graph_wrapper*) graph;
if (graph_wrapper->magic != ONNXIFI_GRAPH_MAGIC) {
return ONNXIFI_STATUS_INVALID_GRAPH;
}
if (graph_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_GRAPH;
}
/* Call onnxSetGraphIO with unwrapped graph handle */
return graph_wrapper->library->onnxSetGraphIO(
graph_wrapper->graph,
inputsCount,
inputDescriptors,
outputsCount,
outputDescriptors);
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI
onnxRunGraph(
onnxGraph graph,
const onnxMemoryFenceV1* inputFence,
onnxMemoryFenceV1* outputFence)
{
if (graph == NULL) {
return ONNXIFI_STATUS_INVALID_GRAPH;
}
if (inputFence == NULL) {
return ONNXIFI_STATUS_INVALID_POINTER;
}
if (outputFence == NULL) {
return ONNXIFI_STATUS_INVALID_POINTER;
}
switch (inputFence->tag) {
case ONNXIFI_TAG_MEMORY_FENCE_V1:
break;
default:
return ONNXIFI_STATUS_UNSUPPORTED_TAG;
}
switch (outputFence->tag) {
case ONNXIFI_TAG_MEMORY_FENCE_V1:
break;
default:
return ONNXIFI_STATUS_UNSUPPORTED_TAG;
}
struct onnxifi_graph_wrapper* graph_wrapper =
(struct onnxifi_graph_wrapper*) graph;
if (graph_wrapper->magic != ONNXIFI_GRAPH_MAGIC) {
return ONNXIFI_STATUS_INVALID_GRAPH;
}
if (graph_wrapper->library == NULL) {
return ONNXIFI_STATUS_INVALID_GRAPH;
}
const onnxMemoryFenceV1* input_fence_ptr = inputFence;
onnxMemoryFenceV1 input_fence_wrapper;
switch (inputFence->type) {
case ONNXIFI_SYNCHRONIZATION_EVENT:
{
if (inputFence->event == NULL) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
const struct onnxifi_event_wrapper* event_wrapper =
(const struct onnxifi_event_wrapper*) inputFence->event;
if (event_wrapper->magic != ONNXIFI_EVENT_MAGIC) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
if (event_wrapper->library != graph_wrapper->library) {
return ONNXIFI_STATUS_INVALID_EVENT;
}
/* Initialize wrapper for input fence */
input_fence_wrapper.tag = ONNXIFI_TAG_MEMORY_FENCE_V1;
input_fence_wrapper.type = ONNXIFI_SYNCHRONIZATION_EVENT;
input_fence_wrapper.event = event_wrapper->event;
input_fence_ptr = &input_fence_wrapper;
break;
}
default:
/* Pass inputFence as is */
break;
}
onnxMemoryFenceV1* output_fence_ptr = outputFence;
onnxMemoryFenceV1 output_fence_wrapper;
struct onnxifi_event_wrapper* output_event = NULL;
switch (outputFence->type) {
case ONNXIFI_SYNCHRONIZATION_EVENT:
{
/* Initialize wrapper for output fence */
output_fence_wrapper.tag = ONNXIFI_TAG_MEMORY_FENCE_V1;
output_fence_wrapper.type = ONNXIFI_SYNCHRONIZATION_EVENT;
/* event will be populated by onnxRunGraph */
output_fence_wrapper.event = NULL;
output_fence_ptr = &output_fence_wrapper;
/*
* Pre-allocate memory for output event wrapper.
* This must be done before onnxRunGraph, so in case allocation fails,
* the function call has no side-effects.
*/
output_event = malloc(sizeof(struct onnxifi_event_wrapper));
if (output_event == NULL) {
return ONNXIFI_STATUS_NO_SYSTEM_MEMORY;
}
memset(output_event, 0, sizeof(struct onnxifi_event_wrapper));
}
}
const onnxStatus status = graph_wrapper->library->onnxRunGraph(
graph_wrapper->graph,
input_fence_ptr,
output_fence_ptr);
if (status == ONNXIFI_STATUS_SUCCESS) {
/* Wrap output event, if needed */
if (output_event != NULL) {
output_event->magic = ONNXIFI_EVENT_MAGIC;
output_event->event = output_fence_wrapper.event;
output_event->library = graph_wrapper->library;
outputFence->event = (onnxEvent) output_event;
}
} else {
/* Deallocate output event wrapper, if needed */
free(output_event);
}
return status;
}
ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI onnxReleaseGraph(
onnxGraph graph)
{
if (graph == NULL) {
return ONNXIFI_STATUS_INVALID_GRAPH;
}
struct onnxifi_graph_wrapper* graph_wrapper =
(struct onnxifi_graph_wrapper*) graph;
if (graph_wrapper->magic != ONNXIFI_GRAPH_MAGIC) {
return ONNXIFI_STATUS_INVALID_GRAPH;
}
const onnxStatus status =
graph_wrapper->library->onnxReleaseGraph(graph_wrapper->graph);
/*
* Note: ReleaseGraph either succeeded, or failed with internal error.
* Either way, it is not safe to use the graph handle again.
*/
memset(graph_wrapper, 0, sizeof(struct onnxifi_graph_wrapper));
free(graph_wrapper);
return status;
}
onnx-1.7.0/onnx/frontend/ 0000775 0000000 0000000 00000000000 13655345213 014011 5 ustar root root onnx-1.7.0/onnx/frontend/__init__.py 0000664 0000000 0000000 00000000000 13655345213 016110 0 ustar root root onnx-1.7.0/onnx/onnxifi_utils.h 0000664 0000000 0000000 00000000574 13655345213 015243 0 ustar root root #include
#include
#include "onnx/onnx_pb.h"
#include "onnx/onnxifi.h"
#include "onnx/proto_utils.h"
namespace ONNX_NAMESPACE {
namespace testing {
onnxTensorDescriptorV1 ProtoToOnnxTensorDescriptor(
const ONNX_NAMESPACE::TensorProto& proto_tensor,
std::vector>& shape_pool);
} // namespace testing
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/onnxifi_ext.h 0000664 0000000 0000000 00000032207 13655345213 014701 0 ustar root root #ifndef ONNXIFI_EXT_H
#define ONNXIFI_EXT_H 1
#include "onnx/onnxifi.h"
#ifdef __cplusplus
extern "C" {
#endif
/**
* Generic ONNXIFI extension function pointer.
*
* The caller should convert this generic function pointer to the function
* pointer specific for an extension function type.
*/
typedef onnxStatus (ONNXIFI_ABI* onnxExtensionFunctionPointer)(void);
/* Function pointer declarations for dynamic loading */
typedef ONNXIFI_CHECK_RESULT onnxStatus
(ONNXIFI_ABI* onnxGetExtensionFunctionAddressFunction)(
onnxBackendID backendID,
const char* name,
onnxExtensionFunctionPointer* function);
/**
* Query function pointer for an ONNXIFI extension function.
*
* The returned function pointer is specific to the provided backend ID, and
* MUST NOT be used with objects created for other backend IDs.
*
* This function is a part of onnx_extension_function extension. Backends which
* implement this function MUST list "onnx_extension_function" in the result of
* onnxGetBackendInfo with ONNXIFI_BACKEND_EXTENSIONS information type.
*
* @param backendID - ID of the backend to query for extension function.
* @param[in] name - name of the extension function to query.
* @param[out] function - pointer to a generic function pointer for an ONNXIFI
* extension function. If the function fails, the
* function pointer is initialized to NULL. The caller
* MUST cast this function pointer to the type specific
* for the extension function before use.
*
* @retval ONNXIFI_STATUS_SUCCESS The function call succeeded and the extension
* function pointer is stored in the location
* specified by function argument.
* @retval ONNXIFI_STATUS_INVALID_ID The function call failed because backendID
* is not an ONNXIFI backend ID.
* @retval ONNXIFI_STATUS_INVALID_POINTER The function call failed because
* name or function argument is NULL.
* @retval ONNXIFI_STATUS_UNIDENTIFIED_NAME The function call failed because
* the backend does not implement
* the function identified by the name.
* @retval ONNXIFI_STATUS_INTERNAL_ERROR The function call failed because the
* backend experienced an unrecovered
* internal error.
* @retval ONNXIFI_STATUS_BACKEND_UNAVAILABLE The function call failed because
* the backend was disconnected or
* uninstalled from the system.
*/
ONNXIFI_PUBLIC ONNXIFI_CHECK_RESULT onnxStatus ONNXIFI_ABI
onnxGetExtensionFunctionAddress(
onnxBackendID backendID,
const char* name,
onnxExtensionFunctionPointer* function);
/* Extension function pointer declarations for dynamic loading */
typedef ONNXIFI_CHECK_RESULT onnxStatus
(ONNXIFI_ABI* onnxSetIOAndRunGraphFunction)(
onnxGraph graph,
uint32_t inputsCount,
const onnxTensorDescriptorV1* inputDescriptors,
uint32_t outputsCount,
const onnxTensorDescriptorV1* outputDescriptors,
onnxMemoryFenceV1* outputFence);
/**
* A combination of onnxSetIO and onnxRunGraph, functionally equals to first run
* onnxSetIO(graph, inputsCount, inputDescriptors, outputsCount,
* outputDescriptors), then run onnxRunGraph(graph, inputFence, outputFence)
* with an internal inputFence.
*
* As two separate functions, it is difficult to do atomic evaluation.
* Therefore, we would like to unify this process and make it evaluable.
*
* @param graph - graph handle created by onnxInitGraph.
* @param inputsCount - number of elements in the inputDescriptors array.
* @param[in] inputDescriptors - descriptors of input tensors for the graph.
* Elements of this array must provide a location
* for each ValueInfoProto.name listed in
* ModelProto.graph.input of the ONNX graph.
* If inputsCount is non-zero, inputDescriptors
* pointer must be non-NULL.
* @param outputsCount - number of elements in the outputDescriptors array.
* Must be greater than zero.
* @param[in] outputDescriptors - descriptors of output tensors for the graph.
* outputDescriptors pointer must be non-NULL.
* Elements of this array must provide a location
* for each ValueInfoProto.name listed in
* ModelProto.graph.output of the ONNX graph.
* @param[out] outputFence - synchronization primitive that signals when graph
* outputs are ready to use by the caller. The type
* of the synchronization primitive always must be
* initialized by the caller. The type of the
* synchronization primitive determines whether it
* is initialized by the user before the call or by
* the backend as a result of this call. Single-shot
* synchronizatiom objects are initialized as a result
* of the call. Reusable synchronization objects are
* generally initialized by the user prior to the
* call.
*
* @retval ONNXIFI_STATUS_SUCCESS The function call succeeded and the all graph
* inputs and outputs were matched to a memory
* location.
* @retval ONNXIFI_STATUS_INVALID_GRAPH The function call failed because
* graph is not an ONNXIFI graph handle.
* @retval ONNXIFI_STATUS_INVALID_POINTER The function call failed because
* outputDescriptors pointer is NULL or
* inputDescriptors pointer is NULL while
* inputsCount is non-zero.
* @retval ONNXIFI_STATUS_INVALID_NAME The function call failed because one of
* the names in tensor descriptors doesn't
* match blob name in ModelProto.graph.input
* or ModelProto.graph.output, or the same
* name appears in more than one tensor
* descriptor.
* @retval ONNXIFI_STATUS_INVALID_SHAPE The function call failed because one of
* the shape dimensions is 0.
* @retval ONNXIFI_STATUS_INVALID_DATATYPE The function call failed because
* one of the data types in
* inputDescriptors or outputDescriptors
* is unknown to the backend.
* @retval ONNXIFI_STATUS_INVALID_MEMORY_TYPE The function call failed because
* one of the memory types in
* inputDescriptors or
* outputDescriptors is unknown to
* the backend.
* @retval ONNXIFI_STATUS_INVALID_MEMORY_LOCATION The function call failed
* because one of the memory
* locations in inputDescriptors
* or outputDescriptors is not
* valid for the specified
* memory type (e.g. NULL pointer
* for ONNXIFI_MEMORY_TYPE_CPU).
* @retval ONNXIFI_STATUS_UNSUPPORTED_TAG The function call failed because one
* of the tags in inputDescriptors or
* outputDescriptors is unknown to the
* backend (tag does not match
* ONNXIFI_TAG_TENSOR_DESCRIPTOR_V1).
* @retval ONNXIFI_STATUS_UNSUPPORTED_SHAPE The function call failed because the
* backend does not support the
* tensor shapes in an input or output
* of one of the operators. The
* problematic tensor shapes could be
* directly specified through
* inputDescriptors or
* outputDescriptors argument,
* or inferred from the inputs by the
* backend. This error code can be
* returned when the backend supports
* variable-size inputs and outputs,
* and the problematic tensor shape was
* provided in the ValueInfoProto as a
* symbolic variable.
* @retval ONNXIFI_STATUS_UNSUPPORTED_MEMORY_TYPE The function call failed
* because the backend does not
* support one of the memory
* types in inputDescriptors or
* outputDescriptors.
* @retval ONNXIFI_STATUS_UNIDENTIFIED_NAME The function call failed because one
* of the ValueInfoProto.name value in
* ModelProto.graph.input or
* ModelProto.graph.output doesn't have
* a match in the inputDescriptors or
* outputDescriptors.
* @retval ONNXIFI_STATUS_MISMATCHING_SHAPE The function call failed because
* the shapes specified through
* inputDescriptors or
* outputDescriptors argument are
* inconsistent with the shapes
* specified in the ONNX model graph.
* @retval ONNXIFI_STATUS_MISMATCHING_DATATYPE The function call failed because
* data types specified through
* inputDescriptors or
* outputDescriptors argument are
* inconsistent with the data types
* specified in the ONNX model
* graph.
* @retval ONNXIFI_STATUS_NO_SYSTEM_MEMORY The function call failed because the
* backend could not allocate enough
* system memory to parse, analyze, and
* initialize the tensor locations.
* @retval ONNXIFI_STATUS_NO_SYSTEM_RESOURCES The function call failed due to
* insufficient non-memory system
* resources (e.g. file handles) to
* initialize the tensor locations.
* @retval ONNXIFI_STATUS_NO_DEVICE_MEMORY The function call failed due to
* insufficient backend-specific memory
* to initialize the tensor locations.
* @retval ONNXIFI_STATUS_NO_DEVICE_RESOURCES The function call failed due to
* insufficient non-memory
* backend-specific resources (e.g.
* command queues) to initialize the
* tensor locations.
* @retval ONNXIFI_STATUS_BACKEND_UNAVAILABLE The function call failed because
* the backend was disconnected or
* uninstalled from the system.
* @retval ONNXIFI_STATUS_INTERNAL_ERROR The function call failed because the
* backend experienced an unrecovered
* internal error.
*/
ONNXIFI_PUBLIC ONNXIFI_CHECK_RESULT onnxStatus ONNXIFI_ABI onnxSetIOAndRunGraph(
onnxGraph graph,
uint32_t inputsCount,
const onnxTensorDescriptorV1* inputDescriptors,
uint32_t outputsCount,
const onnxTensorDescriptorV1* outputDescriptors,
onnxMemoryFenceV1* outputFence);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* !defined(ONNXIFI_EXT_H) */
onnx-1.7.0/onnx/onnxifi_loader.h 0000664 0000000 0000000 00000007043 13655345213 015347 0 ustar root root #ifndef ONNXIFI_LOADER_H
#define ONNXIFI_LOADER_H 1
#include "onnx/onnxifi.h"
#include "onnx/onnxifi_ext.h"
#ifdef ONNXIFI_ENABLE_EXT
#define ONNXIFI_LOADER_FUNCTION_COUNT 16
#else
#define ONNXIFI_LOADER_FUNCTION_COUNT 15
#endif
#define ONNXIFI_LOADER_FLAG_VERSION_MASK 0xFF
#define ONNXIFI_LOADER_FLAG_VERSION_1_0 0x01
#ifndef ONNXIFI_HIDDEN
#if defined(__ELF__)
#define ONNXIFI_HIDDEN __attribute__((__visibility__("hidden")))
#elif defined(__MACH__)
#define ONNXIFI_HIDDEN __attribute__((__visibility__("hidden")))
#else
#define ONNXIFI_HIDDEN
#endif
#endif /* !defined(ONNXIFI_HIDDEN) */
struct onnxifi_library {
/*
* Opaque handle for the loaded ONNXIFI library.
*
* Note: this is the value returned from LoadLibraryW (on Windows), or
* dlopen (on other operating systems and environments).
*/
void* handle;
/*
* Options used for dynamic loading of the ONNXIFI library and its API
* functions. These are the options passed in flags parameter to onnxifi_load.
*/
uint32_t flags;
union {
struct {
onnxGetBackendIDsFunction onnxGetBackendIDs;
onnxReleaseBackendIDFunction onnxReleaseBackendID;
onnxGetBackendInfoFunction onnxGetBackendInfo;
onnxGetBackendCompatibilityFunction onnxGetBackendCompatibility;
onnxInitBackendFunction onnxInitBackend;
onnxReleaseBackendFunction onnxReleaseBackend;
onnxInitEventFunction onnxInitEvent;
onnxSignalEventFunction onnxSignalEvent;
onnxGetEventStateFunction onnxGetEventState;
onnxWaitEventFunction onnxWaitEvent;
onnxReleaseEventFunction onnxReleaseEvent;
onnxInitGraphFunction onnxInitGraph;
onnxSetGraphIOFunction onnxSetGraphIO;
onnxRunGraphFunction onnxRunGraph;
onnxReleaseGraphFunction onnxReleaseGraph;
#ifdef ONNXIFI_ENABLE_EXT
onnxGetExtensionFunctionAddressFunction onnxGetExtensionFunctionAddress;
#endif
};
void* functions[ONNXIFI_LOADER_FUNCTION_COUNT];
};
};
#ifdef __cplusplus
extern "C" {
#endif
/**
* Dynamically load the ONNXIFI library.
*
* @param flags - options for dynamic loading of the ONNXIFI library.
* The ONNXIFI_LOADER_FLAG_VERSION_MASK part of the flags
* specifies which ONNX API function should be loaded. The only
* currently supported value is ONNXIFI_LOADER_FLAG_VERSION_1_0.
* @param path - optional path to the ONNXIFI library to load.
* If this argument is null, the default path is used.
* @param[out] library - the structure representing the dynamic library and
* its API functions. On success, this structure will
* be initialized with valid pointers to implentation.
* On failure, this structure will be zero-initialized.
*
* @return Non-zero if the function succeeds, or zero if the function fails.
*/
ONNXIFI_HIDDEN int ONNXIFI_ABI onnxifi_load(
uint32_t flags,
#ifdef _WIN32
const wchar_t* path,
#else
const char* path,
#endif
struct onnxifi_library* library);
/**
* Unload the dynamically loaded ONNXIFI library.
*
* @param[in,out] library - the structure representing the dynamic library and
* its API functions. If this structure is
* zero-initialized, the function does nothing.
* The function zero-initialized the structure before
* returning.
*/
ONNXIFI_HIDDEN void ONNXIFI_ABI onnxifi_unload(
struct onnxifi_library* library);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* !defined(ONNXIFI_LOADER_H) */
onnx-1.7.0/onnx/onnx.in.proto 0000664 0000000 0000000 00000073021 13655345213 014651 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
syntax = "proto2";
package {PACKAGE_NAME};
// Overview
//
// ONNX is an open specification that is comprised of the following components:
//
// 1) A definition of an extensible computation graph model.
// 2) Definitions of standard data types.
// 3) Definitions of built-in operators.
//
// This document describes the syntax of models and their computation graphs,
// as well as the standard data types. Together, they are referred to as the ONNX
// Intermediate Representation, or 'IR' for short.
//
// The normative semantic specification of the ONNX IR is found in docs/IR.md.
// Definitions of the built-in neural network operators may be found in docs/Operators.md.
// #if ONNX-ML
// Definitions of the built-in classical machine learning operators may be found in
// docs/Operators-ml.md.
// #endif
// Notes
//
// Release
//
// We are still in the very early stage of defining ONNX. The current
// version of ONNX is a starting point. While we are actively working
// towards a complete spec, we would like to get the community involved
// by sharing our working version of ONNX.
//
// Protobuf compatibility
//
// To simplify framework compatibility, ONNX is defined using the subset of protobuf
// that is compatible with both protobuf v2 and v3. This means that we do not use any
// protobuf features that are only available in one of the two versions.
//
// Here are the most notable contortions we have to carry out to work around
// these limitations:
//
// - No 'map' (added protobuf 3.0). We instead represent mappings as lists
// of key-value pairs, where order does not matter and duplicates
// are not allowed.
// Versioning
//
// ONNX versioning is specified in docs/IR.md and elaborated on in docs/Versioning.md
//
// To be compatible with both proto2 and proto3, we will use a version number
// that is not defined by the default value but an explicit enum number.
enum Version {
// proto3 requires the first enum value to be zero.
// We add this just to appease the compiler.
_START_VERSION = 0;
// The version field is always serialized and we will use it to store the
// version that the graph is generated from. This helps us set up version
// control.
// For the IR, we are using simple numbers starting with 0x00000001,
// which was the version we published on Oct 10, 2017.
IR_VERSION_2017_10_10 = 0x0000000000000001;
// IR_VERSION 2 published on Oct 30, 2017
// - Added type discriminator to AttributeProto to support proto3 users
IR_VERSION_2017_10_30 = 0x0000000000000002;
// IR VERSION 3 published on Nov 3, 2017
// - For operator versioning:
// - Added new message OperatorSetIdProto
// - Added opset_import in ModelProto
// - For vendor extensions, added domain in NodeProto
IR_VERSION_2017_11_3 = 0x0000000000000003;
// IR VERSION 4 published on Jan 22, 2019
// - Relax constraint that initializers should be a subset of graph inputs
// - Add type BFLOAT16
IR_VERSION_2019_1_22 = 0x0000000000000004;
// IR VERSION 5 published on March 18, 2019
// - Add message TensorAnnotation.
// - Add quantization annotation in GraphProto to map tensor with its scale and zero point quantization parameters.
IR_VERSION_2019_3_18 = 0x0000000000000005;
// IR VERSION 6 published on Sep 19, 2019
// - Add support for sparse tensor constants stored in model.
// - Add message SparseTensorProto
// - Add sparse initializers
IR_VERSION_2019_9_19 = 0x0000000000000006;
// IR VERSION 7 published on
// - Add support to allow function body graph to rely on multiple external opreator sets.
// - Add a list to promote inference graph's initializers to global and
// mutable variables. Global variables are visible in all graphs of the
// stored models.
// - Add message TrainingInfoProto to store initialization
// method and training algorithm. The execution of TrainingInfoProto
// can modify the values of mutable variables.
// - Make inference graph callable from TrainingInfoProto via GraphCall operator.
IR_VERSION = 0x0000000000000007;
}
// Attributes
//
// A named attribute containing either singular float, integer, string, graph,
// and tensor values, or repeated float, integer, string, graph, and tensor values.
// An AttributeProto MUST contain the name field, and *only one* of the
// following content fields, effectively enforcing a C/C++ union equivalent.
message AttributeProto {
// Note: this enum is structurally identical to the OpSchema::AttrType
// enum defined in schema.h. If you rev one, you likely need to rev the other.
enum AttributeType {
UNDEFINED = 0;
FLOAT = 1;
INT = 2;
STRING = 3;
TENSOR = 4;
GRAPH = 5;
SPARSE_TENSOR = 11;
FLOATS = 6;
INTS = 7;
STRINGS = 8;
TENSORS = 9;
GRAPHS = 10;
SPARSE_TENSORS = 12;
}
// The name field MUST be present for this version of the IR.
optional string name = 1; // namespace Attribute
// if ref_attr_name is not empty, ref_attr_name is the attribute name in parent function.
// In this case, this AttributeProto does not contain data, and it's a reference of attribute
// in parent scope.
// NOTE: This should ONLY be used in function (sub-graph). It's invalid to be used in main graph.
optional string ref_attr_name = 21;
// A human-readable documentation for this attribute. Markdown is allowed.
optional string doc_string = 13;
// The type field MUST be present for this version of the IR.
// For 0.0.1 versions of the IR, this field was not defined, and
// implementations needed to use has_field heuristics to determine
// which value field was in use. For IR_VERSION 0.0.2 or later, this
// field MUST be set and match the f|i|s|t|... field in use. This
// change was made to accommodate proto3 implementations.
optional AttributeType type = 20; // discriminator that indicates which field below is in use
// Exactly ONE of the following fields must be present for this version of the IR
optional float f = 2; // float
optional int64 i = 3; // int
optional bytes s = 4; // UTF-8 string
optional TensorProto t = 5; // tensor value
optional GraphProto g = 6; // graph
optional SparseTensorProto sparse_tensor = 22; // sparse tensor value
// Do not use field below, it's deprecated.
// optional ValueProto v = 12; // value - subsumes everything but graph
repeated float floats = 7; // list of floats
repeated int64 ints = 8; // list of ints
repeated bytes strings = 9; // list of UTF-8 strings
repeated TensorProto tensors = 10; // list of tensors
repeated GraphProto graphs = 11; // list of graph
repeated SparseTensorProto sparse_tensors = 23; // list of sparse tensors
}
// Defines information on value, including the name, the type, and
// the shape of the value.
message ValueInfoProto {
// This field MUST be present in this version of the IR.
optional string name = 1; // namespace Value
// This field MUST be present in this version of the IR for
// inputs and outputs of the top-level graph.
optional TypeProto type = 2;
// A human-readable documentation for this value. Markdown is allowed.
optional string doc_string = 3;
}
// Nodes
//
// Computation graphs are made up of a DAG of nodes, which represent what is
// commonly called a "layer" or "pipeline stage" in machine learning frameworks.
//
// For example, it can be a node of type "Conv" that takes in an image, a filter
// tensor and a bias tensor, and produces the convolved output.
message NodeProto {
repeated string input = 1; // namespace Value
repeated string output = 2; // namespace Value
// An optional identifier for this node in a graph.
// This field MAY be absent in ths version of the IR.
optional string name = 3; // namespace Node
// The symbolic identifier of the Operator to execute.
optional string op_type = 4; // namespace Operator
// The domain of the OperatorSet that specifies the operator named by op_type.
optional string domain = 7; // namespace Domain
// Additional named attributes.
repeated AttributeProto attribute = 5;
// A human-readable documentation for this node. Markdown is allowed.
optional string doc_string = 6;
}
// Training information
// TrainingInfoProto stores information for training a model.
// In particular, this defines two functionalities: an initialization-step
// and a training-algorithm-step. Initialization resets the model
// back to its original state as if no training has been consumed.
// Training algorithm improves the model based on input data.
//
// The semantics of the initialization-step is that the initializers
// in ModelProto.graph and in TrainingInfoProto.algorithm are first
// initialized as specified by the initializers in the graph, and then
// updated by the "initialization_binding" in every instance in
// ModelProto.training_info.
//
// The field "algorithm" defines a computation graph which represents a
// training algorithm's step. After the execution of a
// TrainingInfoProto.algorithm, the initializers specified by "update_binding"
// may be immediately updated. If the targeted training algorithm contains
// consecutive update stages (such as block coordinate descent methods),
// the user needs to create a TrainingInfoProto for each stage.
message TrainingInfoProto {
// This field describes a graph to compute the initial tensors
// upon starting the training process. Initialization graph has no input
// and can have multiple outputs. Usually, trainable tensors in neural
// networks are randomly initialized. To achieve that, for each tensor,
// the user can put a random number operator such as RandomNormal or
// RandomUniform in TrainingInfoProto.initialization.node and assign its
// random output to the specific tensor using "initialization_binding".
// This graph can also set the initializers in "algorithm" in the same
// TrainingInfoProto; a use case is resetting the number of training
// iteration to zero.
//
// By default, this field is an empty graph and its evaluation does not
// produce any output.
optional GraphProto initialization = 1;
// This field represents a training algorithm step. Given required inputs,
// it computes outputs to update initializers in its own or inference graph's
// initializer lists. In general, this graph contains loss node, gradient node,
// optimizer node, increment of iteration count, and some calls to the inference
// graph.
//
// The field algorithm.node is the only place the user can use GraphCall
// operator. The only callable graph is the one stored in ModelProto.graph.
//
// By default, this field is an empty graph and its evaluation does not
// produce any output.
optional GraphProto algorithm = 2;
// This field specifies the bindings from the outputs of "initialization" to
// some initializers in "ModelProto.graph.initializer" and
// the "algorithm.initializer" in the same TrainingInfoProto.
// See "update_binding" below for details.
//
// By default, this field is empty and no initializer would be changed
// by the execution of "initialization".
repeated StringStringEntryProto initialization_binding = 3;
// Gradient-based training is usually an iterative procedure. In one gradient
// descent iteration, we apply
//
// x = x - r * g
//
// where "x" is the optimized tensor, "r" stands for learning rate, and "g" is
// gradient of "x" with respect to a chosen loss. To avoid adding assignments
// into the training graph, we split the update equation into
//
// y = x - r * g
// x = y
//
// The user needs to save "y = x - r * g" into TrainingInfoProto.algorithm. To
// tell that "y" should be assigned to "x", the field "update_binding" may
// contain a key-value pair of strings, "x" (key of StringStringEntryProto)
// and "y" (value of StringStringEntryProto).
// For a neural network with multiple trainable (mutable) tensors, there can
// be multiple key-value pairs in "update_binding".
//
// The initializers appears as keys in "update_binding" are considered
// mutable and globally-visible variables. This implies some behaviors
// as described below.
//
// 1. We have only unique keys in all "update_binding"s so that two global
// variables may not have the same name. This ensures that one
// global variable is assigned up to once.
// 2. The keys must appear in names of "ModelProto.graph.initializer" or
// "TrainingInfoProto.algorithm.initializer".
// 3. The values must be output names of "algorithm".
// 4. If an optional input of a graph is omitted when using GraphCall, the
// global variable with the same name may be used.
// 5. When using GraphCall, the users always can pass values to optional
// inputs of the called graph even if the associated initializers appears
// as keys in "update_binding"s.
// 6. The graphs in TrainingInfoProto's can use global variables as
// their operator inputs.
// 7. Mutable variables are initialized to the value specified by the
// corresponding initializer, and then potentially updated by
// "initializer_binding"s and "update_binding"s in "TrainingInfoProto"s.
//
// This field usually contains names of trainable tensors
// (in ModelProto.graph), optimizer states such as momentums in advanced
// stochastic gradient methods (in TrainingInfoProto.graph),
// and number of training iterations (in TrainingInfoProto.graph).
//
// By default, this field is empty and no initializer would be changed
// by the execution of "algorithm".
repeated StringStringEntryProto update_binding = 4;
}
// Models
//
// ModelProto is a top-level file/container format for bundling a ML model and
// associating its computation graph with metadata.
//
// The semantics of the model are described by the associated GraphProto's.
message ModelProto {
// The version of the IR this model targets. See Version enum above.
// This field MUST be present.
optional int64 ir_version = 1;
// The OperatorSets this model relies on.
// All ModelProtos MUST have at least one entry that
// specifies which version of the ONNX OperatorSet is
// being imported.
//
// All nodes in the ModelProto's graph will bind against the operator
// with the same-domain/same-op_type operator with the HIGHEST version
// in the referenced operator sets.
repeated OperatorSetIdProto opset_import = 8;
// The name of the framework or tool used to generate this model.
// This field SHOULD be present to indicate which implementation/tool/framework
// emitted the model.
optional string producer_name = 2;
// The version of the framework or tool used to generate this model.
// This field SHOULD be present to indicate which implementation/tool/framework
// emitted the model.
optional string producer_version = 3;
// Domain name of the model.
// We use reverse domain names as name space indicators. For example:
// `com.facebook.fair` or `com.microsoft.cognitiveservices`
//
// Together with `model_version` and GraphProto.name, this forms the unique identity of
// the graph.
optional string domain = 4;
// The version of the graph encoded. See Version enum below.
optional int64 model_version = 5;
// A human-readable documentation for this model. Markdown is allowed.
optional string doc_string = 6;
// The parameterized graph that is evaluated to execute the model.
optional GraphProto graph = 7;
// Named metadata values; keys should be distinct.
repeated StringStringEntryProto metadata_props = 14;
// Training-specific information. Sequentially executing all stored
// `TrainingInfoProto.algorithm`s and assigning their outputs following
// the corresponding `TrainingInfoProto.update_binding`s is one training
// iteration. Similarly, to initialize the model
// (as if training hasn't happened), the user should sequentially execute
// all stored `TrainingInfoProto.initialization`s and assigns their outputs
// using `TrainingInfoProto.initialization_binding`s.
//
// If this field is empty, the training behavior of the model is undefined.
repeated TrainingInfoProto training_info = 20;
};
// StringStringEntryProto follows the pattern for cross-proto-version maps.
// See https://developers.google.com/protocol-buffers/docs/proto3#maps
message StringStringEntryProto {
optional string key = 1;
optional string value= 2;
};
message TensorAnnotation {
optional string tensor_name = 1;
// pairs to annotate tensor specified by above.
// The keys used in the mapping below must be pre-defined in ONNX spec.
// For example, for 8-bit linear quantization case, 'SCALE_TENSOR', 'ZERO_POINT_TENSOR' will be pre-defined as
// quantization parameter keys.
repeated StringStringEntryProto quant_parameter_tensor_names = 2;
}
// Graphs
//
// A graph defines the computational logic of a model and is comprised of a parameterized
// list of nodes that form a directed acyclic graph based on their inputs and outputs.
// This is the equivalent of the "network" or "graph" in many deep learning
// frameworks.
message GraphProto {
// The nodes in the graph, sorted topologically.
repeated NodeProto node = 1;
// The name of the graph.
optional string name = 2; // namespace Graph
// A list of named tensor values, used to specify constant inputs of the graph.
// Each TensorProto entry must have a distinct name (within the list) that
// MAY also appear in the input list.
repeated TensorProto initializer = 5;
// Initializers (see above) stored in sparse format.
repeated SparseTensorProto sparse_initializer = 15;
// A human-readable documentation for this graph. Markdown is allowed.
optional string doc_string = 10;
// The inputs and outputs of the graph.
repeated ValueInfoProto input = 11;
repeated ValueInfoProto output = 12;
// Information for the values in the graph. The ValueInfoProto.name's
// must be distinct. It is optional for a value to appear in value_info list.
repeated ValueInfoProto value_info = 13;
// This field carries information to indicate the mapping among a tensor and its
// quantization parameter tensors. For example:
// For tensor 'a', it may have {'SCALE_TENSOR', 'a_scale'} and {'ZERO_POINT_TENSOR', 'a_zero_point'} annotated,
// which means, tensor 'a_scale' and tensor 'a_zero_point' are scale and zero point of tensor 'a' in the model.
repeated TensorAnnotation quantization_annotation = 14;
// DO NOT USE the following fields, they were deprecated from earlier versions.
// repeated string input = 3;
// repeated string output = 4;
// optional int64 ir_version = 6;
// optional int64 producer_version = 7;
// optional string producer_tag = 8;
// optional string domain = 9;
}
// Tensors
//
// A serialized tensor value.
message TensorProto {
enum DataType {
UNDEFINED = 0;
// Basic types.
FLOAT = 1; // float
UINT8 = 2; // uint8_t
INT8 = 3; // int8_t
UINT16 = 4; // uint16_t
INT16 = 5; // int16_t
INT32 = 6; // int32_t
INT64 = 7; // int64_t
STRING = 8; // string
BOOL = 9; // bool
// IEEE754 half-precision floating-point format (16 bits wide).
// This format has 1 sign bit, 5 exponent bits, and 10 mantissa bits.
FLOAT16 = 10;
DOUBLE = 11;
UINT32 = 12;
UINT64 = 13;
COMPLEX64 = 14; // complex with float32 real and imaginary components
COMPLEX128 = 15; // complex with float64 real and imaginary components
// Non-IEEE floating-point format based on IEEE754 single-precision
// floating-point number truncated to 16 bits.
// This format has 1 sign bit, 8 exponent bits, and 7 mantissa bits.
BFLOAT16 = 16;
// Future extensions go here.
}
// The shape of the tensor.
repeated int64 dims = 1;
// The data type of the tensor.
// This field MUST have a valid TensorProto.DataType value
optional int32 data_type = 2;
// For very large tensors, we may want to store them in chunks, in which
// case the following fields will specify the segment that is stored in
// the current TensorProto.
message Segment {
optional int64 begin = 1;
optional int64 end = 2;
}
optional Segment segment = 3;
// Tensor content must be organized in row-major order.
//
// Depending on the data_type field, exactly one of the fields below with
// name ending in _data is used to store the elements of the tensor.
// For float and complex64 values
// Complex64 tensors are encoded as a single array of floats,
// with the real components appearing in odd numbered positions,
// and the corresponding imaginary component appearing in the
// subsequent even numbered position. (e.g., [1.0 + 2.0i, 3.0 + 4.0i]
// is encoded as [1.0, 2.0 ,3.0 ,4.0]
// When this field is present, the data_type field MUST be FLOAT or COMPLEX64.
repeated float float_data = 4 [packed = true];
// For int32, uint8, int8, uint16, int16, bool, and float16 values
// float16 values must be bit-wise converted to an uint16_t prior
// to writing to the buffer.
// When this field is present, the data_type field MUST be
// INT32, INT16, INT8, UINT16, UINT8, BOOL, or FLOAT16
repeated int32 int32_data = 5 [packed = true];
// For strings.
// Each element of string_data is a UTF-8 encoded Unicode
// string. No trailing null, no leading BOM. The protobuf "string"
// scalar type is not used to match ML community conventions.
// When this field is present, the data_type field MUST be STRING
repeated bytes string_data = 6;
// For int64.
// When this field is present, the data_type field MUST be INT64
repeated int64 int64_data = 7 [packed = true];
// Optionally, a name for the tensor.
optional string name = 8; // namespace Value
// A human-readable documentation for this tensor. Markdown is allowed.
optional string doc_string = 12;
// Serializations can either use one of the fields above, or use this
// raw bytes field. The only exception is the string case, where one is
// required to store the content in the repeated bytes string_data field.
//
// When this raw_data field is used to store tensor value, elements MUST
// be stored in as fixed-width, little-endian order.
// Floating-point data types MUST be stored in IEEE 754 format.
// Complex64 elements must be written as two consecutive FLOAT values, real component first.
// Complex128 elements must be written as two consecutive DOUBLE values, real component first.
// Boolean type MUST be written one byte per tensor element (00000001 for true, 00000000 for false).
//
// Note: the advantage of specific field rather than the raw_data field is
// that in some cases (e.g. int data), protobuf does a better packing via
// variable length storage, and may lead to smaller binary footprint.
// When this field is present, the data_type field MUST NOT be STRING or UNDEFINED
optional bytes raw_data = 9;
// Data can be stored inside the protobuf file using type-specific fields or raw_data.
// Alternatively, raw bytes data can be stored in an external file, using the external_data field.
// external_data stores key-value pairs describing data location. Recognized keys are:
// - "location" (required) - POSIX filesystem path relative to the directory where the ONNX
// protobuf model was stored
// - "offset" (optional) - position of byte at which stored data begins. Integer stored as string.
// Offset values SHOULD be multiples 4096 (page size) to enable mmap support.
// - "length" (optional) - number of bytes containing data. Integer stored as string.
// - "checksum" (optional) - SHA1 digest of file specified in under 'location' key.
repeated StringStringEntryProto external_data = 13;
// Location of the data for this tensor. MUST be one of:
// - DEFAULT - data stored inside the protobuf message. Data is stored in raw_data (if set) otherwise in type-specified field.
// - EXTERNAL - data stored in an external location as described by external_data field.
enum DataLocation {
DEFAULT = 0;
EXTERNAL = 1;
}
// If value not set, data is stored in raw_data (if set) otherwise in type-specified field.
optional DataLocation data_location = 14;
// For double
// Complex128 tensors are encoded as a single array of doubles,
// with the real components appearing in odd numbered positions,
// and the corresponding imaginary component appearing in the
// subsequent even numbered position. (e.g., [1.0 + 2.0i, 3.0 + 4.0i]
// is encoded as [1.0, 2.0 ,3.0 ,4.0]
// When this field is present, the data_type field MUST be DOUBLE or COMPLEX128
repeated double double_data = 10 [packed = true];
// For uint64 and uint32 values
// When this field is present, the data_type field MUST be
// UINT32 or UINT64
repeated uint64 uint64_data = 11 [packed = true];
}
// A serialized sparse-tensor value
message SparseTensorProto {
// The sequence of non-default values are encoded as a tensor of shape [NNZ].
// The default-value is zero for numeric tensors, and empty-string for string tensors.
optional TensorProto values = 1;
// The indices of the non-default values, which may be stored in one of two formats.
// (a) Indices can be a tensor of shape [NNZ, rank] with the [i,j]-th value
// corresponding to the j-th index of the i-th value (in the values tensor).
// (b) Indices can be a tensor of shape [NNZ], in which case the i-th value
// must be the linearized-index of the i-th value (in the values tensor).
// The linearized-index can be converted into an index tuple (k_1,...,k_rank)
// using the shape provided below.
// The indices must appear in ascending order without duplication.
// In the first format, the ordering is lexicographic-ordering:
// e.g., index-value [1,4] must appear before [2,1]
optional TensorProto indices = 2;
// The shape of the underlying dense-tensor: [dim_1, dim_2, ... dim_rank]
repeated int64 dims = 3;
}
// Defines a tensor shape. A dimension can be either an integer value
// or a symbolic variable. A symbolic variable represents an unknown
// dimension.
message TensorShapeProto {
message Dimension {
oneof value {
int64 dim_value = 1;
string dim_param = 2; // namespace Shape
};
// Standard denotation can optionally be used to denote tensor
// dimensions with standard semantic descriptions to ensure
// that operations are applied to the correct axis of a tensor.
// Refer to https://github.com/onnx/onnx/blob/master/docs/DimensionDenotation.md#denotation-definition
// for pre-defined dimension denotations.
optional string denotation = 3;
};
repeated Dimension dim = 1;
}
// Types
//
// The standard ONNX data types.
message TypeProto {
message Tensor {
// This field MUST NOT have the value of UNDEFINED
// This field MUST have a valid TensorProto.DataType value
// This field MUST be present for this version of the IR.
optional int32 elem_type = 1;
optional TensorShapeProto shape = 2;
}
// repeated T
message Sequence {
// The type and optional shape of each element of the sequence.
// This field MUST be present for this version of the IR.
optional TypeProto elem_type = 1;
};
// map
message Map {
// This field MUST have a valid TensorProto.DataType value
// This field MUST be present for this version of the IR.
// This field MUST refer to an integral type ([U]INT{8|16|32|64}) or STRING
optional int32 key_type = 1;
// This field MUST be present for this version of the IR.
optional TypeProto value_type = 2;
};
// #if ONNX-ML
message SparseTensor {
// This field MUST NOT have the value of UNDEFINED
// This field MUST have a valid TensorProto.DataType value
// This field MUST be present for this version of the IR.
optional int32 elem_type = 1;
optional TensorShapeProto shape = 2;
}
message Opaque {
// When missing, the domain is the same as the model's.
optional string domain = 1;
// The name is optional but significant when provided.
optional string name = 2;
// parameters that help defining the type
// DEPRECATED do not use.
// repeated TypeProto parameters = 3;
}
// #endif
oneof value {
// The type of a tensor.
Tensor tensor_type = 1;
// NOTE: DNN-only implementations of ONNX MAY elect to not support non-tensor values
// as input and output to graphs and nodes. These types are needed to naturally
// support classical ML operators. DNN operators SHOULD restrict their input
// and output types to tensors.
// The type of a sequence.
Sequence sequence_type = 4;
// The type of a map.
Map map_type = 5;
// #if ONNX-ML
SparseTensor sparse_tensor_type = 8;
Opaque opaque_type = 7;
// #endif
}
// An optional denotation can be used to denote the whole
// type with a standard semantic description as to what is
// stored inside. Refer to https://github.com/onnx/onnx/blob/master/docs/TypeDenotation.md#type-denotation-definition
// for pre-defined type denotations.
optional string denotation = 6;
}
// Operator Sets
//
// OperatorSets are uniquely identified by a (domain, opset_version) pair.
message OperatorSetIdProto {
// The domain of the operator set being identified.
// The empty string ("") or absence of this field implies the operator
// set that is defined as part of the ONNX specification.
// This field MUST be present in this version of the IR when referring to any other operator set.
optional string domain = 1;
// The version of the operator set being identified.
// This field MUST be present in this version of the IR.
optional int64 version = 2;
}
onnx-1.7.0/onnx/onnx-operators-ml.proto 0000664 0000000 0000000 00000016264 13655345213 016674 0 ustar root root //
// WARNING: This file is automatically generated! Please edit onnx.in.proto.
//
// Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
syntax = "proto2";
package onnx;
import "onnx/onnx-ml.proto";
//
// This file contains the proto definitions for OperatorSetProto and
// OperatorProto. OperatorSetProtos are used to describe a versioned
// set of operators that can be used by a ModelProto.
//
// Like ModelProto, OperatorSetProto is defined as a top-level file/wire
// format, however their usage is different.
//
// ModelProto files are used to describe executable graphs that can be
// executed directly by a framework, runtime, or engine.
//
// OperatorSetProto files are used to describe a set of operators that are
// available in a given environment. The file TBD.TBD is the OperatorSetProto
// that describes the ONNX standard operators.
//
// Operator/function status.
enum OperatorStatus {
EXPERIMENTAL = 0;
STABLE = 1;
}
message FunctionProto {
// The name of the function, similar usage of op_type in OperatorProto.
optional string name = 1;
// The first version of a function set which contains this function.
// When there's any breaking change for this function, the function set
// contains the function needs to bump its version, and since_version of
// the updated function will be changed to the updated function set version.
optional int64 since_version = 2;
// This field indicates whether the syntax, semantics, or presence
// of this function is in an experimental or stable stage. Once an
// function is published as STABLE, its syntax and semantics MUST NOT
// change in subsequent versions of the operator set.
// When a function is published as EXPERIMENTAL, the syntax and semantics
// of the function MAY change across operator set versions.
// Functions "become" stable by deprecating the experimental version and
// introducing a new stable function with the same name.
optional OperatorStatus status = 3;
// The inputs and outputs of the function.
repeated string input = 4;
repeated string output = 5;
// The attributes of the function.
repeated string attribute= 6;
// The nodes in the function.
repeated NodeProto node = 7;
// A human-readable documentation for this function. Markdown is allowed.
optional string doc_string = 8;
// The OperatorSets this function body (graph) relies on.
// A FunctionProto body (graph) may implicitly rely on the OperatorSet that
// this function belongs to. It can also explicitly rely on more OperatorSets
// with this field specified.
//
// All nodes in the function body (graph) will bind against the operator
// with the same-domain/same-op_type operator with the HIGHEST version
// in the referenced operator sets. This means at most one version can be relied
// for one domain.
repeated OperatorSetIdProto opset_import = 9;
}
// An OperatorProto represents the immutable specification of the signature
// and semantics of an operator.
//
// Operators are declared as part of an OperatorSet, which also defines the
// domain name for the set.
//
// Operators are uniquely identified by a three part identifier
// (domain, op_type, since_version)
// where
// *domain* is the domain of an operator set that
// contains this operator specification.
//
// *op_type* is the name of the operator as referenced by a
// NodeProto.op_type
//
// *since_version* is the version of the operator set that
// this operator was initially declared in.
//
message OperatorProto {
// The name of the operator within a domain.
// This field MUST be present in this version of the IR.
optional string op_type = 1;
// The version of the operator set that first introduced this
// operator. This value MUST be the same value as the
// opset_version of the operator set that first published this operator.
// Subsequent versions of the operator set MUST NOT alter the signature
// or semantics of the operator once published as STABLE.
// This field MUST be present in this version of the IR.
optional int64 since_version = 2;
// This field indicates whether the syntax, semantics, or presence
// of this operator is in an experimental or stable stage. Once an
// operator is published as STABLE, it's syntax and semantics MUST NOT
// change in subsequent versions of the operator set.
// When an operator is published as EXPERIMENTAL, the syntax and semantics
// of the operator MAY change across operator set versions.
// Operators "become" stable by deprecating the experimental version and
// introducing a new stable operator with the same op_type.
optional OperatorStatus status = 3;
// Eventually we will declare the signature of the operator here
// A human-readable documentation for this operator. Markdown is allowed.
optional string doc_string = 10;
}
// An OperatorSetProto represents an immutable set of immutable operator
// specifications.
//
// The domain of the set (OperatorSetProto.domain) is a reverse-DNS name
// that disambiguates operator sets defined by independent entities.
//
// The version of the set (opset_version) is a monotonically increasing
// integer that indicates changes to the membership of the operator set.
//
//
// Operator sets are uniquely identified by a two part identifier (domain, opset_version)
//
// Like ModelProto, OperatorSetProto is intended as a top-level file/wire format,
// and thus has the standard format headers in addition to the operator set information.
//
message OperatorSetProto {
// All OperatorSetProtos start with a distingushed byte sequence to disambiguate
// protobuf files containing OperatorSets from other content.
// This field MUST be "ONNXOPSET"
// This field MUST be present in this version of the IR
optional string magic = 1;
// All OperatorSetProtos indicate the version of the IR syntax and semantics
// they adhere to. It is always IR_VERSION.
// This field MUST be present in this version of the IR
optional int64 ir_version = 2;
// The prerelease component of the SemVer of the IR.
// This field MAY be absent in this version of the IR
optional string ir_version_prerelease = 3;
// The build metadata component of the SemVer of the IR.
// This field MAY be absent in this version of the IR
optional string ir_build_metadata = 7;
// Domain name of the operator set, in reverse DNS form (e.g., com.acme.dnnops).
optional string domain = 4;
// The version of the set of operators. This is a simple int value
// that is monotonically increasing as new versions of operator set
// are published. All operators in this set MUST have version
// numbers no greater than opset_version.
optional int64 opset_version = 5;
// A human-readable documentation for this set of operators. Markdown is allowed.
optional string doc_string = 6;
// The operators specified by this operator set.
// The (name, version) MUST be unique across all OperatorProtos in operator
repeated OperatorProto operator = 8;
// The functions specified by this operator set.
// The (name, version) MUST be unique across all OperatorProtos/FunctionProtos in operator/functions
repeated FunctionProto functions = 9;
}
// For using protobuf-lite
option optimize_for = LITE_RUNTIME;
onnx-1.7.0/onnx/onnx.proto 0000664 0000000 0000000 00000071471 13655345213 014253 0 ustar root root //
// WARNING: This file is automatically generated! Please edit onnx.in.proto.
//
// Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
syntax = "proto2";
package onnx;
// Overview
//
// ONNX is an open specification that is comprised of the following components:
//
// 1) A definition of an extensible computation graph model.
// 2) Definitions of standard data types.
// 3) Definitions of built-in operators.
//
// This document describes the syntax of models and their computation graphs,
// as well as the standard data types. Together, they are referred to as the ONNX
// Intermediate Representation, or 'IR' for short.
//
// The normative semantic specification of the ONNX IR is found in docs/IR.md.
// Definitions of the built-in neural network operators may be found in docs/Operators.md.
// Notes
//
// Release
//
// We are still in the very early stage of defining ONNX. The current
// version of ONNX is a starting point. While we are actively working
// towards a complete spec, we would like to get the community involved
// by sharing our working version of ONNX.
//
// Protobuf compatibility
//
// To simplify framework compatibility, ONNX is defined using the subset of protobuf
// that is compatible with both protobuf v2 and v3. This means that we do not use any
// protobuf features that are only available in one of the two versions.
//
// Here are the most notable contortions we have to carry out to work around
// these limitations:
//
// - No 'map' (added protobuf 3.0). We instead represent mappings as lists
// of key-value pairs, where order does not matter and duplicates
// are not allowed.
// Versioning
//
// ONNX versioning is specified in docs/IR.md and elaborated on in docs/Versioning.md
//
// To be compatible with both proto2 and proto3, we will use a version number
// that is not defined by the default value but an explicit enum number.
enum Version {
// proto3 requires the first enum value to be zero.
// We add this just to appease the compiler.
_START_VERSION = 0;
// The version field is always serialized and we will use it to store the
// version that the graph is generated from. This helps us set up version
// control.
// For the IR, we are using simple numbers starting with 0x00000001,
// which was the version we published on Oct 10, 2017.
IR_VERSION_2017_10_10 = 0x0000000000000001;
// IR_VERSION 2 published on Oct 30, 2017
// - Added type discriminator to AttributeProto to support proto3 users
IR_VERSION_2017_10_30 = 0x0000000000000002;
// IR VERSION 3 published on Nov 3, 2017
// - For operator versioning:
// - Added new message OperatorSetIdProto
// - Added opset_import in ModelProto
// - For vendor extensions, added domain in NodeProto
IR_VERSION_2017_11_3 = 0x0000000000000003;
// IR VERSION 4 published on Jan 22, 2019
// - Relax constraint that initializers should be a subset of graph inputs
// - Add type BFLOAT16
IR_VERSION_2019_1_22 = 0x0000000000000004;
// IR VERSION 5 published on March 18, 2019
// - Add message TensorAnnotation.
// - Add quantization annotation in GraphProto to map tensor with its scale and zero point quantization parameters.
IR_VERSION_2019_3_18 = 0x0000000000000005;
// IR VERSION 6 published on Sep 19, 2019
// - Add support for sparse tensor constants stored in model.
// - Add message SparseTensorProto
// - Add sparse initializers
IR_VERSION_2019_9_19 = 0x0000000000000006;
// IR VERSION 7 published on
// - Add support to allow function body graph to rely on multiple external opreator sets.
// - Add a list to promote inference graph's initializers to global and
// mutable variables. Global variables are visible in all graphs of the
// stored models.
// - Add message TrainingInfoProto to store initialization
// method and training algorithm. The execution of TrainingInfoProto
// can modify the values of mutable variables.
// - Make inference graph callable from TrainingInfoProto via GraphCall operator.
IR_VERSION = 0x0000000000000007;
}
// Attributes
//
// A named attribute containing either singular float, integer, string, graph,
// and tensor values, or repeated float, integer, string, graph, and tensor values.
// An AttributeProto MUST contain the name field, and *only one* of the
// following content fields, effectively enforcing a C/C++ union equivalent.
message AttributeProto {
// Note: this enum is structurally identical to the OpSchema::AttrType
// enum defined in schema.h. If you rev one, you likely need to rev the other.
enum AttributeType {
UNDEFINED = 0;
FLOAT = 1;
INT = 2;
STRING = 3;
TENSOR = 4;
GRAPH = 5;
SPARSE_TENSOR = 11;
FLOATS = 6;
INTS = 7;
STRINGS = 8;
TENSORS = 9;
GRAPHS = 10;
SPARSE_TENSORS = 12;
}
// The name field MUST be present for this version of the IR.
optional string name = 1; // namespace Attribute
// if ref_attr_name is not empty, ref_attr_name is the attribute name in parent function.
// In this case, this AttributeProto does not contain data, and it's a reference of attribute
// in parent scope.
// NOTE: This should ONLY be used in function (sub-graph). It's invalid to be used in main graph.
optional string ref_attr_name = 21;
// A human-readable documentation for this attribute. Markdown is allowed.
optional string doc_string = 13;
// The type field MUST be present for this version of the IR.
// For 0.0.1 versions of the IR, this field was not defined, and
// implementations needed to use has_field heuristics to determine
// which value field was in use. For IR_VERSION 0.0.2 or later, this
// field MUST be set and match the f|i|s|t|... field in use. This
// change was made to accommodate proto3 implementations.
optional AttributeType type = 20; // discriminator that indicates which field below is in use
// Exactly ONE of the following fields must be present for this version of the IR
optional float f = 2; // float
optional int64 i = 3; // int
optional bytes s = 4; // UTF-8 string
optional TensorProto t = 5; // tensor value
optional GraphProto g = 6; // graph
optional SparseTensorProto sparse_tensor = 22; // sparse tensor value
// Do not use field below, it's deprecated.
// optional ValueProto v = 12; // value - subsumes everything but graph
repeated float floats = 7; // list of floats
repeated int64 ints = 8; // list of ints
repeated bytes strings = 9; // list of UTF-8 strings
repeated TensorProto tensors = 10; // list of tensors
repeated GraphProto graphs = 11; // list of graph
repeated SparseTensorProto sparse_tensors = 23; // list of sparse tensors
}
// Defines information on value, including the name, the type, and
// the shape of the value.
message ValueInfoProto {
// This field MUST be present in this version of the IR.
optional string name = 1; // namespace Value
// This field MUST be present in this version of the IR for
// inputs and outputs of the top-level graph.
optional TypeProto type = 2;
// A human-readable documentation for this value. Markdown is allowed.
optional string doc_string = 3;
}
// Nodes
//
// Computation graphs are made up of a DAG of nodes, which represent what is
// commonly called a "layer" or "pipeline stage" in machine learning frameworks.
//
// For example, it can be a node of type "Conv" that takes in an image, a filter
// tensor and a bias tensor, and produces the convolved output.
message NodeProto {
repeated string input = 1; // namespace Value
repeated string output = 2; // namespace Value
// An optional identifier for this node in a graph.
// This field MAY be absent in ths version of the IR.
optional string name = 3; // namespace Node
// The symbolic identifier of the Operator to execute.
optional string op_type = 4; // namespace Operator
// The domain of the OperatorSet that specifies the operator named by op_type.
optional string domain = 7; // namespace Domain
// Additional named attributes.
repeated AttributeProto attribute = 5;
// A human-readable documentation for this node. Markdown is allowed.
optional string doc_string = 6;
}
// Training information
// TrainingInfoProto stores information for training a model.
// In particular, this defines two functionalities: an initialization-step
// and a training-algorithm-step. Initialization resets the model
// back to its original state as if no training has been consumed.
// Training algorithm improves the model based on input data.
//
// The semantics of the initialization-step is that the initializers
// in ModelProto.graph and in TrainingInfoProto.algorithm are first
// initialized as specified by the initializers in the graph, and then
// updated by the "initialization_binding" in every instance in
// ModelProto.training_info.
//
// The field "algorithm" defines a computation graph which represents a
// training algorithm's step. After the execution of a
// TrainingInfoProto.algorithm, the initializers specified by "update_binding"
// may be immediately updated. If the targeted training algorithm contains
// consecutive update stages (such as block coordinate descent methods),
// the user needs to create a TrainingInfoProto for each stage.
message TrainingInfoProto {
// This field describes a graph to compute the initial tensors
// upon starting the training process. Initialization graph has no input
// and can have multiple outputs. Usually, trainable tensors in neural
// networks are randomly initialized. To achieve that, for each tensor,
// the user can put a random number operator such as RandomNormal or
// RandomUniform in TrainingInfoProto.initialization.node and assign its
// random output to the specific tensor using "initialization_binding".
// This graph can also set the initializers in "algorithm" in the same
// TrainingInfoProto; a use case is resetting the number of training
// iteration to zero.
//
// By default, this field is an empty graph and its evaluation does not
// produce any output.
optional GraphProto initialization = 1;
// This field represents a training algorithm step. Given required inputs,
// it computes outputs to update initializers in its own or inference graph's
// initializer lists. In general, this graph contains loss node, gradient node,
// optimizer node, increment of iteration count, and some calls to the inference
// graph.
//
// The field algorithm.node is the only place the user can use GraphCall
// operator. The only callable graph is the one stored in ModelProto.graph.
//
// By default, this field is an empty graph and its evaluation does not
// produce any output.
optional GraphProto algorithm = 2;
// This field specifies the bindings from the outputs of "initialization" to
// some initializers in "ModelProto.graph.initializer" and
// the "algorithm.initializer" in the same TrainingInfoProto.
// See "update_binding" below for details.
//
// By default, this field is empty and no initializer would be changed
// by the execution of "initialization".
repeated StringStringEntryProto initialization_binding = 3;
// Gradient-based training is usually an iterative procedure. In one gradient
// descent iteration, we apply
//
// x = x - r * g
//
// where "x" is the optimized tensor, "r" stands for learning rate, and "g" is
// gradient of "x" with respect to a chosen loss. To avoid adding assignments
// into the training graph, we split the update equation into
//
// y = x - r * g
// x = y
//
// The user needs to save "y = x - r * g" into TrainingInfoProto.algorithm. To
// tell that "y" should be assigned to "x", the field "update_binding" may
// contain a key-value pair of strings, "x" (key of StringStringEntryProto)
// and "y" (value of StringStringEntryProto).
// For a neural network with multiple trainable (mutable) tensors, there can
// be multiple key-value pairs in "update_binding".
//
// The initializers appears as keys in "update_binding" are considered
// mutable and globally-visible variables. This implies some behaviors
// as described below.
//
// 1. We have only unique keys in all "update_binding"s so that two global
// variables may not have the same name. This ensures that one
// global variable is assigned up to once.
// 2. The keys must appear in names of "ModelProto.graph.initializer" or
// "TrainingInfoProto.algorithm.initializer".
// 3. The values must be output names of "algorithm".
// 4. If an optional input of a graph is omitted when using GraphCall, the
// global variable with the same name may be used.
// 5. When using GraphCall, the users always can pass values to optional
// inputs of the called graph even if the associated initializers appears
// as keys in "update_binding"s.
// 6. The graphs in TrainingInfoProto's can use global variables as
// their operator inputs.
// 7. Mutable variables are initialized to the value specified by the
// corresponding initializer, and then potentially updated by
// "initializer_binding"s and "update_binding"s in "TrainingInfoProto"s.
//
// This field usually contains names of trainable tensors
// (in ModelProto.graph), optimizer states such as momentums in advanced
// stochastic gradient methods (in TrainingInfoProto.graph),
// and number of training iterations (in TrainingInfoProto.graph).
//
// By default, this field is empty and no initializer would be changed
// by the execution of "algorithm".
repeated StringStringEntryProto update_binding = 4;
}
// Models
//
// ModelProto is a top-level file/container format for bundling a ML model and
// associating its computation graph with metadata.
//
// The semantics of the model are described by the associated GraphProto's.
message ModelProto {
// The version of the IR this model targets. See Version enum above.
// This field MUST be present.
optional int64 ir_version = 1;
// The OperatorSets this model relies on.
// All ModelProtos MUST have at least one entry that
// specifies which version of the ONNX OperatorSet is
// being imported.
//
// All nodes in the ModelProto's graph will bind against the operator
// with the same-domain/same-op_type operator with the HIGHEST version
// in the referenced operator sets.
repeated OperatorSetIdProto opset_import = 8;
// The name of the framework or tool used to generate this model.
// This field SHOULD be present to indicate which implementation/tool/framework
// emitted the model.
optional string producer_name = 2;
// The version of the framework or tool used to generate this model.
// This field SHOULD be present to indicate which implementation/tool/framework
// emitted the model.
optional string producer_version = 3;
// Domain name of the model.
// We use reverse domain names as name space indicators. For example:
// `com.facebook.fair` or `com.microsoft.cognitiveservices`
//
// Together with `model_version` and GraphProto.name, this forms the unique identity of
// the graph.
optional string domain = 4;
// The version of the graph encoded. See Version enum below.
optional int64 model_version = 5;
// A human-readable documentation for this model. Markdown is allowed.
optional string doc_string = 6;
// The parameterized graph that is evaluated to execute the model.
optional GraphProto graph = 7;
// Named metadata values; keys should be distinct.
repeated StringStringEntryProto metadata_props = 14;
// Training-specific information. Sequentially executing all stored
// `TrainingInfoProto.algorithm`s and assigning their outputs following
// the corresponding `TrainingInfoProto.update_binding`s is one training
// iteration. Similarly, to initialize the model
// (as if training hasn't happened), the user should sequentially execute
// all stored `TrainingInfoProto.initialization`s and assigns their outputs
// using `TrainingInfoProto.initialization_binding`s.
//
// If this field is empty, the training behavior of the model is undefined.
repeated TrainingInfoProto training_info = 20;
};
// StringStringEntryProto follows the pattern for cross-proto-version maps.
// See https://developers.google.com/protocol-buffers/docs/proto3#maps
message StringStringEntryProto {
optional string key = 1;
optional string value= 2;
};
message TensorAnnotation {
optional string tensor_name = 1;
// pairs to annotate tensor specified by above.
// The keys used in the mapping below must be pre-defined in ONNX spec.
// For example, for 8-bit linear quantization case, 'SCALE_TENSOR', 'ZERO_POINT_TENSOR' will be pre-defined as
// quantization parameter keys.
repeated StringStringEntryProto quant_parameter_tensor_names = 2;
}
// Graphs
//
// A graph defines the computational logic of a model and is comprised of a parameterized
// list of nodes that form a directed acyclic graph based on their inputs and outputs.
// This is the equivalent of the "network" or "graph" in many deep learning
// frameworks.
message GraphProto {
// The nodes in the graph, sorted topologically.
repeated NodeProto node = 1;
// The name of the graph.
optional string name = 2; // namespace Graph
// A list of named tensor values, used to specify constant inputs of the graph.
// Each TensorProto entry must have a distinct name (within the list) that
// MAY also appear in the input list.
repeated TensorProto initializer = 5;
// Initializers (see above) stored in sparse format.
repeated SparseTensorProto sparse_initializer = 15;
// A human-readable documentation for this graph. Markdown is allowed.
optional string doc_string = 10;
// The inputs and outputs of the graph.
repeated ValueInfoProto input = 11;
repeated ValueInfoProto output = 12;
// Information for the values in the graph. The ValueInfoProto.name's
// must be distinct. It is optional for a value to appear in value_info list.
repeated ValueInfoProto value_info = 13;
// This field carries information to indicate the mapping among a tensor and its
// quantization parameter tensors. For example:
// For tensor 'a', it may have {'SCALE_TENSOR', 'a_scale'} and {'ZERO_POINT_TENSOR', 'a_zero_point'} annotated,
// which means, tensor 'a_scale' and tensor 'a_zero_point' are scale and zero point of tensor 'a' in the model.
repeated TensorAnnotation quantization_annotation = 14;
// DO NOT USE the following fields, they were deprecated from earlier versions.
// repeated string input = 3;
// repeated string output = 4;
// optional int64 ir_version = 6;
// optional int64 producer_version = 7;
// optional string producer_tag = 8;
// optional string domain = 9;
}
// Tensors
//
// A serialized tensor value.
message TensorProto {
enum DataType {
UNDEFINED = 0;
// Basic types.
FLOAT = 1; // float
UINT8 = 2; // uint8_t
INT8 = 3; // int8_t
UINT16 = 4; // uint16_t
INT16 = 5; // int16_t
INT32 = 6; // int32_t
INT64 = 7; // int64_t
STRING = 8; // string
BOOL = 9; // bool
// IEEE754 half-precision floating-point format (16 bits wide).
// This format has 1 sign bit, 5 exponent bits, and 10 mantissa bits.
FLOAT16 = 10;
DOUBLE = 11;
UINT32 = 12;
UINT64 = 13;
COMPLEX64 = 14; // complex with float32 real and imaginary components
COMPLEX128 = 15; // complex with float64 real and imaginary components
// Non-IEEE floating-point format based on IEEE754 single-precision
// floating-point number truncated to 16 bits.
// This format has 1 sign bit, 8 exponent bits, and 7 mantissa bits.
BFLOAT16 = 16;
// Future extensions go here.
}
// The shape of the tensor.
repeated int64 dims = 1;
// The data type of the tensor.
// This field MUST have a valid TensorProto.DataType value
optional int32 data_type = 2;
// For very large tensors, we may want to store them in chunks, in which
// case the following fields will specify the segment that is stored in
// the current TensorProto.
message Segment {
optional int64 begin = 1;
optional int64 end = 2;
}
optional Segment segment = 3;
// Tensor content must be organized in row-major order.
//
// Depending on the data_type field, exactly one of the fields below with
// name ending in _data is used to store the elements of the tensor.
// For float and complex64 values
// Complex64 tensors are encoded as a single array of floats,
// with the real components appearing in odd numbered positions,
// and the corresponding imaginary component appearing in the
// subsequent even numbered position. (e.g., [1.0 + 2.0i, 3.0 + 4.0i]
// is encoded as [1.0, 2.0 ,3.0 ,4.0]
// When this field is present, the data_type field MUST be FLOAT or COMPLEX64.
repeated float float_data = 4 [packed = true];
// For int32, uint8, int8, uint16, int16, bool, and float16 values
// float16 values must be bit-wise converted to an uint16_t prior
// to writing to the buffer.
// When this field is present, the data_type field MUST be
// INT32, INT16, INT8, UINT16, UINT8, BOOL, or FLOAT16
repeated int32 int32_data = 5 [packed = true];
// For strings.
// Each element of string_data is a UTF-8 encoded Unicode
// string. No trailing null, no leading BOM. The protobuf "string"
// scalar type is not used to match ML community conventions.
// When this field is present, the data_type field MUST be STRING
repeated bytes string_data = 6;
// For int64.
// When this field is present, the data_type field MUST be INT64
repeated int64 int64_data = 7 [packed = true];
// Optionally, a name for the tensor.
optional string name = 8; // namespace Value
// A human-readable documentation for this tensor. Markdown is allowed.
optional string doc_string = 12;
// Serializations can either use one of the fields above, or use this
// raw bytes field. The only exception is the string case, where one is
// required to store the content in the repeated bytes string_data field.
//
// When this raw_data field is used to store tensor value, elements MUST
// be stored in as fixed-width, little-endian order.
// Floating-point data types MUST be stored in IEEE 754 format.
// Complex64 elements must be written as two consecutive FLOAT values, real component first.
// Complex128 elements must be written as two consecutive DOUBLE values, real component first.
// Boolean type MUST be written one byte per tensor element (00000001 for true, 00000000 for false).
//
// Note: the advantage of specific field rather than the raw_data field is
// that in some cases (e.g. int data), protobuf does a better packing via
// variable length storage, and may lead to smaller binary footprint.
// When this field is present, the data_type field MUST NOT be STRING or UNDEFINED
optional bytes raw_data = 9;
// Data can be stored inside the protobuf file using type-specific fields or raw_data.
// Alternatively, raw bytes data can be stored in an external file, using the external_data field.
// external_data stores key-value pairs describing data location. Recognized keys are:
// - "location" (required) - POSIX filesystem path relative to the directory where the ONNX
// protobuf model was stored
// - "offset" (optional) - position of byte at which stored data begins. Integer stored as string.
// Offset values SHOULD be multiples 4096 (page size) to enable mmap support.
// - "length" (optional) - number of bytes containing data. Integer stored as string.
// - "checksum" (optional) - SHA1 digest of file specified in under 'location' key.
repeated StringStringEntryProto external_data = 13;
// Location of the data for this tensor. MUST be one of:
// - DEFAULT - data stored inside the protobuf message. Data is stored in raw_data (if set) otherwise in type-specified field.
// - EXTERNAL - data stored in an external location as described by external_data field.
enum DataLocation {
DEFAULT = 0;
EXTERNAL = 1;
}
// If value not set, data is stored in raw_data (if set) otherwise in type-specified field.
optional DataLocation data_location = 14;
// For double
// Complex128 tensors are encoded as a single array of doubles,
// with the real components appearing in odd numbered positions,
// and the corresponding imaginary component appearing in the
// subsequent even numbered position. (e.g., [1.0 + 2.0i, 3.0 + 4.0i]
// is encoded as [1.0, 2.0 ,3.0 ,4.0]
// When this field is present, the data_type field MUST be DOUBLE or COMPLEX128
repeated double double_data = 10 [packed = true];
// For uint64 and uint32 values
// When this field is present, the data_type field MUST be
// UINT32 or UINT64
repeated uint64 uint64_data = 11 [packed = true];
}
// A serialized sparse-tensor value
message SparseTensorProto {
// The sequence of non-default values are encoded as a tensor of shape [NNZ].
// The default-value is zero for numeric tensors, and empty-string for string tensors.
optional TensorProto values = 1;
// The indices of the non-default values, which may be stored in one of two formats.
// (a) Indices can be a tensor of shape [NNZ, rank] with the [i,j]-th value
// corresponding to the j-th index of the i-th value (in the values tensor).
// (b) Indices can be a tensor of shape [NNZ], in which case the i-th value
// must be the linearized-index of the i-th value (in the values tensor).
// The linearized-index can be converted into an index tuple (k_1,...,k_rank)
// using the shape provided below.
// The indices must appear in ascending order without duplication.
// In the first format, the ordering is lexicographic-ordering:
// e.g., index-value [1,4] must appear before [2,1]
optional TensorProto indices = 2;
// The shape of the underlying dense-tensor: [dim_1, dim_2, ... dim_rank]
repeated int64 dims = 3;
}
// Defines a tensor shape. A dimension can be either an integer value
// or a symbolic variable. A symbolic variable represents an unknown
// dimension.
message TensorShapeProto {
message Dimension {
oneof value {
int64 dim_value = 1;
string dim_param = 2; // namespace Shape
};
// Standard denotation can optionally be used to denote tensor
// dimensions with standard semantic descriptions to ensure
// that operations are applied to the correct axis of a tensor.
// Refer to https://github.com/onnx/onnx/blob/master/docs/DimensionDenotation.md#denotation-definition
// for pre-defined dimension denotations.
optional string denotation = 3;
};
repeated Dimension dim = 1;
}
// Types
//
// The standard ONNX data types.
message TypeProto {
message Tensor {
// This field MUST NOT have the value of UNDEFINED
// This field MUST have a valid TensorProto.DataType value
// This field MUST be present for this version of the IR.
optional int32 elem_type = 1;
optional TensorShapeProto shape = 2;
}
// repeated T
message Sequence {
// The type and optional shape of each element of the sequence.
// This field MUST be present for this version of the IR.
optional TypeProto elem_type = 1;
};
// map
message Map {
// This field MUST have a valid TensorProto.DataType value
// This field MUST be present for this version of the IR.
// This field MUST refer to an integral type ([U]INT{8|16|32|64}) or STRING
optional int32 key_type = 1;
// This field MUST be present for this version of the IR.
optional TypeProto value_type = 2;
};
oneof value {
// The type of a tensor.
Tensor tensor_type = 1;
// NOTE: DNN-only implementations of ONNX MAY elect to not support non-tensor values
// as input and output to graphs and nodes. These types are needed to naturally
// support classical ML operators. DNN operators SHOULD restrict their input
// and output types to tensors.
// The type of a sequence.
Sequence sequence_type = 4;
// The type of a map.
Map map_type = 5;
}
// An optional denotation can be used to denote the whole
// type with a standard semantic description as to what is
// stored inside. Refer to https://github.com/onnx/onnx/blob/master/docs/TypeDenotation.md#type-denotation-definition
// for pre-defined type denotations.
optional string denotation = 6;
}
// Operator Sets
//
// OperatorSets are uniquely identified by a (domain, opset_version) pair.
message OperatorSetIdProto {
// The domain of the operator set being identified.
// The empty string ("") or absence of this field implies the operator
// set that is defined as part of the ONNX specification.
// This field MUST be present in this version of the IR when referring to any other operator set.
optional string domain = 1;
// The version of the operator set being identified.
// This field MUST be present in this version of the IR.
optional int64 version = 2;
}
// For using protobuf-lite
option optimize_for = LITE_RUNTIME;
onnx-1.7.0/onnx/optimizer.py 0000664 0000000 0000000 00000003315 13655345213 014570 0 ustar root root # ATTENTION: The code in this file is highly EXPERIMENTAL.
# Adventurous users should note that the APIs will probably change.
"""onnx optimizer
This enables users to optimize their models.
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import onnx
import onnx.onnx_cpp2py_export.optimizer as C
from onnx import ModelProto
from typing import Text, Sequence, Optional
"""Apply the optimization on the serialized ModelProto.
Arguments:
input (ModelProto): model
names (list of string): list of optimization names
Return:
return (ModelProto) optimized model
Supported pass names:
-- nop
-- eliminate_identity
-- eliminate_nop_transpose
-- eliminate_nop_pad
-- eliminate_unused_initializer
-- fuse_consecutive_squeezes
-- fuse_consecutive_transposes
-- fuse_add_bias_into_conv
-- fuse_transpose_into_gemm
"""
get_available_passes = C.get_available_passes
def optimize(model, passes=None, fixed_point=False): # type: (ModelProto, Optional[Sequence[Text]], bool) -> ModelProto
if passes is None:
passes = ['eliminate_nop_transpose',
'eliminate_nop_pad',
'fuse_consecutive_transposes',
'fuse_transpose_into_gemm']
if not isinstance(model, ModelProto):
raise ValueError('Optimizer only accepts ModelProto, incorrect type: {}'.format(type(model)))
model_str = model.SerializeToString()
if fixed_point:
optimized_model_str = C.optimize_fixedpoint(model_str, passes)
else:
optimized_model_str = C.optimize(model_str, passes)
return onnx.load_from_string(optimized_model_str)
onnx-1.7.0/onnx/__init__.py 0000664 0000000 0000000 00000014101 13655345213 014300 0 ustar root root from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import os
from .onnx_cpp2py_export import ONNX_ML
from onnx.external_data_helper import load_external_data_for_model, write_external_data_tensors
from .onnx_pb import * # noqa
from .onnx_operators_pb import * # noqa
from .version import version as __version__ # noqa
# Import common subpackages so they're available when you 'import onnx'
import onnx.helper # noqa
import onnx.checker # noqa
import onnx.defs # noqa
import google.protobuf.message
from typing import Union, Text, IO, Optional, cast, TypeVar, Any
from six import string_types
# f should be either readable or a file path
def _load_bytes(f): # type: (Union[IO[bytes], Text]) -> bytes
if hasattr(f, 'read') and callable(cast(IO[bytes], f).read):
s = cast(IO[bytes], f).read()
else:
with open(cast(Text, f), 'rb') as readable:
s = readable.read()
return s
# str should be bytes,
# f should be either writable or a file path
def _save_bytes(str, f): # type: (bytes, Union[IO[bytes], Text]) -> None
if hasattr(f, 'write') and callable(cast(IO[bytes], f).write):
cast(IO[bytes], f).write(str)
else:
with open(cast(Text, f), 'wb') as writable:
writable.write(str)
# f should be either a readable file or a file path
def _get_file_path(f): # type: (Union[IO[bytes], Text]) -> Optional[Text]
if isinstance(f, string_types):
return os.path.abspath(f)
if hasattr(f, 'name'):
return os.path.abspath(f.name)
return None
def _serialize(proto): # type: (Union[bytes, google.protobuf.message.Message]) -> bytes
'''
Serialize a in-memory proto to bytes
@params
proto is a in-memory proto, such as a ModelProto, TensorProto, etc
@return
Serialized proto in bytes
'''
if isinstance(proto, bytes):
return proto
elif hasattr(proto, 'SerializeToString') and callable(proto.SerializeToString):
result = proto.SerializeToString()
return result
else:
raise TypeError('No SerializeToString method is detected. '
'neither proto is a str.\ntype is {}'.format(type(proto)))
_Proto = TypeVar('_Proto', bound=google.protobuf.message.Message)
def _deserialize(s, proto): # type: (bytes, _Proto) -> _Proto
'''
Parse bytes into a in-memory proto
@params
s is bytes containing serialized proto
proto is a in-memory proto object
@return
The proto instance filled in by s
'''
if not isinstance(s, bytes):
raise ValueError('Parameter s must be bytes, but got type: {}'.format(type(s)))
if not (hasattr(proto, 'ParseFromString') and callable(proto.ParseFromString)):
raise ValueError('No ParseFromString method is detected. '
'\ntype is {}'.format(type(proto)))
decoded = cast(Optional[int], proto.ParseFromString(s))
if decoded is not None and decoded != len(s):
raise google.protobuf.message.DecodeError(
"Protobuf decoding consumed too few bytes: {} out of {}".format(
decoded, len(s)))
return proto
def load_model(f, format=None, load_external_data=True): # type: (Union[IO[bytes], Text], Optional[Any], bool) -> ModelProto
'''
Loads a serialized ModelProto into memory
@params
f can be a file-like object (has "read" function) or a string containing a file name
format is for future use
@return
Loaded in-memory ModelProto
'''
s = _load_bytes(f)
model = load_model_from_string(s, format=format)
if load_external_data:
model_filepath = _get_file_path(f)
if model_filepath:
base_dir = os.path.dirname(model_filepath)
load_external_data_for_model(model, base_dir)
return model
def load_tensor(f, format=None): # type: (Union[IO[bytes], Text], Optional[Any]) -> TensorProto
'''
Loads a serialized TensorProto into memory
@params
f can be a file-like object (has "read" function) or a string containing a file name
format is for future use
@return
Loaded in-memory TensorProto
'''
s = _load_bytes(f)
return load_tensor_from_string(s, format=format)
def load_model_from_string(s, format=None): # type: (bytes, Optional[Any]) -> ModelProto
'''
Loads a binary string (bytes) that contains serialized ModelProto
@params
s is a string, which contains serialized ModelProto
format is for future use
@return
Loaded in-memory ModelProto
'''
return _deserialize(s, ModelProto())
def load_tensor_from_string(s, format=None): # type: (bytes, Optional[Any]) -> TensorProto
'''
Loads a binary string (bytes) that contains serialized TensorProto
@params
s is a string, which contains serialized TensorProto
format is for future use
@return
Loaded in-memory TensorProto
'''
return _deserialize(s, TensorProto())
def save_model(proto, f, format=None): # type: (Union[ModelProto, bytes], Union[IO[bytes], Text], Optional[Any]) -> None
'''
Saves the ModelProto to the specified path.
@params
proto should be a in-memory ModelProto
f can be a file-like object (has "write" function) or a string containing a file name
format is for future use
'''
if isinstance(proto, bytes):
proto = _deserialize(proto, ModelProto())
model_filepath = _get_file_path(f)
if model_filepath:
basepath = os.path.dirname(model_filepath)
proto = write_external_data_tensors(proto, basepath)
s = _serialize(proto)
_save_bytes(s, f)
def save_tensor(proto, f): # type: (TensorProto, Union[IO[bytes], Text]) -> None
'''
Saves the TensorProto to the specified path.
@params
proto should be a in-memory TensorProto
f can be a file-like object (has "write" function) or a string containing a file name
format is for future use
'''
s = _serialize(proto)
_save_bytes(s, f)
# For backward compatibility
load = load_model
load_from_string = load_model_from_string
save = save_model
onnx-1.7.0/onnx/onnxifi_utils.cc 0000664 0000000 0000000 00000001571 13655345213 015377 0 ustar root root #include "onnxifi_utils.h"
namespace ONNX_NAMESPACE {
namespace testing {
onnxTensorDescriptorV1 ProtoToOnnxTensorDescriptor(
const ONNX_NAMESPACE::TensorProto& proto_tensor,
std::vector>& shape_pool) {
onnxTensorDescriptorV1 onnx_tensor;
onnx_tensor.tag = ONNXIFI_TAG_TENSOR_DESCRIPTOR_V1;
onnx_tensor.name = proto_tensor.name().c_str();
onnx_tensor.dataType = proto_tensor.data_type();
onnx_tensor.memoryType = ONNXIFI_MEMORY_TYPE_CPU;
std::vector shape_values(
proto_tensor.dims().begin(), proto_tensor.dims().end());
shape_pool.emplace_back(std::move(shape_values));
onnx_tensor.dimensions = (uint32_t)shape_pool.back().size();
onnx_tensor.shape = shape_pool.back().data();
onnx_tensor.buffer = (onnxPointer)proto_tensor.raw_data().data();
return onnx_tensor;
}
} // namespace testing
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/onnx-operators.proto3 0000664 0000000 0000000 00000016053 13655345213 016345 0 ustar root root //
// WARNING: This file is automatically generated! Please edit onnx.in.proto.
//
// Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
syntax = "proto3";
package onnx;
import "onnx/onnx.proto3";
//
// This file contains the proto definitions for OperatorSetProto and
// OperatorProto. OperatorSetProtos are used to describe a versioned
// set of operators that can be used by a ModelProto.
//
// Like ModelProto, OperatorSetProto is defined as a top-level file/wire
// format, however their usage is different.
//
// ModelProto files are used to describe executable graphs that can be
// executed directly by a framework, runtime, or engine.
//
// OperatorSetProto files are used to describe a set of operators that are
// available in a given environment. The file TBD.TBD is the OperatorSetProto
// that describes the ONNX standard operators.
//
// Operator/function status.
enum OperatorStatus {
EXPERIMENTAL = 0;
STABLE = 1;
}
message FunctionProto {
// The name of the function, similar usage of op_type in OperatorProto.
string name = 1;
// The first version of a function set which contains this function.
// When there's any breaking change for this function, the function set
// contains the function needs to bump its version, and since_version of
// the updated function will be changed to the updated function set version.
int64 since_version = 2;
// This field indicates whether the syntax, semantics, or presence
// of this function is in an experimental or stable stage. Once an
// function is published as STABLE, its syntax and semantics MUST NOT
// change in subsequent versions of the operator set.
// When a function is published as EXPERIMENTAL, the syntax and semantics
// of the function MAY change across operator set versions.
// Functions "become" stable by deprecating the experimental version and
// introducing a new stable function with the same name.
OperatorStatus status = 3;
// The inputs and outputs of the function.
repeated string input = 4;
repeated string output = 5;
// The attributes of the function.
repeated string attribute= 6;
// The nodes in the function.
repeated NodeProto node = 7;
// A human-readable documentation for this function. Markdown is allowed.
string doc_string = 8;
// The OperatorSets this function body (graph) relies on.
// A FunctionProto body (graph) may implicitly rely on the OperatorSet that
// this function belongs to. It can also explicitly rely on more OperatorSets
// with this field specified.
//
// All nodes in the function body (graph) will bind against the operator
// with the same-domain/same-op_type operator with the HIGHEST version
// in the referenced operator sets. This means at most one version can be relied
// for one domain.
repeated OperatorSetIdProto opset_import = 9;
}
// An OperatorProto represents the immutable specification of the signature
// and semantics of an operator.
//
// Operators are declared as part of an OperatorSet, which also defines the
// domain name for the set.
//
// Operators are uniquely identified by a three part identifier
// (domain, op_type, since_version)
// where
// *domain* is the domain of an operator set that
// contains this operator specification.
//
// *op_type* is the name of the operator as referenced by a
// NodeProto.op_type
//
// *since_version* is the version of the operator set that
// this operator was initially declared in.
//
message OperatorProto {
// The name of the operator within a domain.
// This field MUST be present in this version of the IR.
string op_type = 1;
// The version of the operator set that first introduced this
// operator. This value MUST be the same value as the
// opset_version of the operator set that first published this operator.
// Subsequent versions of the operator set MUST NOT alter the signature
// or semantics of the operator once published as STABLE.
// This field MUST be present in this version of the IR.
int64 since_version = 2;
// This field indicates whether the syntax, semantics, or presence
// of this operator is in an experimental or stable stage. Once an
// operator is published as STABLE, it's syntax and semantics MUST NOT
// change in subsequent versions of the operator set.
// When an operator is published as EXPERIMENTAL, the syntax and semantics
// of the operator MAY change across operator set versions.
// Operators "become" stable by deprecating the experimental version and
// introducing a new stable operator with the same op_type.
OperatorStatus status = 3;
// Eventually we will declare the signature of the operator here
// A human-readable documentation for this operator. Markdown is allowed.
string doc_string = 10;
}
// An OperatorSetProto represents an immutable set of immutable operator
// specifications.
//
// The domain of the set (OperatorSetProto.domain) is a reverse-DNS name
// that disambiguates operator sets defined by independent entities.
//
// The version of the set (opset_version) is a monotonically increasing
// integer that indicates changes to the membership of the operator set.
//
//
// Operator sets are uniquely identified by a two part identifier (domain, opset_version)
//
// Like ModelProto, OperatorSetProto is intended as a top-level file/wire format,
// and thus has the standard format headers in addition to the operator set information.
//
message OperatorSetProto {
// All OperatorSetProtos start with a distingushed byte sequence to disambiguate
// protobuf files containing OperatorSets from other content.
// This field MUST be "ONNXOPSET"
// This field MUST be present in this version of the IR
string magic = 1;
// All OperatorSetProtos indicate the version of the IR syntax and semantics
// they adhere to. It is always IR_VERSION.
// This field MUST be present in this version of the IR
int64 ir_version = 2;
// The prerelease component of the SemVer of the IR.
// This field MAY be absent in this version of the IR
string ir_version_prerelease = 3;
// The build metadata component of the SemVer of the IR.
// This field MAY be absent in this version of the IR
string ir_build_metadata = 7;
// Domain name of the operator set, in reverse DNS form (e.g., com.acme.dnnops).
string domain = 4;
// The version of the set of operators. This is a simple int value
// that is monotonically increasing as new versions of operator set
// are published. All operators in this set MUST have version
// numbers no greater than opset_version.
int64 opset_version = 5;
// A human-readable documentation for this set of operators. Markdown is allowed.
string doc_string = 6;
// The operators specified by this operator set.
// The (name, version) MUST be unique across all OperatorProtos in operator
repeated OperatorProto operator = 8;
// The functions specified by this operator set.
// The (name, version) MUST be unique across all OperatorProtos/FunctionProtos in operator/functions
repeated FunctionProto functions = 9;
}
// For using protobuf-lite
option optimize_for = LITE_RUNTIME;
onnx-1.7.0/onnx/helper.py 0000664 0000000 0000000 00000052745 13655345213 014040 0 ustar root root from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import collections
import numbers
from six import text_type, integer_types, binary_type
import google.protobuf.message
from onnx import TensorProto, SparseTensorProto, AttributeProto, ValueInfoProto, TensorShapeProto, \
NodeProto, ModelProto, GraphProto, OperatorSetIdProto, TypeProto, IR_VERSION
import onnx.defs as defs
from onnx import mapping
from onnx.mapping import STORAGE_TENSOR_TYPE_TO_FIELD
from typing import Text, Sequence, Any, Optional, Dict, Union, TypeVar, Callable, Tuple, List, cast
import numpy as np # type: ignore
def make_node(
op_type, # type: Text
inputs, # type: Sequence[Text]
outputs, # type: Sequence[Text]
name=None, # type: Optional[Text]
doc_string=None, # type: Optional[Text]
domain=None, # type: Optional[Text]
**kwargs # type: Any
): # type: (...) -> NodeProto
"""Construct a NodeProto.
Arguments:
op_type (string): The name of the operator to construct
inputs (list of string): list of input names
outputs (list of string): list of output names
name (string, default None): optional unique identifier for NodeProto
doc_string (string, default None): optional documentation string for NodeProto
domain (string, default None): optional domain for NodeProto.
If it's None, we will just use default domain (which is empty)
**kwargs (dict): the attributes of the node. The acceptable values
are documented in :func:`make_attribute`.
"""
node = NodeProto()
node.op_type = op_type
node.input.extend(inputs)
node.output.extend(outputs)
if name:
node.name = name
if doc_string:
node.doc_string = doc_string
if domain is not None:
node.domain = domain
if kwargs:
node.attribute.extend(
make_attribute(key, value)
for key, value in sorted(kwargs.items()))
return node
def make_operatorsetid(
domain, # type: Text
version, # type: int
): # type: (...) -> OperatorSetIdProto
"""Construct an OperatorSetIdProto.
Arguments:
domain (string): The domain of the operator set id
version (integer): Version of operator set id
"""
operatorsetid = OperatorSetIdProto()
operatorsetid.domain = domain
operatorsetid.version = version
return operatorsetid
def make_graph(
nodes, # type: Sequence[NodeProto]
name, # type: Text
inputs, # type: Sequence[ValueInfoProto]
outputs, # type: Sequence[ValueInfoProto]
initializer=None, # type: Optional[Sequence[TensorProto]]
doc_string=None, # type: Optional[Text]
value_info=[], # type: Sequence[ValueInfoProto]
): # type: (...) -> GraphProto
if initializer is None:
initializer = []
if value_info is None:
value_info = []
graph = GraphProto()
graph.node.extend(nodes)
graph.name = name
graph.input.extend(inputs)
graph.output.extend(outputs)
graph.initializer.extend(initializer)
graph.value_info.extend(value_info)
if doc_string:
graph.doc_string = doc_string
return graph
def make_opsetid(domain, version): # type: (Text, int) -> OperatorSetIdProto
opsetid = OperatorSetIdProto()
opsetid.domain = domain
opsetid.version = version
return opsetid
def make_model(graph, **kwargs): # type: (GraphProto, **Any) -> ModelProto
model = ModelProto()
# Touch model.ir_version so it is stored as the version from which it is
# generated.
model.ir_version = IR_VERSION
model.graph.CopyFrom(graph)
opset_imports = None # type: Optional[Sequence[OperatorSetIdProto]]
opset_imports = kwargs.pop('opset_imports', None) # type: ignore
if opset_imports is not None:
model.opset_import.extend(opset_imports)
else:
# Default import
imp = model.opset_import.add()
imp.version = defs.onnx_opset_version()
for k, v in kwargs.items():
# TODO: Does this work with repeated fields?
setattr(model, k, v)
return model
def set_model_props(model, dict_value): # type: (ModelProto, Dict[Text, Text]) -> None
del model.metadata_props[:]
for (k, v) in dict_value.items():
entry = model.metadata_props.add()
entry.key = k
entry.value = v
# model.metadata_properties.append(entry)
def split_complex_to_pairs(ca): # type: (Sequence[np.complex64]) -> Sequence[int]
return [(ca[i // 2].real if (i % 2 == 0) else ca[i // 2].imag)
for i in range(len(ca) * 2)]
def make_tensor(
name, # type: Text
data_type, # type: int
dims, # type: Sequence[int]
vals, # type: Any
raw=False # type: bool
): # type: (...) -> TensorProto
'''
Make a TensorProto with specified arguments. If raw is False, this
function will choose the corresponding proto field to store the
values based on data_type. If raw is True, use "raw_data" proto
field to store the values, and values should be of type bytes in
this case.
'''
tensor = TensorProto()
tensor.data_type = data_type
tensor.name = name
if data_type == TensorProto.STRING:
assert not raw, "Can not use raw_data to store string type"
if (data_type == TensorProto.COMPLEX64
or data_type == TensorProto.COMPLEX128):
vals = split_complex_to_pairs(vals)
if raw:
tensor.raw_data = vals
else:
field = mapping.STORAGE_TENSOR_TYPE_TO_FIELD[
mapping.TENSOR_TYPE_TO_STORAGE_TENSOR_TYPE[data_type]]
getattr(tensor, field).extend(vals)
tensor.dims.extend(dims)
return tensor
def make_sparse_tensor(
values, # type: TensorProto
indices, # type: TensorProto
dims # type: Sequence[int]
): # type: (...) -> SparseTensorProto
sparse = SparseTensorProto()
sparse.values.CopyFrom(values)
sparse.indices.CopyFrom(indices)
sparse.dims.extend(dims)
return sparse
def _to_bytes_or_false(val): # type: (Union[Text, bytes]) -> Union[bytes, bool]
"""An internal graph to convert the input to a bytes or to False.
The criteria for conversion is as follows and should be python 2 and 3
compatible:
- If val is py2 str or py3 bytes: return bytes
- If val is py2 unicode or py3 str: return val.decode('utf-8')
- Otherwise, return False
"""
if isinstance(val, bytes):
return val
try:
return val.encode('utf-8')
except AttributeError:
return False
def make_attribute(
key, # type: Text
value, # type: Any
doc_string=None # type: Optional[Text]
): # type: (...) -> AttributeProto
"""Makes an AttributeProto based on the value type."""
attr = AttributeProto()
attr.name = key
if doc_string:
attr.doc_string = doc_string
is_iterable = isinstance(value, collections.Iterable)
bytes_or_false = _to_bytes_or_false(value)
# First, singular cases
# float
if isinstance(value, float):
attr.f = value
attr.type = AttributeProto.FLOAT
# integer
elif isinstance(value, numbers.Integral):
attr.i = cast(int, value)
attr.type = AttributeProto.INT
# string
elif bytes_or_false is not False:
assert isinstance(bytes_or_false, bytes)
attr.s = bytes_or_false
attr.type = AttributeProto.STRING
elif isinstance(value, TensorProto):
attr.t.CopyFrom(value)
attr.type = AttributeProto.TENSOR
elif isinstance(value, SparseTensorProto):
attr.sparse_tensor.CopyFrom(value)
attr.type = AttributeProto.SPARSE_TENSOR
elif isinstance(value, GraphProto):
attr.g.CopyFrom(value)
attr.type = AttributeProto.GRAPH
# third, iterable cases
elif is_iterable:
byte_array = [_to_bytes_or_false(v) for v in value]
if all(isinstance(v, float) for v in value):
attr.floats.extend(value)
attr.type = AttributeProto.FLOATS
elif all(isinstance(v, numbers.Integral) for v in value):
# Turn np.int32/64 into Python built-in int.
attr.ints.extend(int(v) for v in value)
attr.type = AttributeProto.INTS
elif all(map(lambda bytes_or_false: bytes_or_false is not False, byte_array)):
attr.strings.extend(cast(List[bytes], byte_array))
attr.type = AttributeProto.STRINGS
elif all(isinstance(v, TensorProto) for v in value):
attr.tensors.extend(value)
attr.type = AttributeProto.TENSORS
elif all(isinstance(v, SparseTensorProto) for v in value):
attr.sparse_tensors.extend(value)
attr.type = AttributeProto.SPARSE_TENSORS
elif all(isinstance(v, GraphProto) for v in value):
attr.graphs.extend(value)
attr.type = AttributeProto.GRAPHS
else:
raise ValueError(
"You passed in an iterable attribute but I cannot figure out "
"its applicable type.")
else:
raise TypeError(
'value "{}" is not valid attribute data type.'.format(value))
return attr
def get_attribute_value(attr): # type: (AttributeProto) -> Any
if attr.type == AttributeProto.FLOAT:
return attr.f
if attr.type == AttributeProto.INT:
return attr.i
if attr.type == AttributeProto.STRING:
return attr.s
if attr.type == AttributeProto.TENSOR:
return attr.t
if attr.type == AttributeProto.GRAPH:
return attr.g
if attr.type == AttributeProto.FLOATS:
return list(attr.floats)
if attr.type == AttributeProto.INTS:
return list(attr.ints)
if attr.type == AttributeProto.STRINGS:
return list(attr.strings)
if attr.type == AttributeProto.TENSORS:
return list(attr.tensors)
if attr.type == AttributeProto.GRAPHS:
return list(attr.graphs)
raise ValueError("Unsupported ONNX attribute: {}".format(attr))
def make_empty_tensor_value_info(name): # type: (Text) -> ValueInfoProto
value_info_proto = ValueInfoProto()
value_info_proto.name = name
return value_info_proto
def make_tensor_value_info(
name, # type: Text
elem_type, # type: int
shape, # type: Optional[Sequence[Union[Text, int]]]
doc_string="", # type: Text
shape_denotation=None, # type: Optional[List[Text]]
): # type: (...) -> ValueInfoProto
"""Makes a ValueInfoProto based on the data type and shape."""
value_info_proto = ValueInfoProto()
value_info_proto.name = name
if doc_string:
value_info_proto.doc_string = doc_string
tensor_type_proto = value_info_proto.type.tensor_type
tensor_type_proto.elem_type = elem_type
tensor_shape_proto = tensor_type_proto.shape
if shape is not None:
# You might think this is a no-op (extending a normal Python
# list by [] certainly is), but protobuf lists work a little
# differently; if a field is never set, it is omitted from the
# resulting protobuf; a list that is explicitly set to be
# empty will get an (empty) entry in the protobuf. This
# difference is visible to our consumers, so make sure we emit
# an empty shape!
tensor_shape_proto.dim.extend([])
if shape_denotation:
if len(shape_denotation) != len(shape):
raise ValueError(
'Invalid shape_denotation. '
'Must be of the same length as shape.')
for i, d in enumerate(shape):
dim = tensor_shape_proto.dim.add()
if d is None:
pass
elif isinstance(d, integer_types):
dim.dim_value = d
elif isinstance(d, text_type):
dim.dim_param = d
else:
raise ValueError(
'Invalid item in shape: {}. '
'Needs to of integer_types or text_type.'.format(d))
if shape_denotation:
dim.denotation = shape_denotation[i]
return value_info_proto
def make_sequence_value_info(
name, # type: Text
elem_type, # type: int
shape, # type: Optional[Sequence[Union[Text, int]]]
doc_string="", # type: Text
elem_shape_denotation=None, # type: Optional[List[Text]]
): # type: (...) -> ValueInfoProto
"""Makes a ValueInfoProto based on the data type and shape for Sequence."""
value_info_proto = ValueInfoProto()
value_info_proto.name = name
if doc_string:
value_info_proto.doc_string = doc_string
sequence_type_proto = value_info_proto.type.sequence_type
sequence_type_proto.elem_type.tensor_type.elem_type = elem_type
tensor_value_info = make_tensor_value_info(name, elem_type, shape, doc_string, elem_shape_denotation)
if shape is not None:
sequence_type_proto.elem_type.tensor_type.shape.CopyFrom(tensor_value_info.type.tensor_type.shape)
return value_info_proto
def _sanitize_str(s): # type: (Union[Text, bytes]) -> Text
if isinstance(s, text_type):
sanitized = s
elif isinstance(s, binary_type):
sanitized = s.decode('utf-8', errors='ignore')
else:
sanitized = str(s)
if len(sanitized) < 64:
return sanitized
return sanitized[:64] + '...<+len=%d>' % (len(sanitized) - 64)
def printable_attribute(attr, subgraphs=False): # type: (AttributeProto, bool) -> Union[Text, Tuple[Text, List[GraphProto]]]
content = []
content.append(attr.name)
content.append("=")
def str_float(f): # type: (float) -> Text
# NB: Different Python versions print different numbers of trailing
# decimals, specifying this explicitly keeps it consistent for all
# versions
return '{:.15g}'.format(f)
def str_int(i): # type: (int) -> Text
# NB: In Python 2, longs will repr() as '2L', which is ugly and
# unnecessary. Explicitly format it to keep it consistent.
return '{:d}'.format(i)
def str_str(s): # type: (Text) -> Text
return repr(s)
_T = TypeVar('_T') # noqa
def str_list(str_elem, xs): # type: (Callable[[_T], Text], Sequence[_T]) -> Text
return '[' + ', '.join(map(str_elem, xs)) + ']'
# for now, this logic should continue to work as long as we are running on a proto3
# implementation. If/when we switch to proto3, we will need to use attr.type
# To support printing subgraphs, if we find a graph attribute, print out
# its name here and pass the graph itself up to the caller for later
# printing.
graphs = []
if attr.HasField("f"):
content.append(str_float(attr.f))
elif attr.HasField("i"):
content.append(str_int(attr.i))
elif attr.HasField("s"):
# TODO: Bit nervous about Python 2 / Python 3 determinism implications
content.append(repr(_sanitize_str(attr.s)))
elif attr.HasField("t"):
if len(attr.t.dims) > 0:
content.append("")
else:
# special case to print scalars
field = STORAGE_TENSOR_TYPE_TO_FIELD[attr.t.data_type]
content.append(''.format(str(getattr(attr.t, field))))
elif attr.HasField("g"):
content.append("".format(attr.g.name))
graphs.append(attr.g)
elif attr.floats:
content.append(str_list(str_float, attr.floats))
elif attr.ints:
content.append(str_list(str_int, attr.ints))
elif attr.strings:
# TODO: Bit nervous about Python 2 / Python 3 determinism implications
content.append(str(list(map(_sanitize_str, attr.strings))))
elif attr.tensors:
content.append("[, ...]")
elif attr.graphs:
content.append('[')
for i, g in enumerate(attr.graphs):
comma = ',' if i != len(attr.graphs) - 1 else ''
content.append('{}'.format(g.name, comma))
content.append(']')
graphs.extend(attr.graphs)
else:
content.append("")
if subgraphs:
return ' '.join(content), graphs
else:
return ' '.join(content)
def printable_dim(dim): # type: (TensorShapeProto.Dimension) -> Text
which = dim.WhichOneof('value')
assert which is not None
return str(getattr(dim, which))
def printable_type(t): # type: (TypeProto) -> Text
if t.WhichOneof('value') == "tensor_type":
s = TensorProto.DataType.Name(t.tensor_type.elem_type)
if t.tensor_type.HasField('shape'):
if len(t.tensor_type.shape.dim):
s += str(', ' + 'x'.join(map(printable_dim, t.tensor_type.shape.dim)))
else:
s += str(', scalar')
return s
if t.WhichOneof('value') is None:
return ""
return 'Unknown type {}'.format(t.WhichOneof('value'))
def printable_value_info(v): # type: (ValueInfoProto) -> Text
s = '%{}'.format(v.name)
if v.type:
s = '{}[{}]'.format(s, printable_type(v.type))
return s
def printable_tensor_proto(t): # type: (TensorProto) -> Text
s = '%{}['.format(t.name)
s += TensorProto.DataType.Name(t.data_type)
if t.dims is not None:
if len(t.dims):
s += str(', ' + 'x'.join(map(str, t.dims)))
else:
s += str(', scalar')
s += ']'
return s
def printable_node(node, prefix='', subgraphs=False): # type: (NodeProto, Text, bool) -> Union[Text, Tuple[Text, List[GraphProto]]]
content = []
if len(node.output):
content.append(
', '.join(['%{}'.format(name) for name in node.output]))
content.append('=')
# To deal with nested graphs
graphs = [] # type: List[GraphProto]
printed_attrs = []
for attr in node.attribute:
if subgraphs:
printed_attr, gs = printable_attribute(attr, subgraphs)
assert isinstance(gs, list)
graphs.extend(gs)
printed_attrs.append(printed_attr)
else:
printed = printable_attribute(attr)
assert isinstance(printed, Text)
printed_attrs.append(printed)
printed_attributes = ', '.join(sorted(printed_attrs))
printed_inputs = ', '.join(['%{}'.format(name) for name in node.input])
if node.attribute:
content.append("{}[{}]({})".format(node.op_type, printed_attributes, printed_inputs))
else:
content.append("{}({})".format(node.op_type, printed_inputs))
if subgraphs:
return prefix + ' '.join(content), graphs
else:
return prefix + ' '.join(content)
def printable_graph(graph, prefix=''): # type: (GraphProto, Text) -> Text
content = []
indent = prefix + ' '
# header
header = ['graph', graph.name]
initializers = {t.name for t in graph.initializer}
if len(graph.input):
header.append("(")
in_strs = [] # required inputs
in_with_init_strs = [] # optional inputs with initializer providing default value
for inp in graph.input:
if inp.name not in initializers:
in_strs.append(printable_value_info(inp))
else:
in_with_init_strs.append(printable_value_info(inp))
if in_strs:
content.append(prefix + ' '.join(header))
header = []
for line in in_strs:
content.append(prefix + ' ' + line)
header.append(")")
if in_with_init_strs:
header.append("optional inputs with matching initializers (")
content.append(prefix + ' '.join(header))
header = []
for line in in_with_init_strs:
content.append(prefix + ' ' + line)
header.append(")")
# from IR 4 onwards an initializer is not required to have a matching graph input
# so output the name, type and shape of those as well
if len(in_with_init_strs) < len(initializers):
graph_inputs = {i.name for i in graph.input}
init_strs = [printable_tensor_proto(i) for i in graph.initializer
if i.name not in graph_inputs]
header.append("initializers (")
content.append(prefix + ' '.join(header))
header = []
for line in init_strs:
content.append(prefix + ' ' + line)
header.append(")")
header.append('{')
content.append(prefix + ' '.join(header))
graphs = [] # type: List[GraphProto]
# body
for node in graph.node:
pn, gs = printable_node(node, indent, subgraphs=True)
assert isinstance(gs, list)
content.append(pn)
graphs.extend(gs)
# tail
tail = ['return']
if len(graph.output):
tail.append(
', '.join(['%{}'.format(out.name) for out in graph.output]))
content.append(indent + ' '.join(tail))
# closing bracket
content.append(prefix + '}')
for g in graphs:
content.append('\n' + printable_graph(g))
return '\n'.join(content)
def strip_doc_string(proto): # type: (google.protobuf.message.Message) -> None
"""
Empties `doc_string` field on any nested protobuf messages
"""
assert isinstance(proto, google.protobuf.message.Message)
for descriptor in proto.DESCRIPTOR.fields:
if descriptor.name == 'doc_string':
proto.ClearField(descriptor.name)
elif descriptor.type == descriptor.TYPE_MESSAGE:
if descriptor.label == descriptor.LABEL_REPEATED:
for x in getattr(proto, descriptor.name):
strip_doc_string(x)
elif proto.HasField(descriptor.name):
strip_doc_string(getattr(proto, descriptor.name))
onnx-1.7.0/onnx/onnx-ml.proto3 0000664 0000000 0000000 00000072155 13655345213 014744 0 ustar root root //
// WARNING: This file is automatically generated! Please edit onnx.in.proto.
//
// Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
syntax = "proto3";
package onnx;
// Overview
//
// ONNX is an open specification that is comprised of the following components:
//
// 1) A definition of an extensible computation graph model.
// 2) Definitions of standard data types.
// 3) Definitions of built-in operators.
//
// This document describes the syntax of models and their computation graphs,
// as well as the standard data types. Together, they are referred to as the ONNX
// Intermediate Representation, or 'IR' for short.
//
// The normative semantic specification of the ONNX IR is found in docs/IR.md.
// Definitions of the built-in neural network operators may be found in docs/Operators.md.
// Definitions of the built-in classical machine learning operators may be found in
// docs/Operators-ml.md.
// Notes
//
// Release
//
// We are still in the very early stage of defining ONNX. The current
// version of ONNX is a starting point. While we are actively working
// towards a complete spec, we would like to get the community involved
// by sharing our working version of ONNX.
//
// Protobuf compatibility
//
// To simplify framework compatibility, ONNX is defined using the subset of protobuf
// that is compatible with both protobuf v2 and v3. This means that we do not use any
// protobuf features that are only available in one of the two versions.
//
// Here are the most notable contortions we have to carry out to work around
// these limitations:
//
// - No 'map' (added protobuf 3.0). We instead represent mappings as lists
// of key-value pairs, where order does not matter and duplicates
// are not allowed.
// Versioning
//
// ONNX versioning is specified in docs/IR.md and elaborated on in docs/Versioning.md
//
// To be compatible with both proto2 and proto3, we will use a version number
// that is not defined by the default value but an explicit enum number.
enum Version {
// proto3 requires the first enum value to be zero.
// We add this just to appease the compiler.
_START_VERSION = 0;
// The version field is always serialized and we will use it to store the
// version that the graph is generated from. This helps us set up version
// control.
// For the IR, we are using simple numbers starting with 0x00000001,
// which was the version we published on Oct 10, 2017.
IR_VERSION_2017_10_10 = 0x0000000000000001;
// IR_VERSION 2 published on Oct 30, 2017
// - Added type discriminator to AttributeProto to support proto3 users
IR_VERSION_2017_10_30 = 0x0000000000000002;
// IR VERSION 3 published on Nov 3, 2017
// - For operator versioning:
// - Added new message OperatorSetIdProto
// - Added opset_import in ModelProto
// - For vendor extensions, added domain in NodeProto
IR_VERSION_2017_11_3 = 0x0000000000000003;
// IR VERSION 4 published on Jan 22, 2019
// - Relax constraint that initializers should be a subset of graph inputs
// - Add type BFLOAT16
IR_VERSION_2019_1_22 = 0x0000000000000004;
// IR VERSION 5 published on March 18, 2019
// - Add message TensorAnnotation.
// - Add quantization annotation in GraphProto to map tensor with its scale and zero point quantization parameters.
IR_VERSION_2019_3_18 = 0x0000000000000005;
// IR VERSION 6 published on Sep 19, 2019
// - Add support for sparse tensor constants stored in model.
// - Add message SparseTensorProto
// - Add sparse initializers
IR_VERSION_2019_9_19 = 0x0000000000000006;
// IR VERSION 7 published on
// - Add support to allow function body graph to rely on multiple external opreator sets.
// - Add a list to promote inference graph's initializers to global and
// mutable variables. Global variables are visible in all graphs of the
// stored models.
// - Add message TrainingInfoProto to store initialization
// method and training algorithm. The execution of TrainingInfoProto
// can modify the values of mutable variables.
// - Make inference graph callable from TrainingInfoProto via GraphCall operator.
IR_VERSION = 0x0000000000000007;
}
// Attributes
//
// A named attribute containing either singular float, integer, string, graph,
// and tensor values, or repeated float, integer, string, graph, and tensor values.
// An AttributeProto MUST contain the name field, and *only one* of the
// following content fields, effectively enforcing a C/C++ union equivalent.
message AttributeProto {
// Note: this enum is structurally identical to the OpSchema::AttrType
// enum defined in schema.h. If you rev one, you likely need to rev the other.
enum AttributeType {
UNDEFINED = 0;
FLOAT = 1;
INT = 2;
STRING = 3;
TENSOR = 4;
GRAPH = 5;
SPARSE_TENSOR = 11;
FLOATS = 6;
INTS = 7;
STRINGS = 8;
TENSORS = 9;
GRAPHS = 10;
SPARSE_TENSORS = 12;
}
// The name field MUST be present for this version of the IR.
string name = 1; // namespace Attribute
// if ref_attr_name is not empty, ref_attr_name is the attribute name in parent function.
// In this case, this AttributeProto does not contain data, and it's a reference of attribute
// in parent scope.
// NOTE: This should ONLY be used in function (sub-graph). It's invalid to be used in main graph.
string ref_attr_name = 21;
// A human-readable documentation for this attribute. Markdown is allowed.
string doc_string = 13;
// The type field MUST be present for this version of the IR.
// For 0.0.1 versions of the IR, this field was not defined, and
// implementations needed to use has_field heuristics to determine
// which value field was in use. For IR_VERSION 0.0.2 or later, this
// field MUST be set and match the f|i|s|t|... field in use. This
// change was made to accommodate proto3 implementations.
AttributeType type = 20; // discriminator that indicates which field below is in use
// Exactly ONE of the following fields must be present for this version of the IR
float f = 2; // float
int64 i = 3; // int
bytes s = 4; // UTF-8 string
TensorProto t = 5; // tensor value
GraphProto g = 6; // graph
SparseTensorProto sparse_tensor = 22; // sparse tensor value
// Do not use field below, it's deprecated.
// optional ValueProto v = 12; // value - subsumes everything but graph
repeated float floats = 7; // list of floats
repeated int64 ints = 8; // list of ints
repeated bytes strings = 9; // list of UTF-8 strings
repeated TensorProto tensors = 10; // list of tensors
repeated GraphProto graphs = 11; // list of graph
repeated SparseTensorProto sparse_tensors = 23; // list of sparse tensors
}
// Defines information on value, including the name, the type, and
// the shape of the value.
message ValueInfoProto {
// This field MUST be present in this version of the IR.
string name = 1; // namespace Value
// This field MUST be present in this version of the IR for
// inputs and outputs of the top-level graph.
TypeProto type = 2;
// A human-readable documentation for this value. Markdown is allowed.
string doc_string = 3;
}
// Nodes
//
// Computation graphs are made up of a DAG of nodes, which represent what is
// commonly called a "layer" or "pipeline stage" in machine learning frameworks.
//
// For example, it can be a node of type "Conv" that takes in an image, a filter
// tensor and a bias tensor, and produces the convolved output.
message NodeProto {
repeated string input = 1; // namespace Value
repeated string output = 2; // namespace Value
// An optional identifier for this node in a graph.
// This field MAY be absent in ths version of the IR.
string name = 3; // namespace Node
// The symbolic identifier of the Operator to execute.
string op_type = 4; // namespace Operator
// The domain of the OperatorSet that specifies the operator named by op_type.
string domain = 7; // namespace Domain
// Additional named attributes.
repeated AttributeProto attribute = 5;
// A human-readable documentation for this node. Markdown is allowed.
string doc_string = 6;
}
// Training information
// TrainingInfoProto stores information for training a model.
// In particular, this defines two functionalities: an initialization-step
// and a training-algorithm-step. Initialization resets the model
// back to its original state as if no training has been consumed.
// Training algorithm improves the model based on input data.
//
// The semantics of the initialization-step is that the initializers
// in ModelProto.graph and in TrainingInfoProto.algorithm are first
// initialized as specified by the initializers in the graph, and then
// updated by the "initialization_binding" in every instance in
// ModelProto.training_info.
//
// The field "algorithm" defines a computation graph which represents a
// training algorithm's step. After the execution of a
// TrainingInfoProto.algorithm, the initializers specified by "update_binding"
// may be immediately updated. If the targeted training algorithm contains
// consecutive update stages (such as block coordinate descent methods),
// the user needs to create a TrainingInfoProto for each stage.
message TrainingInfoProto {
// This field describes a graph to compute the initial tensors
// upon starting the training process. Initialization graph has no input
// and can have multiple outputs. Usually, trainable tensors in neural
// networks are randomly initialized. To achieve that, for each tensor,
// the user can put a random number operator such as RandomNormal or
// RandomUniform in TrainingInfoProto.initialization.node and assign its
// random output to the specific tensor using "initialization_binding".
// This graph can also set the initializers in "algorithm" in the same
// TrainingInfoProto; a use case is resetting the number of training
// iteration to zero.
//
// By default, this field is an empty graph and its evaluation does not
// produce any output.
GraphProto initialization = 1;
// This field represents a training algorithm step. Given required inputs,
// it computes outputs to update initializers in its own or inference graph's
// initializer lists. In general, this graph contains loss node, gradient node,
// optimizer node, increment of iteration count, and some calls to the inference
// graph.
//
// The field algorithm.node is the only place the user can use GraphCall
// operator. The only callable graph is the one stored in ModelProto.graph.
//
// By default, this field is an empty graph and its evaluation does not
// produce any output.
GraphProto algorithm = 2;
// This field specifies the bindings from the outputs of "initialization" to
// some initializers in "ModelProto.graph.initializer" and
// the "algorithm.initializer" in the same TrainingInfoProto.
// See "update_binding" below for details.
//
// By default, this field is empty and no initializer would be changed
// by the execution of "initialization".
repeated StringStringEntryProto initialization_binding = 3;
// Gradient-based training is usually an iterative procedure. In one gradient
// descent iteration, we apply
//
// x = x - r * g
//
// where "x" is the optimized tensor, "r" stands for learning rate, and "g" is
// gradient of "x" with respect to a chosen loss. To avoid adding assignments
// into the training graph, we split the update equation into
//
// y = x - r * g
// x = y
//
// The user needs to save "y = x - r * g" into TrainingInfoProto.algorithm. To
// tell that "y" should be assigned to "x", the field "update_binding" may
// contain a key-value pair of strings, "x" (key of StringStringEntryProto)
// and "y" (value of StringStringEntryProto).
// For a neural network with multiple trainable (mutable) tensors, there can
// be multiple key-value pairs in "update_binding".
//
// The initializers appears as keys in "update_binding" are considered
// mutable and globally-visible variables. This implies some behaviors
// as described below.
//
// 1. We have only unique keys in all "update_binding"s so that two global
// variables may not have the same name. This ensures that one
// global variable is assigned up to once.
// 2. The keys must appear in names of "ModelProto.graph.initializer" or
// "TrainingInfoProto.algorithm.initializer".
// 3. The values must be output names of "algorithm".
// 4. If an optional input of a graph is omitted when using GraphCall, the
// global variable with the same name may be used.
// 5. When using GraphCall, the users always can pass values to optional
// inputs of the called graph even if the associated initializers appears
// as keys in "update_binding"s.
// 6. The graphs in TrainingInfoProto's can use global variables as
// their operator inputs.
// 7. Mutable variables are initialized to the value specified by the
// corresponding initializer, and then potentially updated by
// "initializer_binding"s and "update_binding"s in "TrainingInfoProto"s.
//
// This field usually contains names of trainable tensors
// (in ModelProto.graph), optimizer states such as momentums in advanced
// stochastic gradient methods (in TrainingInfoProto.graph),
// and number of training iterations (in TrainingInfoProto.graph).
//
// By default, this field is empty and no initializer would be changed
// by the execution of "algorithm".
repeated StringStringEntryProto update_binding = 4;
}
// Models
//
// ModelProto is a top-level file/container format for bundling a ML model and
// associating its computation graph with metadata.
//
// The semantics of the model are described by the associated GraphProto's.
message ModelProto {
// The version of the IR this model targets. See Version enum above.
// This field MUST be present.
int64 ir_version = 1;
// The OperatorSets this model relies on.
// All ModelProtos MUST have at least one entry that
// specifies which version of the ONNX OperatorSet is
// being imported.
//
// All nodes in the ModelProto's graph will bind against the operator
// with the same-domain/same-op_type operator with the HIGHEST version
// in the referenced operator sets.
repeated OperatorSetIdProto opset_import = 8;
// The name of the framework or tool used to generate this model.
// This field SHOULD be present to indicate which implementation/tool/framework
// emitted the model.
string producer_name = 2;
// The version of the framework or tool used to generate this model.
// This field SHOULD be present to indicate which implementation/tool/framework
// emitted the model.
string producer_version = 3;
// Domain name of the model.
// We use reverse domain names as name space indicators. For example:
// `com.facebook.fair` or `com.microsoft.cognitiveservices`
//
// Together with `model_version` and GraphProto.name, this forms the unique identity of
// the graph.
string domain = 4;
// The version of the graph encoded. See Version enum below.
int64 model_version = 5;
// A human-readable documentation for this model. Markdown is allowed.
string doc_string = 6;
// The parameterized graph that is evaluated to execute the model.
GraphProto graph = 7;
// Named metadata values; keys should be distinct.
repeated StringStringEntryProto metadata_props = 14;
// Training-specific information. Sequentially executing all stored
// `TrainingInfoProto.algorithm`s and assigning their outputs following
// the corresponding `TrainingInfoProto.update_binding`s is one training
// iteration. Similarly, to initialize the model
// (as if training hasn't happened), the user should sequentially execute
// all stored `TrainingInfoProto.initialization`s and assigns their outputs
// using `TrainingInfoProto.initialization_binding`s.
//
// If this field is empty, the training behavior of the model is undefined.
repeated TrainingInfoProto training_info = 20;
};
// StringStringEntryProto follows the pattern for cross-proto-version maps.
// See https://developers.google.com/protocol-buffers/docs/proto3#maps
message StringStringEntryProto {
string key = 1;
string value= 2;
};
message TensorAnnotation {
string tensor_name = 1;
// pairs to annotate tensor specified by above.
// The keys used in the mapping below must be pre-defined in ONNX spec.
// For example, for 8-bit linear quantization case, 'SCALE_TENSOR', 'ZERO_POINT_TENSOR' will be pre-defined as
// quantization parameter keys.
repeated StringStringEntryProto quant_parameter_tensor_names = 2;
}
// Graphs
//
// A graph defines the computational logic of a model and is comprised of a parameterized
// list of nodes that form a directed acyclic graph based on their inputs and outputs.
// This is the equivalent of the "network" or "graph" in many deep learning
// frameworks.
message GraphProto {
// The nodes in the graph, sorted topologically.
repeated NodeProto node = 1;
// The name of the graph.
string name = 2; // namespace Graph
// A list of named tensor values, used to specify constant inputs of the graph.
// Each TensorProto entry must have a distinct name (within the list) that
// MAY also appear in the input list.
repeated TensorProto initializer = 5;
// Initializers (see above) stored in sparse format.
repeated SparseTensorProto sparse_initializer = 15;
// A human-readable documentation for this graph. Markdown is allowed.
string doc_string = 10;
// The inputs and outputs of the graph.
repeated ValueInfoProto input = 11;
repeated ValueInfoProto output = 12;
// Information for the values in the graph. The ValueInfoProto.name's
// must be distinct. It is optional for a value to appear in value_info list.
repeated ValueInfoProto value_info = 13;
// This field carries information to indicate the mapping among a tensor and its
// quantization parameter tensors. For example:
// For tensor 'a', it may have {'SCALE_TENSOR', 'a_scale'} and {'ZERO_POINT_TENSOR', 'a_zero_point'} annotated,
// which means, tensor 'a_scale' and tensor 'a_zero_point' are scale and zero point of tensor 'a' in the model.
repeated TensorAnnotation quantization_annotation = 14;
// DO NOT USE the following fields, they were deprecated from earlier versions.
// repeated string input = 3;
// repeated string output = 4;
// optional int64 ir_version = 6;
// optional int64 producer_version = 7;
// optional string producer_tag = 8;
// optional string domain = 9;
}
// Tensors
//
// A serialized tensor value.
message TensorProto {
enum DataType {
UNDEFINED = 0;
// Basic types.
FLOAT = 1; // float
UINT8 = 2; // uint8_t
INT8 = 3; // int8_t
UINT16 = 4; // uint16_t
INT16 = 5; // int16_t
INT32 = 6; // int32_t
INT64 = 7; // int64_t
STRING = 8; // string
BOOL = 9; // bool
// IEEE754 half-precision floating-point format (16 bits wide).
// This format has 1 sign bit, 5 exponent bits, and 10 mantissa bits.
FLOAT16 = 10;
DOUBLE = 11;
UINT32 = 12;
UINT64 = 13;
COMPLEX64 = 14; // complex with float32 real and imaginary components
COMPLEX128 = 15; // complex with float64 real and imaginary components
// Non-IEEE floating-point format based on IEEE754 single-precision
// floating-point number truncated to 16 bits.
// This format has 1 sign bit, 8 exponent bits, and 7 mantissa bits.
BFLOAT16 = 16;
// Future extensions go here.
}
// The shape of the tensor.
repeated int64 dims = 1;
// The data type of the tensor.
// This field MUST have a valid TensorProto.DataType value
int32 data_type = 2;
// For very large tensors, we may want to store them in chunks, in which
// case the following fields will specify the segment that is stored in
// the current TensorProto.
message Segment {
int64 begin = 1;
int64 end = 2;
}
Segment segment = 3;
// Tensor content must be organized in row-major order.
//
// Depending on the data_type field, exactly one of the fields below with
// name ending in _data is used to store the elements of the tensor.
// For float and complex64 values
// Complex64 tensors are encoded as a single array of floats,
// with the real components appearing in odd numbered positions,
// and the corresponding imaginary component appearing in the
// subsequent even numbered position. (e.g., [1.0 + 2.0i, 3.0 + 4.0i]
// is encoded as [1.0, 2.0 ,3.0 ,4.0]
// When this field is present, the data_type field MUST be FLOAT or COMPLEX64.
repeated float float_data = 4 [packed = true];
// For int32, uint8, int8, uint16, int16, bool, and float16 values
// float16 values must be bit-wise converted to an uint16_t prior
// to writing to the buffer.
// When this field is present, the data_type field MUST be
// INT32, INT16, INT8, UINT16, UINT8, BOOL, or FLOAT16
repeated int32 int32_data = 5 [packed = true];
// For strings.
// Each element of string_data is a UTF-8 encoded Unicode
// string. No trailing null, no leading BOM. The protobuf "string"
// scalar type is not used to match ML community conventions.
// When this field is present, the data_type field MUST be STRING
repeated bytes string_data = 6;
// For int64.
// When this field is present, the data_type field MUST be INT64
repeated int64 int64_data = 7 [packed = true];
// Optionally, a name for the tensor.
string name = 8; // namespace Value
// A human-readable documentation for this tensor. Markdown is allowed.
string doc_string = 12;
// Serializations can either use one of the fields above, or use this
// raw bytes field. The only exception is the string case, where one is
// required to store the content in the repeated bytes string_data field.
//
// When this raw_data field is used to store tensor value, elements MUST
// be stored in as fixed-width, little-endian order.
// Floating-point data types MUST be stored in IEEE 754 format.
// Complex64 elements must be written as two consecutive FLOAT values, real component first.
// Complex128 elements must be written as two consecutive DOUBLE values, real component first.
// Boolean type MUST be written one byte per tensor element (00000001 for true, 00000000 for false).
//
// Note: the advantage of specific field rather than the raw_data field is
// that in some cases (e.g. int data), protobuf does a better packing via
// variable length storage, and may lead to smaller binary footprint.
// When this field is present, the data_type field MUST NOT be STRING or UNDEFINED
bytes raw_data = 9;
// Data can be stored inside the protobuf file using type-specific fields or raw_data.
// Alternatively, raw bytes data can be stored in an external file, using the external_data field.
// external_data stores key-value pairs describing data location. Recognized keys are:
// - "location" (required) - POSIX filesystem path relative to the directory where the ONNX
// protobuf model was stored
// - "offset" (optional) - position of byte at which stored data begins. Integer stored as string.
// Offset values SHOULD be multiples 4096 (page size) to enable mmap support.
// - "length" (optional) - number of bytes containing data. Integer stored as string.
// - "checksum" (optional) - SHA1 digest of file specified in under 'location' key.
repeated StringStringEntryProto external_data = 13;
// Location of the data for this tensor. MUST be one of:
// - DEFAULT - data stored inside the protobuf message. Data is stored in raw_data (if set) otherwise in type-specified field.
// - EXTERNAL - data stored in an external location as described by external_data field.
enum DataLocation {
DEFAULT = 0;
EXTERNAL = 1;
}
// If value not set, data is stored in raw_data (if set) otherwise in type-specified field.
DataLocation data_location = 14;
// For double
// Complex128 tensors are encoded as a single array of doubles,
// with the real components appearing in odd numbered positions,
// and the corresponding imaginary component appearing in the
// subsequent even numbered position. (e.g., [1.0 + 2.0i, 3.0 + 4.0i]
// is encoded as [1.0, 2.0 ,3.0 ,4.0]
// When this field is present, the data_type field MUST be DOUBLE or COMPLEX128
repeated double double_data = 10 [packed = true];
// For uint64 and uint32 values
// When this field is present, the data_type field MUST be
// UINT32 or UINT64
repeated uint64 uint64_data = 11 [packed = true];
}
// A serialized sparse-tensor value
message SparseTensorProto {
// The sequence of non-default values are encoded as a tensor of shape [NNZ].
// The default-value is zero for numeric tensors, and empty-string for string tensors.
TensorProto values = 1;
// The indices of the non-default values, which may be stored in one of two formats.
// (a) Indices can be a tensor of shape [NNZ, rank] with the [i,j]-th value
// corresponding to the j-th index of the i-th value (in the values tensor).
// (b) Indices can be a tensor of shape [NNZ], in which case the i-th value
// must be the linearized-index of the i-th value (in the values tensor).
// The linearized-index can be converted into an index tuple (k_1,...,k_rank)
// using the shape provided below.
// The indices must appear in ascending order without duplication.
// In the first format, the ordering is lexicographic-ordering:
// e.g., index-value [1,4] must appear before [2,1]
TensorProto indices = 2;
// The shape of the underlying dense-tensor: [dim_1, dim_2, ... dim_rank]
repeated int64 dims = 3;
}
// Defines a tensor shape. A dimension can be either an integer value
// or a symbolic variable. A symbolic variable represents an unknown
// dimension.
message TensorShapeProto {
message Dimension {
oneof value {
int64 dim_value = 1;
string dim_param = 2; // namespace Shape
};
// Standard denotation can optionally be used to denote tensor
// dimensions with standard semantic descriptions to ensure
// that operations are applied to the correct axis of a tensor.
// Refer to https://github.com/onnx/onnx/blob/master/docs/DimensionDenotation.md#denotation-definition
// for pre-defined dimension denotations.
string denotation = 3;
};
repeated Dimension dim = 1;
}
// Types
//
// The standard ONNX data types.
message TypeProto {
message Tensor {
// This field MUST NOT have the value of UNDEFINED
// This field MUST have a valid TensorProto.DataType value
// This field MUST be present for this version of the IR.
int32 elem_type = 1;
TensorShapeProto shape = 2;
}
// repeated T
message Sequence {
// The type and optional shape of each element of the sequence.
// This field MUST be present for this version of the IR.
TypeProto elem_type = 1;
};
// map
message Map {
// This field MUST have a valid TensorProto.DataType value
// This field MUST be present for this version of the IR.
// This field MUST refer to an integral type ([U]INT{8|16|32|64}) or STRING
int32 key_type = 1;
// This field MUST be present for this version of the IR.
TypeProto value_type = 2;
};
message SparseTensor {
// This field MUST NOT have the value of UNDEFINED
// This field MUST have a valid TensorProto.DataType value
// This field MUST be present for this version of the IR.
int32 elem_type = 1;
TensorShapeProto shape = 2;
}
message Opaque {
// When missing, the domain is the same as the model's.
string domain = 1;
// The name is optional but significant when provided.
string name = 2;
// parameters that help defining the type
// DEPRECATED do not use.
// repeated TypeProto parameters = 3;
}
oneof value {
// The type of a tensor.
Tensor tensor_type = 1;
// NOTE: DNN-only implementations of ONNX MAY elect to not support non-tensor values
// as input and output to graphs and nodes. These types are needed to naturally
// support classical ML operators. DNN operators SHOULD restrict their input
// and output types to tensors.
// The type of a sequence.
Sequence sequence_type = 4;
// The type of a map.
Map map_type = 5;
SparseTensor sparse_tensor_type = 8;
Opaque opaque_type = 7;
}
// An optional denotation can be used to denote the whole
// type with a standard semantic description as to what is
// stored inside. Refer to https://github.com/onnx/onnx/blob/master/docs/TypeDenotation.md#type-denotation-definition
// for pre-defined type denotations.
string denotation = 6;
}
// Operator Sets
//
// OperatorSets are uniquely identified by a (domain, opset_version) pair.
message OperatorSetIdProto {
// The domain of the operator set being identified.
// The empty string ("") or absence of this field implies the operator
// set that is defined as part of the ONNX specification.
// This field MUST be present in this version of the IR when referring to any other operator set.
string domain = 1;
// The version of the operator set being identified.
// This field MUST be present in this version of the IR.
int64 version = 2;
}
// For using protobuf-lite
option optimize_for = LITE_RUNTIME;
onnx-1.7.0/onnx/defs/ 0000775 0000000 0000000 00000000000 13655345213 013113 5 ustar root root onnx-1.7.0/onnx/defs/function.h 0000664 0000000 0000000 00000005470 13655345213 015117 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#pragma once
#include
#include
#include
#include
#include "attr_proto_util.h"
#include "onnx/common/constants.h"
#include "onnx/common/status.h"
#include "onnx/onnx-operators_pb.h"
#include "tensor_proto_util.h"
namespace ONNX_NAMESPACE {
// Helper function to expand a function node given the function proto
void FunctionExpandHelper(
const NodeProto& node,
const FunctionProto& func,
GraphProto& g,
const std::string& node_prefix = "");
class FunctionBodyHelper {
public:
struct AttributeProtoWrapper {
AttributeProto proto;
AttributeProtoWrapper() {}
AttributeProtoWrapper(const AttributeProto& attr_prot) {
proto = attr_prot;
}
template
AttributeProtoWrapper(const std::string& attr_name, T value) {
proto = MakeAttribute(attr_name, value);
}
};
struct NodeDef {
NodeDef(
const std::vector& outputs,
const std::string& op_type,
const std::vector& inputs)
: outputs(outputs), op_type(op_type), inputs(inputs) {}
NodeDef(
const std::vector& outputs,
const std::string& op_type,
const std::vector& inputs,
const std::vector& attributes)
: outputs(outputs),
op_type(op_type),
inputs(inputs),
attributes(attributes) {}
std::vector outputs;
std::string op_type;
std::vector inputs;
std::vector attributes;
};
/*
BuildNodes() is an utility function for easily define a Function Body.
To build a simple node:
{{"Z"}, "Add", {"X", "Y"}} represents Z = Add(X,Y)
To build a node with attribute:
{{"Y"}, "Concat", {"X1", "X2", "X3"}, {{"axis", 1}}}
represents Y = Concat(X1,X2,X3) with axis = 1
The attribute type are infered from the attribute value's c++ type
Supported value types are
int64_t -> int, vector -> ints
float -> float, vector -> floats
string -> string, vector ->strings
For refering an attribute from parent, use:
{MakeRefAttribute("axes", AttributeProto::INTS)}}
For more examples, please find the references of this function
*/
static std::vector BuildNodes(
const std::vector& node_defs);
template
static NodeDef Const(const std::string& name, const T& value) {
return NodeDef{{name}, "Constant", {}, {{"value", ToTensor(value)}}};
}
template
static NodeDef Const(const std::string& name, const std::vector& values) {
return NodeDef{{name}, "Constant", {}, {{"value", ToTensor(values)}}};
}
};
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/defs/operator_sets_training.h 0000664 0000000 0000000 00000001021 13655345213 020042 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#pragma once
#include "onnx/defs/schema.h"
namespace ONNX_NAMESPACE {
// Declare training operators.
// Iterate over schema from ai.onnx.training version 1
class OpSet_OnnxTraining_ver1 {
public:
static void ForEachSchema(std::function /* fn */) {
}
};
// Register training operators.
inline void RegisterOnnxTrainingOperatorSetSchema() {
RegisterOpSetSchema();
}
} // namespace ONNX_NAMESPACE onnx-1.7.0/onnx/defs/reduction/ 0000775 0000000 0000000 00000000000 13655345213 015107 5 ustar root root onnx-1.7.0/onnx/defs/reduction/defs.cc 0000664 0000000 0000000 00000017631 13655345213 016347 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#include
#include
#include "onnx/defs/schema.h"
namespace ONNX_NAMESPACE {
std::vector GetSupportedDataTypesForReductionOps(
bool supports8bit) {
if (supports8bit) {
auto data_types = OpSchema::numeric_types_for_math_reduction();
data_types.push_back("tensor(uint8)");
data_types.push_back("tensor(int8)");
return data_types;
}
return OpSchema::numeric_types_for_math_reduction();
}
std::function ReduceDocGenerator(
const char* name,
bool supports_8bit_datatypes = false) {
return [=](OpSchema& schema) {
std::string doc;
POPULATE_OP_DOC_STR(doc = R"DOC(
Computes the {name} of the input tensor's element along the provided axes. The resulted
tensor has the same rank as the input if keepdims equal 1. If keepdims equal 0, then
the resulted tensor have the reduced dimension pruned.
The above behavior is similar to numpy, with the exception that numpy default keepdims to
False instead of True.)DOC";
ReplaceAll(doc, "{name}", name););
schema.SetDoc(doc.c_str());
schema.Attr(
"axes",
"A list of integers, along which to reduce. The default is to reduce over "
"all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data).",
AttributeProto::INTS,
OPTIONAL_VALUE);
schema.Attr(
"keepdims",
"Keep the reduced dimension or not, default 1 mean keep reduced dimension.",
AttributeProto::INT,
static_cast(1));
schema.Input(0, "data", "An input tensor.", "T");
schema.Output(0, "reduced", "Reduced output tensor.", "T");
schema.TypeConstraint(
"T",
GetSupportedDataTypesForReductionOps(supports_8bit_datatypes),
supports_8bit_datatypes
? "Constrain input and output types to high-precision and 8 bit numeric tensors."
: "Constrain input and output types to high-precision numeric tensors.");
schema.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
int64_t keep_dims = 1;
auto attr_proto = ctx.getAttribute("keepdims");
if (attr_proto) {
keep_dims = attr_proto->i();
}
auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
int64_t input_ndim = input_shape.dim_size();
auto output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
std::vector axes;
auto axes_proto = ctx.getAttribute("axes");
if (axes_proto)
axes.assign(axes_proto->ints().begin(), axes_proto->ints().end());
for (size_t i = 0; i < axes.size(); ++i) {
if (axes[i] < -input_ndim || axes[i] >= input_ndim) {
fail_shape_inference(
"axis must be in [-rank, rank-1]. input rank was ", input_ndim);
}
if (axes[i] < 0)
axes[i] += input_ndim;
}
// do we need handle negative axis?
for (int i = 0; i < input_ndim; ++i) {
// axes empty means reduce all dim
if (!axes.empty() &&
std::find(axes.begin(), axes.end(), i) == axes.end()) {
auto dim = output_shape->add_dim();
dim->CopyFrom(input_shape.dim(i));
} else {
if (keep_dims == 1) {
auto dim = output_shape->add_dim();
dim->set_dim_value(1);
}
}
}
});
};
}
ONNX_OPERATOR_SET_SCHEMA(
ReduceMax,
12,
OpSchema().FillUsing(ReduceDocGenerator("max", true)));
ONNX_OPERATOR_SET_SCHEMA(
ReduceMin,
12,
OpSchema().FillUsing(ReduceDocGenerator("min", true)));
ONNX_OPERATOR_SET_SCHEMA(
ReduceSum,
11,
OpSchema().FillUsing(ReduceDocGenerator("sum")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceSumSquare,
11,
OpSchema().FillUsing(ReduceDocGenerator("sum square")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceMean,
11,
OpSchema().FillUsing(ReduceDocGenerator("mean")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceProd,
11,
OpSchema().FillUsing(ReduceDocGenerator("product")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceLogSum,
11,
OpSchema().FillUsing(ReduceDocGenerator("log sum")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceLogSumExp,
11,
OpSchema().FillUsing(ReduceDocGenerator("log sum exponent")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceL1,
11,
OpSchema().FillUsing(ReduceDocGenerator("L1 norm")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceL2,
11,
OpSchema().FillUsing(ReduceDocGenerator("L2 norm")));
std::function ArgReduceDocGenerator(const char* name) {
return [=](OpSchema& schema) {
std::string doc;
POPULATE_OP_DOC_STR(doc = R"DOC(
Computes the indices of the {name} elements of the input tensor's element along the
provided axis. The resulting tensor has the same rank as the input if keepdims equal 1.
If keepdims equal 0, then the resulting tensor have the reduced dimension pruned.
If select_last_index is True (default False), the index of the last occurrence of the {name}
is selected if the {name} appears more than once in the input. Otherwise the index of the
first occurrence is selected.
The type of the output tensor is integer.)DOC";
ReplaceAll(doc, "{name}", name););
schema.SetDoc(doc.c_str());
schema.Attr(
"axis",
"The axis in which to compute the arg indices. Accepted range is [-r, r-1] where r = rank(data).",
AttributeProto::INT,
static_cast(0));
schema.Attr(
"keepdims",
"Keep the reduced dimension or not, default 1 mean keep reduced dimension.",
AttributeProto::INT,
static_cast(1));
schema.Attr(
"select_last_index",
"Whether to select the last index or the first index if the {name} appears in multiple indices, default is False (first index).",
AttributeProto::INT,
static_cast(0));
schema.Input(0, "data", "An input tensor.", "T");
schema.Output(
0,
"reduced",
"Reduced output tensor with integer data type.",
"tensor(int64)");
schema.TypeConstraint(
"T",
OpSchema::all_numeric_types(),
"Constrain input and output types to all numeric tensors.");
schema.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// set output element type to int64
updateOutputElemType(ctx, 0, TensorProto_DataType_INT64);
if (!hasNInputShapes(ctx, 1)) {
return;
}
auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
auto output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
int64_t input_ndim = input_shape.dim_size();
int64_t axis = 0; // default to 0
auto axis_proto = ctx.getAttribute("axis");
if (axis_proto) {
axis = axis_proto->i();
if (axis < -input_ndim || axis >= input_ndim) {
fail_shape_inference(
"'axis' must be in [-rank(indices), rank(indices)-1]");
}
if (axis < 0)
axis += input_ndim;
}
int64_t keep_dims = 1;
auto attr_proto = ctx.getAttribute("keepdims");
if (attr_proto) {
keep_dims = attr_proto->i();
}
// do we need handle negative axis?
for (int i = 0; i < input_ndim; ++i) {
if (i != axis) {
auto dim = output_shape->add_dim();
dim->CopyFrom(input_shape.dim(i));
} else {
if (keep_dims == 1) {
auto dim = output_shape->add_dim();
dim->set_dim_value(1);
}
}
}
});
};
} // namespace ONNX_NAMESPACE
ONNX_OPERATOR_SET_SCHEMA(
ArgMax,
12,
OpSchema().FillUsing(ArgReduceDocGenerator("max")));
ONNX_OPERATOR_SET_SCHEMA(
ArgMin,
12,
OpSchema().FillUsing(ArgReduceDocGenerator("min")));
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/defs/reduction/old.cc 0000664 0000000 0000000 00000023464 13655345213 016205 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#include
#include
#include "onnx/defs/schema.h"
namespace ONNX_NAMESPACE {
std::function ReduceDocGenerator_opset1(
const char* name,
int opset = 1) {
return [=](OpSchema& schema) {
std::string doc;
POPULATE_OP_DOC_STR(doc = R"DOC(
Computes the {name} of the input tensor's element along the provided axes. The resulted
tensor has the same rank as the input if keepdims equal 1. If keepdims equal 0, then
the resulted tensor have the reduced dimension pruned.
The above behavior is similar to numpy, with the exception that numpy default keepdims to
False instead of True.)DOC";
ReplaceAll(doc, "{name}", name););
schema.SetDoc(doc.c_str());
schema.Attr(
"axes",
opset >= 11
? "A list of integers, along which to reduce. The default is to reduce over "
"all the dimensions of the input tensor. Accepted range is [-r, r-1] where r = rank(data)."
: "A list of integers, along which to reduce. The default is to reduce over "
"all the dimensions of the input tensor.",
AttributeProto::INTS,
OPTIONAL_VALUE);
schema.Attr(
"keepdims",
"Keep the reduced dimension or not, default 1 mean keep reduced dimension.",
AttributeProto::INT,
static_cast(1));
schema.Input(0, "data", "An input tensor.", "T");
schema.Output(0, "reduced", "Reduced output tensor.", "T");
schema.TypeConstraint(
"T",
OpSchema::numeric_types_for_math_reduction(),
"Constrain input and output types to high-precision numeric tensors.");
schema.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
int64_t keep_dims = 1;
auto attr_proto = ctx.getAttribute("keepdims");
if (attr_proto) {
keep_dims = attr_proto->i();
}
auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
int64_t input_ndim = input_shape.dim_size();
auto output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
std::vector axes;
auto axes_proto = ctx.getAttribute("axes");
if (axes_proto)
axes.assign(axes_proto->ints().begin(), axes_proto->ints().end());
for (size_t i = 0; i < axes.size(); ++i) {
if (axes[i] < 0)
axes[i] += input_ndim;
}
// do we need handle negative axis?
for (int i = 0; i < input_ndim; ++i) {
// axes empty means reduce all dim
if (!axes.empty() &&
std::find(axes.begin(), axes.end(), i) == axes.end()) {
auto dim = output_shape->add_dim();
dim->CopyFrom(input_shape.dim(i));
} else {
if (keep_dims == 1) {
auto dim = output_shape->add_dim();
dim->set_dim_value(1);
}
}
}
});
};
}
ONNX_OPERATOR_SET_SCHEMA(
ReduceMax,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("max")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceMin,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("min")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceSum,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("sum")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceSumSquare,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("sum square")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceMean,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("mean")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceProd,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("product")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceLogSum,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("log sum")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceLogSumExp,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("log sum exponent")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceL1,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("L1 norm")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceL2,
1,
OpSchema().FillUsing(ReduceDocGenerator_opset1("L2 norm")));
ONNX_OPERATOR_SET_SCHEMA(
ReduceMax,
11,
OpSchema().FillUsing(ReduceDocGenerator_opset1("max", 11)));
ONNX_OPERATOR_SET_SCHEMA(
ReduceMin,
11,
OpSchema().FillUsing(ReduceDocGenerator_opset1("min", 11)));
std::function ArgReduceDocGenerator_opset1(const char* name) {
return [=](OpSchema& schema) {
std::string doc;
POPULATE_OP_DOC_STR(doc = R"DOC(
Computes the indices of the {name} elements of the input tensor's element along the
provided axis. The resulted tensor has the same rank as the input if keepdims equal 1.
If keepdims equal 0, then the resulted tensor have the reduced dimension pruned.
The type of the output tensor is integer.)DOC";
ReplaceAll(doc, "{name}", name););
schema.SetDoc(doc.c_str());
schema.Attr(
"axis",
"The axis in which to compute the arg indices.",
AttributeProto::INT,
static_cast(0));
schema.Attr(
"keepdims",
"Keep the reduced dimension or not, default 1 mean keep reduced dimension.",
AttributeProto::INT,
static_cast(1));
schema.Input(0, "data", "An input tensor.", "T");
schema.Output(
0,
"reduced",
"Reduced output tensor with integer data type.",
"tensor(int64)");
schema.TypeConstraint(
"T",
OpSchema::all_numeric_types(),
"Constrain input and output types to all numeric tensors.");
schema.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// set output element type to int64
updateOutputElemType(ctx, 0, TensorProto_DataType_INT64);
if (!hasNInputShapes(ctx, 1)) {
return;
}
auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
auto output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
int64_t input_ndim = input_shape.dim_size();
int64_t axis = 0; // default to 0
auto axis_proto = ctx.getAttribute("axis");
if (axis_proto) {
axis = axis_proto->i();
if (axis < 0)
axis += input_ndim;
}
int64_t keep_dims = 1;
auto attr_proto = ctx.getAttribute("keepdims");
if (attr_proto) {
keep_dims = attr_proto->i();
}
// do we need handle negative axis?
for (int i = 0; i < input_ndim; ++i) {
if (i != axis) {
auto dim = output_shape->add_dim();
dim->CopyFrom(input_shape.dim(i));
} else {
if (keep_dims == 1) {
auto dim = output_shape->add_dim();
dim->set_dim_value(1);
}
}
}
});
};
} // namespace ONNX_NAMESPACE
ONNX_OPERATOR_SET_SCHEMA(
ArgMax,
1,
OpSchema().FillUsing(ArgReduceDocGenerator_opset1("max")));
ONNX_OPERATOR_SET_SCHEMA(
ArgMin,
1,
OpSchema().FillUsing(ArgReduceDocGenerator_opset1("min")));
std::function ArgReduceDocGenerator_opset11(const char* name) {
return [=](OpSchema& schema) {
std::string doc = R"DOC(
Computes the indices of the {name} elements of the input tensor's element along the
provided axis. The resulting tensor has the same rank as the input if keepdims equal 1.
If keepdims equal 0, then the resulting tensor have the reduced dimension pruned.
The type of the output tensor is integer.)DOC";
ReplaceAll(doc, "{name}", name);
schema.SetDoc(doc.c_str());
schema.Attr(
"axis",
"The axis in which to compute the arg indices. Accepted range is [-r, r-1] where r = rank(data).",
AttributeProto::INT,
static_cast(0));
schema.Attr(
"keepdims",
"Keep the reduced dimension or not, default 1 mean keep reduced dimension.",
AttributeProto::INT,
static_cast(1));
schema.Input(0, "data", "An input tensor.", "T");
schema.Output(
0,
"reduced",
"Reduced output tensor with integer data type.",
"tensor(int64)");
schema.TypeConstraint(
"T",
OpSchema::all_numeric_types(),
"Constrain input and output types to all numeric tensors.");
schema.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// set output element type to int64
updateOutputElemType(ctx, 0, TensorProto_DataType_INT64);
if (!hasNInputShapes(ctx, 1)) {
return;
}
auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
auto output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
int64_t input_ndim = input_shape.dim_size();
int64_t axis = 0; // default to 0
auto axis_proto = ctx.getAttribute("axis");
if (axis_proto) {
axis = axis_proto->i();
if (axis < -input_ndim || axis >= input_ndim) {
fail_shape_inference(
"'axis' must be in [-rank(indices), rank(indices)-1]");
}
if (axis < 0)
axis += input_ndim;
}
int64_t keep_dims = 1;
auto attr_proto = ctx.getAttribute("keepdims");
if (attr_proto) {
keep_dims = attr_proto->i();
}
// do we need handle negative axis?
for (int i = 0; i < input_ndim; ++i) {
if (i != axis) {
auto dim = output_shape->add_dim();
dim->CopyFrom(input_shape.dim(i));
} else {
if (keep_dims == 1) {
auto dim = output_shape->add_dim();
dim->set_dim_value(1);
}
}
}
});
};
} // namespace ONNX_NAMESPACE
ONNX_OPERATOR_SET_SCHEMA(
ArgMax,
11,
OpSchema().FillUsing(ArgReduceDocGenerator_opset11("max")));
ONNX_OPERATOR_SET_SCHEMA(
ArgMin,
11,
OpSchema().FillUsing(ArgReduceDocGenerator_opset11("min")));
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/defs/object_detection/ 0000775 0000000 0000000 00000000000 13655345213 016417 5 ustar root root onnx-1.7.0/onnx/defs/object_detection/old.cc 0000664 0000000 0000000 00000006741 13655345213 017514 0 ustar root root // Copyright (c) Facebook Inc. and Microsoft Corporation.
// Licensed under the MIT license.
#include "onnx/defs/schema.h"
using namespace ONNX_NAMESPACE;
namespace ONNX_NAMESPACE {
static const char* NonMaxSuppression_doc = R"DOC(
Filter out boxes that have high intersection-over-union (IOU) overlap with previously selected boxes.
Bounding boxes with score less than score_threshold are removed. Bounding box format is indicated by attribute center_point_box.
Note that this algorithm is agnostic to where the origin is in the coordinate system and more generally is invariant to
orthogonal transformations and translations of the coordinate system; thus translating or reflections of the coordinate system
result in the same boxes being selected by the algorithm.
The selected_indices output is a set of integers indexing into the input collection of bounding boxes representing the selected boxes.
The bounding box coordinates corresponding to the selected indices can then be obtained using the Gather or GatherND operation.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
NonMaxSuppression,
10,
OpSchema()
.Input(
0,
"boxes",
"An input tensor with shape [num_batches, spatial_dimension, 4]. The single box data format is indicated by center_point_box.",
"tensor(float)")
.Input(
1,
"scores",
"An input tensor with shape [num_batches, num_classes, spatial_dimension]",
"tensor(float)")
.Input(
2,
"max_output_boxes_per_class",
"Integer representing the maximum number of boxes to be selected per batch per class. It is a scalar. Default to 0, which means no output.",
"tensor(int64)",
OpSchema::Optional)
.Input(
3,
"iou_threshold",
"Float representing the threshold for deciding whether boxes overlap too much with respect to IOU. It is scalar. Value range [0, 1]. Default to 0.",
"tensor(float)",
OpSchema::Optional)
.Input(
4,
"score_threshold",
"Float representing the threshold for deciding when to remove boxes based on score. It is a scalar.",
"tensor(float)",
OpSchema::Optional)
.Output(
0,
"selected_indices",
"selected indices from the boxes tensor. [num_selected_indices, 3], the selected index format is [batch_index, class_index, box_index].",
"tensor(int64)")
.Attr(
"center_point_box",
"Integer indicate the format of the box data. The default is 0. "
"0 - the box data is supplied as [y1, x1, y2, x2] where (y1, x1) and (y2, x2) are the coordinates of any diagonal pair of box corners "
"and the coordinates can be provided as normalized (i.e., lying in the interval [0, 1]) or absolute. Mostly used for TF models. "
"1 - the box data is supplied as [x_center, y_center, width, height]. Mostly used for Pytorch models.",
AttributeProto::INT,
static_cast(0))
.SetDoc(NonMaxSuppression_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
auto selected_indices_type =
ctx.getOutputType(0)->mutable_tensor_type();
selected_indices_type->set_elem_type(
::ONNX_NAMESPACE::TensorProto_DataType::
TensorProto_DataType_INT64);
}));
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/defs/object_detection/defs.cc 0000664 0000000 0000000 00000021621 13655345213 017651 0 ustar root root // Copyright (c) Facebook Inc. and Microsoft Corporation.
// Licensed under the MIT license.
#include "onnx/defs/schema.h"
using namespace ONNX_NAMESPACE;
namespace ONNX_NAMESPACE {
static const char* RoiAlign_ver1_doc = R"DOC(
Region of Interest (RoI) align operation described in the
[Mask R-CNN paper](https://arxiv.org/abs/1703.06870).
RoiAlign consumes an input tensor X and region of interests (rois)
to apply pooling across each RoI; it produces a 4-D tensor of shape
(num_rois, C, output_height, output_width).
RoiAlign is proposed to avoid the misalignment by removing
quantizations while converting from original image into feature
map and from feature map into RoI feature; in each ROI bin,
the value of the sampled locations are computed directly
through bilinear interpolation.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
RoiAlign,
10,
OpSchema()
.SetDoc(RoiAlign_ver1_doc)
.Attr(
"spatial_scale",
"Multiplicative spatial scale factor to translate ROI coordinates "
"from their input spatial scale to the scale used when pooling, "
"i.e., spatial scale of the input feature map X relative to the "
"input image. E.g.; default is 1.0f. ",
AttributeProto::FLOAT,
1.f)
.Attr(
"output_height",
"default 1; Pooled output Y's height.",
AttributeProto::INT,
static_cast(1))
.Attr(
"output_width",
"default 1; Pooled output Y's width.",
AttributeProto::INT,
static_cast(1))
.Attr(
"sampling_ratio",
"Number of sampling points in the interpolation grid used to compute "
"the output value of each pooled output bin. If > 0, then exactly "
"sampling_ratio x sampling_ratio grid points are used. If == 0, then "
"an adaptive number of grid points are used (computed as "
"ceil(roi_width / output_width), and likewise for height). Default is 0.",
AttributeProto::INT,
static_cast(0))
.Attr(
"mode",
"The pooling method. Two modes are supported: 'avg' and 'max'. "
"Default is 'avg'.",
AttributeProto::STRING,
std::string("avg"))
.Input(
0,
"X",
"Input data tensor from the previous operator; "
"4-D feature map of shape (N, C, H, W), "
"where N is the batch size, C is the number of channels, "
"and H and W are the height and the width of the data.",
"T1")
.Input(
1,
"rois",
"RoIs (Regions of Interest) to pool over; rois is "
"2-D input of shape (num_rois, 4) given as "
"[[x1, y1, x2, y2], ...]. "
"The RoIs' coordinates are in the coordinate system of the input image. "
"Each coordinate set has a 1:1 correspondence with the 'batch_indices' input.",
"T1")
.Input(
2,
"batch_indices",
"1-D tensor of shape (num_rois,) with each element denoting "
"the index of the corresponding image in the batch.",
"T2")
.Output(
0,
"Y",
"RoI pooled output, 4-D tensor of shape "
"(num_rois, C, output_height, output_width). The r-th batch element Y[r-1] "
"is a pooled feature map corresponding to the r-th RoI X[r-1].",
"T1")
.TypeConstraint(
"T1",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain types to float tensors.")
.TypeConstraint(
"T2",
{"tensor(int64)"},
"Constrain types to int tensors.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
size_t input_param = 0, rois_param = 1, batch_index_param = 2;
checkInputRank(ctx, input_param, 4);
checkInputRank(ctx, rois_param, 2);
checkInputRank(ctx, batch_index_param, 1);
// Output dimensions, initialized to an unknown-dimension-value
Dim num_rois, C, ht, width;
// Get value of C from dim 1 of input_param, if available
unifyInputDim(ctx, input_param, 1, C);
// Get value of num_rois from dim 0 of rois_param, if available
unifyInputDim(ctx, rois_param, 0, num_rois);
// ... or from dim 0 of batch_index_param, if available
unifyInputDim(ctx, batch_index_param, 0, num_rois);
// Get height from attribute, using default-value of 1
unifyDim(ht, getAttribute(ctx, "output_height", 1));
// Get width from attribute, using default-value of 1
unifyDim(width, getAttribute(ctx, "output_width", 1));
// set output shape:
updateOutputShape(ctx, 0, {num_rois, C, ht, width});
}));
static const char* NonMaxSuppression_doc = R"DOC(
Filter out boxes that have high intersection-over-union (IOU) overlap with previously selected boxes.
Bounding boxes with score less than score_threshold are removed. Bounding box format is indicated by attribute center_point_box.
Note that this algorithm is agnostic to where the origin is in the coordinate system and more generally is invariant to
orthogonal transformations and translations of the coordinate system; thus translating or reflections of the coordinate system
result in the same boxes being selected by the algorithm.
The selected_indices output is a set of integers indexing into the input collection of bounding boxes representing the selected boxes.
The bounding box coordinates corresponding to the selected indices can then be obtained using the Gather or GatherND operation.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
NonMaxSuppression,
11,
OpSchema()
.Input(
0,
"boxes",
"An input tensor with shape [num_batches, spatial_dimension, 4]. The single box data format is indicated by center_point_box.",
"tensor(float)")
.Input(
1,
"scores",
"An input tensor with shape [num_batches, num_classes, spatial_dimension]",
"tensor(float)")
.Input(
2,
"max_output_boxes_per_class",
"Integer representing the maximum number of boxes to be selected per batch per class. It is a scalar. Default to 0, which means no output.",
"tensor(int64)",
OpSchema::Optional)
.Input(
3,
"iou_threshold",
"Float representing the threshold for deciding whether boxes overlap too much with respect to IOU. It is scalar. Value range [0, 1]. Default to 0.",
"tensor(float)",
OpSchema::Optional)
.Input(
4,
"score_threshold",
"Float representing the threshold for deciding when to remove boxes based on score. It is a scalar.",
"tensor(float)",
OpSchema::Optional)
.Output(
0,
"selected_indices",
"selected indices from the boxes tensor. [num_selected_indices, 3], the selected index format is [batch_index, class_index, box_index].",
"tensor(int64)")
.Attr(
"center_point_box",
"Integer indicate the format of the box data. The default is 0. "
"0 - the box data is supplied as [y1, x1, y2, x2] where (y1, x1) and (y2, x2) are the coordinates of any diagonal pair of box corners "
"and the coordinates can be provided as normalized (i.e., lying in the interval [0, 1]) or absolute. Mostly used for TF models. "
"1 - the box data is supplied as [x_center, y_center, width, height]. Mostly used for Pytorch models.",
AttributeProto::INT,
static_cast(0))
.SetDoc(NonMaxSuppression_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Type inference - Output is always of type INT64
auto* selected_indices_type =
ctx.getOutputType(0)->mutable_tensor_type();
selected_indices_type->set_elem_type(
TensorProto_DataType::TensorProto_DataType_INT64);
// Shape inference
// The exact shape cannot be determined as it depends on the input and
// other input configurations for the op But part of the shape can be
// established
auto* selected_indices_shape = getOutputShape(ctx, 0);
selected_indices_shape->clear_dim();
// Output is 2D always
// The value of the first dim is determined by input data
// hence its value cannot be determined statically
selected_indices_shape->add_dim();
// The value of the second dim is 3
selected_indices_shape->add_dim()->set_dim_value(3);
}));
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/defs/attr_proto_util.h 0000664 0000000 0000000 00000003626 13655345213 016525 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#pragma once
#include "onnx/onnx-operators_pb.h"
namespace ONNX_NAMESPACE {
AttributeProto MakeAttribute(const std::string& attr_name, const float& value);
AttributeProto MakeAttribute(
const std::string& attr_name,
const int64_t& value);
AttributeProto MakeAttribute(
const std::string& attr_name,
const std::string& value);
AttributeProto MakeAttribute(
const std::string& attr_name,
const TensorProto& value);
AttributeProto MakeAttribute(
const std::string& attr_name,
const GraphProto& value);
AttributeProto MakeAttribute(
const std::string& attr_name,
const std::vector& values);
AttributeProto MakeAttribute(
const std::string& attr_name,
const std::vector& values);
AttributeProto MakeAttribute(
const std::string& attr_name,
const std::vector& values);
AttributeProto MakeAttribute(
const std::string& attr_name,
const std::vector& values);
AttributeProto MakeAttribute(
const std::string& attr_name,
const std::vector& values);
// Make a "reference" attribute for a node in a function body.
// specifies the attribute name of both the function node and its
// function body node. They're using the same attribute name.
// specifies the attribute type.
AttributeProto MakeRefAttribute(
const std::string& attr_name,
AttributeProto_AttributeType type);
// Make a "reference" attribute for a node in a function body.
// specifies the attribute name of the function body node.
// specifies the referred attribute name of the function
// node.
// specifies the attribute type.
AttributeProto MakeRefAttribute(
const std::string& attr_name,
const std::string& referred_attr_name,
AttributeProto_AttributeType type);
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/defs/tensor/ 0000775 0000000 0000000 00000000000 13655345213 014425 5 ustar root root onnx-1.7.0/onnx/defs/tensor/utils.h 0000664 0000000 0000000 00000001731 13655345213 015740 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#pragma once
#include "onnx/defs/schema.h"
#include "onnx/defs/tensor_proto_util.h"
#include
namespace ONNX_NAMESPACE {
// The below is called by ops after opset 11, inclusively.
void resizeShapeInference(InferenceContext& ctx, bool is_resize_op);
void resizeShapeInferenceHelper(
const TensorShapeProto& input_shape,
const std::vector& scales_data,
TensorShapeProto* output_shape);
void resizeShapeInferenceHelper(
const TensorShapeProto& input_shape,
const std::vector& sizes_data,
TensorShapeProto* output_shape);
// The below is called by ops between opset 7 and opset 10, inclusively.
void resizeShapeInference_opset7_to_10(InferenceContext& ctx);
void resizeShapeInferenceHelper_opset7_to_10(
const TensorShapeProto& input_shape,
const std::vector& scales_data,
TensorShapeProto* output_shape);
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/defs/tensor/old.cc 0000664 0000000 0000000 00000201317 13655345213 015516 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#include
#include
#include
#include "onnx/defs/tensor/utils.h"
namespace ONNX_NAMESPACE {
static const char* Cast_ver1_doc = R"DOC(
The operator casts the elements of a given input tensor to a data type
specified by the 'to' argument and returns an output tensor of the same size in
the converted type. The 'to' argument must be one of the data types specified
in the 'DataType' enum field in the TensorProto message.
NOTE: Casting to and from strings is not supported yet.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Cast,
1,
OpSchema()
.SetDoc(Cast_ver1_doc)
.Attr(
"to",
"The data type to which the elements of the input tensor are cast. "
"Strictly must be one of the types from DataType enum in TensorProto",
AttributeProto::STRING)
.Input(0, "input", "Input tensor to be cast.", "T1")
.Output(
0,
"output",
"Output tensor with the same shape as input with type "
"specified by the 'to' argument",
"T2")
.TypeConstraint(
"T1",
{"tensor(float16)",
"tensor(float)",
"tensor(double)",
"tensor(int8)",
"tensor(int16)",
"tensor(int32)",
"tensor(int64)",
"tensor(uint8)",
"tensor(uint16)",
"tensor(uint32)",
"tensor(uint64)",
"tensor(bool)"},
"Constrain input types. Casting from strings and complex are not supported.")
.TypeConstraint(
"T2",
{"tensor(float16)",
"tensor(float)",
"tensor(double)",
"tensor(int8)",
"tensor(int16)",
"tensor(int32)",
"tensor(int64)",
"tensor(uint8)",
"tensor(uint16)",
"tensor(uint32)",
"tensor(uint64)",
"tensor(bool)"},
"Constrain output types. Casting to strings and complex are not supported."));
static const char* Cast_ver6_doc = R"DOC(
The operator casts the elements of a given input tensor to a data type
specified by the 'to' argument and returns an output tensor of the same size in
the converted type. The 'to' argument must be one of the data types specified
in the 'DataType' enum field in the TensorProto message.
NOTE: Casting to and from strings is not supported yet.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Cast,
6,
OpSchema()
.SetDoc(Cast_ver6_doc)
.Attr(
"to",
"The data type to which the elements of the input tensor are cast. "
"Strictly must be one of the types from DataType enum in TensorProto",
AttributeProto::INT)
.Input(0, "input", "Input tensor to be cast.", "T1")
.Output(
0,
"output",
"Output tensor with the same shape as input with type "
"specified by the 'to' argument",
"T2")
.TypeConstraint(
"T1",
{"tensor(float16)",
"tensor(float)",
"tensor(double)",
"tensor(int8)",
"tensor(int16)",
"tensor(int32)",
"tensor(int64)",
"tensor(uint8)",
"tensor(uint16)",
"tensor(uint32)",
"tensor(uint64)",
"tensor(bool)"},
"Constrain input types. Casting from strings and complex are not supported.")
.TypeConstraint(
"T2",
{"tensor(float16)",
"tensor(float)",
"tensor(double)",
"tensor(int8)",
"tensor(int16)",
"tensor(int32)",
"tensor(int64)",
"tensor(uint8)",
"tensor(uint16)",
"tensor(uint32)",
"tensor(uint64)",
"tensor(bool)"},
"Constrain output types. Casting to strings and complex are not supported.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromAttributeToOutput(ctx, "to", 0);
if (hasNInputShapes(ctx, 1)) {
propagateShapeFromInputToOutput(ctx, 0, 0);
}
}));
static const char* Concat_ver1_doc =
R"DOC(Concatenate a list of tensors into a single tensor)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Concat,
1,
OpSchema()
.Attr(
"axis",
"Which axis to concat on. Default value is 1.",
AttributeProto::INT,
OPTIONAL_VALUE)
.SetDoc(Concat_ver1_doc)
.Input(
0,
"inputs",
"List of tensors for concatenation",
"T",
OpSchema::Variadic)
.Output(0, "concat_result", "Concatenated tensor", "T")
.TypeConstraint(
"T",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain output types to float tensors."));
ONNX_OPERATOR_SET_SCHEMA(
Concat,
4,
OpSchema()
.Attr("axis", "Which axis to concat on", AttributeProto::INT)
.SetDoc("Concatenate a list of tensors into a single tensor")
.Input(
0,
"inputs",
"List of tensors for concatenation",
"T",
OpSchema::Variadic)
.Output(0, "concat_result", "Concatenated tensor", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain output types to any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
auto numInputs = ctx.getNumInputs();
if (numInputs < 1 ||
!hasNInputShapes(ctx, static_cast(numInputs))) {
return;
}
auto rank = ctx.getInputType(0)->tensor_type().shape().dim_size();
auto axisAttr = ctx.getAttribute("axis");
if (!axisAttr) {
fail_shape_inference("Required attribute axis is missing");
}
int axis = static_cast(axisAttr->i());
if (rank <= axis) {
fail_shape_inference("rank must be greater than axis");
}
if (axis < 0) {
return; // TODO: check if negative axis must be supported
}
bool all_lengths_known = true;
int total_length = 0;
auto* output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
for (int64_t i = 0; i < rank; ++i) {
output_shape->add_dim();
}
for (size_t i = 0; i < numInputs; i++) {
const auto& shape = ctx.getInputType(i)->tensor_type().shape();
if (shape.dim_size() != rank)
fail_shape_inference("All inputs to Concat must have same rank");
for (int j = 0; j < rank; j++) {
if (j == axis) {
if (shape.dim(j).has_dim_value()) {
total_length += static_cast(shape.dim(j).dim_value());
} else {
all_lengths_known = false;
}
} else {
auto& output_dim = *output_shape->mutable_dim(j);
const auto& input_dim = shape.dim(j);
mergeInDimensionInfo(input_dim, output_dim, j);
}
}
}
if (all_lengths_known) {
output_shape->mutable_dim(axis)->set_dim_value(total_length);
}
}));
static const char* Split_ver1_doc =
R"DOC(Split a tensor into a list of tensors, along the specified
'axis'. The lengths of the split can be specified using argument 'axis' or
optional second input blob to the operator. Otherwise, the tensor is split
to equal sized parts.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Split,
1,
OpSchema()
.Input(0, "input", "The tensor to split", "T")
.Input(
1,
"split",
"Optional list of output lengths (see also arg 'split')",
"T",
OpSchema::Optional)
.Output(
0,
"outputs...",
"One or more outputs forming list of tensors after splitting",
"T",
OpSchema::Variadic)
.TypeConstraint(
"T",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain input types to float tensors.")
.Attr("axis", "Which axis to split on", AttributeProto::INT, OPTIONAL_VALUE)
.Attr("split", "length of each output", AttributeProto::INTS, OPTIONAL_VALUE)
.SetDoc(Split_ver1_doc));
static const char* Pad_ver1_doc = R"DOC(
Given `data` tensor, paddings, mode, and value.
Example:
Insert 0 paddings to the beginning of the second dimension.
data = [
[1.0, 1.2],
[2.3, 3.4],
[4.5, 5.7],
]
paddings = [0, 0, 2, 0]
output = [
[
[0.0, 0.0, 1.0, 1.2],
[0.0, 0.0, 2.3, 3.4],
[0.0, 0.0, 4.5, 5.7],
],
]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Pad,
1,
OpSchema()
.Attr(
"paddings",
"List of integers indicate the padding element count at the "
"beginning and end of each axis, for 2D it is the number of pixel. "
"`paddings` rank should be double of the input's rank. `paddings` format should be as follow "
"[x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels "
"added at the beginning of axis `i` and xi_end, the number of pixels added at "
"the end of axis `i`.",
AttributeProto::INTS)
.Attr(
"mode",
"Three modes: constant(default), reflect, edge",
AttributeProto::STRING,
std::string("constant"))
.Attr(
"value",
"One float, indicates the value to be filled, default is 0",
AttributeProto::FLOAT,
0.0f)
.SetDoc(Pad_ver1_doc)
.Input(0, "data", "Input tensor.", "T")
.Output(0, "output", "Tensor after padding.", "T")
.TypeConstraint(
"T",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain input and output types to float tensors."));
static const char* Reshape_ver1_doc = R"DOC(
Reshape the input tensor similar to numpy.reshape.
It takes a tensor as input and an argument `shape`. It outputs the reshaped tensor.
At most one dimension of the new shape can be -1. In this case, the value is
inferred from the size of the tensor and the remaining dimensions. A dimension
could also be 0, in which case the actual dimension value is unchanged (i.e. taken
from the input tensor).)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Reshape,
1,
OpSchema()
.SetDoc(Reshape_ver1_doc)
.Attr("shape", "New shape", AttributeProto::INTS, OPTIONAL_VALUE)
// This attribute was added via AllowConsumed API in OpSchema.
// After removing the API, we're now using the Attr API to simulate the
// old definition.
.Attr(
"consumed_inputs",
"legacy optimization attribute.",
AttributeProto::INTS,
OPTIONAL_VALUE)
.Input(0, "data", "An input tensor.", "T")
.Output(0, "reshaped", "Reshaped data.", "T")
.TypeConstraint(
"T",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain input and output types to float tensors."));
static const char* Upsample_ver1_doc = R"DOC(
Upsample the input tensor.
The width and height of the output tensor are:
output_width = floor(input_width * width_scale),
output_height = floor(input_height * height_scale).
Example:
Given `data` tensor, width_scale, height_scale, mode,
Upsample the input 4-D tensor in nearest mode:
data = [[[
[1, 2],
[3, 4]
]]]
width_scale = 2
height_scale = 2
mode = "nearest"
output = [[[
[1, 1, 2, 2],
[1, 1, 2, 2],
[3, 3, 4, 4],
[3, 3, 4, 4]
]]]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Tile,
1,
OpSchema()
.SetDoc("Repeat the elements of a tensor along an axis.")
.Input(0, "input", "Input tensor of any shape.", "T")
.Input(
1,
"tiles",
"Number of repeated copies to make of the input tensor.",
"T")
.Input(2, "axis", "Axis along which to repeat.", "T")
.Output(
0,
"output",
"Output tensor of same shape and type as input.",
"T")
.TypeConstraint(
"T",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain input types to float tensors.")
.TypeConstraint(
"T1",
{"tensor(int64)"},
"Constrain tiles and axis's type to int64 tensors.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
// Only rank of output can be inferred. We can do better if second
// input is a constant, but this requires extending InferenceContext
// interface to get values of constant inputs.
}));
ONNX_OPERATOR_SET_SCHEMA(
Upsample,
1,
OpSchema()
.SetSupportLevel(OpSchema::SupportType::EXPERIMENTAL)
.Attr(
"width_scale",
"The scale along width dimension. It takes value greater than or equal to 1.",
AttributeProto::FLOAT)
.Attr(
"height_scale",
"The scale along height dimension. It takes value greater than or equal to 1.",
AttributeProto::FLOAT)
.Attr(
"mode",
"Two interpolation modes: nearest(default), bilinear",
AttributeProto::STRING,
std::string("nearest"))
.Input(0, "X", "4-D tensor, [N,C,H,W]", "T")
.Output(0, "Y", "4-D tensor after resizing, [N,C,H,W]", "T")
.TypeConstraint(
"T",
{"tensor(bool)",
"tensor(int32)",
"tensor(int64)",
"tensor(float16)",
"tensor(float)",
"tensor(double)"},
"Constrain output types to bool, int32, int64, float16, float, double tensors.")
.SetDoc(Upsample_ver1_doc));
static const char* Upsample_ver7_doc = R"DOC(
Upsample the input tensor.
Each dimension value of the output tensor is:
output_dimension = floor(input_dimension * scale).
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Upsample,
7,
OpSchema()
.Attr(
"scales",
"The scale array along each dimension. It takes value greater than or equal to 1."
" The number of elements of 'scales' should be the same as the rank of input 'X'.",
AttributeProto::FLOATS)
.Attr(
"mode",
"Two interpolation modes: nearest (default), and linear (including bilinear, trilinear, etc)",
AttributeProto::STRING,
std::string("nearest"))
.Input(0, "X", "N-D tensor", "T")
.Output(0, "Y", "N-D tensor after resizing", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.SetDoc(Upsample_ver7_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
if (!hasNInputShapes(ctx, 1)) {
return;
}
propagateElemTypeFromInputToOutput(ctx, 0, 0);
const auto& input_shape = getInputShape(ctx, 0);
auto* output_shape = getOutputShape(ctx, 0);
const auto* scales = ctx.getAttribute("scales");
if (output_shape->dim_size() > 0) {
if (output_shape->dim_size() != input_shape.dim_size()) {
fail_shape_inference(
"Ranks inferred (",
input_shape.dim_size(),
") is not equal to the existing rank value (",
output_shape->dim_size(),
").");
}
} else { // Infer the rank of output anyway
for (int i = 0; i < input_shape.dim_size(); ++i) {
output_shape->add_dim();
}
}
if (nullptr != scales) {
// Infer output shape's dimension value if 'scales' is known.
if (scales->type() == AttributeProto_AttributeType_FLOATS) {
const std::vector scales_data(
scales->floats().begin(), scales->floats().end());
if (scales_data.size() !=
static_cast(input_shape.dim_size())) {
fail_shape_inference(
"Number of elements of attribute 'scales' must be same as rank of input 'X'");
}
resizeShapeInferenceHelper_opset7_to_10(
input_shape, scales_data, output_shape);
} else {
fail_shape_inference("Attribute 'scales' must have floats type.");
} // scales->type() == float
} else {
fail_shape_inference("Attribute 'scales' is required.");
} // nullptr != scales
}));
static const char* Upsample_ver9_doc = R"DOC(
Upsample the input tensor.
Each dimension value of the output tensor is:
output_dimension = floor(input_dimension * scale).
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Upsample,
9,
OpSchema()
.Attr(
"mode",
"Two interpolation modes: nearest (default), and linear (including bilinear, trilinear, etc)",
AttributeProto::STRING,
std::string("nearest"))
.Input(0, "X", "N-D tensor", "T")
.Input(
1,
"scales",
"The scale array along each dimension. It takes value greater than or equal to 1."
" The number of elements of 'scales' should be the same as the rank of input 'X'.",
"tensor(float)")
.Output(0, "Y", "N-D tensor after resizing", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input 'X' and output 'Y' to all tensor types.")
.SetDoc(Upsample_ver9_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
resizeShapeInference_opset7_to_10(ctx);
}));
static const char* Resize_ver10_doc = R"DOC(
Resize the input tensor.
Each dimension value of the output tensor is:
output_dimension = floor(input_dimension * scale).
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Resize,
10,
OpSchema()
.Attr(
"mode",
"Two interpolation modes: nearest (default), and linear (including bilinear, trilinear, etc)",
AttributeProto::STRING,
std::string("nearest"))
.Input(0, "X", "N-D tensor", "T")
.Input(
1,
"scales",
"The scale array along each dimension. It takes value greater than 0. If it's less than 1,"
" it's sampling down, otherwise, it's upsampling. The number of elements of 'scales' should"
" be the same as the rank of input 'X'.",
"tensor(float)")
.Output(0, "Y", "N-D tensor after resizing", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input 'X' and output 'Y' to all tensor types.")
.SetDoc(Resize_ver10_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
resizeShapeInference_opset7_to_10(ctx);
}));
static const char* Slice_ver1_doc = R"DOC(
Produces a slice of the input tensor along multiple axes. Similar to numpy:
https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html
Slices uses `axes`, `starts` and `ends` attributes to specify the start and end
dimension for each axis in the list of axes, it uses this information to
slice the input `data` tensor. If a negative value is passed for any of the
start or end indices, it represent number of elements before the end of that
dimension. If the value passed to start or end is larger than the `n` (the
number of elements in this dimension), it represents `n`. For slicing to the
end of a dimension with unknown size, it is recommended to pass in `INT_MAX`.
If `axes` are omitted, they are set to `[0, ..., ndim-1]`.
Example 1:
data = [
[1, 2, 3, 4],
[5, 6, 7, 8],
]
axes = [0, 1]
starts = [1, 0]
ends = [2, 3]
result = [
[5, 6, 7],
]
Example 2:
data = [
[1, 2, 3, 4],
[5, 6, 7, 8],
]
starts = [0, 1]
ends = [-1, 1000]
result = [
[2, 3, 4],
]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Slice,
1,
OpSchema()
.SetDoc(Slice_ver1_doc)
.Input(0, "data", "Tensor of data to extract slices from.", "T")
.Attr(
"axes",
"Axes that `starts` and `ends` apply to. "
"It's optional. If not present, will be treated as "
"[0, 1, ..., len(`starts`) - 1].",
AttributeProto::INTS,
OPTIONAL_VALUE)
.Attr(
"starts",
"Starting indices of corresponding axis in `axes`",
AttributeProto::INTS)
.Attr(
"ends",
"Ending indices (exclusive) of corresponding axis in axes`",
AttributeProto::INTS)
.Output(0, "output", "Sliced data tensor.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
std::vector starts;
std::vector ends;
if (!getRepeatedAttribute(ctx, "starts", starts) ||
!getRepeatedAttribute(ctx, "ends", ends) ||
starts.size() != ends.size()) {
fail_shape_inference(
"Incorrect or missing attribute value for starts and ends");
;
}
std::vector axes;
if (!getRepeatedAttribute(ctx, "axes", axes)) {
for (int i = 0; (size_t)i < starts.size(); ++i) {
axes.push_back(i);
}
} else if (axes.size() != starts.size()) {
fail_shape_inference("Attribute axes has incorrect length");
;
} else if (!std::is_sorted(axes.begin(), axes.end())) {
// TODO support shape inference for unsorted axes
return;
}
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
for (size_t i = 0, j = 0; (int64_t)i <
ctx.getInputType(0)->tensor_type().shape().dim_size();
++i) {
auto* newdim = ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim();
if (j < axes.size() && static_cast(axes[j]) == i) {
// There's a lot of potential behaviors. For now just
// handle some simple cases.
if (ctx.getInputType(0)
->tensor_type()
.shape()
.dim((int)i)
.has_dim_value() &&
starts[j] >= 0 && ends[j] >= 0) {
auto newval = std::min(
(int64_t)ctx.getInputType(0)
->tensor_type()
.shape()
.dim((int)i)
.dim_value(),
ends[j]) -
starts[j];
if (newval >= 0) {
newdim->set_dim_value(newval);
}
}
++j;
} else {
*newdim = ctx.getInputType(0)->tensor_type().shape().dim((int)i);
}
}
}));
static const char* Slice_ver10_doc = R"DOC(
Produces a slice of the input tensor along multiple axes. Similar to numpy:
https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html
Slices uses `starts`, `ends`, `axes` and `steps` inputs to specify the start and end
dimension and step for each axis in the list of axes, it uses this information to
slice the input `data` tensor. If a negative value is passed for any of the
start or end indices, it represent number of elements before the end of that
dimension. If the value passed to start or end is larger than the `n` (the
number of elements in this dimension), it represents `n`. For slicing to the
end of a dimension with unknown size, it is recommended to pass in `INT_MAX`.
If a negative value is passed for step, it represents slicing backward.
If `axes` are omitted, they are set to `[0, ..., ndim-1]`.
If `steps` are omitted, they are set to `[1, ..., 1]` of length `len(starts)`
Example 1:
data = [
[1, 2, 3, 4],
[5, 6, 7, 8],
]
axes = [0, 1]
starts = [1, 0]
ends = [2, 3]
steps = [1, 2]
result = [
[5, 7],
]
Example 2:
data = [
[1, 2, 3, 4],
[5, 6, 7, 8],
]
starts = [0, 1]
ends = [-1, 1000]
result = [
[2, 3, 4],
]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Slice,
10,
OpSchema()
.SetDoc(Slice_ver10_doc)
.Input(0, "data", "Tensor of data to extract slices from.", "T")
.Input(
1,
"starts",
"1-D tensor of starting indices of corresponding axis in `axes`",
"Tind")
.Input(
2,
"ends",
"1-D tensor of ending indices (exclusive) of corresponding axis in `axes`",
"Tind")
.Input(
3,
"axes",
"1-D tensor of axes that `starts` and `ends` apply to.",
"Tind",
OpSchema::Optional)
.Input(
4,
"steps",
"1-D tensor of slice step of corresponding axis in `axes`. Default to 1. ",
"Tind",
OpSchema::Optional)
.Output(0, "output", "Sliced data tensor.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeConstraint(
"Tind",
{"tensor(int32)", "tensor(int64)"},
"Constrain indices to integer types")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
size_t num_inputs = ctx.getNumInputs();
if (num_inputs != 3 && num_inputs != 4 && num_inputs != 5) {
fail_type_inference(
"Slice op must have either three, four or five inputs.");
}
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
// Shape Inference if
// 1. 2nd and 3rd input data (starts, ends) are available.
// and 2. 4th and 5th optional input (axes, steps) are either not set,
// or set and is initializer.
const TensorProto* startsInitializer = ctx.getInputData(1);
const TensorProto* endsInitializer = ctx.getInputData(2);
const TensorProto* axesInitializer =
hasInputShape(ctx, 3) ? ctx.getInputData(3) : nullptr;
const TensorProto* stepsInitializer =
hasInputShape(ctx, 4) ? ctx.getInputData(4) : nullptr;
if (!startsInitializer || !endsInitializer ||
(hasInputShape(ctx, 3) && !ctx.getInputData(3)) ||
(hasInputShape(ctx, 4) && !ctx.getInputData(4))) {
return;
}
// don't know data_type- can't proceed
if (!startsInitializer->has_data_type())
return;
auto get_initializer_data =
[](const TensorProto* initializer) -> std::vector {
std::vector vec;
if (initializer->data_type() == TensorProto::INT64) {
const auto& data = ParseData(initializer);
vec.insert(vec.end(), data.begin(), data.end());
} else if (initializer->data_type() == TensorProto::INT32) {
const auto& data = ParseData(initializer);
vec.insert(vec.end(), data.begin(), data.end());
} else {
// unaccepted data type
fail_shape_inference(
"Only supports `int32_t` or `int64_t` inputs for starts/ends/axes/steps");
}
return vec;
};
auto clamp = [](int64_t val, int64_t low, int64_t high) -> int64_t {
if (val < low)
return low;
if (val > high)
return high;
return val;
};
std::vector starts = get_initializer_data(startsInitializer);
std::vector ends = get_initializer_data(endsInitializer);
if (starts.size() != ends.size()) {
fail_shape_inference(
"Incorrect or missing input value for starts and ends");
}
const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
const auto input_rank = input_shape.dim_size();
std::vector axes(starts.size());
if (!axesInitializer) {
std::iota(axes.begin(), axes.end(), 0);
} else {
axes = get_initializer_data(axesInitializer);
if (axes.size() != starts.size()) {
fail_shape_inference("Input axes has incorrect length");
}
}
std::vector steps;
if (!stepsInitializer) {
steps = std::vector(starts.size(), 1);
} else {
steps = get_initializer_data(stepsInitializer);
if (steps.size() != axes.size()) {
fail_shape_inference("Input steps has incorrect length");
}
}
for (size_t i = 0; (int64_t)i < input_rank; ++i) {
// first update rank of output dim
auto* output_dim = ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim();
const auto& input_dim = input_shape.dim((int)i);
if (input_dim.has_dim_value()) {
output_dim->set_dim_value(input_dim.dim_value());
} else if (input_dim.has_dim_param()) {
output_dim->set_dim_param(input_dim.dim_param());
}
}
std::unordered_set unique_axes;
size_t axes_size = axes.size();
for (size_t axis_index = 0; axis_index < axes_size; ++axis_index) {
auto axis = axes[axis_index] < 0
? axes[axis_index] + static_cast(input_rank)
: axes[axis_index];
if (axis >= static_cast(input_rank) || axis < 0)
fail_shape_inference("Input axes has invalid data");
if (unique_axes.find(axis) != unique_axes.end())
fail_shape_inference("'axes' has duplicates");
unique_axes.insert(axis);
auto input_dim =
ctx.getInputType(0)->tensor_type().shape().dim((int)axis);
// input dim value is missing - cannot perform shape inference for
// this axis
if (!input_dim.has_dim_value())
continue;
const auto input_dim_value = input_dim.dim_value();
// process step
auto step = steps[axis_index];
if (step == 0)
fail_shape_inference("'step' cannot be 0");
// process start
auto start = starts[axis_index];
if (start < 0)
start += input_dim_value;
if (step < 0)
start = clamp(start, 0, input_dim_value - 1);
else
start = clamp(start, 0, input_dim_value);
// process end
auto end = ends[axis_index];
if (end < 0)
end += input_dim_value;
if (step < 0)
end = clamp(end, -1, input_dim_value);
else
end = clamp(end, 0, input_dim_value);
// find output dim value for this axis
auto temp = static_cast(ceil(1.0 * (end - start) / step));
if (temp < 0)
temp = 0;
// assign output value
ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->mutable_dim((int)axis)
->set_dim_value(temp);
}
}));
static const char* Scatter_ver9_doc = R"DOC(
Given `data`, `updates` and `indices` input tensors of rank r >= 1, write the values provided by `updates`
into the first input, `data`, along `axis` dimension of `data` (by default outer-most one as axis=0) at corresponding `indices`.
For each entry in `updates`, the target index in `data` is specified by corresponding entry in `indices`
for dimension = axis, and index in source for dimension != axis. For instance, in a 2-D tensor case,
data[indices[i][j]][j] = updates[i][j] if axis = 0, or data[i][indices[i][j]] = updates[i][j] if axis = 1,
where i and j are loop counters from 0 up to the respective size in `updates` - 1.
Example 1:
data = [
[0.0, 0.0, 0.0],
[0.0, 0.0, 0.0],
[0.0, 0.0, 0.0],
]
indices = [
[1, 0, 2],
[0, 2, 1],
]
updates = [
[1.0, 1.1, 1.2],
[2.0, 2.1, 2.2],
]
output = [
[2.0, 1.1, 0.0]
[1.0, 0.0, 2.2]
[0.0, 2.1, 1.2]
]
Example 2:
data = [[1.0, 2.0, 3.0, 4.0, 5.0]]
indices = [[1, 3]]
updates = [[1.1, 2.1]]
axis = 1
output = [[1.0, 1.1, 3.0, 2.1, 5.0]]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Scatter,
9,
OpSchema()
.SetDoc(Scatter_ver9_doc)
.Attr(
"axis",
"Which axis to scatter on. Negative value means "
"counting dimensions from the back. Accepted range is [-r, r-1]",
AttributeProto::INT,
static_cast(0))
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(
1,
"indices",
"Tensor of int32/int64 indices, of r >= 1 (same rank as input).",
"Tind")
.Input(
2,
"updates",
"Tensor of rank r >=1 (same rank and shape as indices)",
"T")
.Output(0, "output", "Tensor of rank r >= 1 (same rank as input).", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Input and output types can be of any tensor type.")
.TypeConstraint(
"Tind",
{"tensor(int32)", "tensor(int64)"},
"Constrain indices to integer types")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (hasNInputShapes(ctx, 1)) {
propagateShapeFromInputToOutput(ctx, 0, 0);
}
}));
static const char* DepthToSpace_ver1_doc =
R"DOC(DepthToSpace rearranges (permutes) data from depth into blocks of spatial data.
This is the reverse transformation of SpaceToDepth. More specifically, this op outputs a copy of
the input tensor where values from the depth dimension are moved in spatial blocks to the height
and width dimensions.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
DepthToSpace,
1,
OpSchema()
.Attr(
"blocksize",
"Blocks of [blocksize, blocksize] are moved.",
AttributeProto::INT)
.SetDoc(DepthToSpace_ver1_doc)
.Input(
0,
"input",
"Input tensor of [N,C,H,W], where N is the batch axis, C is the channel or depth"
", H is the height and W is the width.",
"T")
.Output(
0,
"output",
"Output tensor of [N, C/(blocksize * blocksize), H * blocksize, W * blocksize].",
"T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
auto blocksize = getAttribute(ctx, "blocksize", 0);
if (blocksize <= 0)
fail_shape_inference("Blocksize must be positive");
if (hasInputShape(ctx, 0)) {
auto& input_shape = getInputShape(ctx, 0);
if (input_shape.dim_size() == 4) {
// TODO: Clarify what behavior should be if C is not a multiple of
// blocksize*blocksize.
updateOutputShape(
ctx,
0,
{input_shape.dim(0),
input_shape.dim(1) / (blocksize * blocksize),
input_shape.dim(2) * blocksize,
input_shape.dim(3) * blocksize});
} else
fail_shape_inference("Input tensor must be 4-dimensional");
}
}));
static const char* Gather_ver1_doc = R"DOC(
Given `data` tensor of rank r >= 1, and `indices` tensor of rank q, gather
entries of the axis dimension of `data` (by default outer-most one as axis=0) indexed by `indices`, and concatenates
them in an output tensor of rank q + (r - 1).
Example 1:
```
data = [
[1.0, 1.2],
[2.3, 3.4],
[4.5, 5.7],
]
indices = [
[0, 1],
[1, 2],
]
output = [
[
[1.0, 1.2],
[2.3, 3.4],
],
[
[2.3, 3.4],
[4.5, 5.7],
],
]
```
Example 2:
```
data = [
[1.0, 1.2, 1.9],
[2.3, 3.4, 3.9],
[4.5, 5.7, 5.9],
]
indices = [
[0, 2],
]
axis = 1,
output = [
[
[1.0, 1.9],
[2.3, 3.9],
[4.5, 5.9],
],
]
```
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Gather,
1,
OpSchema()
.SetDoc(Gather_ver1_doc)
.Attr(
"axis",
"Which axis to gather on. Negative value means "
"counting dimensions from the back. Accepted range is [-r, r-1]",
AttributeProto::INT,
static_cast(0))
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(
1,
"indices",
"Tensor of int32/int64 indices, of any rank q. All index values are expected to be within bounds. "
"It is an error if any of the index values are out of bounds.",
"Tind")
.Output(0, "output", "Tensor of rank q + (r - 1).", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to any tensor type.")
.TypeConstraint(
"Tind",
{"tensor(int32)", "tensor(int64)"},
"Constrain indices to integer types")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 2)) {
return;
}
const TensorShapeProto& data_shape =
ctx.getInputType(0)->tensor_type().shape();
const TensorShapeProto& indices_shape =
ctx.getInputType(1)->tensor_type().shape();
int r = data_shape.dim_size();
if (r < 1) {
fail_shape_inference("data tensor must have rank >= 1");
}
int q = indices_shape.dim_size();
int axis = static_cast(getAttribute(ctx, "axis", 0));
if (axis < -r || axis >= r) {
fail_shape_inference("axis must be in [-r, r-1]");
}
if (axis < 0) {
axis += r;
}
int out_rank = q + r - 1;
if (out_rank == 0) {
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
}
for (int i = 0; i < out_rank; ++i) {
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() = (i < axis) ? data_shape.dim(i) : // i < axis < r
(i >= axis && i < axis + q) ? indices_shape.dim(i - axis)
: // i - axis < q
data_shape.dim(i - q + 1); // i < out_rank < q + r - 1
}
}));
static const char* Squeeze_ver1_doc = R"DOC(
Remove single-dimensional entries from the shape of a tensor.
Takes a parameter `axes` with a list of axes to squeeze.
If `axes` is not provided, all the single dimensions will be removed from
the shape. If an axis is selected with shape entry not equal to one, an error is raised.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Squeeze,
1,
OpSchema()
.Attr(
"axes",
"List of non-negative integers, indicate the dimensions to squeeze.",
AttributeProto::INTS,
OPTIONAL_VALUE)
.SetDoc(Squeeze_ver1_doc)
.Input(0, "data", "Tensors with at least max(dims) dimensions.", "T")
.Output(0, "squeezed", "Reshaped tensor with same data as input.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
std::vector axes;
if (!getRepeatedAttribute(ctx, "axes", axes)) {
return;
}
if (!ctx.getInputType(0)->tensor_type().has_shape()) {
return;
}
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
for (int i = 0, j = 0; i < input_shape.dim_size(); ++i) {
if (static_cast(j) < axes.size() && axes[j] == i) {
if (input_shape.dim(i).has_dim_value() &&
input_shape.dim(i).dim_value() != 1) {
fail_shape_inference(
"Dimension of input ",
i,
" must be 1 instead of ",
input_shape.dim(i).dim_value());
}
++j;
} else {
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() = input_shape.dim(i);
}
}
}));
static const char* Unsqueeze_ver1_doc = R"DOC(
Insert single-dimensional entries to the shape of a tensor.
Takes one required argument `axes`, a list of dimensions that will be inserted.
Dimension indices in `axes` are as seen in the output tensor. For example:
Given a tensor such that tensor with shape [3, 4, 5], then
Unsqueeze(tensor, axes=[0, 4]) has shape [1, 3, 4, 5, 1]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Unsqueeze,
1,
OpSchema()
.Attr(
"axes",
"List of non-negative integers, indicate the dimensions to be inserted",
AttributeProto::INTS)
.SetDoc(Unsqueeze_ver1_doc)
.Input(0, "data", "Original tensor", "T")
.Output(0, "expanded", "Reshaped tensor with same data as input.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
std::vector axes;
if (!getRepeatedAttribute(ctx, "axes", axes)) {
return;
}
std::sort(axes.begin(), axes.end());
if (!ctx.getInputType(0)->tensor_type().has_shape()) {
return;
}
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
int j = 0;
for (int i = 0;
i < ctx.getInputType(0)->tensor_type().shape().dim_size();
++i) {
while (static_cast(j) < axes.size() &&
axes[j] ==
ctx.getOutputType(0)->tensor_type().shape().dim_size()) {
ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim()
->set_dim_value(1);
++j;
}
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() =
ctx.getInputType(0)->tensor_type().shape().dim(i);
}
while (static_cast(j) < axes.size() &&
axes[j] ==
ctx.getOutputType(0)->tensor_type().shape().dim_size()) {
ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim()
->set_dim_value(1);
++j;
}
}));
static const char* OneHot_ver9_doc = R"DOC(
Produces a one-hot tensor based on inputs.
The locations represented by the index values in the 'indices' input tensor will have 'on_value'
and the other locations will have 'off_value' in the output tensor, where 'on_value' and 'off_value'
are specified as part of required input argument 'values', which is a two-element tensor of format
[off_value, on_value]. The rank of the output tensor will be one greater than the rank of the
input tensor. The additional dimension is for one-hot representation. The additional dimension will
be inserted at the position specified by 'axis'. If 'axis' is not specified then then additional
dimension will be inserted as the innermost dimension, i.e. axis=-1. The size of the additional
dimension is specified by required scalar input 'depth'. The type of the output tensor is the same
as the type of the 'values' input. Any entries in the 'indices' input tensor with values outside
the range [0, depth) will result in one-hot representation with all 'off_value' values in the
output tensor.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
OneHot,
9,
OpSchema()
.SetDoc(OneHot_ver9_doc)
.Attr(
"axis",
"(Optional) Axis along which one-hot representation in added. Default: axis=-1. "
"axis=-1 means that the additional dimension will be inserted as the "
"innermost/last dimension in the output tensor.",
AttributeProto::INT,
static_cast(-1))
.Input(
0,
"indices",
"Input tensor containing indices. The values must be non-negative integers. "
"Any entries in the 'indices' input tensor with values outside the range [0, depth) "
"will result in one-hot representation with all 'off_value' values in the output tensor."
"In case 'indices' is of non-integer type, the values will be casted to int64 before use.",
"T1")
.Input(
1,
"depth",
"Scalar specifying the number of classes in one-hot tensor. This is also the size "
"of the one-hot dimension (specified by 'axis' attribute) added on in the output "
"tensor. The values in the 'indices' input tensor are expected to be "
"in the range [0, depth). "
"In case 'depth' is of non-integer type, it will be casted to int64 before use.",
"T2")
.Input(
2,
"values",
"Rank 1 tensor containing exactly two elements, in the format [off_value, on_value], "
"where 'on_value' is the value used for filling locations specified in 'indices' input "
"tensor, and 'off_value' is the value used for filling locations other than those specified "
"in 'indices' input tensor. ",
"T3")
.Output(
0,
"output",
"Tensor of rank one greater than input tensor 'indices', i.e. rank(output) = rank(indices) + 1. "
"The data type for the elements of the output tensor is the same as the type of input 'values' "
"is used.",
"T3")
.TypeConstraint(
"T1",
OpSchema::all_numeric_types(),
"Constrains input to only numeric types.")
.TypeConstraint(
"T2",
OpSchema::all_numeric_types(),
"Constrains input to only numeric types.")
.TypeConstraint(
"T3",
OpSchema::all_tensor_types(),
"Constrain to any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Check that the node has three inputs.
if (ctx.getNumInputs() != 3) {
fail_type_inference("OneHot node must have three inputs.");
}
// Input 'depth' must be a scalar or a single-element vector.
// TODO: Ideally to match spec for this input only allow Scalar should
// be allowed. Making this change now can affect backward
// compatibility for this op. Since this does not seem like a good
// justification to update version for this op, allowing both scalar
// and 1 element vector for now. In future when version update for
// this op is done we should only allow scalar or chage the spec to
// allow both.
if (hasInputShape(ctx, 1)) {
auto& depth_shape = getInputShape(ctx, 1);
if (depth_shape.dim_size() != 0 && depth_shape.dim_size() != 1) {
fail_type_inference(
"Input 'depth' must be a scalar or rank 1 tensor.");
}
if (depth_shape.dim_size() == 1 &&
depth_shape.dim((int)0).has_dim_value() &&
depth_shape.dim((int)0).dim_value() != 1) {
fail_type_inference(
"Input 'depth' must have exactly one element.");
}
}
// Input 'values' must be a two-element vector.
if (hasInputShape(ctx, 2)) {
auto& values_shape = getInputShape(ctx, 2);
if (values_shape.dim_size() != 1) {
fail_type_inference("Input 'values' must be rank 1 tensor.");
}
if (values_shape.dim((int)0).has_dim_value() &&
values_shape.dim((int)0).dim_value() != 2) {
fail_type_inference(
"Input 'values' must have exactly two elements.");
}
}
// Set output type to be the same as the third input, 'values'.
propagateElemTypeFromInputToOutput(ctx, 2, 0);
// Set the output shape, if input 0 (indices) shape is available.
if (hasInputShape(ctx, 0)) {
const TensorShapeProto& indices_shape =
ctx.getInputType(0)->tensor_type().shape();
int r = indices_shape.dim_size();
if (r < 1) {
fail_shape_inference("Indices tensor must have rank >= 1");
}
int out_rank = r + 1;
int axis = static_cast(getAttribute(ctx, "axis", -1));
if (axis < -out_rank || axis >= out_rank) {
fail_shape_inference(
"'axis' must be in [-rank(indices)-1, rank(indices)]");
}
if (axis < 0) {
axis += out_rank;
}
auto* output_shape = getOutputShape(ctx, 0);
for (int i = 0; i < out_rank; ++i) {
auto* dim = output_shape->add_dim();
if (i < axis) {
if (indices_shape.dim(i).has_dim_value()) {
dim->set_dim_value(indices_shape.dim(i).dim_value());
} else if (indices_shape.dim(i).has_dim_param()) {
dim->set_dim_param(indices_shape.dim(i).dim_param());
}
} else if (i > axis) {
if (indices_shape.dim(i - 1).has_dim_value()) {
dim->set_dim_value(indices_shape.dim(i - 1).dim_value());
} else if (indices_shape.dim(i - 1).has_dim_param()) {
dim->set_dim_param(indices_shape.dim(i - 1).dim_param());
}
}
}
}
}));
static const char* Compress_ver9_doc = R"DOC(
Selects slices from an input tensor along a given axis where condition evaluates to True for each axis index.
In case axis is not provided, input is flattened before elements are selected.
Compress behaves like numpy.compress: https://docs.scipy.org/doc/numpy/reference/generated/numpy.compress.html
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Compress,
9,
OpSchema()
.SetDoc(Compress_ver9_doc)
.Attr(
"axis",
"(Optional) Axis along which to take slices. If not specified, "
"input is flattened before elements being selected.",
AttributeProto::INT,
OPTIONAL_VALUE)
.Input(0, "input", "Tensor of rank r >= 1.", "T")
.Input(
1,
"condition",
"Rank 1 tensor of booleans to indicate which slices or data elements to be selected. "
"Its length can be less than the input length alone the axis "
"or the flattened input size if axis is not specified. "
"In such cases data slices or elements exceeding the condition length are discarded.",
"T1")
.Output(
0,
"output",
"Tensor of rank r if axis is specified. Otherwise output is a Tensor of rank 1.",
"T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeConstraint(
"T1",
{"tensor(bool)"},
"Constrains to boolean tensors."));
static const char* Split_ver2_doc =
R"DOC(Split a tensor into a list of tensors, along the specified
'axis'. Lengths of the parts can be specified using argument 'split'.
Otherwise, the tensor is split to equal sized parts.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Split,
2,
OpSchema()
.Input(0, "input", "The tensor to split", "T")
.Output(
0,
"outputs",
"One or more outputs forming list of tensors after splitting",
"T",
OpSchema::Variadic)
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.Attr(
"axis",
"Which axis to split on. ",
AttributeProto::INT,
static_cast(0))
.Attr("split", "length of each output", AttributeProto::INTS, OPTIONAL_VALUE)
.SetDoc(Split_ver2_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
for (int i = 0; i < static_cast(ctx.getNumOutputs()); ++i) {
propagateElemTypeFromInputToOutput(ctx, 0, i);
}
if (!hasNInputShapes(ctx, 1)) {
return;
}
auto axisAttr = ctx.getAttribute("axis");
int axis = axisAttr ? static_cast(axisAttr->i()) : 0;
if (axis < 0) {
return;
}
std::vector split;
if (!getRepeatedAttribute(ctx, "split", split)) {
if (!ctx.getInputType(0)->tensor_type().has_shape()) {
return;
}
const auto& shape = ctx.getInputType(0)->tensor_type().shape();
if (axis >= shape.dim_size()) {
fail_type_inference("Invalid value of attribute 'axis'");
}
const auto& splitDim = shape.dim(axis);
if (!splitDim.has_dim_value()) {
return;
}
int splitDimValue = static_cast(splitDim.dim_value());
int chunkSize =
splitDimValue / static_cast(ctx.getNumOutputs());
int leftOver = splitDimValue -
(chunkSize * static_cast(ctx.getNumOutputs()));
for (int i = 0; i < static_cast(ctx.getNumOutputs()); i++) {
split.push_back(i < leftOver ? chunkSize + 1 : chunkSize);
}
for (size_t i = 0; i < ctx.getNumOutputs(); i++) {
*ctx.getOutputType(i)->mutable_tensor_type()->mutable_shape() =
shape;
ctx.getOutputType(i)
->mutable_tensor_type()
->mutable_shape()
->mutable_dim(axis)
->set_dim_value(split[i]);
}
}
}));
static const char* Pad_ver2_doc = R"DOC(
Given `data` tensor, pads, mode, and value.
Example:
Insert 0 pads to the beginning of the second dimension.
data = [
[1.0, 1.2],
[2.3, 3.4],
[4.5, 5.7],
]
pads = [0, 2, 0, 0]
output = [
[
[0.0, 0.0, 1.0, 1.2],
[0.0, 0.0, 2.3, 3.4],
[0.0, 0.0, 4.5, 5.7],
],
]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Pad,
2,
OpSchema()
.Attr(
"pads",
"List of integers indicating the number of padding elements to add or remove (if negative) "
"at the beginning and end of each axis. For 2D it is the number of pixels. "
"`pads` rank should be double of the input's rank. `pads` format should be as follow "
"[x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels "
"added at the beginning of axis `i` and xi_end, the number of pixels added at "
"the end of axis `i`.",
AttributeProto::INTS)
.Attr(
"mode",
"Three modes: constant(default), reflect, edge",
AttributeProto::STRING,
std::string("constant"))
.Attr(
"value",
"One float, indicates the value to be filled.",
AttributeProto::FLOAT,
0.0f)
.SetDoc(Pad_ver2_doc)
.Input(0, "data", "Input tensor.", "T")
.Output(0, "output", "Tensor after padding.", "T")
.TypeConstraint(
"T",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain input and output types to float tensors.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
std::vector pads;
if (!getRepeatedAttribute(ctx, "pads", pads))
fail_shape_inference("Attribute value for pads is required");
if (pads.size() != static_cast(input_shape.dim_size() * 2)) {
fail_shape_inference("Attribute pads has incorrect length");
;
}
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
for (size_t i = 0; (int64_t)i < input_shape.dim_size(); ++i) {
auto* newdim = ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim();
if (ctx.getInputType(0)
->tensor_type()
.shape()
.dim((int)i)
.has_dim_value()) {
newdim->set_dim_value(
ctx.getInputType(0)
->tensor_type()
.shape()
.dim((int)i)
.dim_value() +
pads[i] + pads[input_shape.dim_size() + i]);
} else if (pads[i] + pads[input_shape.dim_size() + i] == 0) {
*newdim = input_shape.dim((int)i);
}
}
}));
static const char* GatherND_ver11_doc = R"DOC(
Given `data` tensor of rank `r` >= 1, and `indices` tensor of rank `q` >= 1, this operator gathers
slices of `data` into an output tensor of rank `q + r - indices_shape[-1] - 1`.
`indices` is an q-dimensional integer tensor, best thought of as a `(q-1)`-dimensional tensor of index-tuples into `data`,
where each element defines a slice of `data`
Some salient points about the inputs' rank and shape:
1) r >= 1 and q >= 1 are to be honored. There is no dependency condition to be met between ranks `r` and `q`
2) The `indices_shape[-1]` should have a value between 1 (inclusive) and rank `r` (inclusive)
3) All values in `indices` are expected to be within bounds [-s, s-1] along axis of size `s` (i.e.) `-data_shape[i] <= indices[...,i] <= data_shape[i] - 1`.
It is an error if any of the index values are out of bounds.
The output is computed as follows:
The output tensor is obtained by mapping each index-tuple in the `indices` tensor to the corresponding slice of the input `data`.
1) If `indices_shape[-1] > r` => error condition
2) If `indices_shape[-1] == r`, since the rank of `indices` is `q`, `indices` can be thought of as a `(q-1)`-dimensional tensor
containing 1-D tensors of dimension `r`. Let us think of each such `r` ranked tensor as `indices_slice`.
Each *scalar value* corresponding to `data[indices_slice]` is filled into the corresponding location of the `(q-1)`-dimensional tensor
to form the `output` tensor (Example 1 below)
3) If `indices_shape[-1] < r`, since the rank of `indices` is `q`, `indices` can be thought of as a `(q-1)`-dimensional tensor
containing 1-D tensors of dimension `< r`. Let us think of each such tensors as `indices_slice`.
Each *tensor slice* corresponding to `data[indices_slice , :]` is filled into the corresponding location of the `(q-1)`-dimensional tensor
to form the `output` tensor (Examples 2, 3, and 4 below)
This operator is the inverse of `ScatterND`.
`Example 1`
data = [[0,1],[2,3]] # data_shape = [2, 2]
indices = [[0,0],[1,1]] # indices_shape = [2, 2]
output = [0,3] # output_shape = [2]
`Example 2`
data = [[0,1],[2,3]] # data_shape = [2, 2]
indices = [[1],[0]] # indices_shape = [2, 1]
output = [[2,3],[0,1]] # output_shape = [2, 2]
`Example 3`
data = [[[0,1],[2,3]],[[4,5],[6,7]]] # data_shape = [2, 2, 2]
indices = [[0,1],[1,0]] # indices_shape = [2, 2]
output = [[2,3],[4,5]] # output_shape = [2, 2]
`Example 4`
data = [[[0,1],[2,3]],[[4,5],[6,7]]] # data_shape = [2, 2, 2]
indices = [[[0,1]],[[1,0]]] # indices_shape = [2, 1, 2]
output = [[[2,3]],[[4,5]]] # output_shape = [2, 1, 2]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
GatherND,
11,
OpSchema()
.SetDoc(GatherND_ver11_doc)
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(
1,
"indices",
"Tensor of rank q >= 1. All index values are expected to be within bounds [-s, s-1] "
"along axis of size s. It is an error if any of the index values are out of bounds.",
"tensor(int64)")
.Output(
0,
"output",
"Tensor of rank q + r - indices_shape[-1] - 1.",
"T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Type inference
propagateElemTypeFromInputToOutput(ctx, 0, 0);
// Shape inference
if (!hasNInputShapes(ctx, 2)) {
// cannot proceed with shape or rank inference
return;
}
const auto& data_shape = ctx.getInputType(0)->tensor_type().shape();
const auto data_rank = data_shape.dim_size();
const auto& indices_shape =
ctx.getInputType(1)->tensor_type().shape();
const auto indices_rank = indices_shape.dim_size();
if (data_rank < 1 || indices_rank < 1) {
fail_shape_inference(
"Both `data` and `indices` input tensors in GatherND op "
"need to have rank larger than 0.");
}
// cannot ascertain if the input shapes are valid if shape of
// `indices` is missing last dimension value so return at this point
if (!indices_shape.dim(indices_rank - 1).has_dim_value()) {
return;
}
const auto last_index_dimension =
indices_shape.dim(indices_rank - 1).dim_value();
if (last_index_dimension > data_rank) {
fail_shape_inference(
"Last dimension of `indices` input tensor in GatherND op "
"must not be larger than the rank of `data` tensor");
}
for (int i = 0; i < indices_rank - 1; ++i) {
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() = indices_shape.dim(i);
}
for (int i = static_cast(last_index_dimension); i < data_rank;
++i) {
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() = data_shape.dim(i);
}
}));
} // namespace ONNX_NAMESPACE onnx-1.7.0/onnx/defs/tensor/defs.cc 0000664 0000000 0000000 00000323060 13655345213 015661 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#include "onnx/defs/tensor/utils.h"
#include
#include
#include
namespace ONNX_NAMESPACE {
static const char* Cast_ver9_doc = R"DOC(
The operator casts the elements of a given input tensor to a data type
specified by the 'to' argument and returns an output tensor of the same size in
the converted type. The 'to' argument must be one of the data types specified
in the 'DataType' enum field in the TensorProto message.
Casting from string tensor in plain (e.g., "3.14" and "1000") and scientific numeric representations
(e.g., "1e-5" and "1E8") to float types is supported. For example, converting string "100.5" to an integer may
result 100. There are some string literals reserved for special floating-point values;
"+INF" (and "INF"), "-INF", and "NaN" are positive infinity, negative infinity, and not-a-number, respectively.
Any string which can exactly match "+INF" in a case-insensitive way would be mapped to positive infinite. Similarly,
this case-insensitive rule is applied to "INF" and "NaN". When casting from numeric tensors
to string tensors, plain floating-point representation (such as "314.15926") would be used.
Converting non-numerical-literal string such as "Hello World!" is an undefined behavior. Cases
of converting string representing floating-point arithmetic value, such as "2.718", to INT is an undefined behavior.
Conversion from a numerical type to any numerical type is always allowed.
User must be aware of precision loss and value change caused by range difference between two types.
For example, a 64-bit float 3.1415926459 may be round to a 32-bit float 3.141592. Similarly, converting
an integer 36 to Boolean may produce 1 because we truncate bits which can't be stored in the targeted type.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Cast,
9,
OpSchema()
.SetDoc(Cast_ver9_doc)
.Attr(
"to",
"The data type to which the elements of the input tensor are cast. "
"Strictly must be one of the types from DataType enum in TensorProto",
AttributeProto::INT)
.Input(0, "input", "Input tensor to be cast.", "T1")
.Output(
0,
"output",
"Output tensor with the same shape as input with type "
"specified by the 'to' argument",
"T2")
.TypeConstraint(
"T1",
{"tensor(float16)",
"tensor(float)",
"tensor(double)",
"tensor(int8)",
"tensor(int16)",
"tensor(int32)",
"tensor(int64)",
"tensor(uint8)",
"tensor(uint16)",
"tensor(uint32)",
"tensor(uint64)",
"tensor(bool)",
"tensor(string)"},
"Constrain input types. Casting from complex is not supported.")
.TypeConstraint(
"T2",
{"tensor(float16)",
"tensor(float)",
"tensor(double)",
"tensor(int8)",
"tensor(int16)",
"tensor(int32)",
"tensor(int64)",
"tensor(uint8)",
"tensor(uint16)",
"tensor(uint32)",
"tensor(uint64)",
"tensor(bool)",
"tensor(string)"},
"Constrain output types. Casting to complex is not supported.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromAttributeToOutput(ctx, "to", 0);
if (hasNInputShapes(ctx, 1)) {
propagateShapeFromInputToOutput(ctx, 0, 0);
}
}));
static const char* Reshape_ver5_doc = R"DOC(
Reshape the input tensor similar to numpy.reshape.
First input is the data tensor, second input is a shape tensor which specifies the output shape. It outputs the reshaped tensor.
At most one dimension of the new shape can be -1. In this case, the value is
inferred from the size of the tensor and the remaining dimensions. A dimension
could also be 0, in which case the actual dimension value is unchanged (i.e. taken
from the input tensor).)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Reshape,
5,
OpSchema()
.SetDoc(Reshape_ver5_doc)
.Input(0, "data", "An input tensor.", "T")
.Input(1, "shape", "Specified shape for output.", "tensor(int64)")
.Output(0, "reshaped", "Reshaped data.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Type inference
propagateElemTypeFromInputToOutput(ctx, 0, 0);
// Shape Inference if 2nd input data (the target shape) is available
const TensorProto* targetShapeInitializer = ctx.getInputData(1);
if (!targetShapeInitializer) {
return;
}
// Make targetShape (0 -> same as originalShape, -1 -> inferred).
// The targetShape vector represents the specified shape for output.
std::vector targetShape;
if (targetShapeInitializer->has_raw_data()) {
const std::string& bytes = targetShapeInitializer->raw_data();
targetShape.insert(
targetShape.end(),
reinterpret_cast(bytes.c_str()),
reinterpret_cast(bytes.c_str() + bytes.size()));
} else {
const auto& data = targetShapeInitializer->int64_data();
targetShape.insert(targetShape.end(), data.begin(), data.end());
}
// Iterate through targetShape, adding dimensions in the outputShape
// TensorProto. If the targertShape dimension is -1, we do not set the
// dimension value in this iteration, but we record the Dimension. If
// targertShape dimension is 0, we attempt to propagate the dimension
// value/param. If the value cannot be inferred, we set the flag in
// the unresolveZeros vector. If targetShape dimension is positive, we
// set the dimension value in the outputShape. We track the product of
// the dimensions we are setting outputShape in the outputProduct
// variable. The outputProduct will potentially be used for inferring
// a dimension marked -1.
auto* outputShape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
TensorShapeProto::Dimension* negativeOneDim = nullptr;
const auto& dataInputTensorType = ctx.getInputType(0)->tensor_type();
std::vector unresolvedZeros(targetShape.size(), false);
int64_t outputProduct = 1;
for (int i = 0; i < static_cast(targetShape.size()); ++i) {
// Add a new dimension to outputShape
auto* new_dim = outputShape->add_dim();
if (targetShape[i] == -1) {
// Check if multiple -1's. If not, set negativeOneDim, marking
// this dimension to potentially be filled in later.
if (negativeOneDim) {
fail_shape_inference(
"Target shape may not have multiple -1 dimensions");
}
negativeOneDim = new_dim;
} else if (targetShape[i] == 0) {
// Check if data input has a shape and if the index i is within
// its bounds. If these conditions are satisfied, any dimension
// value/param should be propogated. If dimension value cannot be
// inferred, set the corresponding unresolvedZeros flag to true.
unresolvedZeros[i] = true;
if (dataInputTensorType.has_shape()) {
if (i >= dataInputTensorType.shape().dim_size()) {
fail_shape_inference("Invalid position of 0");
}
if (dataInputTensorType.shape().dim(i).has_dim_value()) {
const auto& dim_value =
dataInputTensorType.shape().dim(i).dim_value();
new_dim->set_dim_value(dim_value);
outputProduct *= dim_value;
unresolvedZeros[i] = false;
} else if (dataInputTensorType.shape().dim(i).has_dim_param()) {
const auto& dim_param =
dataInputTensorType.shape().dim(i).dim_param();
new_dim->set_dim_param(dim_param);
}
}
} else if (targetShape[i] > 0) {
// Set the dimension value to targetShape[i]
new_dim->set_dim_value(targetShape[i]);
outputProduct *= targetShape[i];
} else {
// Check if value is less than -1; fail if so
fail_shape_inference("Invalid dimension value: ", targetShape[i]);
}
}
// If negativeOneDim has been set, we attempt to infer its value. This
// can be done if all dimension values for the data input tensor shape
// are known other than the ones corresponding to unresolvedZeros
// flags.
if (negativeOneDim) {
// First, attempt to compute product of data input shape dimensions
// that are not marked by unresolvedZeros. If not possible, set the
// inputProductValid flag to false.
if (!outputProduct) {
fail_shape_inference("Invalid Target shape product of 0");
}
int64_t inputProduct = 1;
bool inputProductValid = true;
if (!dataInputTensorType.has_shape()) {
inputProductValid = false;
} else {
for (int i = 0; i < dataInputTensorType.shape().dim_size(); ++i) {
if (dataInputTensorType.shape().dim(i).has_dim_value()) {
inputProduct *=
dataInputTensorType.shape().dim(i).dim_value();
} else if (
i >= static_cast(unresolvedZeros.size()) ||
!unresolvedZeros[i]) {
inputProductValid = false;
break;
}
}
}
if (inputProductValid) {
if (inputProduct % outputProduct != 0) {
fail_shape_inference(
"Dimension could not be inferred: incompatible shapes");
}
negativeOneDim->set_dim_value(inputProduct / outputProduct);
}
}
}));
static const char* Shape_ver1_doc = R"DOC(
Takes a tensor as input and outputs an 1D int64 tensor containing the shape of the input tensor.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Shape,
1,
OpSchema()
.SetDoc(Shape_ver1_doc)
.Input(0, "data", "An input tensor.", "T")
.Output(0, "shape", "Shape of the input tensor", "T1")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Input tensor can be of arbitrary type.")
.TypeConstraint(
"T1",
{"tensor(int64)"},
"Constrain output to int64 tensor.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
ctx.getOutputType(0)->mutable_tensor_type()->set_elem_type(
TensorProto::INT64);
if (!hasNInputShapes(ctx, 1)) {
return;
}
if (ctx.getInputType(0)->tensor_type().has_shape()) {
ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim()
->set_dim_value(
ctx.getInputType(0)->tensor_type().shape().dim_size());
}
}));
static const char* Size_ver1_doc = R"DOC(
Takes a tensor as input and outputs a int64 scalar that equals to the total number of elements of the input tensor.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Size,
1,
OpSchema()
.SetDoc(Size_ver1_doc)
.Input(0, "data", "An input tensor.", "T")
.Output(0, "size", "Total number of elements of the input tensor", "T1")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Input tensor can be of arbitrary type.")
.TypeConstraint(
"T1",
{"tensor(int64)"},
"Constrain output to int64 tensor, which should be a scalar though.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
ctx.getOutputType(0)->mutable_tensor_type()->set_elem_type(
TensorProto::INT64);
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
}));
ONNX_OPERATOR_SET_SCHEMA(
Concat,
11,
OpSchema()
.Attr(
"axis",
"Which axis to concat on. A negative value means counting dimensions from the back. "
"Accepted range is [-r, r-1] where r = rank(inputs)..",
AttributeProto::INT)
.SetDoc(
"Concatenate a list of tensors into a single tensor. "
"All input tensors must have the same shape, except for the dimension size of the axis to concatenate on.")
.Input(
0,
"inputs",
"List of tensors for concatenation",
"T",
OpSchema::Variadic)
.Output(0, "concat_result", "Concatenated tensor", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain output types to any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
auto numInputs = ctx.getNumInputs();
if (numInputs < 1 ||
!hasNInputShapes(ctx, static_cast(numInputs))) {
return;
}
auto rank = ctx.getInputType(0)->tensor_type().shape().dim_size();
auto axisAttr = ctx.getAttribute("axis");
if (!axisAttr) {
fail_shape_inference("Required attribute axis is missing");
}
int axis = static_cast(axisAttr->i());
if (axis < -rank || axis >= rank) {
fail_shape_inference("axis must be in [-rank, rank-1].");
}
if (axis < 0) {
axis += rank;
}
if (numInputs == 1) {
propagateShapeFromInputToOutput(ctx, 0, 0);
return;
}
bool all_lengths_known = true;
int total_length = 0;
auto* output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
for (int64_t i = 0; i < rank; ++i) {
output_shape->add_dim();
}
for (size_t i = 0; i < numInputs; i++) {
const auto& shape = ctx.getInputType(i)->tensor_type().shape();
if (shape.dim_size() != rank)
fail_shape_inference("All inputs to Concat must have same rank");
for (int j = 0; j < rank; j++) {
if (j == axis) {
if (shape.dim(j).has_dim_value()) {
total_length += static_cast(shape.dim(j).dim_value());
} else {
all_lengths_known = false;
}
} else {
auto& output_dim = *output_shape->mutable_dim(j);
const auto& input_dim = shape.dim(j);
mergeInDimensionInfo(input_dim, output_dim, j);
}
}
}
if (all_lengths_known) {
output_shape->mutable_dim(axis)->set_dim_value(total_length);
}
}));
static const char* Split_ver11_doc =
R"DOC(Split a tensor into a list of tensors, along the specified
'axis'. Lengths of the parts can be specified using argument 'split'.
Otherwise, the tensor is split to equal sized parts.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Split,
11,
OpSchema()
.Input(0, "input", "The tensor to split", "T")
.Output(
0,
"outputs",
"One or more outputs forming list of tensors after splitting",
"T",
OpSchema::Variadic)
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.Attr(
"axis",
"Which axis to split on. "
"A negative value means counting dimensions from the back. Accepted range is [-rank, rank-1] "
"where r = rank(input).",
AttributeProto::INT,
static_cast(0))
.Attr("split", "length of each output. Values should be >= 0.", AttributeProto::INTS, OPTIONAL_VALUE)
.SetDoc(Split_ver11_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
for (int i = 0; i < static_cast(ctx.getNumOutputs()); ++i) {
propagateElemTypeFromInputToOutput(ctx, 0, i);
}
if (!hasNInputShapes(ctx, 1)) {
return;
}
const auto& shape = ctx.getInputType(0)->tensor_type().shape();
int rank = shape.dim_size();
int axis = static_cast(getAttribute(ctx, "axis", 0));
if (axis < -rank || axis >= rank) {
fail_type_inference(
"Invalid value of attribute 'axis'. Rank=",
rank,
" Value=",
axis);
}
if (axis < 0) {
axis += rank;
}
const auto& split_dim = shape.dim(axis);
if (!split_dim.has_dim_value()) {
for (size_t i = 0; i < ctx.getNumOutputs(); i++) {
*ctx.getOutputType(i)->mutable_tensor_type()->mutable_shape() =
shape;
ctx.getOutputType(i)
->mutable_tensor_type()
->mutable_shape()
->mutable_dim(axis)
->Clear();
}
return;
}
int split_dim_value = static_cast(split_dim.dim_value());
std::vector split;
if (getRepeatedAttribute(ctx, "split", split)) {
if (split.size() != ctx.getNumOutputs()) {
fail_shape_inference(
"Mismatch between number of splits (",
split.size(),
") and outputs (",
ctx.getNumOutputs(),
")");
}
int64_t total_dim = 0;
for (int64_t d : split) {
total_dim += d;
}
if (total_dim != split_dim_value) {
fail_shape_inference(
"Mismatch between the sum of 'split' (",
total_dim,
") and the split dimension of the input (",
split_dim_value,
")");
}
} else {
int num_outputs = static_cast(ctx.getNumOutputs());
if (split_dim_value % num_outputs != 0) {
fail_shape_inference("The input is not evenly splittable");
}
int chunk_size = split_dim_value / num_outputs;
for (int i = 0; i < static_cast(ctx.getNumOutputs()); i++) {
split.push_back(chunk_size);
}
}
for (size_t i = 0; i < ctx.getNumOutputs(); i++) {
*ctx.getOutputType(i)->mutable_tensor_type()->mutable_shape() =
shape;
ctx.getOutputType(i)
->mutable_tensor_type()
->mutable_shape()
->mutable_dim(axis)
->set_dim_value(split[i]);
}
}));
static const char* Slice_ver11_doc = R"DOC(
Produces a slice of the input tensor along multiple axes. Similar to numpy:
https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html
Slices uses `starts`, `ends`, `axes` and `steps` inputs to specify the start and end
dimension and step for each axis in the list of axes, it uses this information to
slice the input `data` tensor. If a negative value is passed for any of the
start or end indices, it represents number of elements before the end of that
dimension. If the value passed to start or end is larger than the `n` (the
number of elements in this dimension), it represents `n`. For slicing to the
end of a dimension with unknown size, it is recommended to pass in `INT_MAX`
when sclicing forward and 'INT_MIN' when slicing backward.
If a negative value is passed for step, it represents slicing backward.
However step value cannot be 0.
If `axes` are omitted, they are set to `[0, ..., ndim-1]`.
If `steps` are omitted, they are set to `[1, ..., 1]` of length `len(starts)`
Example 1:
data = [
[1, 2, 3, 4],
[5, 6, 7, 8],
]
axes = [0, 1]
starts = [1, 0]
ends = [2, 3]
steps = [1, 2]
result = [
[5, 7],
]
Example 2:
data = [
[1, 2, 3, 4],
[5, 6, 7, 8],
]
starts = [0, 1]
ends = [-1, 1000]
result = [
[2, 3, 4],
]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Slice,
11,
OpSchema()
.SetDoc(Slice_ver11_doc)
.Input(0, "data", "Tensor of data to extract slices from.", "T")
.Input(
1,
"starts",
"1-D tensor of starting indices of corresponding axis in `axes`",
"Tind")
.Input(
2,
"ends",
"1-D tensor of ending indices (exclusive) of corresponding axis in `axes`",
"Tind")
.Input(
3,
"axes",
"1-D tensor of axes that `starts` and `ends` apply to. Negative value means counting dimensions "
"from the back. Accepted range is [-r, r-1] where r = rank(data).",
"Tind",
OpSchema::Optional)
.Input(
4,
"steps",
"1-D tensor of slice step of corresponding axis in `axes`. "
"Negative value means slicing backward. 'steps' cannot be 0. "
"Defaults to 1.",
"Tind",
OpSchema::Optional)
.Output(0, "output", "Sliced data tensor.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeConstraint(
"Tind",
{"tensor(int32)", "tensor(int64)"},
"Constrain indices to integer types")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
size_t num_inputs = ctx.getNumInputs();
if (num_inputs != 3 && num_inputs != 4 && num_inputs != 5) {
fail_type_inference(
"Slice op must have either three, four or five inputs.");
}
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
// Shape Inference if
// 1. 2nd and 3rd input data (starts, ends) are available.
// and 2. 4th and 5th optional input (axes, steps) are either not set,
// or set and is initializer.
const TensorProto* startsInitializer = ctx.getInputData(1);
const TensorProto* endsInitializer = ctx.getInputData(2);
const TensorProto* axesInitializer =
hasInputShape(ctx, 3) ? ctx.getInputData(3) : nullptr;
const TensorProto* stepsInitializer =
hasInputShape(ctx, 4) ? ctx.getInputData(4) : nullptr;
if (!startsInitializer || !endsInitializer ||
(hasInputShape(ctx, 3) && !ctx.getInputData(3)) ||
(hasInputShape(ctx, 4) && !ctx.getInputData(4))) {
return;
}
// don't know data_type- can't proceed
if (!startsInitializer->has_data_type())
return;
auto get_initializer_data =
[](const TensorProto* initializer) -> std::vector {
std::vector vec;
if (initializer->data_type() == TensorProto::INT64) {
const auto& data = ParseData(initializer);
vec.insert(vec.end(), data.begin(), data.end());
} else if (initializer->data_type() == TensorProto::INT32) {
const auto& data = ParseData(initializer);
vec.insert(vec.end(), data.begin(), data.end());
} else {
// unaccepted data type
fail_shape_inference(
"Only supports `int32_t` or `int64_t` inputs for starts/ends/axes/steps");
}
return vec;
};
auto clamp = [](int64_t val, int64_t low, int64_t high) -> int64_t {
if (val < low)
return low;
if (val > high)
return high;
return val;
};
std::vector starts = get_initializer_data(startsInitializer);
std::vector ends = get_initializer_data(endsInitializer);
if (starts.size() != ends.size()) {
fail_shape_inference(
"Incorrect or missing input value for starts and ends");
}
const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
const auto input_rank = input_shape.dim_size();
std::vector axes(starts.size());
if (!axesInitializer) {
std::iota(axes.begin(), axes.end(), 0);
} else {
axes = get_initializer_data(axesInitializer);
if (axes.size() != starts.size()) {
fail_shape_inference("Input axes has incorrect length");
}
}
std::vector steps;
if (!stepsInitializer) {
steps = std::vector(starts.size(), 1);
} else {
steps = get_initializer_data(stepsInitializer);
if (steps.size() != axes.size()) {
fail_shape_inference("Input steps has incorrect length");
}
}
for (size_t i = 0; (int64_t)i < input_rank; ++i) {
// first update rank of output dim
auto* output_dim = ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim();
const auto& input_dim = input_shape.dim((int)i);
if (input_dim.has_dim_value()) {
output_dim->set_dim_value(input_dim.dim_value());
} else if (input_dim.has_dim_param()) {
output_dim->set_dim_param(input_dim.dim_param());
}
}
std::unordered_set unique_axes;
size_t axes_size = axes.size();
for (size_t axis_index = 0; axis_index < axes_size; ++axis_index) {
auto axis = axes[axis_index] < 0
? axes[axis_index] + static_cast(input_rank)
: axes[axis_index];
if (axis >= static_cast(input_rank) || axis < 0)
fail_shape_inference("Input axes has invalid data");
if (unique_axes.find(axis) != unique_axes.end())
fail_shape_inference("'axes' has duplicates");
unique_axes.insert(axis);
auto input_dim =
ctx.getInputType(0)->tensor_type().shape().dim((int)axis);
// input dim value is missing - cannot perform shape inference for
// this axis
if (!input_dim.has_dim_value()) {
// Clear any previously propagated dim_param and leave this dimension "empty",
// before moving on to the next dimension
ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->mutable_dim(static_cast(axis))
->clear_dim_param();
continue;
}
const auto input_dim_value = input_dim.dim_value();
// process step
auto step = steps[axis_index];
if (step == 0)
fail_shape_inference("'step' cannot be 0");
// process start
auto start = starts[axis_index];
if (start < 0)
start += input_dim_value;
if (step < 0)
start = clamp(start, 0, input_dim_value - 1);
else
start = clamp(start, 0, input_dim_value);
// process end
auto end = ends[axis_index];
if (end < 0)
end += input_dim_value;
if (step < 0)
end = clamp(end, -1, input_dim_value);
else
end = clamp(end, 0, input_dim_value);
// find output dim value for this axis
auto temp = static_cast(ceil(1.0 * (end - start) / step));
if (temp < 0)
temp = 0;
// assign output value
ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->mutable_dim((int)axis)
->set_dim_value(temp);
}
}));
static const char* Transpose_ver1_doc = R"DOC(
Transpose the input tensor similar to numpy.transpose. For example, when
perm=(1, 0, 2), given an input tensor of shape (1, 2, 3), the output shape
will be (2, 1, 3).
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Transpose,
1,
OpSchema()
.SetDoc(Transpose_ver1_doc)
.Attr(
"perm",
"A list of integers. By default, reverse the dimensions, "
"otherwise permute the axes according to the values given.",
AttributeProto::INTS,
OPTIONAL_VALUE)
.Input(0, "data", "An input tensor.", "T")
.Output(0, "transposed", "Transposed output.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
auto input_type = ctx.getInputType(0);
const TensorShapeProto& shape = input_type->tensor_type().shape();
std::vector perm;
bool has_perm_attr = getRepeatedAttribute(ctx, "perm", perm);
if (!has_perm_attr) {
for (int i = shape.dim_size() - 1; i >= 0; --i)
perm.push_back(i);
} else if (!perm.empty()) {
// check if every index is valid
for (int64_t fromDimIndex : perm)
if (!(0 <= fromDimIndex && fromDimIndex < shape.dim_size())) {
std::ostringstream oss;
oss << "Invalid attribute perm {" << perm[0];
for (size_t i = 1; i != perm.size(); ++i) {
oss << ", " << perm[i];
}
oss << "}, input shape = {";
if (shape.dim_size() > 0) {
oss << shape.dim(0).dim_value();
for (int i = 1; i != shape.dim_size(); ++i) {
oss << ", " << shape.dim(i).dim_value();
}
oss << "}";
}
fail_type_inference(oss.str());
}
}
propagateElemTypeFromInputToOutput(ctx, 0, 0);
for (size_t i = 0; i < perm.size(); ++i) {
appendSingleDimCopiedFromInputTypeToOutputType(
ctx, 0, 0, static_cast(perm[i]));
}
}));
static const char* Scatter_ver11_doc = R"DOC(
This operator is deprecated. Please use ScatterElements, which provides the same functionality.
Scatter takes three inputs `data`, `updates`, and `indices` of the same
rank r >= 1 and an optional attribute axis that identifies an axis of `data`
(by default, the outer-most axis, that is axis 0). The output of the operation
is produced by creating a copy of the input `data`, and then updating its value
to values specified by `updates` at specific index positions specified by
`indices`. Its output shape is the same as the shape of `data`.
For each entry in `updates`, the target index in `data` is obtained by combining
the corresponding entry in `indices` with the index of the entry itself: the
index-value for dimension = axis is obtained from the value of the corresponding
entry in `indices` and the index-value for dimension != axis is obtained from the
index of the entry itself.
For instance, in a 2-D tensor case, the update corresponding to the [i][j] entry
is performed as below:
```
output[indices[i][j]][j] = updates[i][j] if axis = 0,
output[i][indices[i][j]] = updates[i][j] if axis = 1,
```
This operator is the inverse of GatherElements. It is similar to Torch's Scatter operation.
Example 1:
```
data = [
[0.0, 0.0, 0.0],
[0.0, 0.0, 0.0],
[0.0, 0.0, 0.0],
]
indices = [
[1, 0, 2],
[0, 2, 1],
]
updates = [
[1.0, 1.1, 1.2],
[2.0, 2.1, 2.2],
]
output = [
[2.0, 1.1, 0.0]
[1.0, 0.0, 2.2]
[0.0, 2.1, 1.2]
]
```
Example 2:
```
data = [[1.0, 2.0, 3.0, 4.0, 5.0]]
indices = [[1, 3]]
updates = [[1.1, 2.1]]
axis = 1
output = [[1.0, 1.1, 3.0, 2.1, 5.0]]
```
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Scatter,
11,
OpSchema()
.Deprecate()
.SetDoc(Scatter_ver11_doc)
.Attr(
"axis",
"Which axis to scatter on. Negative value means "
"counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data).",
AttributeProto::INT,
static_cast(0))
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(
1,
"indices",
"Tensor of int32/int64 indices, of r >= 1 (same rank as input). All index values are expected to be "
"within bounds [-s, s-1] along axis of size s. It is an error if any of the index values are out of bounds.",
"Tind")
.Input(
2,
"updates",
"Tensor of rank r >=1 (same rank and shape as indices)",
"T")
.Output(0, "output", "Tensor of rank r >= 1 (same rank as input).", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Input and output types can be of any tensor type.")
.TypeConstraint(
"Tind",
{"tensor(int32)", "tensor(int64)"},
"Constrain indices to integer types")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (hasNInputShapes(ctx, 1)) {
propagateShapeFromInputToOutput(ctx, 0, 0);
}
}));
static const char* ScatterND_ver11_doc = R"DOC(
ScatterND takes three inputs `data` tensor of rank r >= 1, `indices` tensor of rank q >= 1,
and `updates` tensor of rank q + r - indices.shape[-1] - 1. The output of the operation
is produced by creating a copy of the input `data`, and then updating its value to values
specified by `updates` at specific index positions specified by `indices`. Its output shape
is the same as the shape of `data`. Note that `indices` should not have duplicate entries.
That is, two or more `updates` for the same index-location is not supported.
`indices` is an integer tensor. Let k denote indices.shape[-1], the last dimension in the shape of `indices`.
`indices` is treated as a (q-1)-dimensional tensor of k-tuples, where each k-tuple is a partial-index into `data`.
Hence, k can be a value at most the rank of `data`. When k equals rank(data), each update entry specifies an
update to a single element of the tensor. When k is less than rank(data) each update entry specifies an
update to a slice of the tensor.
`updates` is treated as a (q-1)-dimensional tensor of replacement-slice-values. Thus, the
first (q-1) dimensions of updates.shape must match the first (q-1) dimensions of indices.shape.
The remaining dimensions of `updates` correspond to the dimensions of the
replacement-slice-values. Each replacement-slice-value is a (r-k) dimensional tensor,
corresponding to the trailing (r-k) dimensions of `data`. Thus, the shape of `updates`
must equal indices.shape[0:q-1] ++ data.shape[k:r-1], where ++ denotes the concatenation
of shapes.
The `output` is calculated via the following equation:
output = np.copy(data)
update_indices = indices.shape[:-1]
for idx in np.ndindex(update_indices):
output[indices[idx]] = updates[idx]
The order of iteration in the above loop is not specified.
In particular, indices should not have duplicate entries: that is, if idx1 != idx2, then indices[idx1] != indices[idx2].
This ensures that the output value does not depend on the iteration order.
This operator is the inverse of GatherND.
Example 1:
```
data = [1, 2, 3, 4, 5, 6, 7, 8]
indices = [[4], [3], [1], [7]]
updates = [9, 10, 11, 12]
output = [1, 11, 3, 10, 9, 6, 7, 12]
```
Example 2:
```
data = [[[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]],
[[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]],
[[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]],
[[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]]]
indices = [[0], [2]]
updates = [[[5, 5, 5, 5], [6, 6, 6, 6], [7, 7, 7, 7], [8, 8, 8, 8]],
[[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]]]
output = [[[5, 5, 5, 5], [6, 6, 6, 6], [7, 7, 7, 7], [8, 8, 8, 8]],
[[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]],
[[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]],
[[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]]]
```
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
ScatterND,
11,
OpSchema()
.SetDoc(ScatterND_ver11_doc)
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(1, "indices", "Tensor of rank q >= 1.", "tensor(int64)")
.Input(
2,
"updates",
"Tensor of rank q + r - indices_shape[-1] - 1.",
"T")
.Output(0, "output", "Tensor of rank r >= 1.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (hasNInputShapes(ctx, 1)) {
propagateShapeFromInputToOutput(ctx, 0, 0);
}
}));
static const char* ScatterElements_ver11_doc = R"DOC(
ScatterElements takes three inputs `data`, `updates`, and `indices` of the same
rank r >= 1 and an optional attribute axis that identifies an axis of `data`
(by default, the outer-most axis, that is axis 0). The output of the operation
is produced by creating a copy of the input `data`, and then updating its value
to values specified by `updates` at specific index positions specified by
`indices`. Its output shape is the same as the shape of `data`.
For each entry in `updates`, the target index in `data` is obtained by combining
the corresponding entry in `indices` with the index of the entry itself: the
index-value for dimension = axis is obtained from the value of the corresponding
entry in `indices` and the index-value for dimension != axis is obtained from the
index of the entry itself.
For instance, in a 2-D tensor case, the update corresponding to the [i][j] entry
is performed as below:
```
output[indices[i][j]][j] = updates[i][j] if axis = 0,
output[i][indices[i][j]] = updates[i][j] if axis = 1,
```
This operator is the inverse of GatherElements. It is similar to Torch's Scatter operation.
Example 1:
```
data = [
[0.0, 0.0, 0.0],
[0.0, 0.0, 0.0],
[0.0, 0.0, 0.0],
]
indices = [
[1, 0, 2],
[0, 2, 1],
]
updates = [
[1.0, 1.1, 1.2],
[2.0, 2.1, 2.2],
]
output = [
[2.0, 1.1, 0.0]
[1.0, 0.0, 2.2]
[0.0, 2.1, 1.2]
]
```
Example 2:
```
data = [[1.0, 2.0, 3.0, 4.0, 5.0]]
indices = [[1, 3]]
updates = [[1.1, 2.1]]
axis = 1
output = [[1.0, 1.1, 3.0, 2.1, 5.0]]
```
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
ScatterElements,
11,
OpSchema()
.SetDoc(ScatterElements_ver11_doc)
.Attr(
"axis",
"Which axis to scatter on. Negative value means "
"counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data).",
AttributeProto::INT,
static_cast(0))
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(
1,
"indices",
"Tensor of int32/int64 indices, of r >= 1 (same rank as input). All index values are expected to be "
"within bounds [-s, s-1] along axis of size s. It is an error if any of the index values are out of bounds.",
"Tind")
.Input(
2,
"updates",
"Tensor of rank r >=1 (same rank and shape as indices)",
"T")
.Output(0, "output", "Tensor of rank r >= 1 (same rank as input).", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Input and output types can be of any tensor type.")
.TypeConstraint(
"Tind",
{"tensor(int32)", "tensor(int64)"},
"Constrain indices to integer types")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (hasNInputShapes(ctx, 1)) {
propagateShapeFromInputToOutput(ctx, 0, 0);
}
}));
static const char* Gather_ver11_doc = R"DOC(
Given `data` tensor of rank r >= 1, and `indices` tensor of rank q, gather
entries of the axis dimension of `data` (by default outer-most one as axis=0) indexed by `indices`, and concatenates
them in an output tensor of rank q + (r - 1).
axis = 0 :
Let
k = indices[i_{0}, ..., i_{q-1}]
Then
output[i_{0}, ..., i_{q-1}, j_{0}, ..., j_{r-2}] = input[k , j_{0}, ..., j_{r-2}]
```
data = [
[1.0, 1.2],
[2.3, 3.4],
[4.5, 5.7],
]
indices = [
[0, 1],
[1, 2],
]
output = [
[
[1.0, 1.2],
[2.3, 3.4],
],
[
[2.3, 3.4],
[4.5, 5.7],
],
]
```
axis = 1 :
Let
k = indices[i_{0}, ..., i_{q-1}]
Then
output[i_{0}, ..., i_{q-1}, j_{0}, ..., j_{r-2}] = input[j_{0}, k, j_{1}, ..., j_{r-2}]
```
data = [
[1.0, 1.2, 1.9],
[2.3, 3.4, 3.9],
[4.5, 5.7, 5.9],
]
indices = [
[0, 2],
]
axis = 1,
output = [
[
[1.0, 1.9],
[2.3, 3.9],
[4.5, 5.9],
],
]
```
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Gather,
11,
OpSchema()
.SetDoc(Gather_ver11_doc)
.Attr(
"axis",
"Which axis to gather on. Negative value means "
"counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data).",
AttributeProto::INT,
static_cast(0))
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(
1,
"indices",
"Tensor of int32/int64 indices, of any rank q. All index values are expected to be within bounds [-s, s-1] "
"along axis of size s. It is an error if any of the index values are out of bounds.",
"Tind")
.Output(0, "output", "Tensor of rank q + (r - 1).", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to any tensor type.")
.TypeConstraint(
"Tind",
{"tensor(int32)", "tensor(int64)"},
"Constrain indices to integer types")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 2)) {
return;
}
const TensorShapeProto& data_shape =
ctx.getInputType(0)->tensor_type().shape();
const TensorShapeProto& indices_shape =
ctx.getInputType(1)->tensor_type().shape();
int r = data_shape.dim_size();
if (r < 1) {
fail_shape_inference("data tensor must have rank >= 1");
}
int q = indices_shape.dim_size();
int axis = static_cast(getAttribute(ctx, "axis", 0));
if (axis < -r || axis >= r) {
fail_shape_inference("axis must be in [-r, r-1]");
}
if (axis < 0) {
axis += r;
}
int out_rank = q + r - 1;
if (out_rank == 0) {
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
}
for (int i = 0; i < out_rank; ++i) {
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() = (i < axis) ? data_shape.dim(i) : // i < axis < r
(i >= axis && i < axis + q) ? indices_shape.dim(i - axis)
: // i - axis < q
data_shape.dim(i - q + 1); // i < out_rank < q + r - 1
}
}));
static const char* GatherElements_ver11_doc = R"DOC(
GatherElements takes two inputs `data` and `indices` of the same rank r >= 1
and an optional attribute `axis` that identifies an axis of `data`
(by default, the outer-most axis, that is axis 0). It is an indexing operation
that produces its output by indexing into the input data tensor at index
positions determined by elements of the `indices` tensor.
Its output shape is the same as the shape of `indices` and consists of one value
(gathered from the `data`) for each element in `indices`.
For instance, in the 3-D case (r = 3), the output produced is determined
by the following equations:
```
out[i][j][k] = input[index[i][j][k]][j][k] if axis = 0,
out[i][j][k] = input[i][index[i][j][k]][k] if axis = 1,
out[i][j][k] = input[i][j][index[i][j][k]] if axis = 2,
```
This operator is also the inverse of ScatterElements. It is similar to Torch's gather operation.
Example 1:
```
data = [
[1, 2],
[3, 4],
]
indices = [
[0, 0],
[1, 0],
]
axis = 1
output = [
[
[1, 1],
[4, 3],
],
]
```
Example 2:
```
data = [
[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
]
indices = [
[1, 2, 0],
[2, 0, 0],
]
axis = 0
output = [
[
[4, 8, 3],
[7, 2, 3],
],
]
```
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
GatherElements,
11,
OpSchema()
.SetDoc(GatherElements_ver11_doc)
.Attr(
"axis",
"Which axis to gather on. Negative value means "
"counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data).",
AttributeProto::INT,
static_cast(0))
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(
1,
"indices",
"Tensor of int32/int64 indices, with the same rank r as the input. All index values are expected to be "
"within bounds [-s, s-1] along axis of size s. It is an error if any of the index values are out of bounds.",
"Tind")
.Output(0, "output", "Tensor of the same shape as indices.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to any tensor type.")
.TypeConstraint(
"Tind",
{"tensor(int32)", "tensor(int64)"},
"Constrain indices to integer types")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
// propagate indices' shape to output if it exists
if (hasInputShape(ctx, 1)) {
propagateShapeFromInputToOutput(ctx, 1, 0);
}
}));
static const char* Squeeze_ver11_doc = R"DOC(
Remove single-dimensional entries from the shape of a tensor.
Takes a parameter `axes` with a list of axes to squeeze.
If `axes` is not provided, all the single dimensions will be removed from
the shape. If an axis is selected with shape entry not equal to one, an error is raised.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Squeeze,
11,
OpSchema()
.Attr(
"axes",
"List of integers indicating the dimensions to squeeze. Negative value means counting dimensions "
"from the back. Accepted range is [-r, r-1] where r = rank(data).",
AttributeProto::INTS,
OPTIONAL_VALUE)
.SetDoc(Squeeze_ver11_doc)
.Input(0, "data", "Tensors with at least max(dims) dimensions.", "T")
.Output(0, "squeezed", "Reshaped tensor with same data as input.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
std::vector axes;
if (!getRepeatedAttribute(ctx, "axes", axes)) {
return;
}
if (!ctx.getInputType(0)->tensor_type().has_shape()) {
return;
}
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
const auto input_ndim = input_shape.dim_size();
std::transform(
axes.begin(),
axes.end(),
axes.begin(),
[&](int64_t axis) -> int64_t {
return axis < 0 ? axis + input_ndim : axis;
});
for (int i = 0, j = 0; i < input_ndim; ++i) {
if (std::find(axes.begin(), axes.end(), i) != axes.end()) {
if (input_shape.dim(i).has_dim_value() &&
input_shape.dim(i).dim_value() != 1) {
fail_shape_inference(
"Dimension of input ",
i,
" must be 1 instead of ",
input_shape.dim(i).dim_value());
}
++j;
} else {
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() = input_shape.dim(i);
}
}
}));
static const char* Unsqueeze_ver11_doc = R"DOC(
Insert single-dimensional entries to the shape of an input tensor (`data`).
Takes one required argument `axes` - which contains a list of dimension indices and this operator will insert a dimension of value `1` into the corresponding index of the output tensor (`expanded`).
For example:
Given an input tensor (`data`) of shape [3, 4, 5], then
Unsqueeze(data, axes=[0, 4]) outputs a tensor (`expanded`) containing same data as `data` but with shape [1, 3, 4, 5, 1].
The attribute `axes` should not contain any duplicate entries. It is an error if it contains duplicates.
The rank of the output tensor (`output_rank`) is the rank of the input tensor (`data`) plus the number of values in `axes`.
Each value in `axes` should be within the (inclusive) range [-output_rank , output_rank - 1].
The order of values in `axes` does not matter and can come in any order.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Unsqueeze,
11,
OpSchema()
.Attr(
"axes",
"List of integers indicating the dimensions to be inserted. Negative value means counting dimensions "
"from the back. Accepted range is [-r, r-1] where r = rank(expanded).",
AttributeProto::INTS)
.SetDoc(Unsqueeze_ver11_doc)
.Input(0, "data", "Original tensor", "T")
.Output(0, "expanded", "Reshaped tensor with same data as input.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 1)) {
return;
}
std::vector axes;
if (!getRepeatedAttribute(ctx, "axes", axes)) {
return;
}
// validate 'axes' for duplicate entries
std::unordered_set unique_values;
for (const auto val : axes) {
if (unique_values.find(val) != unique_values.end()) {
fail_shape_inference(
"'axes' attribute must not contain any duplicates");
}
unique_values.insert(val);
}
if (!ctx.getInputType(0)->tensor_type().has_shape()) {
return;
}
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
const auto input_ndim = input_shape.dim_size();
const auto output_ndim = input_ndim + static_cast(axes.size());
for (auto& axe : axes) {
if (axe < -output_ndim || axe >= output_ndim) {
fail_shape_inference(
"values in 'axes' are beyond the bounds of the computed output shape");
}
if (axe < 0) {
axe += output_ndim;
}
}
// sort after correcting negative axes values (if any) in the previous
// step
std::sort(axes.begin(), axes.end());
int j = 0;
for (int i = 0; i < input_ndim; ++i) {
while (static_cast(j) < axes.size() &&
axes[j] ==
ctx.getOutputType(0)->tensor_type().shape().dim_size()) {
ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim()
->set_dim_value(1);
++j;
}
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() =
ctx.getInputType(0)->tensor_type().shape().dim(i);
}
while (static_cast(j) < axes.size() &&
axes[j] ==
ctx.getOutputType(0)->tensor_type().shape().dim_size()) {
ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim()
->set_dim_value(1);
++j;
}
}));
static const char* SpaceToDepth_ver1_doc =
R"DOC(SpaceToDepth rearranges blocks of spatial data into depth. More specifically,
this op outputs a copy of the input tensor where values from the height and width dimensions
are moved to the depth dimension.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
SpaceToDepth,
1,
OpSchema()
.Attr(
"blocksize",
"Blocks of [blocksize, blocksize] are moved.",
AttributeProto::INT)
.SetDoc(SpaceToDepth_ver1_doc)
.Input(
0,
"input",
"Input tensor of [N,C,H,W], where N is the batch axis, C is the channel or depth"
", H is the height and W is the width.",
"T")
.Output(
0,
"output",
"Output tensor of [N, C * blocksize * blocksize, H/blocksize, W/blocksize].",
"T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
auto blocksize = getAttribute(ctx, "blocksize", 0);
if (blocksize <= 0)
fail_shape_inference("Blocksize must be positive");
if (hasInputShape(ctx, 0)) {
auto& input_shape = getInputShape(ctx, 0);
if (input_shape.dim_size() == 4) {
// TODO: Clarify what behavior should be if H or W is not a
// multiple of blocksize.
updateOutputShape(
ctx,
0,
{input_shape.dim(0),
input_shape.dim(1) * (blocksize * blocksize),
input_shape.dim(2) / blocksize,
input_shape.dim(3) / blocksize});
} else
fail_shape_inference("Input tensor must be 4-dimensional");
}
}));
static const char* DepthToSpace_ver11_doc =
R"DOC(DepthToSpace rearranges (permutes) data from depth into blocks of spatial data.
This is the reverse transformation of SpaceToDepth. More specifically, this op outputs a copy of
the input tensor where values from the depth dimension are moved in spatial blocks to the height
and width dimensions. By default, `mode` = `DCR`.
In the DCR mode, elements along the depth dimension from the input tensor are rearranged in the
following order: depth, column, and then row. The output y is computed from the input x as below:
b, c, h, w = x.shape
tmp = np.reshape(x, [b, blocksize, blocksize, c // (blocksize**2), h, w])
tmp = np.transpose(tmp, [0, 3, 4, 1, 5, 2])
y = np.reshape(tmp, [b, c // (blocksize**2), h * blocksize, w * blocksize])
In the CRD mode, elements along the depth dimension from the input tensor are rearranged in the
following order: column, row, and the depth. The output y is computed from the input x as below:
b, c, h, w = x.shape
tmp = np.reshape(x, [b, c // (blocksize ** 2), blocksize, blocksize, h, w])
tmp = np.transpose(tmp, [0, 1, 4, 2, 5, 3])
y = np.reshape(tmp, [b, c // (blocksize ** 2), h * blocksize, w * blocksize])
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
DepthToSpace,
11,
OpSchema()
.Attr(
"blocksize",
"Blocks of [blocksize, blocksize] are moved.",
AttributeProto::INT)
.Attr(
"mode",
"DCR (default) for depth-column-row order re-arrangement. Use CRD for column-row-depth order.",
AttributeProto::STRING,
std::string("DCR"))
.SetDoc(DepthToSpace_ver11_doc)
.Input(
0,
"input",
"Input tensor of [N,C,H,W], where N is the batch axis, C is the channel or depth"
", H is the height and W is the width.",
"T")
.Output(
0,
"output",
"Output tensor of [N, C/(blocksize * blocksize), H * blocksize, W * blocksize].",
"T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
auto blocksize = getAttribute(ctx, "blocksize", 0);
if (blocksize <= 0)
fail_shape_inference("Blocksize must be positive");
if (hasInputShape(ctx, 0)) {
auto& input_shape = getInputShape(ctx, 0);
if (input_shape.dim_size() == 4) {
// TODO: Clarify what behavior should be if C is not a multiple of
// blocksize*blocksize.
updateOutputShape(
ctx,
0,
{input_shape.dim(0),
input_shape.dim(1) / (blocksize * blocksize),
input_shape.dim(2) * blocksize,
input_shape.dim(3) * blocksize});
} else
fail_shape_inference("Input tensor must be 4-dimensional");
}
}));
static const char* Tile_ver6_doc =
R"DOC(Constructs a tensor by tiling a given tensor.
This is the same as function `tile` in Numpy, but no broadcast.
For example A = [[1, 2], [3, 4]], B = [1, 2], tile(A, B) = [[1, 2, 1, 2], [3, 4, 3, 4]]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Tile,
6,
OpSchema()
.SetDoc(Tile_ver6_doc)
.Input(0, "input", "Input tensor of any shape.", "T")
.Input(
1,
"repeats",
"1D int64 tensor of the same length as input's dimension number, "
"includes numbers of repeated copies along input's dimensions.",
"T1")
.Output(
0,
"output",
"Output tensor of the same dimension and type as tensor input. "
"output_dim[i] = input_dim[i] * repeats[i]",
"T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeConstraint(
"T1",
{"tensor(int64)"},
"Constrain repeat's type to int64 tensors.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Type inference
propagateElemTypeFromInputToOutput(ctx, 0, 0);
// Shape inference
// Needs atleast the first input to proceed
if (!hasNInputShapes(ctx, 1)) {
return;
}
const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
const auto input_rank = input_shape.dim_size();
const auto* repeats_inputs = ctx.getInputData(1);
auto* output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
if (nullptr != repeats_inputs && hasNInputShapes(ctx, 2)) {
// shape inference is possible only when 'repeats' is an initializer
const auto& repeats_shape =
ctx.getInputType(1)->tensor_type().shape();
if (repeats_shape.dim_size() != 1 ||
repeats_inputs->data_type() != TensorProto::INT64)
fail_shape_inference(
"'Repeats' input must be 1D tensor of type int64");
const auto& repeats_data = ParseData(repeats_inputs);
if (repeats_data.size() != static_cast(input_rank))
fail_shape_inference(
"'Repeats' input has incorrect number of values. "
"The number of values in 'repeats' must be equal "
"to the number of input dimensions.");
for (size_t i = 0; (int64_t)i < input_rank; ++i) {
const auto& input_dim = input_shape.dim((int)i);
auto* output_dim = output_shape->add_dim();
if (input_dim.has_dim_value()) {
output_dim->set_dim_value(
input_dim.dim_value() * repeats_data[i]);
}
}
} else {
// Infer output shape's rank in any case (if repeats data is not
// available)
auto* output_shape_0 = getOutputShape(ctx, 0);
for (size_t i = 0; (int64_t)i < input_rank; ++i) {
output_shape_0->add_dim();
}
}
return;
}));
static const char* Upsample_ver10_doc = R"DOC(
Upsample the input tensor.
Each dimension value of the output tensor is:
output_dimension = floor(input_dimension * scale).
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Upsample,
10,
OpSchema()
.Deprecate()
.Attr(
"mode",
"Two interpolation modes: nearest (default), and linear (including bilinear, trilinear, etc)",
AttributeProto::STRING,
std::string("nearest"))
.Input(0, "X", "N-D tensor", "T")
.Input(
1,
"scales",
"The scale array along each dimension. It takes value greater than or equal to 1."
" The number of elements of 'scales' should be the same as the rank of input 'X'.",
"tensor(float)")
.Output(0, "Y", "N-D tensor after resizing", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input 'X' and output 'Y' to all tensor types.")
.SetDoc(Upsample_ver10_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
resizeShapeInference_opset7_to_10(ctx);
}));
static const char* Resize_ver11_doc = R"DOC(
Resize the input tensor. In general, it calculates every value in the output tensor as a weighted average of neighborhood (a.k.a. sampling locations) in the input tensor.
Each dimension value of the output tensor is:
output_dimension = floor(input_dimension * (roi_end - roi_start) * scale) if input \"sizes\" is not specified.
)DOC";
static const char* Resize_attr_coordinate_transformation_mode_doc = R"DOC(
This attribute describes how to transform the coordinate in the resized tensor to the coordinate in the original tensor.
The coordinate of each dimension is transformed individually. Let's describe a case using axis x as an example.
Denote x_resized as the coordinate of axis x in the resized tensor, x_original as the coordinate of axis x in the original tensor, length_original as the length of the original tensor in axis x, length_resized as the length of the resized tensor in axis x, roi_x = (start_x, end_x) of the axis x in input "roi", scale = length_resized / length_original,
if coordinate_transformation_mode is "half_pixel",
x_original = (x_resized + 0.5) / scale - 0.5,
if coordinate_transformation_mode is "pytorch_half_pixel",
x_original = length_resized > 1 ? (x_resized + 0.5) / scale - 0.5 : 0,
if coordinate_transformation_mode is "align_corners",
x_original = x_resized * (length_original - 1) / (length_resized - 1),
if coordinate_transformation_mode is "asymmetric",
x_original = x_resized / scale,
if coordinate_transformation_mode is "tf_half_pixel_for_nn",
x_original = (x_resized + 0.5) / scale,
if coordinate_transformation_mode is "tf_crop_and_resize",
x_original = length_resized > 1 ? start_x * (length_original - 1) + x_resized * (end_x - start_x) * (length_original - 1) / (length_resized - 1) : 0.5 * (start_x + end_x) * (length_original - 1).)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Resize,
11,
OpSchema()
.Attr(
"mode",
"Three interpolation modes: nearest (default), linear and cubic. "
"The \"linear\" mode includes linear interpolation for 1D tensor and N-linear interpolation for N-D tensor (for example, bilinear interpolation for 2D tensor). "
"The \"cubic\" mode includes cubic interpolation for 1D tensor and N-cubic interpolation for N-D tensor (for example, bicubic interpolation for 2D tensor).",
AttributeProto::STRING,
std::string("nearest"))
.Attr(
"cubic_coeff_a",
"The coefficient 'a' used in cubic interpolation. Two common choice are -0.5 (in some cases of TensorFlow) and -0.75"
" (in PyTorch). Check out Equation (4) in https://ieeexplore.ieee.org/document/1163711 for the details. "
"This attribute is valid only if \"mode\" is \"cubic\".",
AttributeProto::FLOAT,
static_cast(-0.75))
.Attr(
"exclude_outside",
"If set to 1, the weight of sampling locations outside the tensor will be set to 0"
" and the weight will be renormalized so that their sum is 1.0. The default value is 0.",
AttributeProto::INT,
static_cast(0))
.Attr(
"coordinate_transformation_mode",
Resize_attr_coordinate_transformation_mode_doc,
AttributeProto::STRING,
std::string("half_pixel"))
.Attr(
"nearest_mode",
"Four modes: round_prefer_floor (default, as known as round half down), round_prefer_ceil (as known as round half up), floor, ceil. Only used by nearest interpolation. It indicates how to get \"nearest\" pixel in input tensor from x_original, so this attribute is valid only if \"mode\" is \"nearest\".",
AttributeProto::STRING,
std::string("round_prefer_floor"))
.Attr(
"extrapolation_value",
"When coordinate_transformation_mode is \"tf_crop_and_resize\" and x_original is outside the range [0, length_original - 1], this value is used as the corresponding output value. Default is 0.0f.",
AttributeProto::FLOAT,
static_cast(0))
.Input(0, "X", "N-D tensor", "T1")
.Input(
1,
"roi",
"1-D tensor given as [start1, ..., startN, end1, ..., endN], where N is the rank of X. The RoIs' coordinates are normalized in the coordinate system of the input image. It only takes effect when coordinate_transformation_mode is \"tf_crop_and_resize\"",
"T2")
.Input(
2,
"scales",
"The scale array along each dimension. It takes value greater than 0. If it's less than 1,"
" it's sampling down, otherwise, it's upsampling. The number of elements of 'scales' should"
" be the same as the rank of input 'X'. Only one of 'scales' and 'sizes' can be specified. If 'size' is needed, the user can use an empty string as the name of 'scales' in this operator's input list.",
"tensor(float)")
.Input(
3,
"sizes",
"The size of the output tensor. The number of elements of 'sizes' should be the same as the"
" rank of input 'X'. Only one of 'scales' and 'sizes' can be specified.",
"tensor(int64)",
OpSchema::Optional)
.Output(0, "Y", "N-D tensor after resizing", "T1")
.TypeConstraint(
"T1",
OpSchema::all_tensor_types(),
"Constrain input 'X' and output 'Y' to all tensor types.")
.TypeConstraint(
"T2",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain roi type to float or double.")
.SetDoc(Resize_ver11_doc)
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
resizeShapeInference(ctx, true);
}));
ONNX_OPERATOR_SET_SCHEMA(
Identity,
1,
OpSchema()
.SetDoc("Identity operator")
.Input(0, "input", "Input tensor", "T")
.Output(0, "output", "Tensor to copy input into.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction(propagateShapeAndTypeFromFirstInput));
static const char* Compress_ver11_doc = R"DOC(
Selects slices from an input tensor along a given axis where condition evaluates to True for each axis index.
In case axis is not provided, input is flattened before elements are selected.
Compress behaves like numpy.compress: https://docs.scipy.org/doc/numpy/reference/generated/numpy.compress.html
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Compress,
11,
OpSchema()
.SetDoc(Compress_ver11_doc)
.Attr(
"axis",
"(Optional) Axis along which to take slices. If not specified, "
"input is flattened before elements being selected. Negative value means counting dimensions "
"from the back. Accepted range is [-r, r-1] where r = rank(input).",
AttributeProto::INT,
OPTIONAL_VALUE)
.Input(0, "input", "Tensor of rank r >= 1.", "T")
.Input(
1,
"condition",
"Rank 1 tensor of booleans to indicate which slices or data elements to be selected. "
"Its length can be less than the input length along the axis "
"or the flattened input size if axis is not specified. "
"In such cases data slices or elements exceeding the condition length are discarded.",
"T1")
.Output(
0,
"output",
"Tensor of rank r if axis is specified. Otherwise output is a Tensor of rank 1.",
"T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeConstraint(
"T1",
{"tensor(bool)"},
"Constrains to boolean tensors.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (hasInputShape(ctx, 0)) {
const TensorShapeProto& indices_shape =
ctx.getInputType(0)->tensor_type().shape();
int r = indices_shape.dim_size();
if (r < 1) {
fail_shape_inference("Indices tensor must have rank >= 1");
}
auto axisAttr = ctx.getAttribute("axis");
if (axisAttr) {
int axis = static_cast(axisAttr->i());
if (axis < -r || axis >= r) {
fail_shape_inference(
"'axis' must be in [-rank(indices), rank(indices)-1]");
}
if (axis < 0) {
axis += r;
}
}
}
}));
static const char* OneHot_ver11_doc = R"DOC(
Produces a one-hot tensor based on inputs.
The locations represented by the index values in the 'indices' input tensor will have 'on_value'
and the other locations will have 'off_value' in the output tensor, where 'on_value' and 'off_value'
are specified as part of required input argument 'values', which is a two-element tensor of format
[off_value, on_value]. The rank of the output tensor will be one greater than the rank of the
input tensor. The additional dimension is for one-hot representation. The additional dimension will
be inserted at the position specified by 'axis'. If 'axis' is not specified then then additional
dimension will be inserted as the innermost dimension, i.e. axis=-1. The size of the additional
dimension is specified by required scalar input 'depth'. The type of the output tensor is the same
as the type of the 'values' input. Any entries in the 'indices' input tensor with values outside
the range [-depth, depth-1] will result in one-hot representation with all 'off_value' values in the
output tensor.
when axis = 0:
output[input[i, j, k], i, j, k] = 1 for all i, j, k and 0 otherwise.
when axis = -1:
output[i, j, k, input[i, j, k]] = 1 for all i, j, k and 0 otherwise.
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
OneHot,
11,
OpSchema()
.SetDoc(OneHot_ver11_doc)
.Attr(
"axis",
"(Optional) Axis along which one-hot representation in added. Default: axis=-1. "
"axis=-1 means that the additional dimension will be inserted as the "
"innermost/last dimension in the output tensor. Negative value means counting dimensions "
"from the back. Accepted range is [-r-1, r] where r = rank(indices).",
AttributeProto::INT,
static_cast(-1))
.Input(
0,
"indices",
"Input tensor containing indices. Any entries in the 'indices' input tensor with "
"values outside the range [-depth, depth-1] will result in one-hot representation with all "
"'off_value' values in the output tensor."
"In case 'indices' is of non-integer type, the values will be casted to int64 before use.",
"T1")
.Input(
1,
"depth",
"Scalar specifying the number of classes in one-hot tensor. This is also the size "
"of the one-hot dimension (specified by 'axis' attribute) added on in the output "
"tensor. The values in the 'indices' input tensor are expected to be "
"in the range [-depth, depth-1]. "
"In case 'depth' is of non-integer type, it will be casted to int64 before use.",
"T2")
.Input(
2,
"values",
"Rank 1 tensor containing exactly two elements, in the format [off_value, on_value], "
"where 'on_value' is the value used for filling locations specified in 'indices' input "
"tensor, and 'off_value' is the value used for filling locations other than those specified "
"in 'indices' input tensor. ",
"T3")
.Output(
0,
"output",
"Tensor of rank one greater than input tensor 'indices', i.e. rank(output) = rank(indices) + 1. "
"The data type for the elements of the output tensor is the same as the type of input 'values' "
"is used.",
"T3")
.TypeConstraint(
"T1",
OpSchema::all_numeric_types(),
"Constrains input to only numeric types.")
.TypeConstraint(
"T2",
OpSchema::all_numeric_types(),
"Constrains input to only numeric types.")
.TypeConstraint(
"T3",
OpSchema::all_tensor_types(),
"Constrain to any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Check that the node has three inputs.
if (ctx.getNumInputs() != 3) {
fail_type_inference("OneHot node must have three inputs.");
}
// Input 'depth' must be a scalar or a single-element vector.
// TODO: Ideally to match spec for this input only Scalar should
// be allowed. Making this change now can affect backward
// compatibility for this op. Since this does not seem like a good
// justification to update version for this op, allowing both scalar
// and 1 element vector for now. In future when version update for
// this op is done we should only allow scalar or chage the spec to
// allow both.
if (hasInputShape(ctx, 1)) {
auto& depth_shape = getInputShape(ctx, 1);
if (depth_shape.dim_size() != 0 && depth_shape.dim_size() != 1) {
fail_type_inference(
"Input 'depth' must be a scalar or rank 1 tensor.");
}
if (depth_shape.dim_size() == 1 &&
depth_shape.dim((int)0).has_dim_value() &&
depth_shape.dim((int)0).dim_value() != 1) {
fail_type_inference(
"Input 'depth' must have exactly one element.");
}
}
// Input 'values' must be a two-element vector.
if (hasInputShape(ctx, 2)) {
auto& values_shape = getInputShape(ctx, 2);
if (values_shape.dim_size() != 1) {
fail_type_inference("Input 'values' must be rank 1 tensor.");
}
if (values_shape.dim((int)0).has_dim_value() &&
values_shape.dim((int)0).dim_value() != 2) {
fail_type_inference(
"Input 'values' must have exactly two elements.");
}
}
// Set output type to be the same as the third input, 'values'.
propagateElemTypeFromInputToOutput(ctx, 2, 0);
// Set the output shape, if input 0 (indices) shape is available.
if (hasInputShape(ctx, 0)) {
const TensorShapeProto& indices_shape =
ctx.getInputType(0)->tensor_type().shape();
int r = indices_shape.dim_size();
if (r < 1) {
fail_shape_inference("Indices tensor must have rank >= 1");
}
int out_rank = r + 1;
int axis = static_cast(getAttribute(ctx, "axis", -1));
if (axis < -out_rank || axis >= out_rank) {
fail_shape_inference(
"'axis' must be in [-rank(indices), rank(indices)-1]");
}
if (axis < 0) {
axis += out_rank;
}
auto* output_shape = getOutputShape(ctx, 0);
for (int i = 0; i < out_rank; ++i) {
auto* dim = output_shape->add_dim();
if (i < axis) {
if (indices_shape.dim(i).has_dim_value()) {
dim->set_dim_value(indices_shape.dim(i).dim_value());
} else if (indices_shape.dim(i).has_dim_param()) {
dim->set_dim_param(indices_shape.dim(i).dim_param());
}
} else if (i > axis) {
if (indices_shape.dim(i - 1).has_dim_value()) {
dim->set_dim_value(indices_shape.dim(i - 1).dim_value());
} else if (indices_shape.dim(i - 1).has_dim_param()) {
dim->set_dim_param(indices_shape.dim(i - 1).dim_param());
}
}
}
}
}));
ONNX_OPERATOR_SET_SCHEMA(
IsNaN,
9,
OpSchema()
.SetDoc(R"DOC(Returns which elements of the input are NaN.)DOC")
.Input(0, "X", "input", "T1")
.Output(0, "Y", "output", "T2")
.TypeConstraint(
"T1",
{"tensor(float16)", "tensor(float)", "tensor(double)"},
"Constrain input types to float tensors.")
.TypeConstraint(
"T2",
{"tensor(bool)"},
"Constrain output types to boolean tensors.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
updateOutputElemType(ctx, 0, TensorProto::BOOL);
if (hasInputShape(ctx, 0)) {
propagateShapeFromInputToOutput(ctx, 0, 0);
}
}));
ONNX_OPERATOR_SET_SCHEMA(
IsInf,
10,
OpSchema()
.SetDoc(R"DOC(Map infinity to true and other values to false.)DOC")
.Input(0, "X", "input", "T1")
.Output(0, "Y", "output", "T2")
.Attr(
"detect_positive",
"(Optional) Whether map positive infinity to true. Default to 1 "
"so that positive infinity induces true. Set this attribute to 0 "
"if positive infinity should be mapped to false.",
AttributeProto::INT,
static_cast(1))
.Attr(
"detect_negative",
"(Optional) Whether map negative infinity to true. Default to 1 "
"so that negative infinity induces true. Set this attribute to 0 "
"if negative infinity should be mapped to false.",
AttributeProto::INT,
static_cast(1))
.TypeConstraint(
"T1",
{"tensor(float)", "tensor(double)"},
"Constrain input types to float tensors.")
.TypeConstraint(
"T2",
{"tensor(bool)"},
"Constrain output types to boolean tensors.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
updateOutputElemType(ctx, 0, TensorProto::BOOL);
if (hasInputShape(ctx, 0)) {
propagateShapeFromInputToOutput(ctx, 0, 0);
}
}));
static const char* Where_ver9_doc = R"DOC(
Return elements, either from X or Y, depending on condition
(with Numpy-style broadcasting support).
Where behaves like numpy.where with three parameters:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.where.html
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Where,
9,
OpSchema()
.SetDoc(Where_ver9_doc)
.Input(
0,
"condition",
"When True (nonzero), yield X, otherwise yield Y",
"B")
.Input(
1,
"X",
"values selected at indices where condition is True",
"T")
.Input(
2,
"Y",
"values selected at indices where condition is False",
"T")
.Output(
0,
"output",
"Tensor of shape equal to the broadcasted shape of condition, X, and Y.",
"T")
.TypeConstraint("B", {"tensor(bool)"}, "Constrain to boolean tensors.")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 1, 0);
if (hasNInputShapes(ctx, 3)) {
std::vector shapes;
shapes.push_back(&ctx.getInputType(0)->tensor_type().shape());
shapes.push_back(&ctx.getInputType(1)->tensor_type().shape());
shapes.push_back(&ctx.getInputType(2)->tensor_type().shape());
multidirectionalBroadcastShapeInference(
shapes,
*ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape());
}
}));
static const char* NonZero_ver9_doc = R"DOC(
Returns the indices of the elements that are non-zero
(in row-major order - by dimension).
NonZero behaves similar to numpy.nonzero:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.nonzero.html
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
NonZero,
9,
OpSchema()
.SetDoc(NonZero_ver9_doc)
.Input(0, "X", "input", "T")
.Output(0, "Y", "output", "tensor(int64)")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain to all tensor types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
updateOutputElemType(ctx, 0, TensorProto::INT64);
}));
static const char* ReverseSequence_ver10_doc = R"DOC(
Reverse batch of sequences having different lengths specified by `sequence_lens`.
For each slice i iterating on batch axis, the operator reverses the first sequence_lens[i] elements on time axis,
and copies elements whose index's beyond sequence_lens[i] to the output. So the output slice i contains reversed
sequences on the first sequence_lens[i] elements, then have original values copied for the other elements.
Example 1:
input = [[0.0, 4.0, 8.0, 12.0],
[1.0, 5.0, 9.0, 13.0],
[2.0, 6.0, 10.0, 14.0],
[3.0, 7.0, 11.0, 15.0]]
sequence_lens = [4, 3, 2, 1]
time_axis = 0
batch_axis = 1
output = [[3.0, 6.0, 9.0, 12.0],
[2.0, 5.0, 8.0, 13.0],
[1.0, 4.0, 10.0, 14.0],
[0.0, 7.0, 11.0, 15.0]]
Example 2:
input = [[0.0, 1.0, 2.0, 3.0 ],
[4.0, 5.0, 6.0, 7.0 ],
[8.0, 9.0, 10.0, 11.0],
[12.0, 13.0, 14.0, 15.0]]
sequence_lens = [1, 2, 3, 4]
time_axis = 1
batch_axis = 0
output = [[0.0, 1.0, 2.0, 3.0 ],
[5.0, 4.0, 6.0, 7.0 ],
[10.0, 9.0, 8.0, 11.0],
[15.0, 14.0, 13.0, 12.0]]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
ReverseSequence,
10,
OpSchema()
.SetDoc(ReverseSequence_ver10_doc)
.Attr(
"time_axis",
"(Optional) Specify which axis is time axis. Must be one of 0 (default), or 1.",
AttributeProto::INT,
static_cast(0))
.Attr(
"batch_axis",
"(Optional) Specify which axis is batch axis. Must be one of 1 (default), or 0.",
AttributeProto::INT,
static_cast(1))
.Input(0, "input", "Tensor of rank r >= 2.", "T")
.Input(
1,
"sequence_lens",
"Tensor specifying lengths of the sequences in a batch. It has shape `[batch_size]`.",
"tensor(int64)")
.Output(0, "Y", "Tensor with same shape of input.", "T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Input and output types can be of any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
propagateElemTypeFromInputToOutput(ctx, 0, 0);
if (!hasNInputShapes(ctx, 2)) {
return;
}
auto& first_input_shape = getInputShape(ctx, 0);
if (first_input_shape.dim_size() < 2) {
fail_shape_inference("'input' must have rank >= 2");
}
auto& seq_len_input_shape = getInputShape(ctx, 1);
if (seq_len_input_shape.dim_size() != 1) {
fail_shape_inference("'sequence_lens' must have rank of 1");
}
propagateShapeFromInputToOutput(ctx, 0, 0);
}));
static const char* Unique_ver11_doc = R"DOC(
Find the unique elements of a tensor. When an optional attribute 'axis' is provided, unique subtensors sliced along the 'axis' are returned.
Otherwise the input tensor is flattened and unique values of the flattened tensor are returned.
This operator returns the unique values or sliced unique subtensors of the input tensor and three optional outputs.
The first output tensor 'Y' contains all unique values or subtensors of the input.
The second optional output tensor 'indices' contains indices of 'Y' elements' first occurance in 'X'..
The third optional output tensor 'inverse_indices' contains, for elements of 'X', its corresponding indices in 'Y'. ".
The fourth optional output tensor 'counts' contains the count of each element of 'Y' in the input.
Outputs are either sorted in ascending order or optionally in the order of the first occurrence of the values in the input.
https://docs.scipy.org/doc/numpy/reference/generated/numpy.unique.html
Example 1:
input_X = [2, 1, 1, 3, 4, 3]
attribute_sorted = 0
attribute_axis = None
output_Y = [2, 1, 3, 4]
output_indices = [0, 1, 3, 4]
output_inverse_indices = [0, 1, 1, 2, 3, 2]
output_counts = [1, 2, 2, 1]
Example 2:
input_X = [[1, 3], [2, 3]]
attribute_sorted = 1
attribute_axis = None
output_Y = [1, 2, 3]
output_indices = [0, 2, 1]
output_inverse_indices = [0, 2, 1, 2]
output_counts = [1, 1, 2]
Example 3:
input_X = [[1, 0, 0], [1, 0, 0], [2, 3, 4]]
attribute_sorted = 1
attribute_axis = 0
output_Y = [[1, 0, 0], [2, 3, 4]]
output_indices = [0, 2]
output_inverse_indices = [0, 0, 1]
output_counts = [2, 1]
Example 4:
input_x = [[[1., 1.], [0., 1.], [2., 1.], [0., 1.]],
[[1., 1.], [0., 1.], [2., 1.], [0., 1.]]]
attribute_sorted = 1
attribute_axis = 1
intermediate data are presented below for better understanding:
there are 4 subtensors sliced along axis 1 of input_x (shape = (2, 4, 2)):
A: [[1, 1], [1, 1]],
[[0, 1], [0, 1]],
[[2, 1], [2, 1]],
[[0, 1], [0, 1]].
there are 3 unique subtensors:
[[1, 1], [1, 1]],
[[0, 1], [0, 1]],
[[2, 1], [2, 1]].
sorted unique subtensors:
B: [[0, 1], [0, 1]],
[[1, 1], [1, 1]],
[[2, 1], [2, 1]].
output_Y is constructed from B:
[[[0. 1.], [1. 1.], [2. 1.]],
[[0. 1.], [1. 1.], [2. 1.]]]
output_indices is to map from B to A:
[1, 0, 2]
output_inverse_indices is to map from A to B:
[1, 0, 2, 0]
output_counts = [2 1 1]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Unique,
11,
OpSchema()
.SetDoc(Unique_ver11_doc)
.Attr(
"sorted",
"(Optional) Whether to sort the unique elements in ascending order before returning as output. "
"Must be one of 0, or 1 (default).",
AttributeProto::INT,
static_cast(1))
.Attr(
"axis",
"(Optional) The dimension to apply unique. If not specified, the unique elements of the "
"flattened input are returned. Negative value means counting dimensions "
"from the back. Accepted range is [-r, r-1] where r = rank(input).",
AttributeProto::INT,
OPTIONAL_VALUE)
.Input(0, "X", "A N-D input tensor that is to be processed.", "T")
.Output(
0,
"Y",
"A tensor of the same type as 'X' "
"containing all the unique values or subtensors sliced along a provided 'axis' in 'X', either sorted "
"or maintained in the same order they occur in input 'X'",
"T")
.Output(
1,
"indices",
"A 1-D INT64 tensor "
"containing indices of 'Y' elements' first occurance in 'X'. "
"When 'axis' is provided, it contains indices to subtensors in input 'X' on the 'axis'. "
"When 'axis' is not provided, it contains indices to values in the flattened input tensor. ",
"tensor(int64)",
OpSchema::Optional)
.Output(
2,
"inverse_indices",
"A 1-D INT64 tensor "
"containing, for elements of 'X', its corresponding indices in 'Y'. "
"When 'axis' is provided, it contains indices to subtensors in output 'Y' on the 'axis'. "
"When 'axis' is not provided, it contains indices to values in output 'Y'. ",
"tensor(int64)",
OpSchema::Optional)
.Output(
3,
"counts",
"A 1-D INT64 tensor containing "
"the count of each element "
"of 'Y' in input 'X'",
"tensor(int64)",
OpSchema::Optional)
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Input can be of any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Type inference
propagateElemTypeFromInputToOutput(ctx, 0, 0);
const TypeProto* xTensorProto = ctx.getInputType(0);
TypeProto* yTensorProto = ctx.getOutputType(0);
TypeProto* indicesTensorProto = nullptr;
TypeProto* inverseIndicesTensorProto = nullptr;
TypeProto* countsTensorProto = nullptr;
// 'indices', 'inverse_indices', and 'counts' are 1-D tensors of
// unknown dimension.
auto num_outputs = ctx.getNumOutputs();
if (num_outputs >= 2) {
indicesTensorProto = ctx.getOutputType(1);
updateOutputElemType(ctx, 1, TensorProto::INT64);
indicesTensorProto->mutable_tensor_type()
->mutable_shape()
->add_dim();
}
if (num_outputs >= 3) {
inverseIndicesTensorProto = ctx.getOutputType(2);
updateOutputElemType(ctx, 2, TensorProto::INT64);
inverseIndicesTensorProto->mutable_tensor_type()
->mutable_shape()
->add_dim();
}
if (num_outputs >= 4) {
countsTensorProto = ctx.getOutputType(3);
updateOutputElemType(ctx, 3, TensorProto::INT64);
countsTensorProto->mutable_tensor_type()
->mutable_shape()
->add_dim();
}
auto axisAttr = ctx.getAttribute("axis");
if (!axisAttr) {
// 'axis' is not provided. Input 'X' is flattened.
// 'Y' is a 1-D tensor of unknown dimension.
yTensorProto->mutable_tensor_type()->mutable_shape()->add_dim();
} else {
// 'axis' is provided.
int axis = static_cast(axisAttr->i());
const TensorShapeProto& input_shape =
xTensorProto->tensor_type().shape();
int rank = input_shape.dim_size();
if (axis < 0)
axis += rank;
if (axis < 0 || axis >= rank)
fail_shape_inference("Invalid value for attribute axis");
// 'Y' has the same shape as 'X' except in the 'axis' dimension
// which is unknown.
for (int i = 0; i < input_shape.dim_size(); i++) {
auto* dim = yTensorProto->mutable_tensor_type()
->mutable_shape()
->add_dim();
if (i != axis) {
*dim = input_shape.dim(i);
}
}
}
}));
static const char* GatherND_ver12_doc = R"DOC(
Given `data` tensor of rank `r` >= 1, `indices` tensor of rank `q` >= 1, and `batch_dims` integer `b`, this operator gathers
slices of `data` into an output tensor of rank `q + r - indices_shape[-1] - 1 - b`.
`indices` is an q-dimensional integer tensor, best thought of as a `(q-1)`-dimensional tensor of index-tuples into `data`,
where each element defines a slice of `data`
`batch_dims` (denoted as `b`) is an integer indicating the number of batch dimensions, i.e the leading `b` number of dimensions of
`data` tensor and `indices` are representing the batches, and the gather starts from the `b+1` dimension.
Some salient points about the inputs' rank and shape:
1) r >= 1 and q >= 1 are to be honored. There is no dependency condition to be met between ranks `r` and `q`
2) The first `b` dimensions of the shape of `indices` tensor and `data` tensor must be equal.
3) b < min(q, r) is to be honored.
4) The `indices_shape[-1]` should have a value between 1 (inclusive) and rank `r-b` (inclusive)
5) All values in `indices` are expected to be within bounds [-s, s-1] along axis of size `s` (i.e.) `-data_shape[i] <= indices[...,i] <= data_shape[i] - 1`.
It is an error if any of the index values are out of bounds.
The output is computed as follows:
The output tensor is obtained by mapping each index-tuple in the `indices` tensor to the corresponding slice of the input `data`.
1) If `indices_shape[-1] > r-b` => error condition
2) If `indices_shape[-1] == r-b`, since the rank of `indices` is `q`, `indices` can be thought of as `N` `(q-b-1)`-dimensional tensors
containing 1-D tensors of dimension `r-b`, where `N` is an integer equals to the product of 1 and all the elements in the batch dimensions
of the indices_shape. Let us think of each such `r-b` ranked tensor as `indices_slice`. Each *scalar value* corresponding to `data[0:b-1,indices_slice]`
is filled into the corresponding location of the `(q-b-1)`-dimensional tensor to form the `output` tensor (Example 1 below)
3) If `indices_shape[-1] < r-b`, since the rank of `indices` is `q`, `indices` can be thought of as `N` `(q-b-1)`-dimensional tensor
containing 1-D tensors of dimension `< r-b`. Let us think of each such tensors as `indices_slice`. Each *tensor slice* corresponding
to `data[0:b-1, indices_slice , :]` is filled into the corresponding location of the `(q-b-1)`-dimensional tensor
to form the `output` tensor (Examples 2, 3, 4 and 5 below)
This operator is the inverse of `ScatterND`.
`Example 1`
batch_dims = 0
data = [[0,1],[2,3]] # data_shape = [2, 2]
indices = [[0,0],[1,1]] # indices_shape = [2, 2]
output = [0,3] # output_shape = [2]
`Example 2`
batch_dims = 0
data = [[0,1],[2,3]] # data_shape = [2, 2]
indices = [[1],[0]] # indices_shape = [2, 1]
output = [[2,3],[0,1]] # output_shape = [2, 2]
`Example 3`
batch_dims = 0
data = [[[0,1],[2,3]],[[4,5],[6,7]]] # data_shape = [2, 2, 2]
indices = [[0,1],[1,0]] # indices_shape = [2, 2]
output = [[2,3],[4,5]] # output_shape = [2, 2]
`Example 4`
batch_dims = 0
data = [[[0,1],[2,3]],[[4,5],[6,7]]] # data_shape = [2, 2, 2]
indices = [[[0,1]],[[1,0]]] # indices_shape = [2, 1, 2]
output = [[[2,3]],[[4,5]]] # output_shape = [2, 1, 2]
`Example 5`
batch_dims = 1
data = [[[0,1],[2,3]],[[4,5],[6,7]]] # data_shape = [2, 2, 2]
indices = [[1],[0]] # indices_shape = [2, 1]
output = [[2,3],[4,5]] # output_shape = [2, 2]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
GatherND,
12,
OpSchema()
.SetDoc(GatherND_ver12_doc)
.Attr(
"batch_dims",
"The number of batch dimensions. The gather of indexing starts from dimension of data[batch_dims:]",
AttributeProto::INT,
static_cast(0))
.Input(0, "data", "Tensor of rank r >= 1.", "T")
.Input(
1,
"indices",
"Tensor of rank q >= 1. All index values are expected to be within bounds [-s, s-1] "
"along axis of size s. It is an error if any of the index values are out of bounds.",
"tensor(int64)")
.Output(
0,
"output",
"Tensor of rank q + r - indices_shape[-1] - 1.",
"T")
.TypeConstraint(
"T",
OpSchema::all_tensor_types(),
"Constrain input and output types to any tensor type.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Type inference
propagateElemTypeFromInputToOutput(ctx, 0, 0);
// Shape inference
if (!hasNInputShapes(ctx, 2)) {
// cannot proceed with shape or rank inference
return;
}
const auto& data_shape = ctx.getInputType(0)->tensor_type().shape();
const auto data_rank = data_shape.dim_size();
const auto& indices_shape =
ctx.getInputType(1)->tensor_type().shape();
const auto indices_rank = indices_shape.dim_size();
int64_t batch_dims_data = getAttribute(ctx, "batch_dims", 0);
if (data_rank < 1 || indices_rank < 1) {
fail_shape_inference(
"Both `data` and `indices` input tensors in GatherND op "
"need to have rank larger than 0.");
}
// cannot ascertain if the input shapes are valid if shape of
// `indices` is missing last dimension value so return at this point
if (!indices_shape.dim(indices_rank - 1).has_dim_value()) {
return;
}
const auto last_index_dimension =
indices_shape.dim(indices_rank - 1).dim_value() + batch_dims_data;
if (last_index_dimension > data_rank) {
fail_shape_inference(
"Last dimension of `indices` input tensor in GatherND op "
"must not be larger than the rank of `data` tensor");
}
for (int i = 0; i < indices_rank - 1; ++i) {
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() = indices_shape.dim(i);
}
for (int i = static_cast(last_index_dimension); i < data_rank;
++i) {
*ctx.getOutputType(0)
->mutable_tensor_type()
->mutable_shape()
->add_dim() = data_shape.dim(i);
}
}));
static const char* Pad_ver11_doc = R"DOC(
Given a tensor containing the data to be padded (`data`), a tensor containing the number of start and end pad values for axis (`pads`), (optionally) a `mode`, and (optionally) `constant_value`,
a padded tensor (`output`) is generated.
The three supported `modes` are (similar to corresponding modes supported by `numpy.pad`):
1) `constant`(default) - pads with a given constant value as specified by `constant_value` (which defaults to 0)
2) `reflect` - pads with the reflection of the vector mirrored on the first and last values of the vector along each axis
3) `edge` - pads with the edge values of array
Example 1 (`constant` mode):
Insert 0 pads to the beginning of the second dimension.
data =
[
[1.0, 1.2],
[2.3, 3.4],
[4.5, 5.7],
]
pads = [0, 2, 0, 0]
mode = 'constant'
constant_value = 0.0
output =
[
[
[0.0, 0.0, 1.0, 1.2],
[0.0, 0.0, 2.3, 3.4],
[0.0, 0.0, 4.5, 5.7],
],
]
Example 2 (`reflect` mode):
data =
[
[1.0, 1.2],
[2.3, 3.4],
[4.5, 5.7],
]
pads = [0, 2, 0, 0]
mode = 'reflect'
output =
[
[
[1.0, 1.2, 1.0, 1.2],
[2.3, 3.4, 2.3, 3.4],
[4.5, 5.7, 4.5, 5.7],
],
]
Example 3 (`edge` mode):
data =
[
[1.0, 1.2],
[2.3, 3.4],
[4.5, 5.7],
]
pads = [0, 2, 0, 0]
mode = 'edge'
output =
[
[
[1.0, 1.0, 1.0, 1.2],
[2.3, 2.3, 2.3, 3.4],
[4.5, 4.5, 4.5, 5.7],
],
]
)DOC";
ONNX_OPERATOR_SET_SCHEMA(
Pad,
11,
OpSchema()
.Attr(
"mode",
"Supported modes: `constant`(default), `reflect`, `edge`",
AttributeProto::STRING,
std::string("constant"))
.SetDoc(Pad_ver11_doc)
.Input(0, "data", "Input tensor.", "T")
.Input(
1,
"pads",
"Tensor of integers indicating the number of padding elements to add or remove (if negative) "
"at the beginning and end of each axis. For 2D input tensor, it is the number of pixels. "
"`pads` should be a 1D tensor of shape [2 * input_rank]. "
"`pads` format should be: [x1_begin, x2_begin,...,x1_end, x2_end,...], "
"where xi_begin is the number of pad values added at the beginning of axis `i` and "
"xi_end, the number of pad values added at the end of axis `i`.",
"tensor(int64)")
.Input(
2,
"constant_value",
"(Optional) A scalar value to be used if the mode chosen is `constant` (by default it is 0).",
"T",
OpSchema::Optional)
.Output(0, "output", "Tensor after padding.", "T")
.TypeConstraint(
"T",
OpSchema::all_numeric_types(),
"Constrains input and output to only numeric types.")
.TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
// Type inference
propagateElemTypeFromInputToOutput(ctx, 0, 0);
// Shape inference needs the input data shape
if (!hasNInputShapes(ctx, 1)) {
return;
}
const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
const auto input_rank = input_shape.dim_size();
// Infer output shape if 'pads' tensor is available
const auto* pads_initializer = ctx.getInputData(1);
if (nullptr != pads_initializer) {
if (pads_initializer->dims_size() != 1 ||
pads_initializer->data_type() != TensorProto::INT64)
fail_shape_inference(
"'pads' input must be a 1D (shape: [2 * input_rank]) tensor of type int64");
const auto& pads_data = ParseData(pads_initializer);
if (pads_data.size() != static_cast(2 * input_rank))
fail_shape_inference("Pads has incorrect number of values");
auto* output_shape =
ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
for (size_t i = 0; static_cast(i) < input_rank; ++i) {
const auto& input_dim = input_shape.dim((int)i);
auto* output_dim = output_shape->add_dim();
if (input_dim.has_dim_value()) {
output_dim->set_dim_value(
input_dim.dim_value() + pads_data[i] +
pads_data[i + input_rank]);
} else if (pads_data[i] + pads_data[i + input_rank] == 0) {
*output_dim = input_dim;
}
}
} else {
// Infer output shapes' rank in any case
auto* output_shape_0 = getOutputShape(ctx, 0);
for (size_t i = 0; static_cast(i) < input_rank; ++i) {
output_shape_0->add_dim();
}
}
return;
}));
} // namespace ONNX_NAMESPACE
onnx-1.7.0/onnx/defs/tensor/utils.cc 0000664 0000000 0000000 00000014265 13655345213 016104 0 ustar root root // Copyright (c) ONNX Project Contributors.
// Licensed under the MIT license.
#include "onnx/defs/tensor/utils.h"
namespace ONNX_NAMESPACE {
void resizeShapeInferenceHelper(
const TensorShapeProto& input_shape,
const std::vector& sizes_data,
TensorShapeProto* output_shape) {
if (!sizes_data.empty()) {
for (int i = 0; i < input_shape.dim_size(); ++i) {
auto* dim = output_shape->mutable_dim(i);
dim->set_dim_value(sizes_data[i]);
}
return;
}
}
void resizeShapeInferenceHelper(
const TensorShapeProto& input_shape,
const std::vector& scales_data,
TensorShapeProto* output_shape) {
for (int i = 0; i < input_shape.dim_size(); ++i) {
auto* dim = output_shape->mutable_dim(i);
// If input_shape has dim_value, we caculate the scaled result
// If input_shape doesn's have one, we leave it here
if (input_shape.dim(i).has_dim_value()) {
int64_t dim_value = static_cast