Larry Gritz
2012-09-29 00:09:29 UTC
At long last, I believe I have finished the work to fully support OpenEXR 2.0 (main new features: "deep" images, multi-part EXR files), including extending OIIO's APIs to handle some of the new concepts. Dealing with deep images took me a while to wrap my head around in terms of how to support it while only minimally change the OIIO public APIs or the simple concepts that they are based on.
The pull request is here: https://github.com/OpenImageIO/oiio/pull/437
And the text of the pull request is below.
Anybody particularly interested in deep images is invited to please grab the "lg-deep" branch in my repo (https://github.com/lgritz/oiio.git) and try it out.
Once this gets approved, that mostly clears the way to formally branch and beta test OpenImageIO 1.1. Only a few other changes are on my list for a final 1.1 release, so hopefully a final release can happen within the next couple weeks.
------
Part 2 of the changes to support OpenEXR 2.0: "deep" images.
(This patch also incorporates OpenImageIO/oiio#435, which has not yet been reviewed. Do them separately or together, I don't care.)
"Deep" images are those that can store more than one sample per pixel. In fact, potentially different numbers for each pixel. OpenEXR 2.0 allows this. This breaks some of OIIO's assumptions in ways that required me to add some new API calls, and generally to make a much larger patch than most features we add. But conceptually, it's not all that tricky.
Alter the core APIs to support "deep" reading and writing:
'bool deep' field in ImageSpec, indicating a deep image.
DeepData helper class for holding the sample counts, pointers, and data block, passed from/to the read* and write* deep functions.
ImageInput: read_native_deep_scanlines and read_native_deep_tiles, which should be overloaded by any format reader supporting deep images, and read_native_deep_image (implemented in terms of the others).
ImageOutput: write_deep_scanlines and write_deep_tiles, which should be overloaded by any format writer supporting deep images, and write_deep_image (implemented in terms of the others).
Very rudimentary ImageBuf support for deep data. A DeepData field has been added internally to ImageBuf. Deep images loaded as ImageBufs will read directly into there, bypassing the traditional pixel storage (or ImageCache) used by non-deep ImageBufs. ImageBuf::Iterator (and ConstIterator) have been modified to be deep-safe, and to be able to access the deep data. You can also access through the ImageBuf directly. Quite a bit of other ImageBuf functionality doesn't work (and isn't meaningful) with deep data, beware.
A small number of ImageBufAlgo functions have been modified to work properly with deep ImageBuf's: computePixelStats(), compare().
OpenEXR reader/writer support for deep images. (Only if built against OpenEXR >= 2.0.)
Some basic utility support for deep images: iinfo and oiiotool can safely load deep images and print information about them, with -a, -v, --hash, --stats working. idiff works with deep images. Almost nothing else is expected to work with deep images at this time. Some more functionality will be added over time, but most image operations aren't very meaningful for deep images, they are kind of their own beast.
AND... like I said, this also was built upon, and thus incorporates, #435, to wit:
This set of changes works toward fully supporting OpenEXR 2.0's "multi-part" files (a partial implementation of multiple images in a single file).
Unfortunately, OpenEXR's library requires that the multi-part files declare the headers for all the parts upon opening the file. This is in contradiction to the way TIFF works, and therefore the way I had organized OIIO's APIs, to allow additional subimages to simply be appended one by one. So to accommodate this, I had to add a new API call to ImageOutput:
bool open (const std::string &name, int subimages, const ImageSpec *specs);
This lets you open with full information about the number and specification of subimages that you will be writing. A new supports("appendsubimage"), which returns true for TIFF, is used to indicate a format that can be output with the old appending method instead of requiring pre-declaration of the subimages.
I also changed oiiotool to use this new call in the appropriate way, and verified that all this stuff is working by adding a test that uses oiiotool to copy all the subimages of one of the new OpenEXR 2.0 test images. In the process, I changed the convention of the exr test images being stored in ../openexr-images-1.5.0 (which seems like a bad idea to have a specific version) to ../openexr-images. So beware that you may need to make a link on your end to ensure that the testsuite runs properly.
It was a much bigger pain to modify 'iconvert' to write multi-part OpenEXR files (or, more properly, to restructure it to use the new pre-declare flavor of the API), so I didn't do that at this time. That means that for now, please use oiiotool rather than iconvert if you anticipate needing to write multi-part OpenEXRs. I'm not sure if that's a serious limitation, if it is I can try to fix it, otherwise I'll get to it when I have a chance.
With this patch, OIIO will build with both OpenEXR 2.x as well as 1.x. Of course, if only 1.x is found at build time, the multi-part support won't be enabled. Also I've added top-level-Makefile OPENEXR_HOME=... and ILMBASE_HOME=... variables that let you override the location of those libraries.
You can merge this Pull Request by running:
git pull https://github.com/lgritz/oiio lg-deep
Or view, comment on, or merge it at:
https://github.com/OpenImageIO/oiio/pull/437
Commit Summary
Add ImageOutput::open(name,subimages,subimagespecs[])
Change oiiotool to use the new "open with subimages specified" API call
testsuite/openexr-v2 tests OpenEXR 2.x features, also change OpenEXR
Support for OpenEXR 2.x "multi-part" files (and build support for Ope?
Fix bugs in OpenEXR multi-part input
Small error reporting improvements, found along the way.
Alter APIs to support 'deep' reading and writing.
Very rudimentary ImageBuf support for deep data.
Document API changes for deep data I/O
OpenEXR 2.0 deep file support.
TIFF: any compression requested that starts with 'zip' gets zip.
Add basic deep file support to utilities.
Update docs
File Changes
M CHANGES (9)
M Makefile (9)
M src/CMakeLists.txt (4)
M src/cmake/externalpackages.cmake (18)
M src/doc/imageinput.tex (104)
M src/doc/imageioapi.tex (28)
M src/doc/imageoutput.tex (392)
M src/doc/openimageio.pdf (0)
M src/doc/openimageio.tex (6)
M src/doc/writingplugins.tex (8)
M src/idiff/idiff.cpp (7)
M src/iinfo/iinfo.cpp (98)
M src/include/imagebuf.h (52)
M src/include/imagebufalgo.h (8)
M src/include/imagecache.h (2)
M src/include/imageio.h (117)
M src/include/version.h.in (5)
M src/libOpenImageIO/formatspec.cpp (8)
M src/libOpenImageIO/imagebuf.cpp (137)
M src/libOpenImageIO/imagebufalgo.cpp (156)
M src/libOpenImageIO/imageinput.cpp (49)
M src/libOpenImageIO/imageio.cpp (65)
M src/libOpenImageIO/imageoutput.cpp (61)
M src/libtexture/imagecache.cpp (4)
M src/oiiotool/oiiotool.cpp (31)
M src/oiiotool/printinfo.cpp (87)
M src/openexr.imageio/exrinput.cpp (646)
M src/openexr.imageio/exroutput.cpp (761)
M src/tiff.imageio/tiffoutput.cpp (4)
M testsuite/openexr-chroma/ref/out.txt (6)
M testsuite/openexr-chroma/run.py (8)
M testsuite/openexr-multires/ref/out.txt (78)
M testsuite/openexr-multires/run.py (4)
M testsuite/openexr-suite/ref/out.txt (110)
M testsuite/openexr-suite/run.py (16)
A testsuite/openexr-v2/ref/out.txt (164)
A testsuite/openexr-v2/run.py (15)
M testsuite/runtest.py (13)
Patch Links
https://github.com/OpenImageIO/oiio/pull/437.patch
https://github.com/OpenImageIO/oiio/pull/437.diff
?
Reply to this email directly or view it on GitHub.
--
Larry Gritz
lg at larrygritz.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openimageio.org/pipermail/oiio-dev-openimageio.org/attachments/20120928/b6456b63/attachment.htm>
The pull request is here: https://github.com/OpenImageIO/oiio/pull/437
And the text of the pull request is below.
Anybody particularly interested in deep images is invited to please grab the "lg-deep" branch in my repo (https://github.com/lgritz/oiio.git) and try it out.
Once this gets approved, that mostly clears the way to formally branch and beta test OpenImageIO 1.1. Only a few other changes are on my list for a final 1.1 release, so hopefully a final release can happen within the next couple weeks.
------
Part 2 of the changes to support OpenEXR 2.0: "deep" images.
(This patch also incorporates OpenImageIO/oiio#435, which has not yet been reviewed. Do them separately or together, I don't care.)
"Deep" images are those that can store more than one sample per pixel. In fact, potentially different numbers for each pixel. OpenEXR 2.0 allows this. This breaks some of OIIO's assumptions in ways that required me to add some new API calls, and generally to make a much larger patch than most features we add. But conceptually, it's not all that tricky.
Alter the core APIs to support "deep" reading and writing:
'bool deep' field in ImageSpec, indicating a deep image.
DeepData helper class for holding the sample counts, pointers, and data block, passed from/to the read* and write* deep functions.
ImageInput: read_native_deep_scanlines and read_native_deep_tiles, which should be overloaded by any format reader supporting deep images, and read_native_deep_image (implemented in terms of the others).
ImageOutput: write_deep_scanlines and write_deep_tiles, which should be overloaded by any format writer supporting deep images, and write_deep_image (implemented in terms of the others).
Very rudimentary ImageBuf support for deep data. A DeepData field has been added internally to ImageBuf. Deep images loaded as ImageBufs will read directly into there, bypassing the traditional pixel storage (or ImageCache) used by non-deep ImageBufs. ImageBuf::Iterator (and ConstIterator) have been modified to be deep-safe, and to be able to access the deep data. You can also access through the ImageBuf directly. Quite a bit of other ImageBuf functionality doesn't work (and isn't meaningful) with deep data, beware.
A small number of ImageBufAlgo functions have been modified to work properly with deep ImageBuf's: computePixelStats(), compare().
OpenEXR reader/writer support for deep images. (Only if built against OpenEXR >= 2.0.)
Some basic utility support for deep images: iinfo and oiiotool can safely load deep images and print information about them, with -a, -v, --hash, --stats working. idiff works with deep images. Almost nothing else is expected to work with deep images at this time. Some more functionality will be added over time, but most image operations aren't very meaningful for deep images, they are kind of their own beast.
AND... like I said, this also was built upon, and thus incorporates, #435, to wit:
This set of changes works toward fully supporting OpenEXR 2.0's "multi-part" files (a partial implementation of multiple images in a single file).
Unfortunately, OpenEXR's library requires that the multi-part files declare the headers for all the parts upon opening the file. This is in contradiction to the way TIFF works, and therefore the way I had organized OIIO's APIs, to allow additional subimages to simply be appended one by one. So to accommodate this, I had to add a new API call to ImageOutput:
bool open (const std::string &name, int subimages, const ImageSpec *specs);
This lets you open with full information about the number and specification of subimages that you will be writing. A new supports("appendsubimage"), which returns true for TIFF, is used to indicate a format that can be output with the old appending method instead of requiring pre-declaration of the subimages.
I also changed oiiotool to use this new call in the appropriate way, and verified that all this stuff is working by adding a test that uses oiiotool to copy all the subimages of one of the new OpenEXR 2.0 test images. In the process, I changed the convention of the exr test images being stored in ../openexr-images-1.5.0 (which seems like a bad idea to have a specific version) to ../openexr-images. So beware that you may need to make a link on your end to ensure that the testsuite runs properly.
It was a much bigger pain to modify 'iconvert' to write multi-part OpenEXR files (or, more properly, to restructure it to use the new pre-declare flavor of the API), so I didn't do that at this time. That means that for now, please use oiiotool rather than iconvert if you anticipate needing to write multi-part OpenEXRs. I'm not sure if that's a serious limitation, if it is I can try to fix it, otherwise I'll get to it when I have a chance.
With this patch, OIIO will build with both OpenEXR 2.x as well as 1.x. Of course, if only 1.x is found at build time, the multi-part support won't be enabled. Also I've added top-level-Makefile OPENEXR_HOME=... and ILMBASE_HOME=... variables that let you override the location of those libraries.
You can merge this Pull Request by running:
git pull https://github.com/lgritz/oiio lg-deep
Or view, comment on, or merge it at:
https://github.com/OpenImageIO/oiio/pull/437
Commit Summary
Add ImageOutput::open(name,subimages,subimagespecs[])
Change oiiotool to use the new "open with subimages specified" API call
testsuite/openexr-v2 tests OpenEXR 2.x features, also change OpenEXR
Support for OpenEXR 2.x "multi-part" files (and build support for Ope?
Fix bugs in OpenEXR multi-part input
Small error reporting improvements, found along the way.
Alter APIs to support 'deep' reading and writing.
Very rudimentary ImageBuf support for deep data.
Document API changes for deep data I/O
OpenEXR 2.0 deep file support.
TIFF: any compression requested that starts with 'zip' gets zip.
Add basic deep file support to utilities.
Update docs
File Changes
M CHANGES (9)
M Makefile (9)
M src/CMakeLists.txt (4)
M src/cmake/externalpackages.cmake (18)
M src/doc/imageinput.tex (104)
M src/doc/imageioapi.tex (28)
M src/doc/imageoutput.tex (392)
M src/doc/openimageio.pdf (0)
M src/doc/openimageio.tex (6)
M src/doc/writingplugins.tex (8)
M src/idiff/idiff.cpp (7)
M src/iinfo/iinfo.cpp (98)
M src/include/imagebuf.h (52)
M src/include/imagebufalgo.h (8)
M src/include/imagecache.h (2)
M src/include/imageio.h (117)
M src/include/version.h.in (5)
M src/libOpenImageIO/formatspec.cpp (8)
M src/libOpenImageIO/imagebuf.cpp (137)
M src/libOpenImageIO/imagebufalgo.cpp (156)
M src/libOpenImageIO/imageinput.cpp (49)
M src/libOpenImageIO/imageio.cpp (65)
M src/libOpenImageIO/imageoutput.cpp (61)
M src/libtexture/imagecache.cpp (4)
M src/oiiotool/oiiotool.cpp (31)
M src/oiiotool/printinfo.cpp (87)
M src/openexr.imageio/exrinput.cpp (646)
M src/openexr.imageio/exroutput.cpp (761)
M src/tiff.imageio/tiffoutput.cpp (4)
M testsuite/openexr-chroma/ref/out.txt (6)
M testsuite/openexr-chroma/run.py (8)
M testsuite/openexr-multires/ref/out.txt (78)
M testsuite/openexr-multires/run.py (4)
M testsuite/openexr-suite/ref/out.txt (110)
M testsuite/openexr-suite/run.py (16)
A testsuite/openexr-v2/ref/out.txt (164)
A testsuite/openexr-v2/run.py (15)
M testsuite/runtest.py (13)
Patch Links
https://github.com/OpenImageIO/oiio/pull/437.patch
https://github.com/OpenImageIO/oiio/pull/437.diff
?
Reply to this email directly or view it on GitHub.
--
Larry Gritz
lg at larrygritz.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openimageio.org/pipermail/oiio-dev-openimageio.org/attachments/20120928/b6456b63/attachment.htm>