Add support for compressed NPZ file

[s-trinh]

Thanks to: rogersce/cnpy#103

---

lagadic/visp-images#34

Reading npz from numpy.savez_compressed works except somehow for string data.
Writing compressed npz file from ViSP can be read from NumPy.

Some functions should not be exposed to the user, e.g. compressData(), BigEndianTest(), parse_npy_header(), ..., with VISP_EXPORT. But removing them is not easy since I don't know how to handle this kind of thing at a library level.

Rename to npz_save_str() for string parameter/data, otherwise when adding a bool parameter for compression flag:

<>/visp/tutorial/misc/npz/tutorial-npz.cpp: In function ‘int main()’:
<>/visp/tutorial/misc/npz/tutorial-npz.cpp:61:25: error: call of overloaded ‘npz_save(const std::string&, const std::string&, __gnu_cxx::__alloc_traits<std::allocator<char>, char>::value_type*, <brace-enclosed initializer list>, const char [2])’ is ambiguous
   61 |     visp::cnpy::npz_save(npz_filename, identifier, &vec_save_string[0], { vec_save_string.size() }, "w");
      |     ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from <>/visp/tutorial/misc/npz/tutorial-npz.cpp:42:
<>/visp/modules/core/include/visp3/core/vpIoTools.h:297:27: note: candidate: ‘void visp::cnpy::npz_save(const std::string&, std::string, const T*, const std::vector<long unsigned int>&, const std::string&, bool) [with T = char; std::string = std::__cxx11::basic_string<char>]’
  297 | template<typename T> void npz_save(const std::string &zipname, std::string fname, const T *data, const std::vector<size_t> &shape,
      |                           ^~~~~~~~
<>/visp/modules/core/include/visp3/core/vpIoTools.h:190:18: note: candidate: ‘void visp::cnpy::npz_save(const std::string&, const std::string&, const std::string&, const std::string&, bool)’
  190 | VISP_EXPORT void npz_save(const std::string &zipname, const std::string &fname, const std::string &data,

[...]

I suspect before when using brace-enclosed initializer list it was maybe not calling the intended function.

[codecov]

Codecov Report

❌ Patch coverage is 29.59184% with 69 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.58%. Comparing base (f84853e) to head (c07b606).
⚠️ Report is 42 commits behind head on master.

Files with missing lines	Patch %	Lines
modules/core/src/tools/file/vpIoTools_npy.cpp	17.64%	50 Missing and 6 partials ⚠️
modules/core/include/visp3/core/vpIoTools.h	56.66%	8 Missing and 5 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1885      +/-   ##
==========================================
+ Coverage   48.42%   48.58%   +0.15%     
==========================================
  Files         532      532              
  Lines       69216    69293      +77     
  Branches    32369    32402      +33     
==========================================
+ Hits        33521    33667     +146     
- Misses      25176    31481    +6305     
+ Partials    10519     4145    -6374     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fspindle]

@s-trinh Thanks for this PR.
I just discover that lines like

crc = vp_mz_crc32(crc, (uint8_t *)data, nels*sizeof(T));

produce a warning with Xcode when building a project for iOS

implicit conversion loses integer precision: 'unsigned long' to 'uint_32_t' (aka 'unsigned int')

Since I'm not familiar with this code, could you investigate...

As a starting point I'm wondering if the function vp_mz_crc32() should not return an uint_32_t and use internally also uint_32_t instead of an unsigned int

static inline uint_32_t vp_mz_crc32(unsigned long crc, const unsigned char *ptr, size_t buf_len)
{
  static const unsigned int s_crc32[16] = { 0, 0x1db71064, 0x3b6e20c8, 0x26d930ac, 0x76dc4190, 0x6b6b51f4, 0x4db26158, 0x5005713c,
    0xedb88320, 0xf00f9344, 0xd6d6a3e8, 0xcb61b38c, 0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c };
  uint_32_t crcu32 = static_cast<uint_32_t>(crc);
  if (!ptr) return 0;
  crcu32 = ~crcu32;
  while (buf_len--) {
    unsigned char b = *ptr++;
    crcu32 = (crcu32 >> 4) ^ s_crc32[(crcu32 & 0xF) ^ (b & 0xF)];
    crcu32 = (crcu32 >> 4) ^ s_crc32[(crcu32 & 0xF) ^ (b >> 4)];
  }
  return ~crcu32;
}

[s-trinh]

visp/modules/core/include/visp3/core/vpIoTools.h

Lines 64 to 78 in 838e63a

	// https://github.com/BinomialLLC/basis_universal/blob/ad9386a4a1cf2a248f7bbd45f543a7448db15267/encoder/basisu_miniz.h#L665
	static inline unsigned long vp_mz_crc32(unsigned long crc, const unsigned char *ptr, size_t buf_len)
	{
	static const unsigned int s_crc32[16] = { 0, 0x1db71064, 0x3b6e20c8, 0x26d930ac, 0x76dc4190, 0x6b6b51f4, 0x4db26158, 0x5005713c,
	0xedb88320, 0xf00f9344, 0xd6d6a3e8, 0xcb61b38c, 0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c };
	unsigned int crcu32 = static_cast<unsigned int>(crc);
	if (!ptr) return 0;
	crcu32 = ~crcu32;
	while (buf_len--) {
	unsigned char b = *ptr++;
	crcu32 = (crcu32 >> 4) ^ s_crc32[(crcu32 & 0xF) ^ (b & 0xF)];
	crcu32 = (crcu32 >> 4) ^ s_crc32[(crcu32 & 0xF) ^ (b >> 4)];
	}
	return ~crcu32;
	}

---

unsigned long type can be 64-bits on some platform. I have pushed a commit to use uint32_t instead.

https://en.cppreference.com/w/cpp/language/types.html

---

stb implementation of crc32:
https://github.com/nothings/stb/blob/f1c79c02822848a9bed4315b12c8c8f3761e1296/stb_image_write.h#L1024-L1071

[fspindle]

@s-trinh Thanks