HDF5 (.h5, .hdf5)
Background & Context

-
- MIME type: application/x-hdf5
- HDF data format Version 5.
- General-purpose format for representing multidimensional datasets and images.
- Datasets with compound data structures are supported.
- Used for storage, management, and exchange of scientific data.
- HDF is an acronym for Hierarchical Data Format.
- Originally developed by the US National Center for Supercomputing Applications (NCSA).
- Currently maintained by The HDF Group.
- Binary file format.
- Incompatible with HDF Version 4 and earlier.
Import & Export

- Import["file.h5"] imports an HDF5 file, returning the names of the datasets stored in the file.
- Import["file.h5",elem] imports the specified element from an HDF5 file.
- The import format can be specified with Import["file","HDF5"] or Import["file",{"HDF5",elem,…}].
- Export["file.h5",expr] exports a numeric array to HDF5.
- Export["file.h5",{expr1,…},{"Datasets", {"dataset1",…}}] creates an HDF5 file, storing the data arrays {expr1,…} as separate datasets.
- Export["file.h5",expr,elem] exports the specified element to an HDF5 file.
- Export["file.h5",elem1->expr1,…,"Rules"] uses rules to specify the elements to be exported.
- See the following reference pages for full general information:
-
Import, Export import from or export to a file CloudImport, CloudExport import from or export to a cloud object ImportString, ExportString import from or export to a string ImportByteArray, ExportByteArray import from or export to a byte array
Import Elements




- General Import elements:
-
"Elements" list of elements and options available in this file "Summary" summary of the file "Rules" list of rules for all available elements - Structure elements:
-
"Datasets" names of all datasets "Groups" names of all groups "StructureGraph" a directed graph showing the structure of the datasets {"StructureGraph, groupname} a graph showing the structure under groupname "StructureGraphLegend" legend for the structure graph "Summary" summary of properties - Names of groups and datasets are given as the absolute paths starting with the root group name "/".
- Import by default uses the "Datasets" element for the HDF5 format.
- Data representation elements:
-
"Data" all datasets imported as an association {"Data",n} or n n dataset
{"Data",dataset} or dataset named dataset {"Data",groupname} an association of all datasets under groupname {"Data",groupname,lev} an association of datasets under groupname up to level lev - The following basic data types are supported:
-
"Integer8" 8-bit integers "Integer16" 16-bit integers "Integer32" 32-bit integers "Integer64" 64-bit integers "UnsignedInteger8" 8-bit unsigned integers "UnsignedInteger16" 16-bit unsigned integers "UnsignedInteger32" 32-bit unsigned integers "UnsignedInteger64" 64-bit unsigned integers "Real32" IEEE single‐precision numbers "Real64" IEEE double‐precision numbers "String" string of ASCII characters - The following structured data types are supported:
-
"ByteArray" a ByteArray of an arbitrary length "Array" an array of any supported data format "Enum" an enumeration "Compound" a compound dataset consisting of any other data format and other compound datasets - Complex numbers are typically stored and imported as compound types.
- Metadata elements:
-
"Attributes" attributes of all groups and datasets "DataEncoding" specifies how each dataset is compressed "DataFormat" type used to represent each dataset "Dimensions" data dimensions of each dataset {"metadata",n} metadata of the n dataset
{"metadata",dataset} metadata of the named dataset - The following data encodings are supported:
-
None no data compression is used "Fletcher32" adds the Fletcher checksum "GZIP" GZIP compression "ScaleOffset" performs a scale and/or offset operation "Shuffle" reorders so that consistent byte positions are placed together "SZIP" SZIP compression (Import only) - A single dataset can have multiple encodings, which will be specified as a list {enc1,enc2,…}.
Export Elements



- General Export element:
-
"Rules" a list of imported elements in the form of elemexpr - Export["file.h5",{elem1->expr1,…},"Rules"] uses rules to specify the elements to be exported.
- Available Export elements:
-
"Attributes" attributes associated to any object "Datasets" datasets and their associated elements "Groups" group names and their associated elements "NamedDataTypes" named data types and their associated elements - With the "Attributes" element, the following expressions can be given:
-
attr attributes associated to the root group "/" {path1attr1,…} attributes attri associated to the specified pathi - Attributes attri should be given in the form "attname"->attval.
- With the "Groups" element, the following expressions can be given:
-
{"gr1","gr2",…} a list of group paths {"gr1"grdata1,…} list of groups and their associated data - Group data grdatai can take the following keys:
-
"Attributes" group attributes "HardLinks" hard links to other objects "SoftLinks" soft links to other objects - Soft and hard links can be specified using "linkname"path.
- With the "Datasets" element, the following expressions can be given:
-
data stores data under "Dataset1" {"name1"data1,…} a list of dataset names and their associated data {"name1"ds1,…} specifies each dataset dsi using a list of rules - Datasets dsi can take the following keys:
-
"Attributes" dataset attributes "Data" array of data "DataFormat" data type "MaxDimensions" list of maximal dimensions for the dataset "SpaceSelection" part of the dataspace where the data will be written - With the "NamedDataTypes" element, the following expressions can be given:
-
{"name1"type1,…} a data type {"name1"<"Type"type1,"Attributes"att1 >,…} an association specifying a type and its attributes - The type specification typei can take the following forms:
-
"simpletype" a simple data type such as "Integer64" <"Class""ByteArray",… > takes "Length" and "Tag" keys <"Class""Array",… > takes "DataFormat" and "Dimensions" keys <"Class""Compound",… > takes a "Structure" key
Options


- Import and Export options:
-
"ComplexKeys" Automatic keys for complex interpretation and export - By default, {"Re","Im"} are used as complex keys. Other settings include:
-
None no complex interpretation (import only) Automatic use automatic keys {key1,key2} use explicit keys - Import option:
-
"TakeElements" All subset of elements to import - "TakeElements" can take the following values:
-
{elem1,elem2,…} list of elements elemi {m;;n;;s,...} elements m through n in steps of s {opt1val1,…} list of suboptions - The following suboptions opti are available for taking elements:
-
"Offset" {0,0,…} the offset along the dimensions of the dataset "Count" All the number of block to be imported along each dimension "Stride" {1,1,…} the step between beginnings of the blocks "Block" {1,1,…} number of elements in each block - Export option:
-
ByteOrdering $ByteOrdering what byte ordering to use OverwriteTarget True whether to overwrite an existing file "AppendMode" "Extend" how to append to existing objects - Using OverwriteTarget->"Append", new objects may be added to an existing file.
- Possible settings for "AppendMode" include:
-
"Extend" extends existing objects, if possible (default) "Overwrite" overwrites exisiting objects "Preserve" preserves existing objects
Examples
open allclose allBasic Examples (1)Summary of the most common use cases
Scope (12)Survey of the scope of standard use cases
Import (7)
Show all elements available in the file:

https://wolfram.com/xid/0knlzss-4nwp2m

By default, a list of dataset names is returned:

https://wolfram.com/xid/0knlzss-zt3k23

Import the file structure, specifying the format explicitly:

https://wolfram.com/xid/0knlzss-u5g197

Import contents of a dataset by specifying its name:

https://wolfram.com/xid/0knlzss-bpbgyz

Import dimensions and data format for all datasets in the file:

https://wolfram.com/xid/0knlzss-geiai5

Import dimensions and data format of a single dataset:

https://wolfram.com/xid/0knlzss-e0wwyt

Import 8-bit RGB raster data and render it as an Image object:

https://wolfram.com/xid/0knlzss-wwn94m

Export (5)

https://wolfram.com/xid/0knlzss-kituy0

Show the datasets contained in this file:

https://wolfram.com/xid/0knlzss-50dj4l


https://wolfram.com/xid/0knlzss-hd2ljh


https://wolfram.com/xid/0knlzss-ojkn4a


https://wolfram.com/xid/0knlzss-kcox1g


https://wolfram.com/xid/0knlzss-kxuo23

Export a named dataset with given data type:

https://wolfram.com/xid/0knlzss-f2g8a3


https://wolfram.com/xid/0knlzss-c0g5ym

Export a named data type and a group that links to it:

https://wolfram.com/xid/0knlzss-kwvvyk


https://wolfram.com/xid/0knlzss-d897lz

Import Elements (22)
Attributes (4)
Import attributes of all objects in the file:

https://wolfram.com/xid/0knlzss-q9buq4

Import attributes of a specific dataset:

https://wolfram.com/xid/0knlzss-5j6o9v

Import attributes of the second dataset in the file:

https://wolfram.com/xid/0knlzss-g85w4v

The order of datasets in the file can be checked by calling:

https://wolfram.com/xid/0knlzss-fap2x7

Import attributes of multiple objects:

https://wolfram.com/xid/0knlzss-2nx19w

Data (4)
Get data from all datasets in the file:

https://wolfram.com/xid/0knlzss-3rd7gl

Import data from the "Complex64" dataset inside the "Complex" group:

https://wolfram.com/xid/0knlzss-byu4o4

Import data from the third dataset in the file:

https://wolfram.com/xid/0knlzss-8wp8ee

Get data from all datasets in a given group:

https://wolfram.com/xid/0knlzss-qjy6pg

Import data from every dataset or group in a list:

https://wolfram.com/xid/0knlzss-zha5jr

DataEncoding (1)
DataFormat (2)
Get data type description for every dataset in the file:

https://wolfram.com/xid/0knlzss-h62h6b

Simple numeric and string types have one-word descriptions:

https://wolfram.com/xid/0knlzss-k123la

"DataFormat" for a compound type shows class and structure:

https://wolfram.com/xid/0knlzss-5drjob

"DataFormat" for enumerated types includes class, base data format and a list of values and names:

https://wolfram.com/xid/0knlzss-9dx01h

"DataFormat" for array types includes class, base data format and dimensions:

https://wolfram.com/xid/0knlzss-4xcogr

"DataFormat" for byte arrays includes class and length:

https://wolfram.com/xid/0knlzss-z0s8zn

Datasets (2)
Dimensions (4)
Get dimensions of all datasets in a file:

https://wolfram.com/xid/0knlzss-2bnom

Import dimensions of all datasets under a given group:

https://wolfram.com/xid/0knlzss-1d7ysw

Dimensions of data with only a single element are indicated by an empty list:

https://wolfram.com/xid/0knlzss-5wqa7v


https://wolfram.com/xid/0knlzss-nu4xwm

Get dimensions of a specific dataset:

https://wolfram.com/xid/0knlzss-swr7p0

Specify the dataset using its index:

https://wolfram.com/xid/0knlzss-gp4ct5

Groups (1)
StructureGraph (3)
Export Elements (28)
Attributes (4)
Attach attributes to the root group:

https://wolfram.com/xid/0knlzss-49xowb

Specify data format for attributes:

https://wolfram.com/xid/0knlzss-4tny94

Export dataset's attributes both in "Datasets" and "Attributes" elements:

https://wolfram.com/xid/0knlzss-8wee6b

Each attribute must have a name that is unique among all attributes attached to the same object:

https://wolfram.com/xid/0knlzss-667ubt

Try to export a different attribute with an already existing name:

https://wolfram.com/xid/0knlzss-xonnge


It is possible to overwrite existing attributes with "AppendMode" set to "Overwrite":

https://wolfram.com/xid/0knlzss-l30arv

Datasets (13)
"Datasets" is the default export element. Data format and dimensions are automatically inferred from the expression:

https://wolfram.com/xid/0knlzss-ltu9qr


https://wolfram.com/xid/0knlzss-znaxye


https://wolfram.com/xid/0knlzss-3779ij

Export expressions into different datasets, each of which can have a full path:

https://wolfram.com/xid/0knlzss-cbjbwa

https://wolfram.com/xid/0knlzss-dzovn3

Inspect the structure of the HDF5 file:

https://wolfram.com/xid/0knlzss-f753ts

Export a dataset with a custom data format:

https://wolfram.com/xid/0knlzss-kfjwib

Export a dataset with an attribute:

https://wolfram.com/xid/0knlzss-nqj7l3

Create a scalar dataset with a single integer:

https://wolfram.com/xid/0knlzss-iw2omo

Create an array of integer numbers:

https://wolfram.com/xid/0knlzss-gz2auq

Export an array of real numbers:

https://wolfram.com/xid/0knlzss-qwxwoo

Export a numeric array with complex numbers:

https://wolfram.com/xid/0knlzss-6qx0yq

Note that complex numbers are exported to a compound dataset:

https://wolfram.com/xid/0knlzss-4ab8gq

Use the "ComplexKeys" option to get complex numbers back:

https://wolfram.com/xid/0knlzss-1r16fu

Create a dataset with strings:

https://wolfram.com/xid/0knlzss-vb1kbq

A ByteArray is stored as a "ByteArray" type (also called the opaque type in HDF5):

https://wolfram.com/xid/0knlzss-7ee5sx

Associations with string keys are exported as elements of a compound type:

https://wolfram.com/xid/0knlzss-dbvimd


https://wolfram.com/xid/0knlzss-yr6f21

Create a dataset of initial size 2×2 that can later be extended to 10 rows and arbitrarily many columns:

https://wolfram.com/xid/0knlzss-spvu1l


https://wolfram.com/xid/0knlzss-lra93i

Overwrite the first row with a list of 5 integers. This will extend the dataset to 5 columns:

https://wolfram.com/xid/0knlzss-wldmfv


https://wolfram.com/xid/0knlzss-08d0

Use the "SpaceSelection" subelement to overwrite the third and fourth columns:

https://wolfram.com/xid/0knlzss-7a3zf6


https://wolfram.com/xid/0knlzss-5qex2c

Create a 3×4 dataset that can be arbitrarily extended:

https://wolfram.com/xid/0knlzss-iqffk7


https://wolfram.com/xid/0knlzss-m9wh27

Append a 3×3 array. This can only be done along the first dimension and Export will automatically detect that:

https://wolfram.com/xid/0knlzss-v0tevy


https://wolfram.com/xid/0knlzss-2d49ng

Sometimes data dimensions do not allow for automatic extensions of the dataset:

https://wolfram.com/xid/0knlzss-vojq68


In this case, "SpaceSelection" can be specified manually:

https://wolfram.com/xid/0knlzss-7cwtv7


https://wolfram.com/xid/0knlzss-favt58

Groups (9)
HDF5 files always have a root group (with path "/"):

https://wolfram.com/xid/0knlzss-s4v9f1


https://wolfram.com/xid/0knlzss-7yi0zt

Export multiple groups by giving a list of paths:

https://wolfram.com/xid/0knlzss-urxgmy

Groups can have links to other groups or datasets:

https://wolfram.com/xid/0knlzss-cfhu1q

Soft link names cannot contain the special character "/":

https://wolfram.com/xid/0knlzss-tyadv6


Soft link target must be a valid path, without "." or "..":

https://wolfram.com/xid/0knlzss-wzacj4


If a soft link target is missing, the link becomes a dangling link:

https://wolfram.com/xid/0knlzss-q2n64o


https://wolfram.com/xid/0knlzss-x601i9

Create a file with two groups, a dataset and a hard link from one group to the dataset:

https://wolfram.com/xid/0knlzss-3jj5f5

A hard link pointing to a nonexistent location cannot be added:

https://wolfram.com/xid/0knlzss-4ascai

Create a new dataset under "/A/newDset":

https://wolfram.com/xid/0knlzss-v9rmvg

Try to redirect the "newDset" link to point to group "B":

https://wolfram.com/xid/0knlzss-w25rbl

It is only possible with "Overwrite" mode:

https://wolfram.com/xid/0knlzss-g8ltxx

Now the access to the second dataset is irreversibly lost. This is a resource leak as the dataset still occupies space in the file.
As with soft links, hard link name is always relative to the group path and must consist of exactly one path element:

https://wolfram.com/xid/0knlzss-ms1cjw

The hard link target must be a valid path, without "." or "..":

https://wolfram.com/xid/0knlzss-tgi93h

NamedDatatypes (2)
Export a simple type to the file:

https://wolfram.com/xid/0knlzss-pkml5r


https://wolfram.com/xid/0knlzss-c87hvl

Append a compound named data type to the same file:

https://wolfram.com/xid/0knlzss-7jsnm9

https://wolfram.com/xid/0knlzss-il1ikv

Use the previously exported type when exporting a dataset:

https://wolfram.com/xid/0knlzss-qcxyf8

Import data format and dimensions of the dataset:

https://wolfram.com/xid/0knlzss-ci2ufi


https://wolfram.com/xid/0knlzss-n6st7j

Named datatypes can carry attributes:

https://wolfram.com/xid/0knlzss-hc2ql4

Import Options (4)
"ComplexKeys" (1)
"TakeElements" (3)
Import three points from a dataset:

https://wolfram.com/xid/0knlzss-dm4eks

This would be equivalent to but more efficient than extracting parts after importing the whole dataset:

https://wolfram.com/xid/0knlzss-53o3as

Import a range of data from a dataset using spans:

https://wolfram.com/xid/0knlzss-uotpmr

Use suboptions for specifying offset, stride, and block:

https://wolfram.com/xid/0knlzss-hueyxc

Export Options (8)
"AppendMode" (1)
When appending data to an existing file using OverwriteTarget"Append", the export behavior for appending data can be specified using the "AppendMode" option. By default, "AppendMode""Extend" is used.

https://wolfram.com/xid/0knlzss-n39x7t


https://wolfram.com/xid/0knlzss-vh4vva


https://wolfram.com/xid/0knlzss-ivnahy

Append some attributes to "gr1/ds1" and a hard link from "gr1" to "gr2":

https://wolfram.com/xid/0knlzss-v1ynbt


https://wolfram.com/xid/0knlzss-wca813

"AppendMode""Extend" does not allow for modifying existing attributes or links:

https://wolfram.com/xid/0knlzss-d8jpol


With "AppendMode""Overwrite", it is possible to overwrite existing objects and add new ones:

https://wolfram.com/xid/0knlzss-z5q5cf


https://wolfram.com/xid/0knlzss-4gys2n

With "AppendMode""Overwrite", it is possible to modify data in existing datasets as long as data format and dimensions match:

https://wolfram.com/xid/0knlzss-d4077o


https://wolfram.com/xid/0knlzss-isvpiu

To append new objects to the file with the guarantee that no existing structures in the file will be modified, use "AppendMode""Preserve":

https://wolfram.com/xid/0knlzss-ydpt5t



ByteOrdering (1)
HDF5 allows you to choose the byte ordering. Create a big-endian file using ByteOrdering-1:

https://wolfram.com/xid/0knlzss-e6v0ht


https://wolfram.com/xid/0knlzss-u1crpc

The two files are actually different:

https://wolfram.com/xid/0knlzss-xhwpc1

Import will work correctly independently of the byte ordering:

https://wolfram.com/xid/0knlzss-5ev5pw

"ComplexKeys" (3)
By default, complex numbers are exported to a compound type using "Re" and "Im" keys:

https://wolfram.com/xid/0knlzss-grxdb4


https://wolfram.com/xid/0knlzss-k3vynl

To import the data as complex numbers, specify the keys in Import:

https://wolfram.com/xid/0knlzss-jddvue


https://wolfram.com/xid/0knlzss-syhk2f


OverwriteTarget (3)
By default, with OverwriteTargetTrue, every call to Export writes a new file:

https://wolfram.com/xid/0knlzss-631guo


https://wolfram.com/xid/0knlzss-t1r38n

With OverwriteTargetFalse, the call to Export will fail if the output file already exists:

https://wolfram.com/xid/0knlzss-l8gve0


https://wolfram.com/xid/0knlzss-718wn4


Export to certain formats, e.g. HDF5, supports the setting OverwriteTarget"Append", which makes Export act on the output file if it already exists, instead of overwriting it:

https://wolfram.com/xid/0knlzss-o7pchh


https://wolfram.com/xid/0knlzss-qdsptb

Properties & Relations (1)Properties of the function, and connections to other functions
An association is, by default, exported as a compound dataset:

https://wolfram.com/xid/0knlzss-0e7e59


https://wolfram.com/xid/0knlzss-n4o618


https://wolfram.com/xid/0knlzss-b57iiw

A list of rules is exported as multiple datasets:

https://wolfram.com/xid/0knlzss-mcz4sf


https://wolfram.com/xid/0knlzss-b4gxdr


https://wolfram.com/xid/0knlzss-v21mt5
