NetInsertSharedArrays

NetInsertSharedArrays[net]

converts all ordinary arrays in net into NetSharedArray objects.

NetInsertSharedArrays[net,"prefix"]

uses a prefix for the names of all newly shared arrays.

Details

  • Each array in the net will be shared using a name that is derived from its position within the net.
  • By sharing all arrays in a net, multiple instances of that net within a larger net will behave identically during training and evaluation.

Examples

open allclose all

Basic Examples  (4)

Convert arrays in a layer into shared arrays:

Convert arrays in a layer into shared arrays, using a specific prefix:

Convert arrays in layers within a net into shared arrays:

Create an LSTM layer whose arrays will be shared every time it is used in a larger net:

Use the LSTM layer on two separate inputs in a graph and combine the resulting states with a CatenateLayer:

Properties & Relations  (1)

Insert shared arrays into a layer:

When used multiple times in a larger network, only one set of weights will be created and stored:

When extracted, the weights for layers 1 and 3 are identical:

The underlying shared array can also be extracted using NetExtract:

The amount of storage required for arrays in the entire net is the same as the storage required for a single layer:

Possible Issues  (1)

The "MovingMean" and "MovingVariance" arrays of BatchNormalizationLayer cannot be shared.

Create a BatchNormalizationLayer with shared arrays:

Train it on some data:

Extract the trained batch normalization layers:

The "Scaling" and "Biases" arrays were shared, but not "MovingMean" or "MovingVariance":

Wolfram Research (2018), NetInsertSharedArrays, Wolfram Language function, https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html (updated 2019).

Text

Wolfram Research (2018), NetInsertSharedArrays, Wolfram Language function, https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html (updated 2019).

CMS

Wolfram Language. 2018. "NetInsertSharedArrays." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2019. https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html.

APA

Wolfram Language. (2018). NetInsertSharedArrays. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html

BibTeX

@misc{reference.wolfram_2024_netinsertsharedarrays, author="Wolfram Research", title="{NetInsertSharedArrays}", year="2019", howpublished="\url{https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html}", note=[Accessed: 22-November-2024 ]}

BibLaTeX

@online{reference.wolfram_2024_netinsertsharedarrays, organization={Wolfram Research}, title={NetInsertSharedArrays}, year={2019}, url={https://reference.wolfram.com/language/ref/NetInsertSharedArrays.html}, note=[Accessed: 22-November-2024 ]}