hls4ml.model.optimizer.passes package

Submodules

hls4ml.model.optimizer.passes.batchnorm_opt module

class hls4ml.model.optimizer.passes.batchnorm_opt.BatchNormOnnxConstantParameters

Bases: OptimizerPass

Remove Constant from the BatchNormalization node parameters (but not input[0])

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Remove Constant from the BatchNormalization node parameters (but not input[0])

TODO: Currently the quantizers are not actually used by the underlying layer.

class hls4ml.model.optimizer.passes.batchnorm_opt.ConstantBatchNormFusion

Bases: OptimizerPass

Merge BatchNorm into Const (after parameters have already been merged in BatchNormalization)

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Remove the batch norm

class hls4ml.model.optimizer.passes.batchnorm_opt.FuseConsecutiveBatchNormalization

Bases: OptimizerPass

OptimizerPass to merge consecutive BatchNormalization layers, only if the earlier one does not have the output type specified. There is a further check on the compatibility to merge: except in cases when merging a scale of 1 or a bias of 0, this does not merge when both scales or both biases are quantized.

Note: Consider restricting this to ApplyAlpha. Batch Normalization-style quantization seems to be ignored.

Note: This optimizer may not be safe if weights are updateable, in particular if a scale can go from ones to other values or if a bias can go from zeros to other values.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.batchnorm_opt.RemoveNopBatchNormalization

Bases: OptimizerPass

OptimizerPass to remove batch normalizations that do nothing (scale 1, bias 0)

Note: This optimizer may not be safe if weights are updateable.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.bn_fuse module

class hls4ml.model.optimizer.passes.bn_fuse.FuseBatchNormalization

Bases: OptimizerPass

OptimizerPass to merge a BatchNormalization layer with Dense or Conv layer, only if the Dense or Conv layer does not have the output type specified. There is a further check on the compatibility to merge: except in cases when merging a weight/scale of 1 or a bias of 0, this optimizer does not merge nodes when both the weight and scale or both biases are quantized.

Note: Consider restricting this to ApplyAlpha. Batch Normalization quantization seems to be ignored.

Note: This optimizer may not be safe if weights are updateable. May need to turn off.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Fuse weight and bias of Dense/Conv1D/Conv2D layer with BN values.

hls4ml.model.optimizer.passes.conv_to_convxd module

class hls4ml.model.optimizer.passes.conv_to_convxd.ConvToConvXD

Bases: OptimizerPass

Convert Conv with constant to a Conv1D or Conv2D layer

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Convert Conv with constant to a Conv1D or Conv2D layer

hls4ml.model.optimizer.passes.conv_to_depthwiseconvxd module

class hls4ml.model.optimizer.passes.conv_to_depthwiseconvxd.ConvToDepthwiseConvXD

Bases: OptimizerPass

Convert Conv with constant to a DepthwiseConv1D or DepthwiseConv2D layer

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Convert Conv with constant to a DepthwiseConv1D or DepthwiseConv2D layer

hls4ml.model.optimizer.passes.convert_to_channels_last module

class hls4ml.model.optimizer.passes.convert_to_channels_last.ChannelsLastConverter

Bases: OptimizerPass

Converts a model from channels_first to channels_last data format by transposing the weights of relevant layers and adding a transpose layer for the inputs and outputs, if necessary

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.convert_to_channels_last.RemoveTransposeBeforeFlatten

Bases: OptimizerPass

After the channels last conversion, model may have a sequence: Transpose -> Flatten -> Dense. In this case we can remove the expensive transpose and instead transpose the weights of the Dense layer.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.expand_layer_group module

class hls4ml.model.optimizer.passes.expand_layer_group.ExpandLayerGroup

Bases: OptimizerPass

Expands LayerGroup (a nested model) into the parent model.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.fuse_biasadd module

class hls4ml.model.optimizer.passes.fuse_biasadd.FuseBiasAdd

Bases: OptimizerPass

Fuses BiasAdd into Dense/Conv2D layer (common in TF models).

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.hgq_proxy_model module

class hls4ml.model.optimizer.passes.hgq_proxy_model.EnforceProxyModelEmbeddedConfig

Bases: OptimizerPass

match(node: Layer)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node: FixedPointQuantizer)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.hgq_proxy_model.FixedPointQuantizer(model, name, attributes, inputs, outputs=None)

Bases: Layer

initialize()

class hls4ml.model.optimizer.passes.hgq_proxy_model.UnaryLUT(model, name, attributes, inputs, outputs=None)

Bases: Layer

initialize()

hls4ml.model.optimizer.passes.hgq_proxy_model.register_hgq_proxy_model()

hls4ml.model.optimizer.passes.hgq_proxy_model.to_hls4ml_fixed(fixed: str)

hls4ml.model.optimizer.passes.hgq_proxy_model.userconf_ifdef(key: str, layer_name: str, model)

hls4ml.model.optimizer.passes.infer_precision module

class hls4ml.model.optimizer.passes.infer_precision.InferPrecisionTypes

Bases: ConfigurableOptimizerPass

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.linear module

class hls4ml.model.optimizer.passes.linear.EliminateLinearActivation

Bases: OptimizerPass

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.linear.MergeLinearActivation

Bases: OptimizerPass

For many objects it’s safe to change the output precision independently of the calculation.

match(node)

Only match if the parent is safe and the precision is not explicitly set.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.matmul_const_to_dense module

class hls4ml.model.optimizer.passes.matmul_const_to_dense.MatmulConstToDense

Bases: OptimizerPass

Convert MatMul with constant to a dense layer. Note, this only supports the second input being the constant. If needed, one could add transposes to make that be the case in other yet to be written optimizers.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Substitute Matmul + Constant for a single dense

hls4ml.model.optimizer.passes.merge_const module

class hls4ml.model.optimizer.passes.merge_const.MergeToApplyAlpha

Bases: OptimizerPass

Convert Add, Sub, Mul, or Div Merges with constant to ApplyAlpha

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.merge_const.MergeToApplyAlphaDiv

Bases: OptimizerPass

Convert Div Merges with constant to ApplyAlpha

TODO: propagate precision

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.merge_const.MergeTwoConstants

Bases: OptimizerPass

Merge of two constants makes another constant

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Merge of two constants makes another constant.

Note: full precision is used in the calculation, and precision is not propagated. The precision

hls4ml.model.optimizer.passes.move_scales module

This file includes optimizations related to moving the ApplyAphas across MatMul and Conv nodes.

TODO: Check that biases are properly handled. (Attempt to do it via Merge)

class hls4ml.model.optimizer.passes.move_scales.BiasDownAdd

Bases: OptimizerPass

Shift a ApplyAlpha with only bias below a Merge (Add)

match(node)

Match if there is only one ApplyAlpha. If there are two, if the scale of both is 0, they would match the ScaleDownAdd, so this optimizer does not need to handle that case.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.move_scales.ScaleDownAdd

Bases: OptimizerPass

Shift an identical ApplyAlpha below a Merge (Add)

match(node)

Check to see if we have an add with two ApplyAlphas with identical scale

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.move_scales.ScaleDownConv

Bases: OptimizerPass

Shift an ApplyAlpha on a Conv with 2-3 inputs

match(node)

Shift an ApplyAlpha from the Weight

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.move_scales.ScaleDownMatMul

Bases: OptimizerPass

Shift an ApplyAlpha below a MatMul

match(node)

Check to see if we have a MatMul with at least one input ApplyAlpha. Note, if both are this optimizer runs twice.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.multi_dense module

class hls4ml.model.optimizer.passes.multi_dense.ReplaceMultidimensionalDenseWithConv

Bases: OptimizerPass

This matches all multidimensional Dense layers and changes them to a convolution. Note: the convolution may subsequently be changed to a pointwise convolution for bakends that implement special pointwise convolutions.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.qkeras module

class hls4ml.model.optimizer.passes.qkeras.ExtractTernaryThreshold

Bases: OptimizerPass

The input value (threshold) at which the output of a a ternary activation changes is configurable. This pass extracts that threshold point, inserting a BatchNormalization layer to execute the scaling. That BatchNormalization layer is then expected to be fused into a BatchNormalizationQuantizedTanh layer configured with the correct threshold.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.qkeras.OutputRoundingSaturationMode

Bases: ConfigurableOptimizerPass

Set the Rounding and Saturation mode of the output (and accumulator, if applicable) of the layers specific in layer list. The layer list is empty by default. To specify which layer to apply this pass to, perform e.g.: hls4ml.model.optimizer.get_optimizer(‘output_rounding_saturation_mode’).configure(layers=[‘Dense’, ‘Activation’]) The Rounding and Saturation modes are ‘None’ by default (so use the compiler defaults) To set which mode to use: hls4ml.model.optimizer.get_optimizer(‘output_rounding_saturation_mode’).configure(rounding_mode=’AP_RND_CONV’) hls4ml.model.optimizer.get_optimizer(‘output_rounding_saturation_mode’).configure(saturation_mode=’AP_SAT’)

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

precision_string_modify(pstr)

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.qkeras.QKerasFactorizeAlpha

Bases: OptimizerPass

OptimizerPass for extracting alpha “scale” from QKeras quantized layer. The weights of the Q{Dense, Conv} layer are scaled to the common data type, and an ‘ApplyAlpha’ layer is inserted to reapply the scale.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.qkeras.register_qkeras()

hls4ml.model.optimizer.passes.quant_opt module

This file includes optimizations related to quant nodes.

As a first step, QuantConstantParameters converts the extra inputs to attributes.

The next step differs between the case of (1) (positive) power-of-2 scale and zero offset, or (2) other cases. In the first case no explicit scaling is required, so a Quant node logically becomes a linear activation. (Cases when the scale is a power of 2 not equal to one are implicitly scaled with fixed precision types.) When the activation is applied to a constant weight, the activation is immediately merged with the weight, quantizing the weights. In case (2), we need to explicitly scale and unscale, so the Quant node becomes 3 nodes, an ApplyAlpha node to apply a scale/shift, a Linear node to apply the quantization, and another ApplyAlpha to unscale/shift. We depend on optimization steps to move the unscaling ApplyAlpha down as needed so that we can do integer or fixed-point calculations. When the Quant is a applied to a weight, the scaling and Linear nodes are immediately merged into the Constant.

class hls4ml.model.optimizer.passes.quant_opt.ConstQuantToConstAlpha

Bases: OptimizerPass

This is for the case when scale is not power-of-2 or zeropt is not 0. It is a a 1:3 transformation of a Quant to an ApplyAlpha (to scale), Activation, ApplyAlpho (to unscale), but an input consts allows for optimization, so the ApplyAlpha (to scale), Activation are optimized away right away.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Change Constant + Quant node to Constant, ApplyAlpha

class hls4ml.model.optimizer.passes.quant_opt.FuseQuantWithConstant

Bases: OptimizerPass

This is for the case when scale is a positive power of 2 and zeropt is 0.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Fuse Quant with Constant.

class hls4ml.model.optimizer.passes.quant_opt.QuantConstantParameters

Bases: OptimizerPass

Remove Constant from the Qaunt node parameters (but not input[0])

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Remove Constant from the Quant node parameters (but not input[0])

class hls4ml.model.optimizer.passes.quant_opt.QuantToActivation

Bases: OptimizerPass

This is for the case when scale is a (positive) power of 2 and zeropt is 0. It is a a 1:1 transformation of a Quant to an Activation.

As an optimization, this is not called when the input is constant.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Change quant node to Activation

class hls4ml.model.optimizer.passes.quant_opt.QuantToAlphaActivationAlpha

Bases: OptimizerPass

This is for the case when scale is not power-of-2 or zeropt is not 0. It is a a 1:3 transformation of a Quant to an ApplyAlpha (to scale), Activatio, ApplyAlpho (to rescale).

NOTE: It needs to be scheduled after QuantToActivation (or we need to make the match criteria stricter)

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Change quant node to ApplyAlhpa, Activation, ApplyAlpha

hls4ml.model.optimizer.passes.reshape_const module

class hls4ml.model.optimizer.passes.reshape_const.ReshapeConstant

Bases: OptimizerPass

ONNX has the target shape come as an input, not a parameter. This removes the Constant input from new shape input. (Non-constant inputs are not supported.) The constant value was already used; this is just a cleanup uptimization.

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Remove Constant from new shape input. Note, input shape node is already used on initialize

hls4ml.model.optimizer.passes.seperable_to_dw_conv module

This optimizer converts a seperable convolution to a depthwise followed by a regular convolution. For backends with a custom pointwise implementations the regular convolution will subsequently be converted to a pointwise convolution by a different optimizer.

class hls4ml.model.optimizer.passes.seperable_to_dw_conv.SeperableToDepthwiseAndConv

Bases: OptimizerPass

Convert Seperable to DepthwiseConv + Conv (potentially later Pointwise)

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.stamp module

class hls4ml.model.optimizer.passes.stamp.MakeStamp

Bases: ModelOptimizerPass

transform(model)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

hls4ml.model.optimizer.passes.transpose_opt module

class hls4ml.model.optimizer.passes.transpose_opt.RemoveNopTranspose

Bases: OptimizerPass

Remove a transpose layer if it doesn’t do anything to a 1D array. i.e, 1D input and perm = [0]

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

class hls4ml.model.optimizer.passes.transpose_opt.RemoveSingleChannelTranspose

Bases: OptimizerPass

Remove transpose of inputs if the number of channels is 1 as for io_parallel this doesn’t affect the array representation used

match(node)

Predicate to match on a given node.

Parameters:

node (Layer) – Node in the model graph to try matching the optimizer on.

transform(model, node)

Transformation to apply if matching was successful.

Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).

Parameters:

• model (ModelGraph) – Model to optimize

• node (Layer) – The matched node in the model graph.

Module contents