hls4ml.model.optimizer.passes package
Submodules
hls4ml.model.optimizer.passes.batchnorm_opt module
- class hls4ml.model.optimizer.passes.batchnorm_opt.BatchNormOnnxConstantParameters
Bases:
OptimizerPass
Remove Constant from the BatchNormalization node parameters (but not input[0])
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Remove Constant from the BatchNormalization node parameters (but not input[0])
TODO: Currently the quantizers are not actually used by the underlying layer.
- class hls4ml.model.optimizer.passes.batchnorm_opt.ConstantBatchNormFusion
Bases:
OptimizerPass
Merge BatchNorm into Const (after parameters have already been merged in BatchNormalization)
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Remove the batch norm
- class hls4ml.model.optimizer.passes.batchnorm_opt.FuseConsecutiveBatchNormalization
Bases:
OptimizerPass
OptimizerPass to merge consecutive BatchNormalization layers, only if the earlier one does not have the output type specified. There is a further check on the compatibility to merge: except in cases when merging a scale of 1 or a bias of 0, this does not merge when both scales or both biases are quantized.
Note: Consider restricting this to ApplyAlpha. Batch Normalization-style quantization seems to be ignored.
Note: This optimizer may not be safe if weights are updateable, in particular if a scale can go from ones to other values or if a bias can go from zeros to other values.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.batchnorm_opt.RemoveNopBatchNormalization
Bases:
OptimizerPass
OptimizerPass to remove batch normalizations that do nothing (scale 1, bias 0)
Note: This optimizer may not be safe if weights are updateable.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.bn_fuse module
- class hls4ml.model.optimizer.passes.bn_fuse.FuseBatchNormalization
Bases:
OptimizerPass
OptimizerPass to merge a BatchNormalization layer with Dense or Conv layer, only if the Dense or Conv layer does not have the output type specified. There is a further check on the compatibility to merge: except in cases when merging a weight/scale of 1 or a bias of 0, this optimizer does not merge nodes when both the weight and scale or both biases are quantized.
Note: Consider restricting this to ApplyAlpha. Batch Normalization quantization seems to be ignored.
Note: This optimizer may not be safe if weights are updateable. May need to turn off.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Fuse weight and bias of Dense/Conv1D/Conv2D layer with BN values.
hls4ml.model.optimizer.passes.conv_to_convxd module
- class hls4ml.model.optimizer.passes.conv_to_convxd.ConvToConvXD
Bases:
OptimizerPass
Convert Conv with constant to a Conv1D or Conv2D layer
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Convert Conv with constant to a Conv1D or Conv2D layer
hls4ml.model.optimizer.passes.conv_to_depthwiseconvxd module
- class hls4ml.model.optimizer.passes.conv_to_depthwiseconvxd.ConvToDepthwiseConvXD
Bases:
OptimizerPass
Convert Conv with constant to a DepthwiseConv1D or DepthwiseConv2D layer
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Convert Conv with constant to a DepthwiseConv1D or DepthwiseConv2D layer
hls4ml.model.optimizer.passes.convert_to_channels_last module
- class hls4ml.model.optimizer.passes.convert_to_channels_last.ChannelsLastConverter
Bases:
OptimizerPass
Converts a model from channels_first to channels_last data format by transposing the weights of relevant layers and adding a transpose layer for the inputs and outputs, if necessary
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.convert_to_channels_last.RemoveTransposeBeforeFlatten
Bases:
OptimizerPass
After the channels last conversion, model may have a sequence: Transpose -> Flatten -> Dense. In this case we can remove the expensive transpose and instead transpose the weights of the Dense layer.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.expand_layer_group module
- class hls4ml.model.optimizer.passes.expand_layer_group.ExpandLayerGroup
Bases:
OptimizerPass
Expands LayerGroup (a nested model) into the parent model.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.fuse_biasadd module
- class hls4ml.model.optimizer.passes.fuse_biasadd.FuseBiasAdd
Bases:
OptimizerPass
Fuses BiasAdd into Dense/Conv2D layer (common in TF models).
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.hgq_proxy_model module
- class hls4ml.model.optimizer.passes.hgq_proxy_model.EnforceProxyModelEmbeddedConfig
Bases:
OptimizerPass
- match(node: Layer)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node: FixedPointQuantizer)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.hgq_proxy_model.FixedPointQuantizer(model, name, attributes, inputs, outputs=None)
Bases:
Layer
- initialize()
- class hls4ml.model.optimizer.passes.hgq_proxy_model.UnaryLUT(model, name, attributes, inputs, outputs=None)
Bases:
Layer
- initialize()
- hls4ml.model.optimizer.passes.hgq_proxy_model.register_hgq_proxy_model()
- hls4ml.model.optimizer.passes.hgq_proxy_model.to_hls4ml_fixed(fixed: str)
- hls4ml.model.optimizer.passes.hgq_proxy_model.userconf_ifdef(key: str, layer_name: str, model)
hls4ml.model.optimizer.passes.infer_precision module
- class hls4ml.model.optimizer.passes.infer_precision.InferPrecisionTypes
Bases:
ConfigurableOptimizerPass
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.linear module
- class hls4ml.model.optimizer.passes.linear.EliminateLinearActivation
Bases:
OptimizerPass
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.linear.MergeLinearActivation
Bases:
OptimizerPass
For many objects it’s safe to change the output precision independently of the calculation.
- match(node)
Only match if the parent is safe and the precision is not explicitly set.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.matmul_const_to_dense module
- class hls4ml.model.optimizer.passes.matmul_const_to_dense.MatmulConstToDense
Bases:
OptimizerPass
Convert MatMul with constant to a dense layer. Note, this only supports the second input being the constant. If needed, one could add transposes to make that be the case in other yet to be written optimizers.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Substitute Matmul + Constant for a single dense
hls4ml.model.optimizer.passes.merge_const module
- class hls4ml.model.optimizer.passes.merge_const.MergeToApplyAlpha
Bases:
OptimizerPass
Convert Add, Sub, Mul, or Div Merges with constant to ApplyAlpha
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.merge_const.MergeToApplyAlphaDiv
Bases:
OptimizerPass
Convert Div Merges with constant to ApplyAlpha
TODO: propagate precision
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.merge_const.MergeTwoConstants
Bases:
OptimizerPass
Merge of two constants makes another constant
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Merge of two constants makes another constant.
Note: full precision is used in the calculation, and precision is not propagated. The precision
hls4ml.model.optimizer.passes.move_scales module
This file includes optimizations related to moving the ApplyAphas across MatMul and Conv nodes.
TODO: Check that biases are properly handled. (Attempt to do it via Merge)
- class hls4ml.model.optimizer.passes.move_scales.BiasDownAdd
Bases:
OptimizerPass
Shift a ApplyAlpha with only bias below a Merge (Add)
- match(node)
Match if there is only one ApplyAlpha. If there are two, if the scale of both is 0, they would match the ScaleDownAdd, so this optimizer does not need to handle that case.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.move_scales.ScaleDownAdd
Bases:
OptimizerPass
Shift an identical ApplyAlpha below a Merge (Add)
- match(node)
Check to see if we have an add with two ApplyAlphas with identical scale
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.move_scales.ScaleDownConv
Bases:
OptimizerPass
Shift an ApplyAlpha on a Conv with 2-3 inputs
- match(node)
Shift an ApplyAlpha from the Weight
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.move_scales.ScaleDownMatMul
Bases:
OptimizerPass
Shift an ApplyAlpha below a MatMul
- match(node)
Check to see if we have a MatMul with at least one input ApplyAlpha. Note, if both are this optimizer runs twice.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.multi_dense module
- class hls4ml.model.optimizer.passes.multi_dense.ReplaceMultidimensionalDenseWithConv
Bases:
OptimizerPass
This matches all multidimensional Dense layers and changes them to a convolution. Note: the convolution may subsequently be changed to a pointwise convolution for bakends that implement special pointwise convolutions.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.qkeras module
- class hls4ml.model.optimizer.passes.qkeras.ExtractTernaryThreshold
Bases:
OptimizerPass
The input value (threshold) at which the output of a a ternary activation changes is configurable. This pass extracts that threshold point, inserting a BatchNormalization layer to execute the scaling. That BatchNormalization layer is then expected to be fused into a BatchNormalizationQuantizedTanh layer configured with the correct threshold.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.qkeras.OutputRoundingSaturationMode
Bases:
ConfigurableOptimizerPass
Set the Rounding and Saturation mode of the output (and accumulator, if applicable) of the layers specific in layer list. The layer list is empty by default. To specify which layer to apply this pass to, perform e.g.: hls4ml.model.optimizer.get_optimizer(‘output_rounding_saturation_mode’).configure(layers=[‘Dense’, ‘Activation’]) The Rounding and Saturation modes are ‘None’ by default (so use the compiler defaults) To set which mode to use: hls4ml.model.optimizer.get_optimizer(‘output_rounding_saturation_mode’).configure(rounding_mode=’AP_RND_CONV’) hls4ml.model.optimizer.get_optimizer(‘output_rounding_saturation_mode’).configure(saturation_mode=’AP_SAT’)
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- precision_string_modify(pstr)
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.qkeras.QKerasFactorizeAlpha
Bases:
OptimizerPass
OptimizerPass for extracting alpha “scale” from QKeras quantized layer. The weights of the Q{Dense, Conv} layer are scaled to the common data type, and an ‘ApplyAlpha’ layer is inserted to reapply the scale.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- hls4ml.model.optimizer.passes.qkeras.register_qkeras()
hls4ml.model.optimizer.passes.quant_opt module
This file includes optimizations related to quant nodes.
As a first step, QuantConstantParameters converts the extra inputs to attributes.
The next step differs between the case of (1) (positive) power-of-2 scale and zero offset, or (2) other cases. In the first case no explicit scaling is required, so a Quant node logically becomes a linear activation. (Cases when the scale is a power of 2 not equal to one are implicitly scaled with fixed precision types.) When the activation is applied to a constant weight, the activation is immediately merged with the weight, quantizing the weights. In case (2), we need to explicitly scale and unscale, so the Quant node becomes 3 nodes, an ApplyAlpha node to apply a scale/shift, a Linear node to apply the quantization, and another ApplyAlpha to unscale/shift. We depend on optimization steps to move the unscaling ApplyAlpha down as needed so that we can do integer or fixed-point calculations. When the Quant is a applied to a weight, the scaling and Linear nodes are immediately merged into the Constant.
- class hls4ml.model.optimizer.passes.quant_opt.ConstQuantToConstAlpha
Bases:
OptimizerPass
This is for the case when scale is not power-of-2 or zeropt is not 0. It is a a 1:3 transformation of a Quant to an ApplyAlpha (to scale), Activation, ApplyAlpho (to unscale), but an input consts allows for optimization, so the ApplyAlpha (to scale), Activation are optimized away right away.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Change Constant + Quant node to Constant, ApplyAlpha
- class hls4ml.model.optimizer.passes.quant_opt.FuseQuantWithConstant
Bases:
OptimizerPass
This is for the case when scale is a positive power of 2 and zeropt is 0.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Fuse Quant with Constant.
- class hls4ml.model.optimizer.passes.quant_opt.QuantConstantParameters
Bases:
OptimizerPass
Remove Constant from the Qaunt node parameters (but not input[0])
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Remove Constant from the Quant node parameters (but not input[0])
- class hls4ml.model.optimizer.passes.quant_opt.QuantToActivation
Bases:
OptimizerPass
This is for the case when scale is a (positive) power of 2 and zeropt is 0. It is a a 1:1 transformation of a Quant to an Activation.
As an optimization, this is not called when the input is constant.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Change quant node to Activation
- class hls4ml.model.optimizer.passes.quant_opt.QuantToAlphaActivationAlpha
Bases:
OptimizerPass
This is for the case when scale is not power-of-2 or zeropt is not 0. It is a a 1:3 transformation of a Quant to an ApplyAlpha (to scale), Activatio, ApplyAlpho (to rescale).
NOTE: It needs to be scheduled after QuantToActivation (or we need to make the match criteria stricter)
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Change quant node to ApplyAlhpa, Activation, ApplyAlpha
hls4ml.model.optimizer.passes.reshape_const module
- class hls4ml.model.optimizer.passes.reshape_const.ReshapeConstant
Bases:
OptimizerPass
ONNX has the target shape come as an input, not a parameter. This removes the Constant input from new shape input. (Non-constant inputs are not supported.) The constant value was already used; this is just a cleanup uptimization.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Remove Constant from new shape input. Note, input shape node is already used on initialize
hls4ml.model.optimizer.passes.resize_remove_constants module
- class hls4ml.model.optimizer.passes.resize_remove_constants.ResizeRemoveConstants
Bases:
OptimizerPass
This optimizer is intended to clean the Resize node from RoI and Scales parameters that if left cause issues in hls4ml.
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Remove RoI and Scale Constant from new shape input.
hls4ml.model.optimizer.passes.seperable_to_dw_conv module
This optimizer converts a separable convolution to a depthwise followed by a regular convolution. For backends with a custom pointwise implementations the regular convolution will subsequently be converted to a pointwise convolution by a different optimizer.
- class hls4ml.model.optimizer.passes.seperable_to_dw_conv.SeparableToDepthwiseAndConv
Bases:
OptimizerPass
Convert Separable to DepthwiseConv + Conv (potentially later Pointwise)
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.stamp module
- class hls4ml.model.optimizer.passes.stamp.MakeStamp
Bases:
ModelOptimizerPass
- transform(model)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
hls4ml.model.optimizer.passes.transpose_opt module
- class hls4ml.model.optimizer.passes.transpose_opt.RemoveNopTranspose
Bases:
OptimizerPass
Remove a transpose layer if it doesn’t do anything to a 1D array. i.e, 1D input and perm = [0]
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.
- class hls4ml.model.optimizer.passes.transpose_opt.RemoveSingleChannelTranspose
Bases:
OptimizerPass
Remove transpose of inputs if the number of channels is 1 as for io_parallel this doesn’t affect the array representation used
- match(node)
Predicate to match on a given node.
- Parameters:
node (Layer) – Node in the model graph to try matching the optimizer on.
- transform(model, node)
Transformation to apply if matching was successful.
Transform should return a boolean value indicating if the model graph was altered (by adding/removing nodes).
- Parameters:
model (ModelGraph) – Model to optimize
node (Layer) – The matched node in the model graph.