Frontends and Backends
hls4ml there is a a concept of a frontend to parse the input NN into an internal model graph, and a backend that controls
what type of output is produced from the graph. Frontends and backends can be independently chosen. Examples of frontends are the
parsers for Keras or ONNX, and examples of backends are Vivado HLS, Intel HLS, and Vitis HLS. See Status and Features for the
currently supported frontends and backends.
hls4ml supports multiple styles for handling data between layers, known as the
Data is passed in parallel between the layers. This is good for MLP networks and small CNNs. Synthesis may fail for larger networks.
Data is passed one “pixel” at a time. Each pixel is an array of channels, which are always sent in parallel. This method for sending
data between layers is recommended for larger CNNs. For
Dense layers, all the inputs are streamed in parallel as a single array.
io_stream IO type, each layer is connected with the subsequent layer through first-in first-out (FIFO) buffers.
The implementation of the FIFO buffers contribute to the overall resource utilization of the design, impacting in particular the BRAM or LUT utilization.
Because the neural networks can have complex architectures generally, it is hard to know a priori the correct depth of each FIFO buffer.
hls4ml choses the most conservative possible depth for each FIFO buffer, which can result in a an unnecessary overutilization of resources.
In order to reduce the impact on the resources used for FIFO buffer implementation, we have a FIFO depth optimization flow. This is described in the FIFO Buffer Depth Optimization section.